Before you start: what do you know about SEO split-testing? If you’re unfamiliar with the principles of statistical SEO split-testing and how SplitSignal works, we suggest you start here or request a demo. 

First, we asked our Twitter followers to vote:

Here’s what other SEO professionals have to share about this test:

Bostjan Tanko, SEO Specialist at APOLLO Insurance:

In my opinion, an article or landing page with more than one H1 is confusing for Google to determine what the page is about.
In the example you have added: https://khn.org/ is a different story since their home page acts as a portal for different pages, so in this case, it won’t make a difference, and it is fine/normal to have more than one H1.

Vladimir Gertner, Senior Project Manager at Soft Road Apps:

Well, the <span> element is there to define styling. It doesn’t necessarily influence the rankings in and of itself, as it’s not semantic. It’s not used to add more weight to the article.
Also, John Muller said that multiple H1s aren’t a problem as far as SEO goes. Since we can’t trust anything that man says I would say we can expect a positive change.
As far as actual official data goes, this should be no significant change. Maybe 4% up given that there are changes done, and they aren’t negative at all.
However, if you are using <span> instead of a heading tag, I wouldn’t expect perfect results. H2, for example, sends a message to Google that it’s how the article is designed to be, and G crawlers check for H2s.
So, if you didn’t include any heading tags and just used <span> instead of a heading tag, I would say, not ideal.

Find out if our followers were right by reading the complete analysis of this test.

The Case Study

Hey Everyone, welcome back to a new and interesting case study. 

The website we’re looking at is a non-English content aggregation website whose individual articles are not formatted correctly. The primary issue is that rather than using appropriately nested heading tags (H1, H2, H3, etc.), every heading is an H1.

The Hypothesis

We hypothesized that by leaving the top H1 heading in place and converting the others to <span> tags, we would see an improvement in clicks by eliminating confusion as to which heading was the preferred heading.

It seems straightforward; what do you think the result will be? Read on, and let’s find out!

The Test

The SEO experiment was set up and configured using our SEO testing tool, SplitSignal. This isn’t a particularly large website, and it has a little over 50,000 indexed pages, so we selected 335 total pages, 158 for the test control, and 177 for the test variant. How does SplitSignal split pages?

On all 177 test pages, we left the top H1 in place and dynamically converted the remainders to <span> tags. The test ran for 21 days with a confidence level of 99%, with 100% of the test pages being recrawled by Googlebot during the period for testing.

The Results

After the twenty-one-day period, we observed a statistically significant, NEGATIVE result for the pages with the converted H1 headings.

We asked you earlier what you thought would happen. The enterprise SEOs at LOCOMOTIVE Agency came in after the experiment and have given their explanation of why they believe this happened. Still, we'd love to hear your opinion in the comments of what you originally thought and whether you think it could be something different?

Overall, the test pages received 2.4% fewer clicks during the test period, a loss of 2,680 clicks. 

LOCOMOTIVE Agency Analysis

As we’ve said before, it’s getting harder to surprise us, and as long-time SEOs, this result was one we would have anticipated based on knowing how Google often has to correct for or make assumptions about development mistakes.

For as long as there have been web standards, and search engines trying to use algorithms, machine learning, machine vision, etc. to understand websites, web developers have not always followed those standards to the letter. This leaves search engines with the unenviable task of trying to figure out, what the heck we mean when we did something in an unconventional fashion.

Thankfully, as we see in this test, Google has gotten pretty good at it over the years, and was likely already ignoring the extra H1 headings, and making assumptions about them being H2s, H3s, and so on.

Removing the H1s and converting them to an inline element like a <span> tag, removed the priority given to them by the heading element, albeit applied incorrectly, and resulted in a loss of relevance of those headings to the overall topical authority of the page, and therefore a decline in average ranking, and causally, a decline in clicks to the pages.

It would have been interesting to see a similar test that, rather than converting the H1s to <span>, rather converted them to appropriately nested headings, however, as each article is different, this would have been difficult to achieve without making blind assumptions moving forward. Still, a test might reveal more about the assumptions Google is making about this page.

What do you think?

Have your next SEO split-test analyzed by the technical SEO experts at LOCOMOTIVE Agency.