Before you start: what do you know about SEO split-testing? If you’re unfamiliar with the principles of statistical SEO split-testing and how SplitSignal works, we're suggesting you start here or request a demo of SplitSignal. 

First, we asked our Twitter followers to vote:

Only 31% of our followers guessed it right, the test result was positive.

Read the full case study to find out why.

The Case Study

If you've been involved in writing or optimizing content from an SEO perspective, you've probably thought about the most optimal heading structure for your content. Google recently reaffirmed that to understand what a web page's content is about, they look at different things, such as headings, to figure out what is actually being emphasized. Proper use of semantic HTML can make content more meaningful. In return, it helps search engines understand the page a little better.

Headings are defined in the HTML of a page. They provide hierarchy and help users and search engines read and understand textual content. Headings show which parts of your content are paramount. There are six types of headings: H1 through H6, with H1 being the most important and H6 the least important.

For a major ecommerce company in the Netherlands, we restructured the headings on their category pages to see if it would make a difference in terms of organic performance.

The Hypothesis

The website in question had its heading structure set up like this, where we’ve changed the H3 to H2:

We saw that the website’s H3 subheadings were not necessarily subtopics of the previous H2 heading but, from a hierarchical perspective, subtopics of the pages’ H1 heading.

We wanted to validate whether restructuring the H3 headings on their category pages to H2 headings would positively affect organic traffic, so that’s what we did.

The Test

We used SplitSignal to set up and analyze the test. 335 category pages were selected as either variant or control. For the test, we changed the website’s H3 headings to H2 headings, with the idea that it would give search engines a better representation of web page content.

The Results

We saw that organic traffic (clicks) to the tested pages outperformed our modelled control group. This means that the traffic to the variant group is performing better than predicted and, the test is positive. 

After running the test for 21 days, we saw a 4.5% increase in clicks, with a 99% confidence level. This indicates that the increase in clicks is attributable to the optimization we have made.

Why?

As mentioned at the beginning, proper use of semantic HTML can make content more meaningful. The hierarchical heading structure was not optimal for the website in question. They nested certain headings as H3 headings, where it would make more sense to use an H2 heading, as the headings were subtopics of the pages’ H1 heading. By optimizing the heading structure, we were able to give search engines the right signals.

Analysis of the data shows that the pages tested started to perform better in search results. Not only did we see an increase in the number of clicks, but we also saw an increase in the number of impressions, suggesting that the pages tested became more relevant to different queries that the pages were ranking for. 

This test validates that as an SEO professional, you need to think about and experiment with your heading structure to give search engines (and users) the best possible answer. In return, you can get more out of your organic performance. 

Given the impact this can have on your overall search performance, thoroughly testing these “small” optimizations is a good idea. For the website in question, this was a real eye-opener and paved the way for much more detailed testing.

Sign up for our newsletter with the most recent case studies via the request form.

Have your next SEO split-test analyzed by OrangeValley Agency.