Before you start: if you’re unfamiliar with the principles of statistical SEO split-testing and how SplitSignal works, we’re suggesting you start here or request a demo of SplitSignal. 

First, we asked our Twitter followers to vote:

This time the majority of our followers were RIGHT! Read further to find out why.

But first, let’s see what SEO professionals had to say about the result of this test:

Uroš Stanimirović, Content Writer & Editor at Brid.TV:

Frankly, I believe it would depend on what Google decides to do with it.
If nothing, it shouldn’t impact organic CTR.
If it turns the result into a featured snippet, it would probably reduce CTR (if the result was previously in position 1).

Trevor Stolber, Digital Marketer at STOLBER.com:

Based on some recent language in Google’s documentation you would say yes. I think this will have minimal impact—possibly even showing no significant change.

Follow us on LinkedIn to share your thoughts on the next test.

The Case Study

Structured data is widely known for its search results features and enhancements in the form of rich snippets. However, structured data is much more than just a “final sprinkle” to get a nice search result. With structured data, you can move from “strings” to “things.”

Structured data is a standardized format for providing information about a page and classifying the page content. It gives meaning to “things” (entities) and explains the context and relationships between different pieces of content across the web. 

Google says it uses structured data it finds on the web to understand the content of a page and gather information about the web and the world in general. From an SEO perspective, the purpose of structured data is to help search engines better understand the meaning of the content (and link it to user intent) so that your content is chosen as the best answer to a search query. 

For an online marketplace for software applications, we wanted to give Google more context about the content on their listing pages. Because these pages are not just a collection of links but a (summary) collection page full of software application listings, referring to software applications living elsewhere on the site, all of which relate to the list page and other linked software applications on the website.

The Hypothesis

The website in question wanted to test if adding ItemList structured data to their listing pages would have a significant impact on organic traffic. 

ItemList structured data is used to mark a list of “things.” A list can be of any type. In this case, we have nested SoftwareApplication(s), and list items must be of the same schema type. 

We hypothesized that by adding ItemList structured data with nested SoftwareApplication(s) we would be able to give Google more context about the page and its relationship to SoftwareApplication(s) across the website. While this wouldn’t result in a rich snippet, we hypothesized it would have a positive effect on organic traffic to the tested pages.

The Test

We used SplitSignal to set up the test. About 1,500 listing pages were selected as either control or variant. 

To implement ItemList structured data on the variant pages, we created a custom script that loops through the HTML of the software application items on a listing page. The script grabbed all relevant information on the page for each software application and placed it in standardized schema.org format.

We started the test and ran it for 30 days. We were able to determine that Googlebot visited 80% of the pages tested.

The Results

Adding ItemList structured data with nested SoftwareApplication(s) on listing pages resulted in a 4.6% increase in clicks.

After just over two weeks, we were sure that the effect we saw was significant. When the blue shaded area performs below or above the x=0 axis, the test is statistically significant at the 95% level. This means that we can be confident that the effect we see is due to the change we made and not to other (external) factors.

Note that we are not comparing the actual control group pages to our tested pages. but rather a forecast based on historical data. The model predicts the counterfactual response that would have occurred had no intervention taken place. We compare this with the actual data. We use a set of control pages to give the model context for trends and external influences. If something else changes during our test (e.g., seasonality), the model will detect and take it into account. By filtering these external factors, we gain insight into the true impact of an SEO change.

Why

This test shows that enriching your search results with rich snippets is just the tip of the iceberg. Using structured data to give search engines more context and semantically connect web pages can positively impact SEO performance.

Data analysis shows that this test did not only affect clicks to the tested pages. Compared to our modeled control group, we also found an increase in impressions. Further analysis showed that this was due to better positioning and increased visibility on a broader set of search terms.

We believe this change has increased the relevance of the tested pages. Google could better evaluate a page and its content and more accurately determine whether it matched a Google user’s intent.

We have decided to roll out similar setups of this test across multiple websites. You can expect more on this topic in future case studies.

Keep in mind that something that works for one website may not work for another. The only way to know for sure is to test what works for you!