Before you start: if you’re unfamiliar with the principles of statistical SEO split-testing and how SplitSignal works, we're suggesting you start here or request a demo of SplitSignal. 

First, we asked our Twitter followers to vote:

Here’s what other SEO Experts from the All-Stars Facebook group have to share about this test and the outcome:

Trevor Stolber, Digital Marketer at STOLBER.com

I think organic clicks will be down slightly for the variant. There are many studies that show a number (and interestingly an odd number performs better) for list-type content works best. This is different, but I feel like the intent here is to look for a lot of choices - so a large number like 10664 would perform better.

Edward Roberts, SEO Manager at Hovia

I’m going with an uptick in organic clicks from the variant. I think angry dog images & pictures do a better job of appealing to users searching for related terms such as angry dog graphics, wallpaper, stock images, etc. Also, there’s a definite disconnect between the 10,664 angry dog photos of the control’s title and the CTA in the description to browse 41 photos.

Kevin Maguire, Head of SEO at ONMARK

The removal of the number will not only improve CTR but will also improve organic rankings. Having the primary keyword at the beginning of the meta title will enhance its importance to the search engine. Removing the number for the search user presents a more concise answer to their search intent.

Mike Friedman, SEO Consultant at TheSEOPub.com

I'm going to guess in this case the CTR went up. I know numbers can improve CTRs according to some studies, but 10,664 of them sounds like a rabbit hole I don't have time for. The rankings probably moved up a little too, putting the keyword closer to the beginning.

Let's dig into the case study to find out who was right this time.

The Case Study

Search engines are a major source of traffic for almost all businesses and organizations. On the one hand, traffic and therefore search behavior are constantly changing. On the other hand, all changes to the website also have an impact. To anticipate these moves, validating these changes is important, especially when you run a site with hundreds of thousands of templated pages, where even small changes can have a huge impact.

For a large photo stock website, OrangeValley and the SplitSignal team researched how people search for stock photos and photos and found some great opportunities that we wanted to validate before rolling this out on the website.

The Hypothesis

The website in question had their page titles set up like this and changed to this:

As you can see, the focus for these page titles is on “photos,” “angry dog photos,” and “angry dog photos.” What we found is that in addition to “stock photos,” most photo searches are used with keywords such as “images” or “photos.” In short, search volumes with these types of keywords are much higher compared to just ‘photo’ related searches. So we wanted to validate whether including these keywords in the page title would have a positive effect on organic traffic.

The Test

We used SplitSignal to set up and analyze the test. 764 category pages were selected as either variant or control. We kicked off the test and ran it for 30 days. We were able to determine that Googlebot visited 100% of the tested pages.

The Results

The test ran from March 3 to April 2. The above image shows the development/progress of the variant (orange line) compared to the modeled control group (blue line). We saw that the traffic to the variant pages performed better than the control group, which means the test is positive. Including the keywords people usually use when searching for photos in the page title resulted in an 18% increase in organic traffic!

Note that we are not comparing the actual control group pages to our variant pages but rather a forecast based on historical data. We compare this with the actual data. We use a set of control pages to give the model context for trends and external influences. If something else changes during our test (e.g., seasonality), the model will detect and take it into account. By filtering these external factors, we gain insight into what the impact of an SEO change really is.

The test was statistically significant at the 95% level. On March 17, we were confident that the increase we are seeing is due to the change we made and not to other (external) factors.

Why

In retrospect, it seems like a no-brainer to implement this change. Winning additional organic traffic confirmed the hypothesis and it also enabled the client to build a strong business case to implement this page title change across all categories. 

Optimizing the page titles with keywords that people often use when searching for photos not only increased organic clicks to the tested pages but also increased the pages’ visibility across a wider spectrum of search queries, suggesting that the tested pages became more relevant for different search queries that the pages were ranked for. In addition, the search results of the tested pages more closely matched the actual search query that users usually use when searching for photos, therefore increasing relevance. 

Split testing not only helps our clients discover ways they can improve their organic traffic, but it can also help prove the impact of SEO changes by prioritizing them in the development queue. This test was done on their most important page template so in terms of priorities this will be a high priority SEO optimization for the website.

The increase in traffic we saw for this website has been very valuable, but it is always important to keep in mind that something that works for one website may not work for another. The only way to know for sure is to test what works for you!