Before you start: what do you know about SEO split-testing? If you're unfamiliar with the principles of statistical SEO split-testing and how SplitSignal works, we're suggesting you start here or request a demo of SplitSignal. 

First, we asked our Twitter followers to vote:

60% of our followers guessed it right, the test result was positive.

Read the full case study to find out why.

The Case Study

Writing relevant page titles today involves much more than just adding the main keyword(s) you want to rank for. Understanding and playing into a user’s search intent and relevant CTAs combined with secondary keywords are just as important to drive relevant and converting traffic to your website.

For a major ecommerce company in Australia, we added “Buy” to their category page titles to see if this made their search results more attractive and effective.

Category pages on an ecommerce website are often the most important pages for generating traffic and revenue. Small optimizations aimed at finding the best answer for both users and search engines can ultimately have a significant impact on your traffic and bottom line, as we'll show with this case study.

The Hypothesis

The website in question had its page titles set up like this:

We wanted to validate whether adding “Buy” to the page title of their category pages would have a positive effect on organic traffic, so that’s what we did.

The Test

We used SplitSignal to set up and analyze the test. 813 category pages were selected as either variant or control. We added “Buy” to the beginning of the page title for each variant page:

The Results

The image above shows the development/progress of the variant (orange line) compared to the predicted control group (blue line). 

Looking at the diagrams, we can see two important things:

1. The organic traffic (clicks) to the tested pages outperformed our modelled control group. This means that the traffic to the variant group is performing better than predicted and shows that the test is positive. 
2. The cumulative view not only shows the additional organic traffic to the tested pages but also shows that the test performed is significant. When all three curves perform below (negative) or above (positive) the y=0 (cumulative gradient) axis, the test is statistically significant. That means we can be sure that the increase we are seeing is due to the change we made and not due to other factors.

Also, note that we are not comparing the actual control group pages to our variant pages but rather a forecast based on historical data. We compare this with the actual data. We use a set of control pages to give the model context for trends and external influences. If something else changes during our test (e.g., seasonality), the model will detect and take it into account. By filtering these external factors, we gain insight into what the impact of an SEO change really is.

After running the test for 21 days, we saw an 8.3% increase in clicks, not bad at all for a “small” change.

Why?

As said in the beginning, optimizing page titles is not just about including your main keyword(s). There is much more to writing appealing and effective page titles. This test is another great example of the importance of responding to search intent and including a CTA.

Analyzing Search Console data shows that this test is a combination of the above but also increases visibility by including a secondary keyword (which also helps respond to search intent and having a CTA). During the test, we saw an increase in CTR to the tested pages, which is the result of a more attractive and relevant title. But we also saw an increase in impressions. By looking further into the data, we saw that this is the result of a more relevant search result on “buy” related searches.

This test shows that as an SEO, you need to think about and experiment with factors such as intent. Knowing what users expect and what users find appealing can make a big difference in how users interact with your search result snippets. Within search, you always compete with other search results. Finding ways to stand out and be the most relevant answer is now more important than ever. If not, you might be leaving money on the table.

However, it is important to carefully test these factors. If you don't get it right at first, it could hurt your performance, as shown by other tests we've published. Experimenting and learning more about Google and your users is essential to be the best and most relevant answer in search.

Have your next SEO split-test analyzed by OrangeValley Agency.