Before you start: if you’re unfamiliar with the principles of statistical SEO split-testing and how SplitSignal works, we’re suggesting you start here or request a demo of SplitSignal. 

First, we asked our Twitter followers to vote:

This time the majority of our followers were RIGHT! But did they think it would have such an impact on organic clicks? I doubt it. Continue reading to find out the actual result.

But first, let’s see what SEO professionals had to say about the result of this test:

Craig Martin, Agency SEO Lead at Online Asset Partners:

Assuming it holds a similar ranking, I expect CTR to be up. “Best” adds an emotional punch, against “60+” which sounds like decision fatigue waiting to happen.
But which one stands out against other titles in the SERP? That will also play into it.

Trevor Stolber, Digital Marketer at STOLBER.com:

Interesting one, I’m leaning towards slight positive improvement. Depending on how well the title and page match the intent “best” may be better than 60+ but possibly not. This may work well for a subset of queries. Leaning towards slight positive improvement

Follow us on LinkedIn to share your thoughts on the next test.

The Case Study

Page titles are typically the first thing users see when they come across your page in search engine results. It is a crucial element in helping search engines understand the content of your page and determining its relevance to a user’s search query. In search, you are always competing with other pages for attention, so it is essential to write a page title that is both relevant and compelling. This can often require a lot of testing and trade-offs between including relevant keywords and crafting a title that will grab users’ attention and improve your click-through rate. A well-written page title can improve your page’s ranking in search results and increase the likelihood of clicking on your page.

In general, it is essential to regularly test and optimize your page titles to ensure that they accurately reflect the content of your page and are compelling enough to attract users. This can be especially important during the holiday season when there is often an increase in online shopping and competition for attention on search engines. Testing different versions of your page title and analyzing the results can help you determine the most effective way to improve your page’s visibility and ranking in search results.

A marketplace website conducted a test on the page titles for the gifts section of their website to see if incorporating season-related keywords and addressing people’s search intent would improve visibility and traffic from search engines. They were interested in determining the effectiveness of using seasonal language in page titles to attract more users during the holiday season. 

The Hypothesis

The website in question analyzed search behavior and hypothesized that words like “best” and “ideas” in relation to gifts in their page titles would increase organic traffic to the tested pages. 

People search more frequently during the holiday season for keywords like “XXX gift ideas” and “best XXX gifts.” This is due to the increase in online shopping and gift-giving during this time of year. As a result, we believe it is essential for businesses to optimize their pages to match their pages’ content better with users’ search queries. This can help to improve the relevance and ranking of their pages in search results and increase the likelihood that users will click on their page. 

To verify this hypothesis, we tested the following template on their listing pages to see if it would positively impact organic traffic.

The Test

To conduct our test, we used SplitSignal to change the page titles for a selection of listing pages on the website. Around 9,000 listing pages were selected as either control or variant. We launched the test and ran it for 30 days. We found that Googlebot visited 97% of the tested pages.

We used a script to monitor whether Google would display the desired changes to the page titles for the tested pages in the search results. We were able to confirm that Google showed the changes.

The Results

We found that changing the page titles to the “Best XX Gift Ideas” format resulted in an estimated 38.4% increase in clicks to the tested pages. This result was highly positive and exceeded everyone’s expectations. It suggests that using this page title format effectively increased the number of clicks to the tested pages.

After only one week of testing, we could determine that the effect we observed was statistically significant. When the blue shaded area falls below or above the x=0 axis, the test is considered statistically significant at the 95% level. This means that we can be confident that the new page title format will have a positive impact on organic traffic to the listing pages of the website.

The effect of the new page title format was significant and meaningful, with a confidence level of 99% by the end of the test period. This further strengthens our confidence in the efficacy of this page title strategy in terms of improving organic traffic to the website.

Note that we are not comparing the actual control group pages to our tested pages. But rather a forecast based on historical data. The model predicts the counterfactual response that would have occurred had no intervention occurred. We compare this with the actual data. We use a set of control pages to give the model context for trends and external influences. If something else changes during our test (e.g., seasonality), the model will detect and consider it. By filtering these external factors, we gain insight into the true impact of an SEO change.

Why

The results of this test were extremely positive, with an estimated 38.4% increase in clicks to the tested pages. Analysis of the data helped us to understand better the factors that contributed to the success of this page title strategy.

The data showed that this test’s effects were not limited to an increase in clicks to the tested pages. We also observed a statistically significant increase in impressions compared to our modeled control group using the SEO A/B test analyzer. Also, you can learn more about the analyzer here. 

Further analysis revealed this was due to better positioning and a significant increase in visibility in a broader range of search terms. These results suggest that the new page title format not only improved the click-through rate to the tested pages, but also made them more visible and improved their rankings in search results for a broader range of search queries.

One of the key findings of this test was that the new page title format not only improved the ranking and visibility of the tested pages on search queries that included the keywords “best” and “ideas” but also on keywords with similar intent. This suggest using these keywords in the page titles helped to more accurately match the content of the pages with the intent of users’ search queries, which improved the pages’ relevance and ranking in search results. Additionally, persuasive language like “best” may have helped grab users’ attention and increase the click-through rate of the pages. These findings highlight the importance of carefully selecting and incorporating keywords in page titles to improve search engine rankings and drive more traffic to your website.

With this test, we could determine that incorporating seasonal and intent-related keywords into the page titles was a valuable strategy for improving rankings and driving more organic traffic to the website. The impact of this test was significant, and it required relatively low effort to make the actual changes to the site. These results highlight the importance of regularly testing and optimizing page titles to improve the visibility and ranking of your pages on search engines.

We will publish more case studies about holiday season-related SEO tests in the coming weeks. These case studies will provide an in-depth analysis of the results of various tests we conducted to determine the most effective strategies for improving rankings and driving more organic traffic to websites during the holiday season. We believe these case studies will provide valuable insights for businesses looking to improve their performance during this busy time of year. Stay tuned for more information about our upcoming case studies on holiday season SEO.

It’s important to remember that what works for one website may not work for another. The only way to determine the most effective strategies for your website is to conduct tests and analyze the results by experimenting with different approaches and measuring the outcomes to understand better what works best for your website and target audience. This will help you make informed decisions about optimizing your website and drive more organic traffic to your site.