You Have to Test Your SEO Theories—These Case Studies Prove That

In this article, we will show you SEO tests that show how important it is to test your own theories. If you are looking for a TL;DR—here it is:

We have hundreds of different SEO changes on dozens of websites. And the exact same change will not always bring the exact same result, no matter how closely you try to replicate what you’ve done before.
So, before you go on and try to copy the homework of the SEO expert next to you, make sure to test their approach (or your own theories) in real time on your own website. It can save you a lot of money, time, and effort in the long run. 

The Big Issue

Improving the performance of new or poorly-managed pages isn’t too difficult for most SEO experts. However, we have all had to work with clients who have already done a decent job, and improving their performance isn’t always too easy. 

Allow us to set the stage for you. A famous brand with millions of visitors has contacted you to hire you as their new SEO director. It is now your job to improve the rankings and CTR of their product pages. But, when you start looking at all of the typical errors, you don’t find anything (We know; it never happens, but humor us for a few moments.)

So, you think: “Well, we can always increase the number of articles on the website.”

And yes, you could. And, in most cases, it does help out. But, the larger the website is, the smaller impact you get from new articles (relatively speaking).

So, what can you do if you’ve been hired by a brand that already has thousands of articles that are well-done and that followed best practices as best as they could? You can start experimenting. 

The fact is, the only way to outperform websites that follow best practices is to go off the beaten path and test your own ideas 

This is where SEO Split Testing comes into play. 

SEO Split Testing

Split Testing is a powerful tool that helps marketers get the most out of their efforts. In essence, it allows you to run two versions of a campaign to see which one performs better. Once you have that data, you can roll it out on a larger scale and boost your results.

But how does that apply to SEO? Well, split testing with SEO can be very tricky. Thankfully, Semrush has released a new, powerful tool called Split Signal that can do almost all of the work for you (at least the manual part). 

We won’t go into details on how to run an SEO Split-Test. We have done that before. Instead, we will focus on why it’s important to do your own testing. 

Back to the Point

Why were we even taking the time to talk about all of this? Well, in most walks of life, people try their best to learn from the experiences of others. If you want to increase your bench press, you can search online to see what others with strong pecs have done. If you want to learn how to play the guitar, you can reach out to others who have done so before you.

But not everyone’s body is the same, and not everyone has the same amount of musical talent. So, just because an exercise worked for someone doesn’t make it so that it will work for you too.

Unfortunately, the same goes for your website. A change that does wonders for your competitor might do you no good at all. In fact, it could even cause significant problems for your website. 

We have seen that with our tests before. We make the same change on three different websites, and we get three different results. So, in hopes of preventing new SEO myths from arising, let’s take a look at tests that seemed very definitive but were later shown to be not quite the key to hacking SEO we were hoping they had been. 

Adding CTA to Title

This almost seems like a no-brainer. Add CTAs to your titles to increase your CTR! It is a statement that many have heard since the first day they started doing SEO. And, as we have mentioned, it worked perfectly for some. After all, that’s how SEO myths are created. However, the miracle doesn’t quite work for everyone. 

1. Successful Test

Back in October 2021, we ran a test that was elegant in its simplicity. All we did was use our tool SplitSignal to add the word “Buy” to the beginning of over 800 product pages on a website. That website saw an increase of 8.3% in their total clicks off of that change alone. 

So, this is a cut-and-dry? Right? We all “know” that adding CTAs to your titles is good for you, and adding a single word to a title increase performance by over 8%. What are we waiting for?
Well, as we have already implied heavily in this article, that’s not how it works. Onto example number two. 

2. The Negative Result

Not long after that test, in July 2022, we decided to run it back. This time, instead of adding the word “Buy” to the beginning of the title, our CTA of the day was “Order.”

We let people try and guess what the result would be, and exactly ZERO PERCENT of people guessed that the traffic would go down. And that’s exactly what happened. 

All experts were already enamored with the idea of having such a simple hack that nobody even considered this test could have a negative result.

Now, you can read both of those blogs to figure out what the major differences between the two websites were, but that’s not the point of this article you are reading today. 

Adding Dates to Your Titles—Dating Your Content

Now, telling people to do this one isn’t nearly as ubiquitous as “adding CTAs to your title tags” was. But, we do have a unique vantage point thanks to previous tests done by the SplitSignal team. Namely, they have done this particular test three times, and, what would you know, they managed to get all three different results. 

And remember, John Muller went out of his way for this one to tell us that adding the published date shouldn’t make a change. 

1. The Significant Rise

In October 2022, the change we saw tested was on a fishing travel website. In this test, the SplitSignal team added the word “Daily” to the beginning of the title and the month and year of publishing to the end. The 929 pages that got this fresh coat of paint suddenly skyrocketed in terms of clicks. The effect size of this minute change was an increase of almost 14%.  

2. No Significant Change

Since this isn’t a lovely fairytale with three bears, the porridge in the middle turns out to be lukewarm at best. In August, just two months before the very positive test above, we tried adding the word “Updated” and the year (2022) to the beginning of the title tag. But this time around, the effort didn’t really amount to much. A 1.5% change with a low confidence level is not something most people would celebrate. 

3. The Soft Drop

The final case study we would like to discuss today is also the latest one. We published it in November 2022, about a month after the “great success.” 

To go into a bit more detail—we simply added the current month and year to the title tag of a travel site. Well, to about 900 product pages of that website. We kept another 900 pages as control. 

The change was as close as possible to the one made for the October test. 

Based on what we’ve seen in the past, we all expected the results to increase, at least slightly. In fact, the tests were so similar we were pretty sure this would be a slam dunk.  

Well, you can guess how surprised we were when we saw that the clicks went down by 6.6% in only 16 days we let the test run. And the more we waited, the worse the results were becoming, so we decided to pull the plug a bit early on this one to avoid any more losses. 

The Conclusion—You Have to Test Your Own Theories

What do these examples tell us? Relying solely on what has or hasn’t worked for others is not always good enough in SEO. Not only do the rules change every few months with the “dreaded” Google core updates, but it also seems like our competitors never get tired and are always looking for ways to get the better of us.

Either one of these case studies could have influenced you to make a change and see a huge drop in your rankings. Or, it could have made you risk-averse and stopped you from making that one small change that would skyrocket your results by 14% in only a few minutes of work. 

As you can guess, neither of those options is great. And if you were running a small website, it wouldn’t be too much of a problem. But, if your job is to make sure that thousands of product pages of your (or your client’s) website see a steady increase in performance, you will have to take some risks and stay on top of the SEO game.

The only way to do that safely and avoid spending tens of thousands of dollars (or more) in developer hours and fixes to failed theories is to test them first with a tool like SplitSignal. 

So, reach out to SplitSignal representatives to start testing today.