Before you start: what do you know about SEO split-testing? If you’re unfamiliar with the principles of statistical SEO split-testing and how SplitSignal works, we’re suggesting you start here or request a demo of SplitSignal. 

First, we asked our Twitter followers to vote:

0% of our followers guessed it right! The result was actually negative.

Read the full case study to find out why.

The Case

Greetings SEO devotees, it’s a great day to be testing your hypotheses.

Depeche Mode had a great lyric, “Words like violence, break the silence…”, and they can have a huge impact, for the positive or negative in our SEO work. The unexpected result, even when it goes against what we would expect, is still interesting to chase down.

For today’s test review, we’ll be looking at a cannabis ecommerce/directory hybrid website, and, can we influence the local listings of dispensaries, by tweaking title tags with some location information.

The Hypothesis

We hypothesized, by adding the city and state of a local dispensary to that dispensary’s listing page, we’d see an increase in clicks. Prior to our test, the meta title only shows the name of the local dispensary, and a “Dispensary Menu | BRAND” where BRAND is the name of the brand website we’re testing.

The Test

The SEO experiment was set up and configured using SplitSignal. A percentage of the dispensary directory listings were selected as the test variant, while an equal number of listing pages were selected as the control group. 

We modified approximately half the page titles (variant pages) from a format like in Control to the Variant example:

The test ran for 21 days with 99% of the variant pages being crawled by Google during that time.

The Results

After three weeks, we observed a statistically significant, NEGATIVE result for the website. 

Pause here and give yourself a moment to form a hypothesis as to why that’s the case? Our expert SEO agency below has an opinion, but we’d love to hear your opinion in the comments.

Overall, the test pages received 4.9% less clicks during the test period, a loss. 

LOCOMOTIVE Agency Analysis

You’ve heard our opinion on a lot of case studies if you’ve been following these case study breakdowns. As Brian mentioned above, we’d love to hear more of what you think. Why do you think this was a negative result?

A loss of clicks can suggest several different things, but in this instance, a close review of Google Search Console in our Google Data Studio dashboard we use for studying these tests, it is pretty obvious the loss in clicks was due primarily to a decrease in ranking for the test group.

In my opinion, this is counterintuitively due to the overoptimization of the title tag. As Google discovered more of these changed title tags, the rankings declined over a 4-6 week period (even though the test ran 21 days, it takes a little longer for Google to recrawl pages when the test is over).

The primary purpose of these local pages is to present a “menu” of products to the consumer for e-commerce purchase, and by removing “Menu” and adding locality information, the title no longer fit the intent of the page, which is to make a purchase, not visit the local business. This resulted in Google probably picking up on the intent mismatch and attempt at overoptimization.

As you no doubt know, every SEO can have different opinions. What’s yours?

Have your next SEO split-test analyzed by the technical SEO experts at LOCOMOTIVE Agency.