Before you start: what do you know about SEO split-testing? If you’re unfamiliar with the principles of statistical SEO split-testing and how SplitSignal works, we suggest you start here or request a demo. 

First, we asked our Twitter followers to vote:

Here’s what other SEO professionals have to share about this test:

Vladimir Gertner, Senior Project Manager at Soft Road Apps:

Hmmm... I will think about it. I am not thrilled by the test variant, but I really dislike the original meta description.
There’s also the fact that it's Walmart. People know and trust the brand. And people who shop at Walmart already know that they are shopping at Walmart. So, it also depends on what search query brought them to the SERP.
I will go with positive. It feels like Walmart has a “new deal” that includes free shipping.

Nina De la Cruz, Content Marketing Strategy at Getsitecontrol:

You can’t beat “Free shipping on everything,” so my assumption is that the “AFTER” version boosted organic clicks.

Find out if our followers were right by reading the complete analysis of this test.

The Case Study

Greetings and welcome to the most recent SplitSignal case study. This one features a well-known e-retailer and their meta descriptions.

The website we’re looking at is an English-language retailer of discounted products whose product category meta descriptions are well configured but a little on the long side in some instances, so we decided to test shortening them a bit.

The Hypothesis

Because this is a discount website, and the prevalence of free shipping with other competitors, we hypothesized replacing the existing meta descriptions with the primary category name H1, followed by “ - Free Shipping on Everything* - Shop Now!” we could possibly see an increase in click-through rate from the SERPs.

It seems fairly straightforward, what do you think the end result will be? Read on and let’s find out!

The Test

The SEO experiment was set up and configured using our SEO testing tool, SplitSignal. This is a larger website with over 1.5 million indexed URIs, so we selected 1,673 total pages, 846 for the test control, and 827 for the test variant. 

The test ran for 30 days with a confidence level of 99% with 99% (all but one) of the test pages being recrawled by Googlebot during the time period for testing.

The Results

After the thirty-day test period, we observed a statistically significant, NEGATIVE result for the pages with the converted meta descriptions.

As we always do, we asked you at the outset what you felt would happen. The technical SEOs at LOCOMOTIVE Agency came in after the experiment and have given their explanation why they believe this happened, but we’d love to hear your opinion in the comments of what you originally thought, and whether you think it could be something different?.

Overall, the test pages received 4.7% less clicks during the test period, a loss of 1,179 clicks. 

LOCOMOTIVE Agency Analysis

One of the things we noticed from the outset was the tested website had pretty well-written H1s. The primary category/sub-category was mentioned a couple of times in the meta, and it had strong calls to action that would appeal to the shoppers' aesthetic tastes (oftentimes) as well as the discounted nature of the products and shipping.

By shortening the meta description to only the product category name, and simplifying the call to action, we feel the psychological appeal of the meta description messaging was lost resulting in the observed loss in clicks when compared to the competitors surrounding the SERP result.

One of the things I love about meta description tests is it shows how important they still are, even if they have no effect on SEO rankings, they can still have a significant impact, just like in this instance, on other elements like click-thru.

What do you think?

Have your next SEO split-test analyzed by the technical SEO experts at LOCOMOTIVE Agency.