SEO Testing: Driving Higher Organic CTR and Rankings Through Systematic Experimentation [Case Study]

By John Reinesch

The web is awash with often conflicting advice on the subject of search engine optimization.

Do bolding keywords generate higher or lower rankings? What about embedding dates in URLs? Do my pages actually answer searcher’s questions?

Knowing the difference between solid advice and search superstition is sometimes impossible. Instead of throwing up our hands in frustration, we should put these assumptions to the test. By applying the scientific method to the factors that influence search rankings, we can learn what works and what doesn’t. Here’s how we approach SEO testing in our work as a marketing technology consultancy.

Why We Test SEO

SEO testing is about finding more sophisticated and effective ways to boost the search rankings of existing content. By applying SEO and messaging optimization on one group of content and maintaining an unaltered control group, tests can discover the optimizations and messaging strategy that reliably improves search rankings.

seo case study

Ranking higher in Google search results requires us to understand what Google’s search algorithm, and it’s audience of billions, prefers. 

Google’s algorithm feeds off roughly 300 ranking factors (The weight of which even Google engineers don’t know, as the ML search engine has the final say).

What’s more, making SEO testing a systematic part of our content production workflow allows us to see exponential gains over the long term.

It’s expensive and risky to make site-wide changes to content. Tests allow us to validate various approaches on subsets of the website’s content before making a site-wide commitment.

SEO Testing Best Practices 

In SEO tests, we apply the scientific method to ensure reliable results, minimizing interference from outside factors. As such, it should only test one variable at a time and follow the basic principles of scientific experiments.   

Keep the following tips in mind:

Hypothesize

Just like a proper scientific experiment, you’ll first want to hypothesize your expectations for the experiment. For instance, “I believe that removing dates from URLs will boost clicks and rankings.” Write it down and come back to it after the results have come in.

Educate yourself on industry-specific SEO criteria

Google’s search variables differ from topic to topic, industry to industry. 

Ecommerce sites, for instance, are rewarded for avoiding heavy advertising, much more so than overall search results. Only three percent of top ten eCommerce pages use AdSense or Ad Links, whereas 9 percent of overall results contain such advertising.

In contrast, health-based content tends towards higher relevance scores, owing to the sensitive nature of health advice. Quality and accuracy are even more important here. Those publishing in the space should expect traffic from college-educated users who are expecting to read in-depth and are likely to bounce if the information contained is not relevant or original.

Familiarize yourself with your industry’s specifics and develop tests unique to your industry.

Qualifying pages for inclusion into SEO tests 

SEO testing requires a degree of statistical significance, so the pages you select should draw enough traffic that any changes could be attributed to the experimental variable. Poor performing pages may require substantial changes to core content.

Ensure parity between control and experimental groups

Just like scientific experiments, SEO tests require that you divide qualified pages into two groups: an experimental group to test an SEO change and a control group that remains unchanged. Both groups should closely resemble each other to avoid any interference. Sort your pages by traffic, and assign them to groups on an alternating basis. 

Familiarize yourself with major SEO tests

Theoretically, there are as many SEO tests as there are ranking variables. There’s no need to try each and every one, though. These seven test highly influential factors:

  • Meta-Title/Description
  • H1
  • Images
  • Calls to action
  • Outbound links
  • Body content

Test timeline

It often takes quite a bit before Google crawls your new page-level change. That means you’ll want to wait a bit before measuring the results. Stick to the following timeline before declaring victory or defeat:

  • Two weeks for page title and meta description changes
  • Four weeks for content improvements
  • Six weeks for new links

Case Study: Food & Nutrition Blog 

We ran a number of tests on our nutrition-advice client’s blog posts (With help from our favorite testing tool, SEOTesting).

Title and meta description tests are probably our favorites, and for good reason: experts generally agree that these are among the highest weighted SEO variables, and they’re the first elements searchers see (Just like IRL, first impressions matter).

The idea behind this is twofold. Of course, we’re hoping Google finds this language more compelling, but we’re mainly concerned with competing with other search results. Sure, higher-ranked pages generally receive more traffic, but if our titles and meta tags better stand out, they can receive a higher click-through rate. Since Google also places substantial weight on this metric, this boost can snowball into higher rankings, effectively leapfrogging the competition.

Direct, unique, and emotionally-compelling copy tends to work best on search results. Focus on benefits first, and consider the user’s perspective and needs when writing. Using their point of view is a great way to make a direct appeal and earn clicks. You can use tools like Advanced Marketing Institute’s Headline Analyzer to evaluate your drafts.

It may be tempting to use click-bait headlines to gain attention. However, search engines punish such writing. Be sure any promises made in link text are kept upon arrival.

For instance, one notable test concerned a post targeting the keyphrase “keto snacks.” This post had managed to make it to the ninth result on the first page, but we thought we could do better.

We changed the headline and meta description to make them more attractive and added the bracketed phrases to help them stand out in results (Note that we’ve approximated the changes we made to protect our client’s privacy). 

Title: [50+ Recipes] Keto Snacks Perfect for Achieving Ketosis

Meta:—High-Fat Cheeses —Combine mascarpone and cream cheese with other listed snacks to increase your fat intake while keeping protein consumption low. 

And it worked!

Results poured in a few days after; clicks drove up 70 percent, and our rank increased by three. Now the page is the sixth result on the first page of search results. 

As you can see, the new text helped the page “pop,” especially compared to Amazon’s extremely basic headline. 

This experiment taught us a valuable lesson. Even a slight change in rankings can have a profound impact on traffic, making the effort worthwhile.

Case study (continued): Page-level test measuring secondary keywords

We conducted another noteworthy experiment with the same client, this time concerning a different page-level test—keyword optimization across multiple key phrases. 

While we generally tie our pages to a single keyword, it’s often necessary to include a range of related keywords to achieve the highest possible ranking. It’s typically pretty simple to ascertain how a given page performs given this targeted keyword. Still, it’s much more difficult to see overall how it performs across a wider range of keywords.

This page-level test achieves just that. It allows us to take a page, zoom out, and see how changes we made to body content, meta descriptions, and headlines contributes to SEO in a birds-eye view.

It’s particularly valuable because optimizations may not necessarily result in higher rankings for the targeted keyword, but this doesn’t mean that gains across other related keywords were made. 

In one test, we made changes to a post concerning the best and worst keto diets, particularly to the body content, meta descriptions, and headline.

We saw substantial results shortly after making the changes live.

The post’s click-through rate increased 1.6.% while growing daily clicks by close to 70%. While the daily click rate—232—may not appear substantial, this growth really compounds over time. Extrapolate that over a month—assuming a 31 day month—and that’s an additional 2,188 clicks.

After analyzing our new keyword report, we indeed found that supplementary keyphrases were driving the improved results. “Best keto oils” was only the eight strongest keywords on the page, which was preceded by the more complementary keyphrases we added in with our content update. The top three keyphrases included “is sesame oil keto,” “sesame oil keto,” and “keto oil.”

Knowledge you can count on

They say knowledge is power, but we marketers know we can’t believe every tip we read. SEO testing is a thrilling new means of testing our assumptions, not only about what search engines prefer but more importantly about what our visitors want. Knowledge is power, but certain knowledge is much more powerful.

There is more where this came from…

The best articles from this blog are available all in one place – our book. Now on it’s 6th edition.

Content Chemistry, The Illustrated Handbook for Content Marketing, is packed with practical tips, real-world examples, and expert insights. A must-read for anyone looking to build a content strategy that drives real business impact. Check out the reviews on Amazon.

Buy now direct $29.95