The 7 Core A/B Testing Strategies That Are Fundamentally Essential To CRO Success

One of these A/B testing strategies is right for your website, and will lead to bigger wins faster.

We have used analysis and testing to find significant increases in revenue and leads for hundreds of companies. For each one, we fashion unique AB testing strategies for each defining where to start and what to test.

However, we will virtually always build out that unique testing strategy off one of seven core strategies that I consider fundamenal to CRO success.

If you are beginning your own split testing or conversion optimization process, this is your guide to AB testing strategies. For each these seven strategies, I’m going to show you:

1. When to use it
2. Where on the site to test it
3. What to test
4. Pitfalls to avoid
5. A real-life example

If you have more questions about your testing strategy, contact us and we’ll be more than happy to answer any questions I don’t cover here.

Let’s get started.

1. Gum Trampoline

We employ the gum trampoline approach when bounce rates are high, especially from new visitors. The bounce rate is the number of visitors who visit a site and leave after only a few seconds. Bouncers only see one page typically.

As the name implies, we want to use these AB testing strategies to slow the bouncing behavior, like putting gum on a trampoline.

We want more visitors to stick to our site and not bounce.

When to Use It

You have a high bounce rate on your entry pages. This approach is especially important if your paid traffic (PPC or display ads) is not buying.

You have run out of paid traffic for a given set of keywords.

Where to Test

Most of your attention will be focused on landing pages. For lead generation, these may be dedicated landing pages. For ecommerce sites, these may be category pages or product pages.

What to Test

The key components of any landing page include:

1. The offer that matches the ad, link or social post.
2. The form that allows the visitor to take action. This may be just a button.
3. The proof you use on the page that it’s a good decision.
4. The trust you build, especially from third-party sources.
5. The images you use to show the product or service. Always have relevant images.

Be Careful

Reducing bounce rate can increase leads and revenue. However, it can also increase the number of unqualified visitors entering the site or becoming prospects.

Example

In the following example, there is a disconnect between the expectation set by the advertisement (left side) and the landing page visitors see when they click on the ad (right side).

Paid ads are often a fantastic tool for bringing in qualified traffic, but if the landing page isn’t matched to the ad, visitors are likely to immediately bounce from the page rather than attempting to hunt for the treasure promised in the ad.

In order to apply gum to this trampoline, Zumba would need to take ad click-throughs to a page a featuring “The New Wonderland Collection”, preferably with the same model used in the ads. The landing needs to be designed specifically for the type of user who would be intrigued by the ad.

2. Completion Optimization

The Completion strategy begins testing at the call to action. For a lead-generation site, the Completion strategy will begin with the action page or signup process. For an ecommerce site, we start with the checkout process.

When to Use It

The Completion strategy is used for sites that have a high number of transactions and want to decrease the abandonment rate. The abandonment rate is the percentage of visitors who start a checkout or registration process, but don’t complete it. They abandon the process before they’re done.

Where to Test

This process starts at the end of the process, in the shopping cart or registration process.

What to Test

There are lots of things that could be impacting your abandonment rate.

• Do you need to build trust with credit logos, security logos, testimonials or proof points?
• Are you showing the cart contents on every step?
• Do you require the visitor to create an account to purchase?
• Do your visitors prefer a one-step checkout or a multi-step checkout?
• Have you addressed your return policy?
• Are you asking for unnecessary information?

Once you have reduced the abandonment rates, you can begin testing further upstream, to get more visitors into your optimized purchase or signup process.

Be Careful

Testing in the cart can be very expensive. Any test treatments that underperform the control are costing you real leads and sales. Also, cart abandonment often has its roots further upstream. Pages on your site that make false promises or leave out key information may be causing your abandonment rates to rise.

For example, if you don’t talk about shipping fees before checkout, you may have lots of people staring the purchase process just to find out what your shipping fees are.

Example

As we’ve talked about before, best practices are essentially guesses in CRO. We know, as a general rule, that lowering checkout friction tends to improve conversion rates and lower abandonment. But sometimes, it’s actually perceived friction that impacts the checkout experience above and beyond the real level of friction.

For example, one of our clients upgraded their website and checkout experience in accordance with best practices.

• The process was reduced from multiple steps to a single step.
• The order is shown, including the product images.
• The “Risk-free Guarantee” at the top and “Doctor Trusted” bug on the right reinforces the purchase.
• Trust symbols are placed near the call-to action button.
• All costs have been addressed, including shipping and taxes.

The new checkout process should have performed better, yet it ended up having a significantly higher bounce rate than the previous checkout process.

Why?

After looking at previous checkout experience, we realized that despite it actually requiring more steps (and clicks) on the part of the user, the process was broken up in a such a way that the user perceived less friction along the way. Information was hidden behind each step, so that they user never ultimately felt the friction.

Step #1:

Paypal payment method step 1

Step #2:

Paypal billing information

This is just one of many reasons running AB tests is mandatory, and it’s also a good example of how beneficial it can be for certian business to start with the checkout process, as dicated by the LT strategy.

3. Flow Optimization

The Flow approach is essentially the opposite of the Completion strategy. With this strategy, you’re trying to get more visitors into the purchase process before you start optimizing the checkout or registration process.

When to Use It

This strategy is typically best for sites with fewer transactions. The goal is to increase visits to the cart or registration process so we start Completion testing at the bottom of the funnel.

Where to Test

Testing starts on entry pages, the pages on which visitors enter the site. This will typically include  the home page and landing pages for lead-generating sites. For ecommerce sites category pages and product pages get intense scrutiny to increase Add to Cart actions.

What to Test

With this strategy, we are most often trying to understand what is missing from the product or service presentation.

• What questions are going unanswered?
• What objections aren’t being addressed?
• What information isn’t presented that visitors need?
• Is the pricing wrong for the value presented?

We will test headlines, copy, images and calls to action when we begin the GT strategy.

Be Careful

Even though we aren’t optimizing the checkout or registration process, avoid testing clicks or engagement metrics. Always use purchases or leads generated as the primary metric in your tests. It’s too easy to get unqualified visitors to add something to cart only to see abandonment rates skyrocket.

Example

Businesses that benefit from the GT strategy typically need to relook at their central value proposition on poorly converting landing pages.

For example, when Groove decided it’s 2.3% homepage conversion rate wasn’t going to cut it anymore, it began the optmization process by revamping its value proposition. The existing page was very bland, with a stock photo and a weak headline that didn’t do anything to address the benefits of the service.

Groove SaaS and eCommerce Customer Support Value Proposition

The new page included a benefits-driven headline and a well-produced video of a real customer describing his positive experience with Groove. As a result, the page revamp more than doubled homepage conversions.

Groove created a ‘copy first’ landing page based on feedback from customers

The point here is that fixing your checkout process isn’t going to do you a ton of good if you aren’t getting a whole lot of people there in the first place. If initial conversions are low, it’s better to start with optimizing your core value proposition than go fishing for problems on the backend of your funnel.

4. Minesweeper

Minesweeper optimization strategies use clues from several tests to determine where additional revenue might be hiding.

Some sites are like the Minesweeper game that has shipped with Windows operating systems for decades. In the game you hunt for open squares and avoid mines. The location of minds is hinted at by numbered squares.

In this game, you don’t know where to look until you start playing. But it’s not random. This is like an exploratory testing strategy.

When to Use It

This testing strategy is for sites that seem to be working against the visitor at every turn. We see this when visit lengths are low or people leave products in the cart at high rates. Use it when things are broken all over the site, then dive into one of the other strategies.

As testing progresses, we get clues about what is really keeping visitors from completing a transaction. The picture slowly resolves as we collect data from around the site.

Where to Test

This strategy starts on the pages where the data says the problems lie.

What to Test

By its nature, it is hard to generalize about this testing strategy. As an example, we may believe that people are having trouble finding the solution or product they are looking for. Issues related to findability, or “discoverability” may include navigation tests, site search fixes, and changes to categories or category names.

Be Careful

This is our least-often used strategy. It is too scattershot to be used frequently. We prefer the data to lead us down tunnels where we mine veins of gold.

However, this is the most common of optimization strategies used by inexperienced marketers. It is one of the reasons that conversion projects get abandoned. The random nature of this approach means that there will be many tests that don’t help much and fewer big wins.

Example

You wouldn’t expect a company pulling in $2.1 Billion in annual revenue to have major breaks in it’s website, yet that’s exactly what I discovered a few years back while attempting to make a purchase from Fry’s Electronics. Whenever I selected the “In-store Pickup” option, I was taken to the following error screen.

This is one of the most important buttons on the site, doubly so near Christmas when shipping gifts becomes an iffy proposition. Even worse, errors like this often aren’t isolated.

While finding a major error like this doesn’t necessarily mean you need to begin the Minesweeper optimization strategy, it’s always important to fix broken pieces of a site before you even begin to look at optimization strategies.

5. Big Rocks

Adding new features — “big rocks” — to a site can fundamentally change its effectiveness.

Almost every site has a primary issue. After analysis, you will see that there are questions about authority and credibility that go unanswered. You might find that issues with the layout are keeping many visitors from taking action.

The Big Rocks testing strategy adds fundamental components to the site in an effort to give visitors what they are looking for.

When to Use It

This strategy is used for sites that have a long history of optimization and ample evidence that an important component is missing.

Where to Test

These tests are usually site-wide. They involve adding fundamental features to the site.

What to Test

Some examples of big rocks include:

• Ratings and Reviews for ecommerce sites
• Live Chat
• Product Demo Videos
• Faceted Site Search
• Recommendation Engines
• Progressive Forms
• Exit-intent Popovers

Be Careful

These tools are difficult to test. Once implemented, they cannot be easily removed from the site. Be sure you have evidence from your visitors that they want the rock. Don’t believe the claims of higher conversions made by the Big Rock company salespeople. Your audience is different.

Example

A good example of the Big Rocks strategy in action comes from Adore Me, a millennial-targeted lingerie retailer that catipulted it’s sales by installing Yotpo’s social-based review system. The company was relying primarily on email and phone for customer feedback and identified ratings and user reviews as its “big rock” to target.

The revamped customer engagement system helped spawn tens of thousands of new reviews and also facilitated a flood of user-generated content on sites like Instagram without Adore Me even having to purchase Instagram ads. Type in #AdoreMe and you’ll find thousands of unsponsored user-generated posts like these:

This is a great example of a how certain AB testing strategies can help facilitate different business models. The key is identify the big opportunities and then focusing on creating real, engaging solutions in those areas.

6. Big Swings

Taking big swings can lead to home runs, but can also obscure the reasons for wins.

A “Big Swing” is any test that changes more than one variable and often changes several. It’s called a big swing because it’s when we swing for the fences with a redesigned page.

When to Use It

Like the Big Rock strategy, this strategy is most often used on a site that has a mature conversion optimization program. When we begin to find the local maxima for a site, it gets harder to find winning hypotheses. If evidence suggests that a fundamental change is needed, we’ll take a big swing and completely redesign a page or set of pages based on what we’ve learned.

Sometimes we start with a Big Swing if we feel that the value proposition for a site is fundamentally broken.

Where to Test

We often take big swings on key entry pages such as the home page or landing pages. For ecommerce sites, you may want to try redesigning the product page template for your site.

What to Test

Big Swings are often related to layout and messaging. All at once, several things may change on a page:

• Copy
• Images
• Layout
• Design Style
• Calls to Action

Be Careful

Big swings don’t tell you much about your audience. When you change more than one thing, the changes can offset each other. Perhaps making the headline bigger increased the conversion rate on a page, but the new image decreased the conversion rate. When you change both, you may not see the change.

Example

Neil Patel is one of those marketers who likes to use the Big Swings strategy on a regular basis. For example, he has tried complete homepage redesigns for Crazy Egg on several occasions.

The first big redesign changed things from a short-form landing page to a very long-form page and resulted in a 30% increase in conversions.

The next big redesign scrapped the long page for another short page, but this time with concise, targeted copy and a video-driven value proposition. This new page improved conversions by another 13%.

And of course, Neil didn’t stop there. Crazy Egg’s homepage has changed yet again, with the current iteration simply inviting users to enter their website’s URL and see a Crazy Egg’s user testing tools in action on their own site. How well is it converting? No clue, but if I know Neil, I can promise you the current page is Crazy Egg’s highest performer to date.

Sometimes the only way to improve conversions is to swing for the fences and try something new.

7. Nuclear Option

I’ll mention the nuclear option here, which is a full site redesign. There are only two good reasons to do an entire site redesign:

1. You’re changing to a new backend platform.
2. You’re redoing your company or product branding.

All other redesign efforts should be done with conversion optimization tests, like Wasp Barcode.

We even recommend creating a separate mobile site rather than using responsive web design.

You should speak to a Conversion Scientist before you embark on a redesign project.

Which A/B Testing Strategy Is Right For You?

Every website is different. The approach you take when testing a site should ultimately be determined by the data you have. Once you settle on a direction it can help you find bigger wins sooner.

Photo Credit: megankhines via Compfight cc, Genista via Compfight cc, chemisti via Compfight cc

• Author
• Recent Posts

Brian Massey

Conversion Scientist™ at Conversion Sciences

Brian Massey is the Founder and Conversion Scientist™ at Conversion Sciences, a sought-after international speaker, and author of Your Customer Creation Equation. He is passionate about transforming websites through a steady diet of visitor profiling, purposeful content, analytics, and AB testing.

Expertise: conversion rate optimization, AB testing, marketing strategy, digital marketing, analytics, landing page design, computer programming, entrepreneurship

Speaking: IBM, Inbound, LeadsCon, Content Marketing World, Affiliate Summit, and more

Books and Training: Your Conversion Creation Equation, Video CRO, Marketing Videos that Convert, CRO Best Practices

As Seen On: ClickZ, Search Engine Land, Marketing Land, Conversion Sciences Blog, Intended Consequences podcast

Education: Texas A&M University, BS Computer Science and Management

Latest posts by Brian Massey (see all)

• Creating a Landing Page that Converts: 6 Essential Elements [Infographic] - December 4, 2024
• Confirmation Bias: What It Is and How It’s Hurting Your Website Conversions - August 20, 2024
• The Conversion Optimization Process for High Converting Websites - August 20, 2024