Intro to AB Testing: What Is An A/B Test?

What is AB Testing? How does split testing work? Who should run AB tests? Discover the Conversion Scientists’ secrets to running an A/B test.

AB testing, also referred to as “split”, A/B test or “ABn” testing, is the process of testing multiple variations of a web page in order to identify higher-performing variations and improve the page’s conversion rate.

As the web has become increasingly competitive and traffic has become increasingly expensive, the rate at which online businesses are able to convert incoming visitors to customers has become more and more important.

In fact, it has led to an entirely new industry, called Conversion Rate Optimization (CRO), and the centerpiece of this new CRO industry is AB testing.

More than any other thing a business can do, AB testing reveals what will increase online revenue and by how much. This is why it’s a crucial tool for today’s online business.

What Is Conversion AB Testing?

An A/B test is an experiment in which a web page (Page A) is compared against a new variation of that page (Page B) by alternately displaying both versions to a live audience.

The number of visitors who convert on each page is recorded as a percentage of conversions per visitor, referred to as the “conversion rate”. The conversion rates for each page variation are then compared against each other to determine which page performs better.

What Is a Split Test? How does ab testing work? Who should run AB tests? Discover the Conversion Scientists’ secrets to AB testing.

What Is An A/B Test?

Using the above image as an example, since Page B has a higher conversion rate, it would be selected as the winning test and replace the original as the permanent page displayed to visitors.

(There are several very important statistical requirements Page B would have to meet in order to truly be declared the winner, but we’re keeping it simple for the purposes of this article. If you prefer to go deep on this topic, check out our ultimate a/b testing guide: everything you need in one place)

How Does Split Testing Work?

Split testing is a conceptually simple process, and thanks to an abundance of high-powered software tools, it is now very easy for marketers to run A/B tests on a regular basis.

1. Select A Page To Improve

The process begins by identifying the page that you want to improve. Online landing pages are commonly tested, but you can test any page of a website. AB testing can even be applied to email, display ads and any number of things..

2. Hypothesize A Better Variation of the Page

Once you have selected your target page, it’s time to create a new variation that can be compared against the original. Your new page will be based on your best hypothesis about what will convert with your target audience, so the better you understand that audience, the better results you will get from AB testing.

Recommended reading: How to Create a Testing Hypothesis That Drive Real Profits: 8 Simple Steps

3. Display Both Pages To A Live Audience via the A/B Test Tool

The next step is to display both pages to a live audience. In order to keep everything else equal, you’ll want to use split testing software to alternately display Page A (original) and Page B (variation) via the same URL.

4. Collect A/B Test Conversion Data

Collect data on both pages. Monitor how many visitors are viewing each page, where they are clicking, and how often they are taking the desired action (usually converting into leads or sales). Tests must be run long enough to achieve statistically significant results.

5. Select The Winning Page

Once one page has proven to have a statistically higher conversion rate, implement it as the permanent page for that URL. The AB testing process is now complete, and a new one can be started by returning to Step #2 and hypothesizing a new page variation.

And Now, the What is AB Testing Video Guide

Who Should Run AB Tests?

Now that you understand what an A/B test is, the next question is whether or not YOU should invest in running A/B tests on your webpages.

There are three primary factors that determine whether AB testing is right for your website:

  1. Number of transactions (purchases, leads or subscribers) per month.
  2. The speed with which you want to test.
  3. The average value of each sale, lead or subscriber to the business.

We’ve created a very helpful calculator called the Conversion Upside Calculator to help you understand what each small increase in your conversion rate will deliver in additional annual income.

Based on how much you stand to earn from improvements, you can decide whether it makes sense to purchase a suite of AB testing tools and experiment on your own or hire a dedicated CRO agency to maximize your results.

Want To Learn More About AB Testing?

21 Quick and Easy CRO Copywriting Hacks to Skyrocket Conversions

21 Quick and Easy CRO Copywriting Hacks

Keep these proven copywriting hacks in mind to make your copy convert.

  • 43 Pages with Examples
  • Assumptive Phrasing
  • "We" vs. "You"
  • Pattern Interrupts
  • The Power of Three

"*" indicates required fields

This field is for validation purposes and should be left unchanged.

Jacob McMillen
15 replies
  1. Tim McCaskill says:

    Hi Jacob,
    Great write up! In regards to deciding if conversion testing is right for us or not, I am guessing that for high ticket items like software services, even a smaller number of sales compared to other lower ticket, higher number of transactions item should suffice to tip the scale in the favor of implementing conversion testing. Because even one conversion as a result of testing can lead to a five-to-six figure client.


Trackbacks & Pingbacks

  1. […] the deal. You simply cannot A/B test effectively without a sound understanding of A/B testing […]

  2. […] the deal. You simply cannot A/B test effectively without a sound understanding of A/B testing […]

  3. […] your test. You will use your Google Analytics goals to ultimately determine which variation of your AB test is the […]

  4. […] already know how to run an AB test. Applying that practice to your headlines is a simple 4-step […]

  5. […] How do you find the right time? Two words: AB testing. […]

  6. […] you how to identify a poor value proposition, hypothesize a series of better alternatives, and split test them to identify the wining combination of copy, video and […]

  7. […] we run a split test with Optimizely, the software uploads scripts and data into all of our visitors’ browsers to […]

  8. […] Split tests are a favored tool of economists, direct response mailers and website optimizers. The primary challenge when creating a split test is being sure that you’re measuring exactly what you think you’re measuring. […]

  9. […] test. The image is one variable and the headline is a second variable. Technically, this is an AB test with two variables changing. If Variation (B) generated more leads, we wouldn’t know if the […]

  10. […] duration of an AB test is a function of two […]

  11. […] Tools for running the actual AB tests […]

  12. […] the deal. You simply cannot AB test effectively without a sound understanding of […]

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *