A/B Test Intro: What Is An A/B Test?


Sign Up For Our Free CRO Course

AB testing, also referred to as “split” or “ABn” testing, is the process of testing multiple variations of a web page in order to identify higher-performing variations and improve the page’s conversion rate.

As the web has become increasingly competitive and traffic has become increasingly expensive, the rate at which online businesses are able to convert incoming visitors to customers has become more and more important.

In fact, it has led to an entirely new industry, called Conversion Rate Optimization (CRO), and the centerpiece of this new CRO industry is AB testing.

More than any other thing a business can do, AB testing reveals what will increase online revenue and by how much. This is why we recommend it.

What Is An A/B Test?

An AB test is an experiment in which a web page (Page A) is compared against a new variation of that page (Page B) by alternately displaying both versions to a live audience.

The number of visitors who convert on each page is recorded as a percentage of conversions per visitor, referred to as the “conversion rate”. The conversion rates for each page variation are then compared against each other to determine which page performs better.


Using the above image as an example, since Page B has a higher conversion rate, it would be selected as the winning test and replace the original as the permanent page displayed to visitors.

(There are several very important statistical requirements Page B would have to meet in order to truly be declared the winner, but we’re keeping it simple for the purposes of this article)

How Does Split Testing Work?

Split testing is a conceptually simple process, and thanks to an abundance of high-powered software tools, it is now very easy for marketers to run AB tests on a regular basis.

1. Select A Page To Improve

The process begins by identifying the page that you want to improve. Online landing pages are commonly tested, but you can test any page of a website. AB testing can even be applied to email, display ads and any number of things..

2. Hypothesize A Better Variation

Once you have selected your target page, it’s time to create a new variation that can be compared against the original. Your new page will be based on your best hypothesis about what will convert with your target audience, so the better you understand that audience, the better results you will get from AB testing.

3. Display Both Pages To A Live Audience

The next step is to display both pages to a live audience. In order to keep everything else equal, you’ll want to use split testing software to alternately display Page A (original) and Page B (variation) via the same URL.

4. Collect Conversion Data

Collect data on both pages. Monitor how many visitors are viewing each page, where they are clicking, and how often they are taking the desired action (usually converting into leads or sales). Tests must be run long enough to achieve statistically significant results.

5. Select The Winning Page

Once one page has proven to have a statistically higher conversion rate, implement it as the permanent page for that URL. The A/B test is now complete, and a new one can be started by returning to Step #2 and hypothesizing a new page variation.

Who Should Run AB Tests?

Now that you understand what an AB test is, the next question is whether or not YOU should invest in running AB tests on your webpages.

There are three primary factors that determine whether AB testing is right for your website:

  1. Number of transactions (purchases, leads or subscribers) per month.
  2. The speed with which you want to test.
  3. The average value of each sale, lead or subscriber to the business.

We’ve created a very helpful calculator called the Conversion Upside Calculator to help you understand what each small increase in your conversion rate will deliver in additional annual income.

Based on how much you stand to earn from improvements, you can decide whether it makes sense to purchase a suite of AB testing tools and experiment on your own or hire a dedicated CRO agency to maximize your results.

Want To Learn More About AB Testing?

Want to learn more about AB testing? Conversion Sciences has created a free CRO training course to help you learn everything you need to know.

Click Here to Sign Up



Categories: AB Testing