Intro to AB Testing: What Is An A/B Test?
What is AB Testing? How does split testing work? Who should run AB tests? Discover the Conversion Scientists’ secrets to running an A/B test.
AB testing, also referred to as “split”, A/B test or “ABn” testing, is the process of testing multiple variations of a web page in order to identify higher-performing variations and improve the page’s conversion rate.
As the web has become increasingly competitive and traffic has become increasingly expensive, the rate at which online businesses are able to convert incoming visitors to customers has become more and more important.
In fact, it has led to an entirely new industry, called Conversion Rate Optimization (CRO), and the centerpiece of this new CRO industry is AB testing.
More than any other thing a business can do, AB testing reveals what will increase online revenue and by how much. This is why it’s a crucial tool for today’s online business.
What Is Conversion AB Testing?
An A/B test is an experiment in which a web page (Page A) is compared against a new variation of that page (Page B) by alternately displaying both versions to a live audience.
The number of visitors who convert on each page is recorded as a percentage of conversions per visitor, referred to as the “conversion rate”. The conversion rates for each page variation are then compared against each other to determine which page performs better.
What Is An A/B Test?
Using the above image as an example, since Page B has a higher conversion rate, it would be selected as the winning test and replace the original as the permanent page displayed to visitors.
(There are several very important statistical requirements Page B would have to meet in order to truly be declared the winner, but we’re keeping it simple for the purposes of this article. If you prefer to go deep on this topic, check out our ultimate a/b testing guide: everything you need in one place)
How Does Split Testing Work?
Split testing is a conceptually simple process, and thanks to an abundance of high-powered software tools, it is now very easy for marketers to run A/B tests on a regular basis.
1. Select A Page To Improve
The process begins by identifying the page that you want to improve. Online landing pages are commonly tested, but you can test any page of a website. AB testing can even be applied to email, display ads and any number of things..
2. Hypothesize A Better Variation of the Page
Once you have selected your target page, it’s time to create a new variation that can be compared against the original. Your new page will be based on your best hypothesis about what will convert with your target audience, so the better you understand that audience, the better results you will get from AB testing.
Recommended reading: How to Create a Testing Hypothesis That Drive Real Profits: 8 Simple Steps
3. Display Both Pages To A Live Audience via the A/B Test Tool
The next step is to display both pages to a live audience. In order to keep everything else equal, you’ll want to use split testing software to alternately display Page A (original) and Page B (variation) via the same URL.
4. Collect A/B Test Conversion Data
Collect data on both pages. Monitor how many visitors are viewing each page, where they are clicking, and how often they are taking the desired action (usually converting into leads or sales). Tests must be run long enough to achieve statistically significant results.
5. Select The Winning Page
Once one page has proven to have a statistically higher conversion rate, implement it as the permanent page for that URL. The AB testing process is now complete, and a new one can be started by returning to Step #2 and hypothesizing a new page variation.
And Now, the What is AB Testing Video Guide
Who Should Run AB Tests?
Now that you understand what an A/B test is, the next question is whether or not YOU should invest in running A/B tests on your webpages.
There are three primary factors that determine whether AB testing is right for your website:
- Number of transactions (purchases, leads or subscribers) per month.
- The speed with which you want to test.
- The average value of each sale, lead or subscriber to the business.
We’ve created a very helpful calculator called the Conversion Upside Calculator to help you understand what each small increase in your conversion rate will deliver in additional annual income.
Based on how much you stand to earn from improvements, you can decide whether it makes sense to purchase a suite of AB testing tools and experiment on your own or hire a dedicated CRO agency to maximize your results.
Want To Learn More About AB Testing?
21 Quick and Easy CRO Copywriting Hacks
Keep these proven copywriting hacks in mind to make your copy convert.
- 43 Pages with Examples
- Assumptive Phrasing
- "We" vs. "You"
- Pattern Interrupts
- The Power of Three
- The 20 Most Recommended AB Testing Tools By Leading CRO Experts (2023 Update) - June 13, 2021
- AB Testing Research: Do Your Conversion Homework - December 8, 2018
- 8 Elements of a High Converting Squeeze Page - June 21, 2018
Conversion Sciences LLC
Hi Jacob,
Great write up! In regards to deciding if conversion testing is right for us or not, I am guessing that for high ticket items like software services, even a smaller number of sales compared to other lower ticket, higher number of transactions item should suffice to tip the scale in the favor of implementing conversion testing. Because even one conversion as a result of testing can lead to a five-to-six figure client.
Thanks,
Tim
Hi Tim,
That’s true, but at the same time, it’s very difficult to run accurate AB tests at low volume. I discuss this in depth here: https://conversionsciences.com/ab-testing-statistics/