testing inspiration

How do you decide which elements of your site to test? This question is at the heart of website optimization.
A better question is, “How do you determine what NOT to test?”
It’s relatively easy to come up with ideas that might increase your conversion rate. We typically come up with 50, 75, 100 or more ideas for each of our client sites. Filtering through this list is the hard part.
Here’s the approach we take at Conversion Sciences (my employer).

Step One: Look For Evidence

You should never test anything if you don’t have some evidence that it is a problem. These ideas are called hypotheses for a reason. A hypothesis is an educated guess, an informed fabrication, a data-based brain fart.
So you need to educate, inform and find data on your ideas, or they don’t qualify as hypotheses. They’re just happy thoughts.
The first benefit of looking for evidence is that you might be able to eliminate a hypothesis. You might find evidence that it’s NOT a problem.
Here’s an example hypothesis for the product page of an e-commerce site: “If we put an ‘Add to Cart’ button at the bottom of the  page, more visitors will add an item to their cart.”
Sounds reasonable. Yet, if few people are scrolling down the page, this hypothesis won’t hold water.
We can look at attention data, or “heat map” data generated by click-tracking and scroll-tracking software such as CrazyEgg. This will tell us how far visitors are scrolling on the product pages of the site.
If they aren’t scrolling far, then we may save this hypothesis for another time.
When we’re identifying what to test, we give each hypothesis a rating from 1 to 5 for how much evidence there is.
A rating of “1” means there’s no evidence, that the hypothesis is just an idea. A rating of “5” tells us that there is overwhelming evidence that there is a problem this hypothesis could address.
I’ve written and talked about the sources of data that are available to help you with this.

Step Two: Rate The Traffic

We want to avoid optimizing the wrong parts of the site. Our hypothesis list should have ideas for site-wide improvement, as well as page-specific enhancements.
Changing the order of the site’s navigation, for example, is a site-wide change. Adding trust symbols to the checkout page is page-specific. If we were to rate the value of the traffic on a scale of 1 to 5 again, what would we give these two scenarios? They both might get a 5.
A site-wide change, such as adjusting the navigation, has an impact on 100% of the visitors. That’s a 5 in my book. Accordingly, changing a page that is only seen by 20% of visitors or less gets a 1.
Visitors to the checkout page often account for a small percentage of viewers. Why give them a 5? Because what this traffic lacks in volume it makes up for in opportunity.
Visitors who are checking out have demonstrated significant buying intent. These visitors are very valuable to us.
Other pages may not get much attention. The “About Us” and “FAQ” pages may not be so interesting to us. They might get a 2 or 3.
Favor hypotheses that have an impact on the most, or most interesting, visitors.

Step Three: How Hard Is It To Test?

For each of our hypotheses, we want to understand what the level of effort might be. It’s easy to change the text of a guarantee or offer. It’s much more difficult to add live chat to a site.
If we use our 1-to-5 scale again, we might give the change in the copy a 1 or a 2. Adding live chat requires hiring a live chat vendor, doing integration and staffing for our chatty visitors. This is a 5 in my book.
You don’t want to favor simple tests for simplicity’s sake. Don’t rush off and test button color just because it’s a 1 on your level-of-effort scale.
Likewise, hold off on swinging for the fences until the low-hanging fruit has been found. Leave your 5s for another time.

Step Four: What Does Experience Tell You?

Finally, gauge the impact you think this hypothesis will have. This is based on your knowledge of your prospects. It is based on what you’ve learned from previous tests you’ve done.
It is based on your experience as an online marketing team. It is based on research you’ve done, such as reading this column.
How about a scale from 1 to 5 again? If you rate a hypothesis as a 1, you’re saying that this is an arbitrary idea. If it has a big impact, that will be a surprise.
If you rate your hypothesis as a 5, you’re saying you believe this change will have a significant impact on the visitors and the site. You’re expecting a big win.
Our intuition can often lead us astray. You will find yourself rating hypotheses higher on this impact scale, not because of your experience, but because you want to try them. Or you might favor one because you like the idea.
These kinds of sentiments don’t belong in a scientific environment like the one we create. However, we cannot ignore the intuition of experienced business people.
This is only one of the four factors we weigh, the others being proof, traffic value, and level of effort. A high impact score may tip a hypothesis into the top 10, but only if it has good ratings in other categories.
Once a hypothesis has been proven or disproved, there is no more role for intuition. When the data is there, we favor the data. However, when deciding what to test, we like to mix in a little gut.

Step Five: Bucket The Winners

Once we have ratings for each of the five areas, we can weight a hypothesis. We simply add together the values for Proof, Impact and Visits/Buyer Affected. Then subtract the level of effort (LOE). Here’s what part of a hypothesis list may look like:

The top ten hypotheses reveal an interesting pattern when you bucket them.

The top 10 hypotheses reveal an interesting pattern when you bucket them.

We take one more step and put each of our top hypotheses into one of five buckets:

  1. User Experience: For hypotheses that would alter the layout, design or other user interface and user experience issues.
  2. Credibility and Authority: For hypotheses that address trust and credibility issues of the business and the site.
  3. Social Proof: For hypotheses that build trust by showing others’ experiences.
  4. Value Proposition: For hypotheses that address the overall messaging and value proposition. Quality, availability, pricing, shipping, business experience, etc.
  5. Risk Reversal: For hypotheses involving warranties, guarantees and other assurances of safety.

It’s important to have these buckets because when we look at the top ten hypotheses shown in the figure, we see that six out of the ten are “User Experience” issues. This gives us a hint about the overall challenge with the site. It’s not well-designed for conversion.
We may spend our initial efforts finding out what kind of user experience these visitors want since our analysis says that the site doesn’t seem to be giving them what they want.
This is a simplified version of our process. If you’d like a copy of the “ROI Prioritized Hypothesis List” spreadsheet we use daily, send me an email at TheLab@ConversionSciences.com.


Brian Massey is the Conversion Scientist at Conversion Sciences and author of Your Customer Creation Equation: Unexpected Website Formulas of The Conversion Scientist. Conversion Sciences specializes in A/B Testing of websites. Follow Brian on Twitter @bmassey

There are a surprising number of AB tests smart marketers can run on their landing pages to ensure they’re getting the highest conversion rates possible. You already know you can experiment with different headlines, subheads, call to action text and colors, and you know to introduce design elements that draw the eye where you want visitors to look.
In this post, we’re going to share nine creative tests that go beyond the basics to expand on what you’re already doing. They might even inspire you to come up with your own imaginative AB tests.

1. Create Targeted Landing Pages

This is an AB test you would apply at specific times, such as when you launch an advertising campaign. Your advertising campaign would be targeted to a specific audience, and your ad variations would use your generic landing page and your targeted landing page.
For example, if Shopify did an AB test between their generic vs. targeted landing pages in an advertising campaign, the control would apply to anyone looking for an ecommerce solution.

Shopify's homepage with a non-specific target would be the control in this AB Test

Shopify’s homepage with a non-specific target would be the control in this AB test

For a variation, they could use their landing page targeted to booksellers.

This landing page has a spefic group of users in mind

This landing page has a spefic group of users in mind

To maximize your conversions with this test, analyze your customer database to determine who your best customers are and create targeted landing pages focused on those specific groups of customers. If you do a quick search of Shopify’s targeted landing pages on Google, you can see they have done this and created targeted landing pages for their top customer groups.

There are different landing pages to accommodate different demographics and groups of users

There are different landing pages to accommodate different demographics and groups of users

You can use these as examples of how to create targeted landing pages for your top customer groups and demographics.

2. Experiment with Animated Headlines

Headlines can make or break your landing page, as they are the first words that capture your visitor’s attention. It’s a simple AB test that can make a dramatic impact on conversions, and it can be done using most AB testing tools including Marketizator.
The key to the headline AB test is to change nothing but the headline. For example, you can see how ActiveCampaign changed from a simple headline about their features…

ActiveCampaign's headline about features

ActiveCampaign’s headline about features

…to a better headline about the benefits their customers can expect when using their service.

ActiveCampaign's headline about benefits

ActiveCampaign’s headline about benefits

This change proved to be a winner, as they have kept it since 2015 with a constantly changing message about their benefits.

This animated headline with benefits proved to be a winner

This animated headline with benefits proved to be a winner

3. Vary Featured Homepage Products

In most cases, your homepage will be the most popular landing page on your website. Hence, it will be a page where you should do extensive AB testing. One test you can do to see if you can increase conversions is simply changing your featured product. Nest does this by swapping out its popular monitoring camera (formerly Dropcam)…

Nest featuring its Nest Cam on the homepage

Nest featuring its Nest Cam on the homepage

…with its popular thermostat controller.

Nest featuring its thermostat on its homepage

Nest featuring its thermostat on its homepage

At the time of writing this article, this test was currently running (as noted by the ?alt=1 that appears at the end of their homepage URL on occasion), so we’ll have to see which one wins in the end. In some cases, you may want to change your featured product based on the one that is currently getting featured coverage in the media and whether that coverage is positive or negative.

4. Explore Different Stories

Do stories resonate with your customers and if so, which stories translate into the most sales? Find out through AB testing. Apple did this in this past by running multiple campaigns on their website, social media, and television ads featuring stories about musicians…

Apple's story appealing to musicians

Apple’s story appealing to musicians

Apple's story appealing to explorers

Apple’s story appealing to explorers

Apple's story appealing to environmentalists

Apple’s story appealing to environmentalists

Apple's story appealing to parents

Apple’s story appealing to parents

…and many other customer groups. The goal was to show how their products could aid in the telling of everyone’s story, no matter what you did or how much of an impact you made in the world. Apple ultimately went back to a homepage focused on its latest products, but without AB testing, Apple couldn’t just assume that approach would convert the highest.

5. Shift the Focus to One Product at a Time

Landing pages have better conversion rates when there is one, clear call-to-action; sometimes that CTA is buying a product. But when your top landing page is your homepage, you can’t focus on just one product, right?
Maybe you can. In the past, Logitech had a pretty standard homepage that offered up all of its products all at once to homepage visitors.

Logitech's original, more traditional homepage

Logitech’s original, more traditional homepage

But now, Logitech gives visitors a tour through their top products, one at a time.

Logitech's new homepage focusing on a single product

Logitech’s new homepage focusing on a single product

In a few moments, you are completely immersed in a particular product and its main benefit, thanks to this new landing page. It’s definitely an AB test you will want to try if you have a few products you can highlight in this fashion.

6. Transform Your Hero Image to Video

You know that great image of your product that you use to compel people to sign up or buy? Why not convert it into a video that dives even deeper into exactly what happens with that feature of your product. MailChimp made that change with their landing page, going from a screenshot of their newsletter designer…

Mailchimp's landing page with an image of newsletter editor

Mailchimp’s landing page with an image of newsletter editor

…to a video of how their newsletter designer worked.

Mailchimp's landing page with a video of newsletter editor

Mailchimp’s landing page with a video of newsletter editor

Instead of hoping that an image would convince visitors that their newsletter designer was easy to use, as the landing page claimed, the video was right there to prove it.
While their current landing page features a different, shorter animation, it still features one that demonstrates the ease of use of their newsletter designer, thus proving that since 2012, video and animation on the landing page beats a screenshot for conversions.

7. Try a Change of Scenery

Sometimes it’s not text or functionality that will make your site convert better, it’s simply imagery that matches the story of your value proposition. Zillow experimented with this idea by changing the background of its search. One variation was a neighborhood overview with home sale prices, which actually contradicts the line below the image about looking for rentals.

Zillow's background showing homes with their prices isn't relevant to people looking for rental property, especially with a message at the bottom of the page specifically written for renters

Zillow’s background showing homes with their prices isn’t relevant to people looking for rental property, especially with a message at the bottom of the page specifically written for renters

Another variation used an image of a specific home, which could appeal to both for sale and for rent searchers.

Zillow's new image says home whether you're buying or renting

Zillow’s new image says “home” whether you’re buying or renting

It seems that they have stuck with the individual home view as it works with what most searchers are looking for.

8. Rearrange the Elements

It may not be your product, your service, your copy, your colors, or other elements on your page that are lowering your conversions. It may simply be the arrangement. Just like when you rearrange all the furniture in your house because it just doesn’t quite feel right, you might want to do the same with your landing page. Take AgoraPulse, for example. They went from this…

Agora Pulse homepage before

AgoraPulse homepage before

…to this.
Agora Pulse homepage after

AgoraPulse homepage after

It’s easy to see why the latter layout works. It flows right into starting your free trial after a simple and convincing headline and subhead. And for visitors still not convinced they should convert, there’s simple video and bullet points to convince them to click that call to action button.

9. Copy Your Competitors

The most creative AB tests might be ones you don’t run on your own website. In addition to AB testing tools, there are tools that will alert you to when your competitors make changes to their websites, potentially based off of their own AB testing experiments.
Rival IQ monitors your competitor’s website to see if changes have been made to it recently. The entry level plan allows you to track up to 15 companies. You’re able to track each company’s website design history along with their social media accounts, organic search rankings, and paid search traffic.

In the website history dashboard, you can view a variety of web pages from your competitor's websites.

In the website history dashboard, you can view a variety of web pages from your competitor’s websites.

Depending on how long the company has been in Rival IQ’s database, you can get a couple of year’s worth of design history from each company.

ActiveCampaign's design history over a few years

ActiveCampaign’s design history over a few years

When you click on a particular month, you see the breakdown of when particular changes occurred as well as the ability to click upon a particular design to see the full landing page.

ActiveCampaign's homepage design history over Novemer and December 2014

ActiveCampaign’s homepage design history over Novemer and December 2014

This will give you an idea of what AB tests a competitor has run in the past and, based on the length of time a competitor has stuck with a particular design, will let you know which test was the presumed winner in terms of increasing their conversions for their homepage, pricing page, features page, or other significant pages.

ActiveCampaign's pricing page history over November and December 2014

ActiveCampaign’s pricing page history over November and December 2014

In addition, you can sign up for email alerts when your competitors make major changes to their website. This will let you know when your competitors have run new tests on their website and made changes based on their results. Or you may even see the tests themselves in action as the pages change from their original to alternative versions.
If you have a lot of competitors, and you’re not sure which to monitor, you can use the BuiltWith Google Chrome extension to find out if a particular competitor’s website is using AB testing software. Chances are, the ones that are will be the ones that will be making frequent changes.
BuiltWith browser extention for analytics and tracking

BuiltWith browser extention for analytics and tracking

What are your AB testing ideas?

If you’ve already done the standard AB tests on your landing pages and found the best headlines, subheads, and call to action buttons, then we hope that these additional creative AB tests will further help you increase conversions on your website as it may have for the brands mentioned here. Have you been running or reading about some interesting AB tests? We’d love to hear about them in the comments.

About the Author

Kristi Hines HeadshotKristi Hines is a freelance writer, blogger, and social media enthusiast. You can follow her latest tweets about business and marketing @kikolani or posts on Facebook @kristihinespage to stay informed.
Feature image by Serge Saint via Compfight cc and adapted for this post.