Conversion Sciences offers a pretty amazing ability to our clients: A completely “turnkey” testing service. By “turnkey” we mean that our clients don’t need to do anything to their website in order for us to analyze the site, design creative, develop code, QA, test, review and route traffic to winners.
Why is this? Don’t we need new pages designed? Interactions with IT? Release schedules? Sprints?
The reason we have this “phenomenal cosmic power” is that our AB testing tools take control of each visitor’s browser. The changes are made on the visitors’ devices, so the website doesn’t have to be modified.
Well, not until we’ve verified that a change brings in big money.
While this makes us feel all high and mighty, it comes with a big drawback.
And then there’s this.
First the web page is loaded as is from the webserver. Some visitors will see only this default version of the website.
Phenomenal Cosmic AB Testing Power
We change the headline to something different.
We change the order of a rotating carousel, or slider.
We hide elements…
and insert them as well.
We insert video.
We completely change the look of a website.
If we wanted, we could deliver a different website to every visitor who arrives. If you’re thinking “personalization” at this point, then you have an idea of where our industry is heading.
AB testing produces the data that makes personalization truly work.
Here are my instagraphic notes from a presentation by Chris Gibbins at Conversion Conference 2016 on using AB testing and personalization.
AB Testing Flicker, Flash, Blink and Ghosting
This has been called a “flash”, “flicker”, and a “blink”. It can have a significant effect on test results.
Was it the new headline we tested or was it the flicker that made more people buy?
The Human Eye is Drawn to Motion
Our eyes and brains have evolved to make motion important. When something moves, our eyes and brains work hard to figure out if the thing moving is edible, if it will eat us, or if we can mate with it. Technology has moved faster than evolution. Even though we’re looking at a website, where there is no immediate source of food, fear or fornication, our lizard brains continue to favor motion.
Imagine this scenario. We are testing the text on a button. That means that for some of our visitors, the button will change. Others will see the default button text. Then we’ll see who buys the most.
If there is a flicker, flash or blink when the text changes, the button will be immediately more visible to those visitors who see it. More of them will see the call to action. This may make more of them consider buying than those who simply scrolled past. If this new treatment generates more revenue, we are left with the question, “Was it the text or was it the motion that created the lift in sales?”
We won’t know until we push the new text onto the server and do a post-rollout analysis to see the real lift. At this point, we may find that the text didn’t increase purchases. It’s a real bitch.
How many buttons have been change because of flicker?
AB Testing Software Tries to Eliminate Blinking
The AB testing software that we use works to eliminate this blinking and flashing. The issue is so important that Convert Insights has patented their method for eliminating “blink”.
AB testing software like Optimizely, Visual Website Optimizer, Adobe Test, and Marketizator, load asynchronously, meaning that they load our changes as the page loads. This makes it easier for changes to be made before the visitor sees the page.
Eliminating Flash Flicker and Blink in AB Tests
In addition to this, our developers can do things that reduce and eliminate flicker and blink. Every test you do has different issues, and so a variety of tactics can be used to address them.
Avoid Google Tag Manager
Make Changes with CSS
Modal Dialogs and Overlays
Modal dialogs usually don’t display until the visitor takes an action. They can be edited before they are shown without flashing.
Use a Timer for DOM Changes
For the AB Testing Software to Load Our Changes Immediately
The AB testing software provides a synchronous loading mode. Optimizely and VWO use different approaches for this.
Rethink the Test
Sometimes, we have to go back to the drawing board and design a test that doesn’t cause flash-inducing changes. We will refactor a test to eliminate items that cause a flash or flicker.
Delay Display of the Content
We can delay the display of the content until the DOM is loaded and all elements have been changed. This causes a different kind of issue, however. The visitor sees a blank page or blank section for longer if they get the page with the change.
Insert Placeholders Using Fast CSS
Optimizing for Mobile
Mobile pages load more slowly. This is because visitors don’t always have a high-quality connection when visiting from the festival, or from inside the walls of a bank while standing in line. For mobile devices, flash can be an even bigger issues.
Fortunately, the tactics we use on the desktop also work on mobile websites. But don’t forget to QA your test treatments on 3G and 4G connections. You may find flicker and blink on mobile devices that didn’t appear on your desktop.
If you’d like our team to take over the effort of developing your AB tests, contact us for a free consultation and an overview of our process.