Tides of Your Website and Tests that Lie [AUDIO]

Don't stop tests early.

What if this test had been stopped on Thursday?

We encourage every online marketer to begin experimenting with split testing as soon as possible. It is a skillset that empowers marketing to really understand their visitors and generate more revenue from existing traffic.

One of the first hurdles you will have to get through is the “damn lying test” phenomenon. This is when a test has been run and has found a way to significantly increase results. The confidence level reported by the testing tool is 95%. You are going to be a hero.

Then you roll out the change to the site and nothing happens. Conversions don’t go up. Revenue per visit doesn’t move. What’s going on?

It’s the Damn Lying Test. One of the reasons tests lie is that they didn’t take into account the ebb and flow of traffic on your site. Tests must take into account peak and off-peak times. Form most sites, a test must run for a full week and end on seven-day boundaries.

How can you discover the rise and fall of tides in your website?

In Ride The Tides Of Your Website To Make More Money With A/B Split Testing, I discuss the various cycles and how to detect them on your site using analytics.

Listen to me read the column

The Conversion Scientist Podcast

Subscribe to the Podcast
If you enjoyed this, please leave a review on iTunes!

I provide graphs and examples to illustrate a variety of website tides:

  • Intra-day cycles
  • Weekly tides
  • Monthly cycles
  • Yearly seasons
  • Device-dependent cycles
  • Sales cycles
  • Artificial cycles

Understanding the rise and fall of the tides in your website will help you design better tests that deliver results you can take to the bank. Use your knowledge of website tides and some discipline to steadily increase the profitability of your site.

Special thanks to Craig Sullivan for inspiring this column with his Digital Elite Camp presentation 20 Simple Ways to Mess Up Your AB Testing.

Brian Massey
0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *