The benefits of split testing are clear – you get more conversions, higher ROI, greater insight into the marketing messages that you should be delivering, etc.
But, like anything else you do online, you have to be extremely careful when testing. It’s just as easy to improve a conversion rate with a simple headline test as it is to worsen a conversion rate – or worse, ruin your brand – by testing the wrong thing at the wrong time.
To help prevent you from making these errors, here are 6 split testing mistakes that businesses make all the time:
1. Overselling
When you’re split testing, it’s extremely tempting to strongly sell your customers – and for the most part, stating value propositions, listing customer testimonials, explaining benefits, etc is all standard best practice.
But, you have to be extremely careful not to compromise conversion increase with the impacts on your brand.
Here’s an example:
Man! Do you trust this guy?
While there are still some basics on this page to be tested (having multiple CTAs, etc), it’s clear that this landing page has many of the standard optimization best-practices built-in, namely:
Benefits
In the “you’ll learn how to” section, he does a great job listing out the three main benefits of downloading the report.
Call to Action
Emphasizing that the report is “Free” is also a best practice (as is the capitalization on the word “Free”)
While this online marketer may have optimized this page for maximal conversion, at what cost did he do this?
These types of landing pages – where product pages are intense and feel oversold – are common in the affiliate space, and completely ignore the benefits of brand. While selling your product to your customers will lead to a short-term lift in conversion rate, it will have long-term negative impacts on your brand that will often outweigh the short-term benefit.
There are other ways to lift a conversion rate – for example, through stunning design:
Do you notice how this page lifts both brand and value?
2. Not Testing Broadly Enough
It’s never been easier for someone to test small variables such as:
- Background Image
- Call to Action
- Headline
Etc. etc. While these tests are certainly valuable – and can lead to a conversion increase all of these tests have their limits. There is only so much of an uplift that you can get by testing these local variables and sometimes, to get the 100%+ lift that you’re looking for, you need to think outside of the box.
SmartShoot did an incredible test based on Dan Ariely’s pricing in “predictably irrational”. Here’s the test:
For the new page, they decided to “make up” an in between package. The intention here wasn’t to show customers a new pricing plan that they could afford, but to simply make the value of both packages clear.
Here’s the new page with the made up version:
The new page led to nearly a 3x increase in conversion rate – with a made-up plan!
While simple changes can lead to a conversion uplift, drastic changes can also make a large conversion improvement. If you’re only testing background images, copy, etc you are missing out on potential opportunities for lift.
3. Testing Randomly without Insight
When we work with clients at UpliftROI.com, one of the first things we ask them is – what have you tested before? And what’s your testing plan?
Many of our clients have tested things before like A/B split testing, but very, very few of them can share a logical plan with us outlining what variables they’ve decided to test and why.
The most important step in your testing process is to layout a logical flow of variables to test, based on customer research.
For example, for a typical e-commerce website, you’re going to need to layout and test the following variables:
Shipping
What is the optimal price for us to offer free shipping at? Where do we offer this? How fast should shipping be?
Product Descriptions
What’s the optimal copy for each product to sell at maximal conversion rate?
Price
Do we have the lowest price? Highest? How do we compete with competitors on price to maximize conversion rate?
Testing these at random isn’t a good idea because of how much these variables can interact with conversion. For example, you need to be extremely careful when testing the price of your products with the price of free shipping, as both of these variables can interact to affect your conversion rate.
4. Assuming Statistical Significance Means More Revenue
Just because a software program tells you that you’ve improved conversion at a 95% confidence interval does *not* mean that you have actually boosted revenue.
This mistake is the most painful and even experts like Neil Patel have performed split tests that didn’t impact revenue.
But how is this possible?
Traffic sources for Fast-Growing Online Businesses can change rapidly.
While it’s possible that your new variation design achieved a lift – at a statistically confident level – it’s possible that it may not achieve that same lift say 2-4 weeks (or even months) later when traffic sources change.
In my experience, this is one of the most frustrating things about conversion rate optimization, both for beginners and professionals, so it’s important to give yourself enough time to prove that the tests are actually driving an increase in revenue.
A great rule of thumb to prevent you from making this mistake is to let your new variation get between 100 and 200 conversions (or more if you have a large website). This should show you beyond a doubt that your new page is driving an uplift in revenue.
5. Not having enough Data & under-estimating Time required
Before you decide to test anything, you need to make sure that you have enough data to prove significance.
We are constantly inundated with requests to optimize websites with only 1,000 to 2,000 unique visitors, and while these sites can be optimized in some cases, they are most often not worth it.
Similarly, I would recommend that you always overestimate the amount of time that it will take you to get an increase in conversion rate. While it could take you as few as a few days to prove that you have achieved higher conversion at statistical confidence, we generally recommend a minimum of 2 weeks (or 100 – 200 conversions) in order to be absolutely sure. When selling CRO into your boss, always under promise and overdeliver.
Summary
A/B Testing is a mixture of an art and a science. Be sure to layout a logical plan with closely measured results in order to maximize the return on your time and minimize simple mistakes.
Author Bio
A/B Testing is Andy’s passion. Prior to starting Uplift he worked at both Google and Facebook in sales and marketing roles.