Is A/B Testing On Your Low Traffic Website a Waste of Time?

In a previous post we examined the question, should you really be A/B testing? Today we’ll dive deeper into the subject and look at A/B testing with low traffic websites.

To test or not to test? It’s a question that’s commonly discussed within the CRO realm for SMB. Some experts believe if your traffic volume is low that you should be focusing on gaining more visitors first, while some say you should always be A/B testing. So which is it?

Let’s examine further.

It is generally accepted that testing and optimizing your marketing tactics is key to growing your business. And if you have sufficient data, A/B testing is a surefire way of determining if a web tactic, design, call to action (CTA), or headline is boosting conversions. But when time is of the essence, and budget is limited, A/B testing can seem daunting.

If you don’t have a million daily visitors or thousands of conversions per month, A/B testing can be tricky but not entirely futile. Take a look at the graph below from VWO to get a sense of how much traffic and time is needed to achieve successful A/B test results.

The conclusion is that if you get less than 500 visitors/day/variation, you will need more than 400 days of testto get conclusive results. Not ideal.

That is to say that, yes, it’s still possible, but no, it’s not the best use of your time. When you’re short on traffic, conducting classic A/B tests can take months to attain enough certainty to confidently act on the insight.

This said, if you still want to begin optimizing your low traffic website and start gaining valuable customer insights, you’ll have to switch up your thinking and get creative with A/B tests or shift your focus to alternative methods.

How Much Traffic Should I Have?

A good rule of thumb is to have roughly 1,000 visitors per week to the page that you want to test or about 50 conversions per week. If you don’t have anywhere near those numbers, and don’t have enough resources to take risks with A/B testing, instead focus on building up your traffic volume, increasing your conversions, and learning more about your customers.

To do this, start by leveraging;

• SEO and CRO with a digital agency
• Email campaigns using your existing subscriber lists
• Heat mapping with tools like Hotjar or CrazyEgg
• Paid traffic with Google AdWords or Facebook, LinkedIn, Instagram, and Twitter Ads
• User testing with services like usertesting.com and whatusersdo.com
• Customer surveys

However, if you do have sufficient traffic/conversions and want to get the most from A/B testing, or you’re not quite there with your traffic/conversions but want to take risks and try some low traffic A/B testing hacks, try the following.

Think Micro, Not Macro

Testing for macro conversions (big ticket conversions) like purchases, sign-ups, and revenue are typically harder to test on low traffic websites. When you’re only working with hundreds of visitors it might be best to test for micro conversions. These can include visitor engagement, progressing from the homepage to the product page, and clicking the “add to cart” button.

Changing your idea of what a conversion is can help you achieve significance faster. For example, you might get 1000x more visitors clicking through to your product page from your homepage than you would visitors actually buying a product. Conducting tests around conversions like that allows you to achieve significant results quicker and enables you to uncover valuable insight into your customer's behavior, which will help you drive more traffic to your product page and in turn drive more sales.

Go Big or Go Home

Low traffic sites should be testing big, high-impact changes. Think entire landing pages, website themes, navigation, pricing plans, and messagin. These high-impact changes often result in huge variances in conversion levels which requires less time in order to achieve statistically significant results.

A/B testing isn’t so much a solution as it is a way of thinking and a way of uncovering key insights into your customer and their behaviour. When you’re preparing your tests, think about what would impact your customers the most.

For example, if your audience is comprised of price sensitive college students, offer up a free-shipping deal to see how that affects conversions. Or if your audience is affluent, image-conscious millennials, create a limited time release of one of your products, as this might intrigue them and spur more purchasing. Get the idea?

Bob and Lush, a premium dog food supplier, knew their audience and were able to see a 13.5% lift in conversions after implementing and testing several high-impact changes to their dog food supplier shopping cart page.

As you start to better understand your customers you can implement tests that will cause drastically different results. The more drastic the results the quicker you can perform successful A/B tests ideal for low traffic scenarios.

Take Risks to Get Results

Another way to achieve results quicker for low traffic websites is to experiment with lowering the level of significance. The greater the statistical risk you are willing to take, the quicker you’ll achieve statistical significance.

Using the online calculator from Optimizely you can see the difference in dropping your confidence level from 95% to 80%. This small change can shave weeks off your total test duration.

To Test Or Not To Test?

Wait until you have at least the recommended 1000 visitors a week or more, otherwise you will be wasting your time. If you don’t yet have that, work on getting more traffic to your site. On the other hand, if you are getting enough visitors, you can begin to run some A/B tests but remember to run tests that are going to have a major impact.