A/B testing is one of the most reliable ways to improve an ecommerce site, but it only works when you have enough traffic to trust the results. If your store sees thousands of visitors a month rather than millions, a classic split test can run for months without telling you anything useful. That does not mean you should stop optimizing. It means you need a different plan.

We first asked whether you should really be A/B testing. This article looks at what low traffic means for testing today, when it is still worth doing, and what to do instead when it is not.

Why low traffic makes A/B testing hard

An A/B test needs enough visitors and conversions in each variation to tell a real difference from random noise. The smaller the change you are trying to detect, the more traffic you need.

Here is a simple example. Say your store converts at 2% and you want to know whether a new product page lifts that by 10%, from 2.0% to 2.2%. At a standard 95% confidence level and 80% statistical power, you need roughly 80,000 visitors in each variation, or about 160,000 in total. If that page sees 20,000 visitors a month, the test would take the better part of a year, and your season, promotions and traffic mix would all change before it finished.

That is why so many low-traffic tests end in "no significant difference". The idea may have been good. The test simply never had enough data to prove it.

When A/B testing still makes sense

Testing is worth the effort when a page has steady traffic, a clear goal and a change big enough to move the numbers. Your homepage, top category pages, best-selling product pages and cart usually qualify first. Run the numbers through a sample size calculator before you build anything. If the answer is longer than about four to six weeks, change the test or choose a different method.

How to A/B test with less traffic

• Measure micro conversions. Purchases are rare, but add to cart, checkout starts and product page clicks happen far more often. With an add to cart rate closer to 8%, the same 10% lift needs about 19,000 visitors per variation, roughly a quarter of the traffic.
• Test bold changes. Swapping a button colour rarely produces a lift big enough to detect. Test a new product page layout, a different free shipping threshold, a simplified checkout or a new value proposition. If the true lift is 20% rather than 10%, you need about a quarter of the sample.
• Test sitewide elements. A change that appears on every product page, such as delivery messaging or review placement, pools traffic from across your catalog instead of relying on one page.
• Set your rules before you start. Decide your confidence level, minimum detectable effect and test length up front, and do not stop a test early because it looks like a winner. If you need to check results as you go, use a tool that supports sequential or Bayesian testing, which is designed for that.
• Choose tools that fit your size. Google Optimize, which many smaller brands relied on, shut down in 2023. Product analytics platforms like PostHog now combine experiments, funnels and session replays in one place, often at a price that suits a growing brand.

What to do instead of A/B testing

When the numbers do not support a test, focus on research and fixes that are likely to help. These methods work well at any traffic level.

• Fix what is broken first. JavaScript errors, failed payments and slow pages quietly cost sales. An error monitoring tool like Noibu shows which issues affect revenue so you can fix them before you test anything.
• Watch real sessions. Session replays and heatmaps show where shoppers hesitate, rage click or drop off. PostHog and Microsoft Clarity both make this easy to set up.
• Run a heuristic review. An expert review against proven usability principles finds problems quickly without any traffic at all. We explain how in how heuristic analysis improves ecommerce UX and 10 heuristics to optimize your checkout.
• Talk to your customers. A handful of moderated user tests on a key journey often reveals the biggest usability issues. Short post-purchase and exit surveys tell you what nearly stopped people from buying.
• Use AI to find patterns. AI tools can now sort through product reviews, support tickets, on-site search terms and survey answers in minutes, surfacing the recurring questions and complaints that are worth fixing first.
• Measure before and after. When you ship a change without a test, record a clear baseline and compare over a comparable period. It is less precise than a controlled test, but far better than guessing.

So is A/B testing a waste of time on a low-traffic site?

Not always, but it is often the wrong first step. If your key pages cannot reach a result within a few weeks, you will learn more by fixing errors, studying real behaviour and making bold, evidence-based changes. As traffic grows, bring in A/B testing on your highest-impact pages and use it to validate the big decisions.

Either way, the goal is the same. Understand your customers, remove what gets in their way, and keep improving. That is the approach behind our conversion rate optimization and customer journey and UX design work. For more ideas, read our ecommerce UX design tips, see our work or get in touch.