Why sample size matters
A test with too few visitors cannot reliably separate a real improvement from random variation. Deciding the sample size in advance, and running the test until you reach it, protects you from stopping early on a lucky streak. It also tells you whether a test is realistic for your traffic before you spend time building it.
Here p₁ is your current conversion rate, p₂ is the improved rate you want to detect, and p̄ is their average. The z values come from your confidence level and power: 1.96 for 95% confidence and 0.84 for 80% power.
Worked example
With a 5% conversion rate, to detect a 10% relative improvement (5% to 5.5%) at 95% confidence and 80% power, you need about 31,234 visitors per variation. For a test of one control and one variation that is about 62,468 visitors in total. At 1,000 visitors a day, that is roughly 63 days.
Choosing the minimum effect
Halving the effect you want to detect roughly quadruples the visitors you need. If your traffic is low, test bigger changes, because small tweaks will not give clear answers in a reasonable time. Confidence of 95% and power of 80% are common defaults. Higher values give more certainty and need more visitors.
Each extra variation needs the full sample, so the total grows with every version you add. Once the test ends, check the result in the A/B test significance calculator, and estimate its value with the CRO uplift calculator.
Frequently asked questions
How many visitors do I need for an A/B test?
It depends on your current conversion rate and the smallest lift you want to detect. A 5% rate and a 10% relative lift needs about 31,000 visitors per variation at 95% confidence and 80% power.
What is statistical power?
Power is the chance your test detects a real improvement of the size you chose. At 80% power, you would catch such an improvement in about four out of five tests.
What is a minimum detectable effect?
It is the smallest improvement you care about finding. A smaller effect needs a much bigger sample, so choose the smallest change that would be worth acting on.
What if I do not have enough traffic?
Test larger changes, run the test for longer, or test on a page with more traffic. Very small tests rarely produce a clear answer.