A/B Test Calculator
Enter visitors and conversions for your control and variant to find out if the difference in conversion rate is statistically significant - or just noise.
How A/B test significance is calculated
Z = (p2 - p1) / sqrt(p_pool x (1 - p_pool) x (1/n1 + 1/n2))The calculator uses a two-proportion z-test to determine whether the difference in conversion rates between control and variant is likely to be real or due to random chance.
It pools the conversion rates from both groups to estimate the baseline rate, calculates the standard error of the difference, and divides the observed difference by that error to get a z-score. The z-score maps to a p-value - the probability of seeing a difference this large by chance if there is actually no effect.
A p-value below 0.05 means there is less than a 5% chance the result is random - this is the standard 95% confidence threshold. A p-value below 0.10 gives 90% confidence. Below either threshold, you can be more confident the variant is genuinely different from the control.
- Visitors
- The number of users who were exposed to each variant.
- Conversions
- The number of goal completions in each group.
- Uplift
- The relative change in conversion rate from control to variant (positive = variant wins).
- Confidence
- How certain you can be that the result is not random. 95%+ is the standard threshold for most decisions.