Statistical significance for two proportions
The pooled two-proportion z-test evaluates a null hypothesis that two population proportions are equal. It uses observed successes and sample sizes from two independent groups.
Choose the alternative hypothesis before interpreting the p-value. Statistical significance is not the same as practical importance, and valid inference still depends on sampling and experiment design.
How to use the two-proportion z-test calculator
- Enter both groups: Provide whole-number successes and sample sizes.
- Choose the alternative: Select two-sided, group 2 greater, or group 2 less.
- Set α: Use a significance level chosen before examining results.
- Run the test: Review p-value, effect estimates, and any approximation warning.
Formula and variables
The null-hypothesis standard error uses the pooled proportion p̂ = (x₁+x₂)/(n₁+n₂).
z = (p̂₂ − p̂₁)/√[p̂(1−p̂)(1/n₁ + 1/n₂)]- x — Successes
- Observed successes in a group
- n — Sample size
- Independent observations in a group
- p̂ — Sample proportion
- Observed successes divided by sample size
- α — Significance level
- Preselected rejection threshold
Conversion-rate comparison
Control has 100 conversions from 1,000 users and variant has 130 from 1,000.
- Control
- 100 / 1,000
- Variant
- 130 / 1,000
- Alternative
- Two-sided
- p̂₁ = 0.10; p̂₂ = 0.13
- Compute the pooled standard error and z statistic
Result: The two-sided p-value is approximately 0.0365.
At α = 0.05 the test rejects equal proportions, while the 3-point absolute difference describes effect size.
Understanding your results
Interpret p-value with effect size
The p-value describes compatibility with the null model, not the size or value of an effect.
- Absolute difference is p̂₂ − p̂₁.
- Relative lift uses group 1 as baseline.
- A one-sided alternative must be selected in advance.
- Small expected counts call for an exact or alternative method.
Assumptions
- Groups are independent and observations within groups are suitably independent.
- The large-sample normal approximation is appropriate.
- The analysis plan and alternative hypothesis were selected without outcome-driven switching.
Limitations
- Does not provide an exact test, confidence interval, power analysis, sequential-testing correction, or multiple-comparison correction.
- Does not correct confounding, selection bias, attrition, peeking, or instrumentation problems.
- Relative lift is undefined when group 1 has zero successes.
Common mistakes
- Entering rates instead of success counts.
- Changing from two-sided to one-sided after seeing results.
- Equating p < α with a large or important effect.
- Ignoring repeated looks or multiple tested variants.
Practical use cases
A/B experiments
Compare two large-sample conversion rates after a valid randomized experiment.
Statistics coursework
Check pooled two-proportion z-test arithmetic.
Frequently asked questions
What does p < 0.05 mean?
Under the null model and assumptions, a result at least this extreme has probability below 0.05; it does not prove the alternative or measure effect size.
When should I avoid this z-test?
Avoid relying on the normal approximation when expected success or failure counts are small or observations are dependent.
Sources and review
- Binomial Proportion Test — NIST/SEMATECH e-Handbook of Statistical Methods. Accessed 2026-07-14.
Reviewed 2026-07-14.