A/B test calculator
How this works
- Treatments3 against the originalApple
- Longest test90 daysApple
- Confidence Apple recommendsAt least 90%Apple
Why 90% and not the textbook 95%
Apple allows 3 treatments against the original, runs a test for up to 90 days, and recommends waiting for at least 90% confidence — which is why 90% is the default here rather than the textbook 95%. App Store Connect reports the confidence for your test; this works out what the test needed in the first place.
Decide the size before you look
Checking a running test repeatedly and stopping the moment it crosses the line inflates the false positive rate well past the number on screen, however the maths is corrected.
What this cannot see
This models impressions to downloads. It cannot see whether a treatment brings worse users — a screenshot set that wins on install rate and loses on retention is a real outcome, and only your own analytics will show it.
Nothing typed here leaves the browser: there is no fetch on this page, and no App Store account it could read. Your numbers stay in this tab, the address bar and this browser's storage.