Peeking & Sequential Testing: Why Early Stops Lie
This lesson is part of the course
In this lesson
You replay 20 A/A tests that were checked four times each, measure how peeking inflates the false positive rate and fix it with a Pocock boundary.
Buy the course once for $9.99 to unlock every lesson and lab, the final exam and the certificate – lifetime access, no subscription.
Lessons in this course
- Metrics, Hypothesis & Minimum Detectable Effect Locked
- Sample Size for a Conversion Test (80% Power) Locked
- Test Duration: From Sample Size to Full Weeks Locked
- Analysing a Conversion Test: z-Test, p-Value & Confidence Interval Locked
- Revenue per Visitor: Welch t-Test on Skewed Data Locked
- Many Variants: The Multiple-Comparison Problem & Bonferroni Locked
- Sample Ratio Mismatch: Chi-Square Check of the Traffic Split Locked
- Peeking & Sequential Testing: Why Early Stops Lie Locked