Three Piles of Data: Train, Validation & Test Splits from a Shuffle Number
This lesson is part of the course
In this lesson
You assign 20 days to train, validation and test with MOD on a given shuffle number, count the days per split and compare the share of busy days in each split with the whole dataset.
Buy the course once for $9.99 to unlock every lesson and lab, the final exam and the certificate – lifetime access, no subscription.
Lessons in this course
- Three Piles of Data: Train, Validation & Test Splits from a Shuffle Number Locked
- k-Fold Cross-Validation: Assigning Folds with MOD and Stratifying by Class Locked
- A Toy Model per Fold: k-Nearest-Neighbour Predictions for a Held-Out Day Locked
- Fold Scores: Mean Absolute Error per Fold, the CV Estimate and Its Spread Locked
- Grid Search: A Setting × Fold Table, the Best k and the One-Standard-Error Rule Locked
- Under- and Overfitting: Training vs Validation Error by Model Complexity Locked
- Data Leakage: Scaling Before Splitting, Duplicates Across Folds and Rolling-Origin Checks Locked
- The Final Report: Choose by CV, Retrain on All Pool Days and Score the Test Set Once Locked