The Final Report: Choose by CV, Retrain on All Pool Days and Score the Test Set Once
This lesson is part of the course
In this lesson
You average the fold scores of four candidate models, pick the winner by its CV MAE, score the retrained model once on the four locked test days and chart the fold scores for the report.
Buy the course once for $9.99 to unlock every lesson and lab, the final exam and the certificate – lifetime access, no subscription.
Lessons in this course
- Three Piles of Data: Train, Validation & Test Splits from a Shuffle Number Locked
- k-Fold Cross-Validation: Assigning Folds with MOD and Stratifying by Class Locked
- A Toy Model per Fold: k-Nearest-Neighbour Predictions for a Held-Out Day Locked
- Fold Scores: Mean Absolute Error per Fold, the CV Estimate and Its Spread Locked
- Grid Search: A Setting × Fold Table, the Best k and the One-Standard-Error Rule Locked
- Under- and Overfitting: Training vs Validation Error by Model Complexity Locked
- Data Leakage: Scaling Before Splitting, Duplicates Across Folds and Rolling-Origin Checks Locked
- The Final Report: Choose by CV, Retrain on All Pool Days and Score the Test Set Once Locked