Metrics That Respect the Minority: Balanced Accuracy, the Matthews Coefficient, F1 & Fβ
This lesson is part of the course
In this lesson
You score four toy models on the same test set with balanced accuracy, the Matthews correlation coefficient and Fβ, and see that every metric can crown a different winner.
Buy the course once for $9.99 to unlock every lesson and lab, the final exam and the certificate – lifetime access.
Lessons in this course
- The Accuracy Trap: 3 Faults in 100 Runs, Confusion Counts & the «Always OK» ModelFree
- Metrics That Respect the Minority: Balanced Accuracy, the Matthews Coefficient, F1 & Fβ Locked
- Random Undersampling & Oversampling: Given Draw Tables, New Class Shares & What Gets Lost Locked
- Synthetic Minority Samples: Neighbour Distances, the k Nearest & New Points by Interpolation Locked
- Class Weights: n ÷ (k × n_class), a Weighted Log-Loss & a New Best Cut-Off Locked
- Cost-Based Thresholds: A Cost Matrix, Expected Cost per Run & the Cheapest Cut-Off Locked
- Probability Correction After Resampling: Training vs Real Prevalence & a Calibration Table Locked
- Capstone – Comparing Strategies: Balanced Accuracy, MCC, Recall, Expected Cost & a Ranked Summary Locked