Imbalanced Data by Hand: The Accuracy Trap, Balanced Metrics, Resampling, Synthetic Minority Samples, Class Weights, Cost-Based Thresholds & Probability Correction

What you'll build

What you will be able to do

Each lab ends with a concrete result that you build yourself and the checker confirms, cell by cell.

• Count true and false alarms of a simple vibration rule with COUNTIFS, compare its accuracy with a model that never raises an alarm and see why the recall of the rare class tells the real story.
• Score four toy models on the same test set with balanced accuracy, the Matthews correlation coefficient and Fβ, and see that every metric can crown a different winner.
• Undersample the OK runs and oversample the fault runs with given draw tables, compute the new class shares and see how each method changes the data the model learns from.
• Measure distances between fault points with SQRT(SUMPRODUCT), find the nearest neighbours with SMALL, create synthetic fault points between neighbours with given gaps and flag the one that lands among the OK runs.
• Compute balanced class weights, apply them to the log-loss of single predictions and to the error count on a threshold grid, and watch the best cut-off move down.
• Price every cell of the confusion matrix, compute the expected cost per run for ten thresholds, pick the cheapest one with MIN and INDEX/MATCH and compare it with the break-even formula.
• Correct the fault probabilities of a model trained on a 50/50 resampled set back to the real prevalence of 1 % and compare raw and corrected probabilities with the observed fault rates.
• Score five ways of handling the imbalance on one fixed test set with recall, balanced accuracy, MCC and expected cost, rank them by cost and chart the result.