Support Vector Machines by Hand: Separating Lines, Margins, Hinge Loss, Soft Margins, Kernel Tables & Support Vectors

What you'll build

What you will be able to do

Each lab ends with a concrete result that you build yourself and the checker confirms, cell by cell.

• Score ten tiles against a given separating line, turn each score into a predicted class with IF, measure the accuracy and plot the toy data set as a scatter chart.
• Compare two lines that both separate the same ten tiles, measure their functional and geometric margins, and rescale the better line so that its margin width 2 ÷ ‖w‖ can be read off directly.
• Score a new batch of ten tiles with the rescaled line, compute the hinge loss MAX(0, 1 − y·score) for each, label every tile as outside the margin, inside the margin or misclassified and add up the total loss.
• Score three candidate lines on the same eight tiles with the soft-margin objective ½‖w‖² + C·Σ hinge for three values of C and see how the best line changes from wide to narrow as C grows.
• Flag the tiles that violate the margin, take one sub-gradient step for w and b with a fixed learning rate by hand, compute the objective for five epochs of a reference run and chart how it falls.
• Find the tiles that sit on the edge of the street, rebuild the weights from the given α values as Σ α·y·x and the bias from the support vectors alone, and see why all other tiles could be removed without moving the line.
• Lift a ring-shaped toy set into a new feature z = x1² + x2² where one straight cut separates it, then build a polynomial and a radial kernel table that measure similarity without writing out any new features.
• Classify six test tiles with a radial-kernel SVM from four given support vectors and α values, find the tile it gets wrong, then combine three one-vs-rest machines into a glaze-finish classifier and measure its accuracy.