AI Model Training & Fine-Tuning Budgets: Tokens, Compute, GPU-Hours, Experiment Grids & Scaling Curves

What you'll build

What you will be able to do

Each lab ends with a concrete result that you build yourself and the checker confirms, cell by cell.

• Turn a fictional training corpus of six sources into raw tokens, remove the duplicates source by source and split off a validation set, so you know how many training tokens the run will really see.
• Estimate the training compute of four fictional model sizes with the 6 × parameters × tokens rule of thumb, write the huge results in scientific notation and compare the plan with a tokens-per-parameter heuristic.
• Convert training FLOPs into GPU-hours with peak speed and utilisation, turn them into wall-clock days for the GPUs you have and size the cluster in whole nodes so the largest run finishes before its deadline.
• Price the same training run on three fictional GPU plans, including a minimum commitment and the extra hours lost to interruptions, and find the cheapest plan and its saving.
• Price a fictional fine-tuning job from examples, tokens and epochs, count the trainable parameters of a low-rank adapter and compare the GPU memory it needs with full fine-tuning.
• Count the runs of a fictional hyper-parameter grid, estimate its GPU-hours with early stopping, price the sweep and work out how many seeds per configuration the next budget allows.
• Take six fictional pilot runs, fit a power law between compute and validation loss with LN and SLOPE, predict the loss of the big run with FORECAST.LINEAR and EXP, and chart the curve.
• Add the costs that are easy to forget (checkpoint storage and evaluation runs) to the fictional Kestrel-XL budget, put a contingency on top and show in a chart where the money goes.