Fine-Tuning Budgets: Epochs, Low-Rank Adapters & Memory per Parameter
This lesson is part of the course
In this lesson
You price a fictional fine-tuning job from examples, tokens and epochs, count the trainable parameters of a low-rank adapter and compare the GPU memory it needs with full fine-tuning.
Buy the course once for $9.99 to unlock every lesson and lab, the final exam and the certificate – lifetime access, no subscription.
Lessons in this course
- Dataset Tokens: Documents, Words, Deduplication & the Validation Split Locked
- Training Compute: The 6 × Parameters × Tokens Rule & Scientific Notation Locked
- GPU-Hours, Utilisation & Cluster Size for a Deadline Locked
- On-Demand, Reserved or Interruptible: Pricing the GPU-Hours Locked
- Fine-Tuning Budgets: Epochs, Low-Rank Adapters & Memory per Parameter Locked
- Experiment Grids: Learning Rates × Batch Sizes × Seeds and Early Stopping Locked
- Scaling Curves: Fitting a Power Law and Predicting Loss at a Bigger Budget Locked
- The Project Budget: Checkpoints, Storage, Evaluations, Contingency & the Chart Locked