Learning Rate: Five Steps, Three Speeds
This lesson is part of the course
In this lesson
You train the same one-weight model with three learning rates for five steps, chart the loss curves and see crawling, converging and exploding training.
Buy the course once for $9.99 to unlock every lesson and lab, the final exam and the certificate – lifetime access, no subscription.
Lessons in this course
- A Single Neuron: Weighted Sum + Bias Locked
- Activation Functions: Step, Sigmoid, tanh & ReLU Locked
- Forward Pass of a 2-2-1 Network Locked
- Loss Functions: MSE vs Cross-Entropy Locked
- Backpropagation: One Training Step by Hand Locked
- Learning Rate: Five Steps, Three Speeds Locked
- Softmax & Multi-Class Probabilities Locked
- Overfitting: Train vs Test Error & Early Stopping Locked