Activation Functions: Step, Sigmoid, tanh & ReLU
This lesson is part of the course
In this lesson
You build the four classic activation functions side by side and the sigmoid's slope, which backpropagation needs later.
Buy the course once for $9.99 to unlock every lesson and lab, the final exam and the certificate – lifetime access, no subscription.
Lessons in this course
- A Single Neuron: Weighted Sum + Bias Locked
- Activation Functions: Step, Sigmoid, tanh & ReLU Locked
- Forward Pass of a 2-2-1 Network Locked
- Loss Functions: MSE vs Cross-Entropy Locked
- Backpropagation: One Training Step by Hand Locked
- Learning Rate: Five Steps, Three Speeds Locked
- Softmax & Multi-Class Probabilities Locked
- Overfitting: Train vs Test Error & Early Stopping Locked