Q-Learning by Hand: Eight Updates of a Q-Table
This lesson is part of the course
In this lesson
You replay eight logged steps of Robo through the Q-learning rule: look up the best next value, build the TD target and TD error, and update the Q-table column by column as the dock's reward travels back along the aisle.
Buy the course once for $9.99 to unlock every lesson and lab, the final exam and the certificate – lifetime access, no subscription.
Lessons in this course
- Multi-Armed Bandits: Sample Averages & the Incremental Update Locked
- Epsilon-Greedy: Explore or Exploit, Visit by Visit Locked
- Upper Confidence Bound (UCB): Optimism with a Bonus Locked
- Returns & Discounting: G = R + γR + γ²R + … Locked
- Bellman Expectation: Evaluate a Random Policy Sweep by Sweep Locked
- Value Iteration: Take the Best Move, Sweep to Convergence Locked
- Q-Learning by Hand: Eight Updates of a Q-Table Locked
- Policy Extraction: From V* to the Best Action, and Random vs Optimal Locked