Returns & Discounting: G = R + γR + γ²R + …
This lesson is part of the course
In this lesson
You calculate the discounted return of one episode of Robo the warehouse robot forwards (discount factors) and backwards (G(t) = R + γ·G(t+1)) and see why both give the same number.
Buy the course once for $9.99 to unlock every lesson and lab, the final exam and the certificate – lifetime access, no subscription.
Lessons in this course
- Multi-Armed Bandits: Sample Averages & the Incremental Update Locked
- Epsilon-Greedy: Explore or Exploit, Visit by Visit Locked
- Upper Confidence Bound (UCB): Optimism with a Bonus Locked
- Returns & Discounting: G = R + γR + γ²R + … Locked
- Bellman Expectation: Evaluate a Random Policy Sweep by Sweep Locked
- Value Iteration: Take the Best Move, Sweep to Convergence Locked
- Q-Learning by Hand: Eight Updates of a Q-Table Locked
- Policy Extraction: From V* to the Best Action, and Random vs Optimal Locked