Next-Token Probabilities: Temperature, Greedy Choice, Top-k & Sampling
This lesson is part of the course
In this lesson
You turn eight logits into next-token probabilities at a temperature, pick the greedy token, find the top-k cut-off and sample a token with a fixed random number.
Buy the course once for $9.99 to unlock every lesson and lab, the final exam and the certificate – lifetime access, no subscription.
Lessons in this course
- Tokens: Splitting a Prompt, Vocabulary Ids & Token Budgets Locked
- Embeddings: Looking Up Token Vectors & Dot Products Locked
- Positional Signals: Sine & Cosine Waves Added to Embeddings Locked
- Queries, Keys & Values: Weight Tables and SUMPRODUCT Locked
- Attention Scores: Scaling by √d, the Causal Mask & Softmax per Row Locked
- Weighted Values: The Attention Output & Which Token Looks Where Locked
- Next-Token Probabilities: Temperature, Greedy Choice, Top-k & Sampling Locked
- Context Length: Re-Sent History, Turn Costs & Why Long Prompts Weigh More Locked