Embeddings: Looking Up Token Vectors & Dot Products
This lesson is part of the course
In this lesson
You look up a 4-number vector for each token of the prompt, compare tokens with dot products and count the parameters of an embedding table.
Buy the course once for $9.99 to unlock every lesson and lab, the final exam and the certificate – lifetime access, no subscription.
Lessons in this course
- Tokens: Splitting a Prompt, Vocabulary Ids & Token Budgets Locked
- Embeddings: Looking Up Token Vectors & Dot Products Locked
- Positional Signals: Sine & Cosine Waves Added to Embeddings Locked
- Queries, Keys & Values: Weight Tables and SUMPRODUCT Locked
- Attention Scores: Scaling by √d, the Causal Mask & Softmax per Row Locked
- Weighted Values: The Attention Output & Which Token Looks Where Locked
- Next-Token Probabilities: Temperature, Greedy Choice, Top-k & Sampling Locked
- Context Length: Re-Sent History, Turn Costs & Why Long Prompts Weigh More Locked