Characters and Frequencies: A Toy Word List, Characters per Word and the Starting Vocabulary
This lesson is part of the course
In this lesson
You split a toy corpus of six words into one character per cell with MID, count characters per word with LEN, weight everything by how often each word occurs and build the starting vocabulary of five characters.
Buy the course once for $9.99 to unlock every lesson and lab, the final exam and the certificate – lifetime access, no subscription.
Lessons in this course
- Characters and Frequencies: A Toy Word List, Characters per Word and the Starting Vocabulary Locked
- Pair Counts: Adjacent Pairs per Word, One Token per Cell, Weighted by Word Frequency Locked
- The First Merge: Rewriting Every Word with IF on Neighbouring Cells and Closing the Gaps Locked
- Several Merges: A Merge Table, the Vocabulary After Each Step and Tokens per Word Shrinking Locked
- Tokenizing New Words: Applying the Merges in Order, Unknown Characters and Fallbacks Locked
- Compression: Tokens per Word, Characters per Token and the Effect of More Merges Locked
- Vocabulary Trade-offs: Vocabulary × Embedding Width = Parameters, Sequence Length vs Vocabulary Size Locked
- Token Budgets & Dashboard: Tokens per Toy Document, Cost at a Fictional Price per 1,000 Tokens and a Chart Locked