Smoothing: Add-One & Add-k So Unseen Words Still Get a Chance
This lesson is part of the course
In this lesson
You apply add-one smoothing to the next-word table, compare it with a gentler add-k, measure how much probability moves to unseen words and decide which k distorts the counts less.
Buy the course once for $9.99 to unlock every lesson and lab, the final exam and the certificate – lifetime access, no subscription.
Lessons in this course
- Corpus & Tokens: Counting Words, Vocabulary Size & Unigram Probabilities Locked
- Bigrams: Pairs of Neighbouring Words & the Next-Word Table Locked
- Probabilities: P(next | word) from Counts & the Probability of a Sentence Locked
- Smoothing: Add-One & Add-k So Unseen Words Still Get a Chance Locked
- Backoff & Interpolation: Mixing Bigram and Unigram, Choosing λ on Held-Out Text Locked
- Sentence Scores & Perplexity: Comparing Two Models on the Same Test Text Locked
- Autocomplete: Top-3 Suggestions, Breaking Ties & Falling Back for Unseen Contexts Locked
- Generation: Cumulative Probabilities, Random Numbers & a Sentence the Corpus Never Had Locked