Backoff & Interpolation: Mixing Bigram and Unigram, Choosing λ on Held-Out Text
This lesson is part of the course
In this lesson
You mix bigram and unigram probabilities with four values of λ for a sentence the model has never seen, score each mix with the sum of log-probabilities, pick the best λ and compare it with a simple backoff score.
Buy the course once for $9.99 to unlock every lesson and lab, the final exam and the certificate – lifetime access, no subscription.
Lessons in this course
- Corpus & Tokens: Counting Words, Vocabulary Size & Unigram Probabilities Locked
- Bigrams: Pairs of Neighbouring Words & the Next-Word Table Locked
- Probabilities: P(next | word) from Counts & the Probability of a Sentence Locked
- Smoothing: Add-One & Add-k So Unseen Words Still Get a Chance Locked
- Backoff & Interpolation: Mixing Bigram and Unigram, Choosing λ on Held-Out Text Locked
- Sentence Scores & Perplexity: Comparing Two Models on the Same Test Text Locked
- Autocomplete: Top-3 Suggestions, Breaking Ties & Falling Back for Unseen Contexts Locked
- Generation: Cumulative Probabilities, Random Numbers & a Sentence the Corpus Never Had Locked