Capstone: Do the Metrics Agree with People? Rank Correlation, Pearson & a System Comparison
This lesson is part of the course
In this lesson
You check ten scored outputs against two raters: rank agreement with RANK.EQ and 1 − 6·Σd²/(n(n² − 1)), Pearson correlation with SUMPRODUCT, a per-system comparison with AVERAGEIFS and a scatter chart of metric against rating.
Buy the course once for $9.99 to unlock every lesson and lab, the final exam and the certificate – lifetime access, no subscription.
Lessons in this course
- Tokens as Rows: Unigram Counts, Matches, Precision & Recall Locked
- N-Gram Precision with Clipping: Bigrams, Trigrams & Several References Locked
- BLEU by Hand: Geometric Mean, Add-One Smoothing, Brevity Penalty & Corpus Level Locked
- ROUGE-1 & ROUGE-2: Recall, Precision & F1 against Several References Locked
- ROUGE-L: Longest Common Subsequence, Recall, Precision & the F-Measure with β Locked
- Question-Answer Scores: Exact Match, Token F1 & Partial Credit Locked
- Length & Paraphrase Effects: Short Outputs, Same Meaning in Other Words & Character N-Grams Locked
- Capstone: Do the Metrics Agree with People? Rank Correlation, Pearson & a System Comparison Locked