Scoring Generated Text by Hand: N-Gram Overlap, Clipped Precision, BLEU & Brevity Penalty, ROUGE Recall, Token F1, Exact Match & Agreement with Human Ratings

What you'll build

What you will be able to do

Each lab ends with a concrete result that you build yourself and the checker confirms, cell by cell.

• Put a generated sentence and its reference into token rows, count every token in both texts with COUNTIF and turn the matches into a first precision and recall.
• Build trigrams with &, find each bigram's highest count in any reference, clip the candidate counts and calculate a clipped bigram precision for an output that repeats itself.
• Smooth four clipped precisions, combine them with a geometric mean written as EXP(AVERAGE(LN(…))), apply the brevity penalty and compare a sentence-level score with a corpus-level score for three sentences.
• Score a weather summary against three references: unigram overlap, ROUGE-1 recall and F1 per reference, ROUGE-2 recall from given bigram overlaps and the best score over all references.
• Turn given LCS lengths into ROUGE-L recall, precision and an F-measure weighted with β, and compare ROUGE-L with ROUGE-2 on sentences whose parts were moved around.
• Score eight short answers from a toy question-answering assistant with exact match and token F1, then compare the exact-match rate with the mean F1 and see where both miss the meaning.
• Compare a full, a very short and a paraphrased summary against one reference, watch precision, recall and the brevity penalty react, and score a spelling variant with character bigrams built from MID and LEN.
• Check ten scored outputs against two raters: rank agreement with RANK.EQ and 1 − 6·Σd²/(n(n² − 1)), Pearson correlation with SUMPRODUCT, a per-system comparison with AVERAGEIFS and a scatter chart of metric against rating.