De-duplication: Cluster IDs, the Golden Record and the Score Distribution
This lesson is part of the course
In this lesson
You turn matched records into clusters with MINIFS, pick the most recently updated record of each cluster as the survivor for the golden record, count the real customers and chart the score distribution.
Buy the course once for $9.99 to unlock every lesson and lab, the final exam and the certificate – lifetime access, no subscription.
Lessons in this course
- Normalising Company Names: TRIM, LOWER, SUBSTITUTE and Legal Suffixes Locked
- Blocking Keys: First Letter + Postcode to Limit Comparisons Locked
- Exact vs Normalised Matching: How Many True Matches Do You Find? Locked
- Character Bigram Similarity: Jaccard and Dice with MID and SEQUENCE Locked
- Token Overlap for Company Names: Word Jaccard and the Overlap Coefficient Locked
- Combined Match Score: Weights and Two Thresholds (Match, Review, Non-Match) Locked
- Precision and Recall of the Matcher against a Labelled Sample Locked
- De-duplication: Cluster IDs, the Golden Record and the Score Distribution Locked