A higher match rate can mean that a system found more valid relationships. It can also mean that it combined records belonging to different people. The number alone doesn't distinguish the two.
To evaluate identity resolution accuracy, compare the system's decisions with reviewed evidence. Measure incorrect merges and missed matches separately, explain the sample and keep uncertain cases visible.
Key Takeaways
Define the entity and matching policy before measuring accuracy.
Evaluate false merges and missed matches separately; match rate isn't an accuracy measure.
Use both a representative sample and a separate difficult-case set.
Human-reviewed labels need consistent criteria and a way to resolve disagreement.
Repeat the evaluation when source data, matching policies or important business requirements change.
Define what counts as the same customer
Start with the entity your workflow requires. A person, a household and an account are different units, and the same contact detail may be shared among several of them.
Document how your team handles shared addresses, recycled phone numbers, family accounts and changed names. Identify the evidence reviewers may use and the situations where they should mark a case uncertain.
Our explanation of identity resolution techniques describes matching approaches. The evaluation still needs your organization's definition of a correct relationship.
Separate false merges from missed matches
A false merge occurs when records belonging to different entities are grouped together. A missed match occurs when records belonging to the same entity remain separate.
Those errors have different consequences. A false merge may attach one person's transactions or attributes to another. A missed match may leave part of a customer's history outside the profile used for analysis or targeting.
For reviewed record-pair decisions, precision is the share of predicted matches that are correct. Recall is the share of actual matches the system finds. State the evaluation unit: pair-based measurements and customer-cluster reviews can reveal different problems. Specify how matching and nonmatching pairs were selected and labeled. Results from a challenge set or a selected review sample should not be presented as a population-wide score without a sampling method that supports that conclusion.
Build two evaluation sets
Use a representative sample to understand performance on the population your team intends to process. Preserve relevant differences across sources, regions and record completeness.
Create a separate challenge set containing known hard cases. Include shared contact details, conflicting identifiers, changed email addresses and sparse records. Use it to test particular failure modes, rather than reporting its error rate as if it represented the whole customer base.
Keep configuration or training examples separate from the held-out evaluation set. Otherwise, a system can appear to improve without demonstrating better decisions on unseen examples.
Make human review repeatable
Give reviewers written criteria and enough source context to make an informed decision. Ask them to record whether a relationship is supported, unsupported or uncertain, with a short reason.
Review a subset independently with more than one reviewer. Discuss disagreements and refine the criteria before interpreting the results. Do not force ambiguous records into a confident label simply to complete the scorecard.
Preserve the sample, labels, configuration version and review date. That makes later comparisons more meaningful than a screenshot of a headline score.
Inspect the customer clusters behind the metrics
A pairwise score can miss the practical effect of combining many records. Review complete customer groupings, especially those containing unusual combinations of names, addresses or source IDs.
Also investigate cases where a likely customer is split across multiple profiles. Trace the decisions to the available identifiers and matching policy.
Amperity's Stitch review documentation describes benchmark checks for potentially overclustered or underclustered records and human review of examples. These checks help direct investigation; a warning is not itself a confirmed identity error.
Compare vendors on equivalent evidence
Use comparable inputs and the same evaluation rules. Ask vendors to explain which records were excluded, which required manual intervention and which cases remain unresolved.
Separate identity quality from audience reach, identifier availability and destination match rate. A profile can be correct while lacking an identifier a destination accepts. Conversely, a successful destination match doesn't prove the complete profile was assembled correctly.
Our guide to choosing a CDP connects that evidence to the wider buying decision.
Check the downstream consequences
Review how identity differences change customer counts, purchase histories, loyalty tiers and audience membership. Use cases to prioritize the errors your team needs to investigate, without redefining correctness to excuse a failure.
The operational impact matters alongside the measurement. Teams need a way to inspect a relationship, explain it and review the effect of changing the matching policy. Customer data unification is useful when the resulting profiles can be trusted for their intended purpose.
Keep the evaluation alive after purchase
Recheck quality when a new source introduces different identifiers or when a policy changes. Compare the new results with the saved baseline and investigate meaningful shifts.
Build that review into normal operation rather than treating an evaluation score as a permanent property of the platform.
Explore our identity resolution capabilities with representative records and difficult cases from your own customer data.
