Entity resolution: why your lead lists are full of duplicates

Entity resolution decides which raw records describe the same real-world thing — one project, one company, one property — and merges them into a single canonical record with aliases, identifiers, and history. Done wrong, it either inflates your data with duplicates or corrupts it with false merges.

Here is the problem in one fictional but painfully typical example. Three sources each publish something about the same development:

• A county permit portal: Maitland Commons Multifamily Phase II, permit BLD-2026-040112, value $14.2M.
• A city planning agenda: Maitland Commons Apartments — Phase 2, case DEV-24-0966, same parcel.
• A business journal: “Corelake Residential breaks ground on Maitland Commons.”

Store them as-is and you have three “projects,” every count you report is inflated, and a salesperson calls the same developer three times. Merge them carelessly and you risk gluing the wrong records together — which is worse, because a false merge looks like clean data while being wrong.

Why fuzzy matching alone corrupts data

The tempting shortcut is string similarity: if two names are close enough, merge them. But commercial names are adversarially similar.Riverside Medical Pavilion and Riverside Medical Plazaare two different buildings. “Phase I” and “Phase II” differ by one character and millions of dollars. A subsidiary and its parent share most of a name and none of a bank account. Every false merge silently poisons downstream counts, scores, and outreach — and unlike a duplicate, it doesn't look wrong.

Match in order of evidence strength

A reliable resolution pass works down a ladder:

• 1. Shared external identifiers. Permit numbers, parcel ids, case numbers, registration numbers. If two records share a real identifier, they are the same thing — this is the only class of evidence strong enough to merge on by itself.
• 2. Exact normalized names. Lowercase, strip punctuation and whitespace, then require equality — within the same entity type and scope (the same feed, the same region). Normalization removes formatting noise without inventing similarity.
• 3. Known aliases. Names previously captured on the canonical record match like the canonical name does. Aliases are earned (observed in a source), never guessed.
• 4. Everything else → a separate record. Ambiguity becomes a new entity that can be merged later by a human or by a stronger signal — recoverable, unlike a bad merge.

In the Maitland example: the permit and the planning case link through the shared parcel id; the news story matches the captured alias “Maitland Commons”; and all three end up as one entity, three source references, two aliases.

What a good merge actually records

• Aliases — the variant name is appended, so the next occurrence matches at step 3.
• Identifiers — new external ids are added to the record, strengthening every future match.
• Source references— the new source is appended as evidence. Corroboration across independent sources is exactly what should raise a record's confidence (see the verification post).
• A version, if fields changed — old value, new value, when, and from which monitoring run. History is the difference between a database and a snapshot.

Entities are only half the graph

Resolution pays off twice when records connect: the project links to its property (by parcel), its developer, and — once identified — its contractor as typed relationships. A single contractor_identified event then arrives with its full commercial context attached: which project, what value, whose site, which developer to call.

How Ballast resolves entities

Ballast implements this ladder as its Entity Resolution agent: identifier match first, then exact normalized names and earned aliases within the feed, with uncertain matches deliberately kept separate. Merges append evidence and aliases, changes write versions, and relationships build the project graph — all inspectable on every record (“view evidence”, version history, corroborating sources). It is less clever than aggressive fuzzy matching, on purpose: in commercial data, boring and traceable beats clever and silently wrong.