The most useful place for intelligence in a data tool is usually a small box labeled “these two records might be the same.” Everything around that box can be ordinary comparison code.
I arrived at that preference while thinking about reconciliation: one local table is treated as canonical, another represents a separate system, and the names, descriptions, and identifiers don't line up perfectly. It’s tempting to hand both sets to a model and ask it to clean everything up. The result will look wonderfully decisive right up until I need to explain why two records were merged.
Reconciliation already contains a lot of work computers can do exactly. Schema checks, normalization, exact joins, field comparisons, and change-plan generation don't become better when a model improvises them. Intelligence earns a narrow role only after those rules have exhausted what they can honestly establish.
Make the comparison mechanical
The run begins with two read-only snapshots selected explicitly for comparison. One has the canonical record identity. The other has the values that need to be reconciled. Before any matching happens, a normal parser validates required columns, types, encoding, duplicate keys, and allowed value shapes.
Invalid structure stops the run. I don't ask a model to repair a malformed table because that would mix data recovery with identity matching. If a required identifier is duplicated, the useful result is a named structural error, not a plausible choice between rows.
Next comes deterministic normalization. Whitespace is trimmed. Case is folded where case has no business meaning. Phone-like or code-like fields receive documented formatting rules. Empty strings become a single explicit null representation. Dates are parsed through a fixed set of accepted forms. The original values remain available beside the normalized ones so the comparison doesn't erase how each source actually represented the record.
Then exact keys do as much work as they can. A stable external identifier wins. Where policy allows, a unique normalized value from a trusted field can establish a match. Every rule has precedence and a reason. If two rules point to different records, the pair becomes a conflict instead of letting whichever function ran last settle the identity.
The output at this stage is a reconciliation set with plain categories: exact match, only in the canonical set, only in the comparison set, conflicting values, and ambiguous identity. No language model is needed to count or explain those conditions. More importantly, rerunning the same snapshots under the same rule version produces the same set.
Let intelligence propose only the hard joins
Ambiguous identity is where rigid rules can become dishonest. A shortened organization name may refer to a longer legal name. Two descriptions may differ in wording while pointing at the same thing. A transposed token may defeat an exact comparison without changing the intended record.
I give only those unresolved candidates to the intelligent step. Candidate generation is deterministic first: compatible record type, nonconflicting strong fields, and a bounded similarity rule reduce the possible pairs. The model doesn't browse the entire dataset or invent a new record. It sees a small candidate group and the specific non-sensitive fields allowed for comparison.
Its output is constrained to candidate identity, supporting fields, conflicting fields, and one of three proposals: likely match, likely distinct, or abstain. “Likely” matters. The proposal doesn't update the canonical table, and a polished explanation doesn't upgrade it into a fact.
This is a better use of intelligence than asking for a complete reconciled dataset. The model works on language and irregularity, where it can save reading. It cannot alter exact matches, rewrite source values, decide that a conflict is harmless, or manufacture the identifier needed to make the join convenient.
There are still hard cases. Two records can sound similar and remain distinct. A model can anchor on one shared phrase and underweight a conflicting attribute. That’s why the candidate view presents the original values, normalized values, matching-rule results, and model proposal together. I want the uncertainty made smaller, not decorated.
Turn judgment into a durable rule
Confirmed ambiguous matches go into an explicit mapping table maintained outside the model response. The table says that source identity A corresponds to canonical identity B and records the basis for that decision at a safe level. Rejected pairs can be recorded too when repeated comparison would otherwise keep proposing the same bad match.
The reconciliation then runs again from the beginning. This is an important bit of stubbornness. I don't patch the model’s accepted pair into the middle of an old result. The deterministic engine loads the mapping table as another versioned rule, rebuilds the joins, and generates a new comparison.
Only that engine can produce the apply plan. The plan lists field-level changes, their source and destination identities, and preconditions on the values it expects to replace. If either side changed after the snapshots were selected, application stops. A human decision about identity shouldn't become permission to overwrite newer data.
Read-back uses the same mechanics. After an approved plan is applied through the appropriate system, another snapshot is compared against the intended result. Exact fields should agree under the declared normalization rules, unresolved conflicts should remain visible, and no unapproved identity pair should have appeared.
This gives me evidence with useful boundaries. I can tell whether a run failed because the source shape was invalid, an exact rule conflicted, an identity remained ambiguous, a human mapping changed, or an apply precondition no longer held. “The AI reconciled it” tells me none of those things.
There’s more code here than in one broad model call. There are also fewer places where a fluent guess can become durable state. The parser owns shape. The rule engine owns exact matches and differences. The model proposes meaning only for bounded ambiguous pairs. The mapping table preserves accepted judgment. The apply step enforces preconditions.
I don't need the whole tool to be smart. I need the uncertain seam to be visible enough that intelligence can help without inheriting the database. When the next mismatch appears, I’d rather add one defensible mapping than ask a model to remember what it probably meant last time.