How Could a CLI Compare Two Data Snapshots?
From the earlier reference collection. Read the current software blog.
Compare two snapshots by stable identifier to report added, removed, and changed records with enough detail to investigate each difference.
Pick the key that identifies a record
A snapshot comparison needs a stable identifier for each record, such as a stop ID or a device key. Load each input into a mapping from that key to its record. If a key appears more than once in either snapshot, report the ambiguity rather than letting a later row overwrite an earlier one.
Keep identifiers as text and normalize only fields whose format is explicitly defined. Trimming whitespace or changing case may merge records that the source considers different. The comparison should preserve the values as supplied unless the user configures a normalization rule.
Classify the differences
Keys found only in the new snapshot are additions; keys found only in the old one are removals. For keys present in both, compare their fields and mark changed records. A record with the same key and identical relevant fields is unchanged. State whether every field participates, or whether the user has selected a subset.
For example, if both snapshots contain stop A17 and its name is the same but its platform field changes, report that record as changed and identify the field. If B04 occurs only in the newer file, list it as added. This concise example helps check that set membership and field comparison are separate steps.
Make changes readable and stable
Sort results by identifier or retain first-seen order, and document the choice. For each changed record, show field names and old and new values, with a clear representation for missing fields. Avoid printing huge or sensitive records by default; offer a focused output mode when records contain more data than a person can review.
A summary count for added, removed, changed, and unchanged records helps scan large comparisons. Include both input filenames and the selected key so the report can be reproduced. If the snapshots were created at different times, show their timestamps when that metadata is available, but do not infer capture times from filenames.
Treat the diff as evidence, not explanation
A difference report shows that records vary; it does not explain why. A changed sensor reading may reflect a new observation, while a changed public directory entry may reflect a correction or a new source export. Avoid describing a change as an error unless a separate rule establishes that.
Test the comparison with identical files, one addition, one removal, one field edit, and duplicate keys. Those cases exercise the central logic. The useful result is a stable, reviewable account of what changed, with enough context for a human to decide what the differences mean.