Can You Build a Feed Field Diff Tool?
From the earlier reference collection. Read the current software blog.
A field diff tool compares table headers from two feed versions and reports additions, removals, and ordering changes without conflating them.
Compare the same tables
A feed field diff tool is a narrow way to see how a dataset’s table structure changed between two versions. For GTFS schedule data, each text file represents a table and its header row names the fields. The GTFS schedule reference defines those files and their field descriptions, so it can help interpret a reported change. The tool itself should first compare the headers exactly as published.
Match tables by filename across the two versions. Report a table that exists only in the newer archive separately from a table present in both whose fields changed. If a table is absent in one version, do not report every field from the other version as a set of removals without clearly labeling the table-level absence.
Separate changes from ordering
For a table present in both versions, compare field names as sets to identify additions and removals, then compare their sequences to detect reordering. A simple set comparison alone misses an ordering change; a sequence comparison alone can make a moved field look like a removal and addition.
Suppose the old header is trip_id,arrival_time,stop_id and the new header is trip_id,stop_id,arrival_time. The field set is unchanged, but the order differs. Report “order changed” and show both sequences. If the new header also adds pickup_type, report that addition independently. This distinction keeps the summary useful to both developers and analysts.
Parse headers safely
Read the first logical CSV record with a CSV parser, not a line split, even if most field names contain no commas. Handle byte-order marks or whitespace only according to an explicit normalization policy. Preserve the raw header for diagnostics, and show when normalization altered a value. Duplicate field names should be called out because they make later column lookup ambiguous.
Do not attempt to infer semantic equivalence from similar names. stop_id and stop_code may look related but can have different meanings. The diff should report structural facts; interpreting whether a change is backward-compatible requires the relevant specification and application context.
Make output reproducible
Print the two version labels or archive names, the table filename, and each detected change. Use a stable order for tables and field names so repeated runs produce readable, comparable output. If neither headers nor table presence changed, say so explicitly; a blank report could mean either no difference or a tool failure.
Test unchanged headers, an addition, a removal, reordered fields, a missing table, a duplicate name, and an empty file. A useful summary might say: stop_times.txt: added pickup_type; field order changed. Keep the full before-and-after header available in a verbose mode. This focused tool can make structural review faster while leaving record-level and semantic validation to separate checks.