Can a Script Find Duplicate Stop Identifiers?
From the earlier reference collection. Read the current software blog.
A focused table scan can detect repeated stop IDs and report every affected row without altering the feed.
Treat identifiers as text
A duplicate-stop check reads stops.txt, remembers each stop_id, and reports values that occur more than once. Read identifiers as strings: leading zeroes and other formatting are part of the value as published. Numeric conversion can collapse distinct identifiers or change the text that should be reported.
Use a CSV parser and verify the stop_id header exists. Define row locations consistently, such as physical file line numbers including the header or data-record numbers beginning at one. State which convention the tool uses so a person can find the reported record.
Collect locations before reporting
For each record, map its identifier to a list of row locations. After the scan, print identifiers associated with more than one location. Reporting all locations makes the result actionable: S12 appears on data rows 3 and 18 tells an editor where to inspect both records.
Do not flag repeated blank identifiers as an ordinary duplicate ID. Report blanks as a separate missing-value problem, since two absent values do not establish that the same stop was intentionally identified twice. If the table contains malformed rows, explain whether the scan stopped or continued and list the affected locations.
Understand what the check does not prove
The presence of a duplicate identifier is a useful consistency warning, but it does not by itself show which row is wrong. A copied row, two records that should have distinct IDs, or a source-specific issue can all produce a repeated value. Keep the script’s output descriptive and leave the correction to someone who can inspect the feed context.
Similarly, distinct IDs do not guarantee that every stop is represented correctly. A separate check could compare references from stop_times.txt with the IDs defined in stops.txt, but that is a different question. Keeping each diagnostic narrow makes its result easier to interpret.
Make the report repeatable
Sort duplicate IDs for stable output, or preserve their first-seen order and document that choice. Include the input filename, total records scanned, number of duplicate identifiers, and any blank or malformed rows. A no-duplicates result is useful when it clearly says how many records were examined.
Try a small fixture with one repeated ID, one unique ID, and two blank values. Confirm that only the repeated nonblank ID appears in the duplicate list, while blanks have their own count. The final utility should read and report; it should never rewrite the source table as a side effect.