How Would You Make a Feed Version Notifier?
From the earlier reference collection. Read the current software blog.
A small notifier compares a feed’s published identity with a stored prior identity and reports a change without confusing failures with updates.
Choose the identity to compare
A version notifier needs a stable identifier that represents the published dataset version. Depending on the feed, that might be a version field, a release identifier, or a publication timestamp. Decide which field is authoritative before writing comparison logic. A file’s retrieval time is not necessarily its version: downloading the same unchanged dataset twice should not look like two new releases.
For a transit schedule, for example, the dataset may include feed information that identifies a version or publication date. The GTFS schedule reference describes the files and fields in a feed, including feed_info.txt; consult the specification and the publisher’s own practice to determine which available value is meaningful. Do not assume that every publisher populates an optional field or changes it consistently.
Separate retrieval from comparison
Build the notifier in clear stages: retrieve the feed metadata, validate that the response is usable, extract the chosen identifier, and compare it with the last recorded identifier. If retrieval fails or the identifier is missing, report an error state and retain the previous value. Treating an empty response as a new version can overwrite the only useful baseline and create a false notification.
Suppose the saved value is 2026-10-12 and the newly retrieved value is 2026-10-20. Report that the identifier changed from the former to the latter, then save the new value. If the response again contains 2026-10-20, report no change. Include the source name and check time in the output so that an operator can distinguish a current observation from an old one.
Make state updates safe
Keep the saved state small and durable, such as a JSON file or a database row containing the source key, last identifier, and time it was checked. Write updates atomically where possible: write a temporary file, flush it, and replace the previous state only after the new record is complete. If the process stops halfway through a write, the old baseline should remain readable.
If multiple jobs may check the same source at once, protect the read-compare-write sequence with a lock or a database transaction. Otherwise two processes can both see the same old version and issue duplicate notices. Decide how notification delivery is retried: state persistence and message delivery are separate actions, so record enough context to avoid silently losing a notice after a crash.
Report changes with context
A useful message states what changed and what the person can do next: identify the feed, show the prior and current identifiers, and link to the publisher’s source page if one is configured. Avoid claiming that a version change means the records themselves changed in a particular way; the identifier is evidence of a publisher-declared revision, not an explanation of its contents.
Test unchanged data, a new identifier, a missing identifier, malformed metadata, and a retrieval failure. Also test a rollback to an earlier identifier; publishers can restore an old release, and the notifier should report the observed change rather than presume identifiers only increase. This keeps the tool small while making its behavior predictable.