How Can You Check Whether a Feed Archive Is Corrupt?
From the earlier reference collection. Read the current software blog.
Check an archive’s integrity and identify a damaged member before letting ingestion treat its contents as valid input.
Separate archive integrity from data validity
A compressed archive has at least two questions to answer. First, can its stored members be read and decompressed successfully? Second, do those members contain records that make sense to the application? A successful integrity check answers only the first question. A clean archive can still contain a missing table, malformed CSV, or inconsistent identifiers.
Keep these stages separate in the command’s messages. “Archive integrity check passed” is a useful result, but it should not be presented as proof that a feed is ready to ingest.
Test before extracting or importing
For a ZIP file, a command such as unzip -t feed.zip checks members without proceeding with normal application ingestion. Review its output for the specific member it could not test. A program can offer the same sequence in code: enumerate members, read each member fully through the archive library, and stop if decompression or checksum verification fails. Avoid printing only a generic “bad archive” message when the library identifies a member name.
For other archive formats, choose their matching integrity-check command or library method. Do not infer that a ZIP-specific option applies to tar, gzip, or another container. Some formats wrap one file; others contain many independently named members, so the useful diagnostic differs.
Report the failure with context
A useful error includes the archive path, the failing member when available, and the stage that failed. For example: feed.zip: checksum failure while reading routes.csv. Keep that distinct from a later message such as routes.csv: required column route_id is absent. This lets an operator decide whether to obtain a fresh download or investigate a producer's data.
If the archive is supplied by a download step, preserve the original file for diagnosis and avoid replacing a known-good feed with a failed candidate. A checksum failure can result from a truncated transfer or damaged storage, but the integrity check alone does not establish which occurred.
Make validation a gate
Structure the workflow as check, then ingest: return a nonzero status on any member read failure and do not launch the importer. If validation is a separate command, make its success explicit in logs or orchestration. Test the failure path with a deliberately damaged copy and verify that the report names the failing member when the format supports it.
This two-stage approach provides a clear boundary. Archive checks establish that the container can be read; format validation establishes whether its contents meet the feed's requirements. Neither result should be silently substituted for the other.