Designing Retry Policies for Public Information Fetchers
From the earlier reference collection. Read the current software blog.
A robust fetcher separates retryable failures from permanent ones, uses bounded backoff, and keeps the last successful result visible with its age and provenance.
Classify before retrying
A refresh can fail because the network timed out, a server temporarily refused the connection, the response could not be parsed, or the request was rejected. These cases deserve different handling. A timeout may succeed on another attempt; a malformed document may indicate a changed format that repeated requests will not fix. Treating every failure alike wastes requests and obscures the real fault.
Represent the outcome explicitly: success, retryable transport failure, permanent request failure, or invalid content. For example, retry a connection timeout, but record a rejected request or schema mismatch for investigation. Be cautious about status codes: the source’s documented behavior determines whether a response is temporary or permanent.
Bound the waiting loop
Use exponential backoff with a cap, and add jitter so many clients do not retry at exactly the same moments. A simple schedule might wait about one, two, four, then eight seconds, with a maximum delay and a maximum attempt count. If the fourth attempt still times out, stop this refresh cycle and schedule a later one; do not let one fetcher wait forever.
Keep retry policy separate from the user-facing display. A command-line importer might return a failure status after exhausting attempts, while a dashboard continues to serve the previous successful snapshot. The next scheduled run can try again without turning a temporary outage into an empty page.
Preserve useful cached evidence
Store each successful payload alongside when it was retrieved, which source supplied it, and any version or request context needed to interpret it. Keep the previous snapshot until a replacement has been fetched and validated completely. If parsing fails halfway through, do not overwrite known-good data with a partial result.
Suppose an outage status refresh fails at 10:05, while the last valid information was fetched at 09:50. Display the cached status with “retrieved at 09:50” and indicate that the latest refresh failed. The timestamp describes retrieval, not necessarily when the underlying outage began or ended. That distinction lets a reader judge freshness without mistaking old data for a current confirmation.
Make failure visible and actionable
Record the attempt time, failure class, attempt number, and next planned retry. Avoid logging credentials or full request headers. Alert only when the condition matters operationally, such as repeated failures or a snapshot becoming older than the service’s intended freshness window.
A public outage information page is a useful place to apply this design: compare the displayed retrieval age with what a person needs to know, and check whether the page makes stale information apparent. The resource can help readers reason about the value of preserved context; it does not imply any particular feed format or update guarantee.