How Can Duplicate Protocol Messages Be Handled Safely?
From the earlier reference collection. Read the current software blog.
Retries can deliver the same message more than once, so receivers need stable identity and idempotent effects. A careful storage sequence prevents duplicate work from corrupting state.
Duplicates are a normal failure mode
A sender may transmit a request, then lose the connection before learning whether the receiver committed it. Retrying is often the only practical way to recover. The receiver can therefore see the same logical operation twice even though the sender initiated it once.
This can happen in a social activity delivery, a payment-adjacent workflow, or an ordinary job queue. The safe design question is not how to guarantee that transport delivers exactly once, but how to make repeated delivery harmless or detectable.
Give each operation a stable identity
A message should carry an identifier that remains the same when the sender retries the same operation. The receiver can record that identifier and associate it with the result. A new identifier for every retry defeats deduplication because the receiver cannot tell that the messages represent one logical action.
For example, a server may accept an activity with identifier activity-42. On first receipt, it stores the identifier and creates the corresponding local record. If the same activity arrives later, the server finds the recorded identifier and returns an appropriate response without creating another record.
Make the effect idempotent
An operation is idempotent when applying it repeatedly has the same lasting effect as applying it once. Setting a record’s state to “archived” can be idempotent if repeated updates do not create extra consequences. Incrementing a counter is not idempotent by itself: processing the same message twice increments it twice.
A receiver can often implement idempotency with a unique constraint on the message identifier and a transaction that records the identifier together with the state change. If the process crashes midway, the transaction should prevent a partial state where the effect occurred but the deduplication record did not—or the reverse.
Define what a retry means
Not every repeated message is identical, and not every protocol treats identifiers the same way. Specify whether the identifier identifies a message, an operation, or a resource, and define how long deduplication records are retained. If old identifiers expire, a very late retry may be processed again unless another rule prevents it.
Test the first delivery, a repeated delivery, a retry after a simulated timeout, and a restart between attempts. Then inspect both the stored result and external side effects. Stable identifiers and idempotent processing make retries a recovery tool instead of a source of accidental duplication.