Idempotency keys and safe retry semantics

Overview

Idempotency keys solve one precise failure: a client sends a non-idempotent write request, the operation may have committed, but the client does not receive a reliable answer. The cause can be a TCP reset, a timeout, a proxy failure after forwarding the request, a process crash after commit but before response, or a mobile network handoff. Retrying the same HTTP POST without coordination can create a second order, payment, message, account, or workflow instance. Not retrying can leave the client unable to converge on the state of the system. The ambiguity is not whether the request was syntactically valid; it is whether a side effect already happened.

An idempotency key converts an ambiguous retry into a lookup of a previously claimed operation. The client sends an application-generated token in the HTTP Idempotency-Key field. The server records that token, scoped to the target operation, before or atomically with the write. A later request with the same key is not treated as a fresh command. If the original completed, the server returns the stored outcome or a representation derived from the already-created resource. If it is still in progress, the server reports a conflict or waits according to local policy. If the same key is attached to a materially different request, the server rejects it.

How it works

At the systems level, the core object is not the header but a durable deduplication record. A typical record contains a scope such as tenant or account, the Idempotency-Key value, the HTTP method and route or operation name, a request fingerprint, a processing state, a response status, optional response metadata, a pointer to the created resource, and an expiration time. The database enforces uniqueness on the scope and key, commonly through a unique index. The first request inserts or claims the record; duplicate requests read the record and follow the state machine rather than executing the write again.

The important ordering rule is that the idempotency record and the write must not be separable in a way that loses the association. A minimal design inserts a PENDING record, executes the write, and updates the record to COMPLETED in the same database transaction when the write and the idempotency table share a store. When side effects leave the database, such as event publication, the usual pattern is an outbox table committed with the write, followed by asynchronous delivery. That prevents the idempotency table from claiming success while an external side effect is neither replayable nor traceable.

Fingerprinting is what prevents accidental key reuse from hiding data loss. The fingerprint should cover the semantic operation: method, normalized route, relevant content type, and a canonical representation or cryptographic digest of the body. It should not depend on volatile headers such as Date or tracing identifiers. Storing only a digest, for example a SHA-256 hash defined by FIPS 180-4, reduces the exposure of sensitive request bodies, but it makes canonicalization correctness more important. If the digest differs for the same scoped key, the retry is not a retry; it is a conflicting command.

What the standard says

The Idempotency-Key field is not yet an Internet Standard. It is specified in an IETF HTTPAPI Working Group Internet-Draft, draft-ietf-httpapi-idempotency-key-header (version 07, October 2025), which has since expired without being published as an RFC. The draft registers the Idempotency-Key field in the IANA HTTP Field Name Registry and defines the field value as an HTTP Structured Field Values string, using the syntax now given by RFC 9651, which obsoletes RFC 8941. It states that a client MUST NOT include more than one Idempotency-Key header field in the same request, and that the field is meant for requests that need idempotent processing, especially non-idempotent methods such as POST and PATCH. Because the draft is not a ratified standard, the durable contract in practice is each server's documented behavior.

The draft assigns obligations to both sides. The client is responsible for generating a unique key and MUST NOT reuse a key with a different request payload. The server defines the scope in which uniqueness applies, may derive and compare an idempotency fingerprint, and should publish an expiration policy because retention is application-specific. For error handling it describes 400 Bad Request when a required key is missing, 422 Unprocessable Content when the key is reused with a different payload, and 409 Conflict when a retry arrives while the original request is still being processed. When problem details are returned, the current specification is RFC 9457, which obsoletes RFC 7807.

RFC 9110, “HTTP Semantics,” remains the baseline for method behavior. It defines safe methods and idempotent methods; GET, HEAD, OPTIONS, and TRACE are safe, and PUT, DELETE, and the safe methods are idempotent. POST is not defined as idempotent by HTTP semantics, so a generic intermediary cannot assume that retrying POST is safe. Idempotency-Key does not change the method definition globally. It gives the origin server and client a resource-specific contract for recognizing retries of the same unsafe request.

For browser clients, the relevant specification is the WHATWG Fetch Standard. Idempotency-Key is not a CORS-safelisted request-header name; the safelisted names are limited to headers such as Accept, Accept-Language, Content-Language, certain Content-Type values, and Range under documented constraints. A cross-origin browser request that sets Idempotency-Key therefore requires a successful CORS preflight, and the response to that preflight must allow the header through Access-Control-Allow-Headers. This is a deployment property, not an idempotency algorithm.

Trade-offs

The main measurable cost is state retention. Every protected write creates at least one additional durable row or key-value entry until its expiration time. The storage footprint is approximately the number of protected requests during the retention window multiplied by the stored key, scope, fingerprint, state, timestamps, and any cached response material. The Idempotency-Key Internet-Draft deliberately does not define a universal retention threshold; it leaves the expiration policy to the resource server. That is correct because retry windows differ between interactive browser requests, queue consumers, batch jobs, and offline clients.

Latency can increase because the first request must claim the idempotency record before executing the write, and duplicates must read it before deciding what to do. A strongly consistent unique constraint prevents duplicate execution but introduces contention for the same scoped key. A weakly consistent cache is faster but cannot by itself prove that two concurrent first attempts will not both execute. The observable trade-off is between write-path coordination and duplicate-side-effect risk. If correctness matters, the uniqueness check belongs in the same consistency boundary as the write or in a system that provides equivalent compare-and-set semantics.

Key quality is another measurable dimension. Random UUIDs are common because RFC 9562, which supersedes RFC 4122, defines UUID version 4 with 122 random bits after version and variant bits are fixed. The standard does not require UUIDs for Idempotency-Key, but high-entropy keys reduce accidental collisions. Low-entropy keys, timestamps, counters shared across clients, or user-visible identifiers can collide within the server’s scope and convert unrelated writes into conflicts. The measurable signal is the rate of 422 mismatches and duplicate-key conflicts per operation family.

Failure modes

The most serious failure mode is a non-atomic claim. If the business row is committed and the idempotency row is not, a retry can execute the write again because the deduplication record is absent. If the idempotency row is committed as completed before the business write is durable, a retry can receive a false success. Diagnosis starts with transaction logs or audit tables: compare creation times of the idempotency record, the domain row, and any outbox event under the same correlation key. A correct trace shows one claim and one associated committed write.

A second common failure mode is inconsistent fingerprinting. JSON bodies can vary in object member order, insignificant whitespace, numeric representation, defaulted fields, or character normalization. If one server hashes raw bytes and another hashes a parsed canonical form, retries routed to different instances can produce false 422 responses. Conversely, an overly narrow fingerprint can allow different commands to share a key. Diagnosis requires logging the fingerprint algorithm version and a non-sensitive digest, not the full payload, so mismatches can be compared without expanding data exposure.

Operational failures often appear as spikes in 409 Conflict, 422 Unprocessable Content, or unexpected new-resource counts after client timeouts. A 409 cluster can indicate long-running operations, lock leakage, worker crashes that leave PENDING records, or retry intervals shorter than the server’s processing time. A 422 cluster usually indicates client key reuse, request mutation during retry, or a server-side canonicalization change. Duplicate writes despite repeated keys indicate missing uniqueness constraints, too narrow a uniqueness scope, reads from stale replicas, or failover to a region that does not share the idempotency store.

Expiration is a subtle failure mode because the same implementation can be correct inside the retention window and unsafe outside it. If a client retries after the server has purged the key, the request can be treated as new unless the domain model has its own natural uniqueness constraint. Diagnosis depends on comparing retry age with the published expiration policy. The Idempotency-Key Internet-Draft leaves the retention interval to the server, so clients cannot infer a universal safe retry duration from the specification.

A minimal implementation

A correct minimal implementation has four pieces. First, require Idempotency-Key on selected unsafe operations, usually POST endpoints that create resources or initiate irreversible workflows. Second, validate that the field is a single RFC 8941 string value and reject missing required keys with 400. Third, compute a stable fingerprint for the intended operation. Fourth, use a durable table with a unique constraint on scope plus key. The table needs at least key, scope, fingerprint, state, status code or resource reference, creation time, and expiration time.

The request algorithm is small. Start a transaction. Try to insert a PENDING record for the scoped key and fingerprint. If the insert succeeds, perform the write, store the resulting resource identifier or response summary, mark the record COMPLETED, and commit. If the insert fails because the key already exists, read the existing record. If the fingerprint differs, return 422. If the state is PENDING, return 409 or wait under a bounded policy. If the state is COMPLETED, return the stored result or reconstruct the response from the stored resource reference. Background cleanup may delete expired records only according to the published retention rule.

The implementation should be deliberately narrow. Idempotency-Key is not a distributed transaction protocol, not a substitute for authorization, and not a way to make arbitrary side effects reversible. It prevents duplicate execution of the same recognized command inside a defined scope and time window. For systems that emit events or call downstream web services, the same idea must be extended at each boundary: durable operation identifiers, unique constraints or compare-and-set, and observable states. Without those boundaries, the HTTP layer can be idempotent while downstream effects still duplicate.