Idempotency keys: retrying an API call without charging the customer twice

A client sends a POST to create a charge. Twenty seconds later the connection times out. The client knows one thing only: it did not get a response. It does not know whether the server never saw the request, saw it and crashed before writing, wrote it and crashed before responding, or wrote it and responded into a dropped connection.

Three of those four cases mean the charge exists. Retrying blindly creates a second one. Not retrying leaves a customer who paid and got nothing. Idempotency keys let the client retry safely without having to know which case it is in.

What idempotent means here

An operation is idempotent if performing it several times has the same effect as performing it once. HTTP already assigns this property to some methods:

The definition is about the state on the server, not the response code. A second DELETE returning 404 is still idempotent; the resource is absent either way.

PATCH is the subtle one. {"status": "shipped"} is idempotent. {"op": "increment", "field": "retries"} is not. If your PATCH accepts relative operations, it needs the same protection as POST. GET vs POST vs PUT vs PATCH covers the distinction in more depth.

The mechanism

The client generates a unique key per logical operation and sends it with the request:

POST /v1/charges HTTP/1.1
Idempotency-Key: 5f8a1c2e-3d4b-4a7e-9f01-2b6c8d3e5a91
Content-Type: application/json

{"amount": 2500, "currency": "usd", "customer": "cus_9182"}

The server, on receiving a request with a key:

1. Attempts to claim the key, atomically.
2. If the claim succeeds, this is the first attempt. Perform the operation, store the response against the key, return it.
3. If the key already exists with a stored response, return that stored response without performing anything.
4. If the key exists but is still in flight, return 409 so the client backs off and retries.

The client reuses the same key for every retry of that one operation, and generates a fresh key for the next distinct operation. A UUID v4 is the normal choice: the client can generate one with no coordination and collisions are not a practical concern.

Scope the key, and fingerprint the request

Two details separate a working implementation from a dangerous one.

Scope keys per account. The uniqueness constraint should be on (account ID, endpoint, key), not on the key alone. A key is client generated, so a global namespace means one tenant can guess or collide with another tenant's key and receive their stored response. That is a data leak dressed as a caching feature.

Store a fingerprint of the request body. Hash the body when you first store the key. On a replay, compare the hash. If the key matches but the body differs, the client has reused a key for a different operation, which is a bug on their side:

HTTP/1.1 422 Unprocessable Entity

{"error": "idempotency_key_reuse",
 "message": "This key was used with a different request body."}

Returning the original response instead would mean a client asking to charge 50 dollars gets told the 25 dollar charge succeeded. Fail loudly.

The concurrency case

Most implementations handle the sequential retry and get the concurrent one wrong. Two retries arriving at the same time, on different servers, both check "does this key exist", both see no, and both proceed.

A check followed by an insert is a race. The claim has to be atomic:

CREATE TABLE idempotency_keys (
  account_id   bigint      NOT NULL,
  key          text        NOT NULL,
  request_hash text        NOT NULL,
  state        text        NOT NULL,   -- 'in_progress' | 'complete'
  status_code  int,
  response     jsonb,
  created_at   timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (account_id, key)
);

Insert the row as in_progress first and let the primary key reject the duplicate. The loser of that race does not proceed; it reads the existing row and either returns the stored response or, if the row is still in_progress, returns 409 with a Retry-After header.

Write the key row and the effect in the same database transaction wherever possible. If the key row commits and the charge does not, retries will return a success response for a charge that never happened. If the charge commits and the key row does not, retries will charge again. The transaction is the whole point.

When the effect lives in an external system that cannot join your transaction, such as a payment processor, the pattern shifts: record the intent, call the processor with your own idempotency key passed through, and reconcile. Most processors accept one for exactly this reason.

Expiry

Keys are not kept forever. Twenty four hours is the common window, and it is what Stripe uses. Clients should finish retrying long before that, and a key retried three days later almost certainly belongs to a different intent.

Document the window explicitly, because it changes client behaviour: after expiry, the same key is a new operation, not a replay. A daily cleanup job deleting rows older than the window keeps the table small.

The client side

A key alone does not make retries safe if the retry policy is wrong.

Generate the key before the first attempt, outside the retry loop. A key generated inside the loop is a new key each time, which defeats the whole mechanism. This is the single most common client bug.

Retry only what is retryable. Connection errors, timeouts, 429 and 5xx. Not 400, 401, 403, 404 or 422; those will fail identically forever.

Back off exponentially with jitter. Doubling the delay without jitter synchronises every client that failed at the same moment into a thundering herd on the recovering server.

const base = 200
const delay = Math.random() * Math.min(30000, base * 2 ** attempt)

Full jitter, as above, spreads the retries across the whole window. It is materially better than a fixed delay plus a small random offset.

Respect Retry-After. If the server tells you when to come back, that instruction beats your own calculation. HTTP 429 rate limiting and backoff covers the header format.

Cap the attempts and surface the failure. Infinite retry turns a brief outage into a permanent load problem.

What to return on a replay

Return the original response: same status code, same body. A replayed create should return 201 and the same resource, not 200 or 409. The client's code path for success ran once and should run once again identically.

Adding a response header to mark the replay is useful for debugging and harmless to clients:

Idempotent-Replay: true

An IETF draft standardising the Idempotency-Key header exists, and the field name is already de facto standard across Stripe, PayPal, Adyen and others. Use that spelling rather than inventing X-Request-Id semantics on top of a header that means something else.

Testing it

The failure mode you care about is invisible in a normal happy path test. Exercise it deliberately: send the same request with the same key twice and assert one record was created, send it with the same key and a changed body and assert a 422, fire two copies concurrently and assert one 201 and one 409 or duplicate response, and let a key expire and assert the next use creates a new record.

The API tester is a quick way to replay a request with an identical key by hand and inspect exactly what comes back, including whether the status code on the second attempt matches the first. Requests go from your browser straight to the target, so nothing about the call passes through a third party.

Related reading: webhooks vs polling covers the same delivery problem from the receiving side, verify webhook signatures with HMAC covers authenticating those retries, and REST vs GraphQL covers where the method semantics above stop applying.