Skip to content

Idempotency

You send a POST to create a sale. The connection drops before the response comes back. And now: was the sale created or not?

With no help, both outcomes are bad. If you retry, you may create two sales. If you do not retry, you may have created none.

Send an Idempotency-Key header with a value of your own, unique for that operation:

Terminal window
curl -X POST "https://api.fatureihoje.com/public/v1/clients" \
-H "Authorization: Bearer $FH_API_KEY" \
-H "Content-Type: application/json" \
-H "Idempotency-Key: 6b7f0e8a-4c21-4d9e-9f3a-71c5d8b2e044" \
-d '{ "name": "Ana Souza" }'

The header is the same on every creation route.

The API stores the response along with that key. If the same key arrives again with the same request, it returns the stored response, with the same status and the same body, without running anything again. The replayed response comes with the Idempotent-Replayed: true header, and that is how you know the effect happened the first time.

So the retry rule becomes simple: repeat the same request, with the same key, as many times as you need.

  • Whenever a POST creates something, and above all when you retry the call after a network error, a timeout, a 500 or a 503.
  • In a queue and in a job that runs again after failing. Generate the key together with the task and store it with the task, not at call time: generating a new key on every attempt cancels the mechanism.

The value can be any string. One version 4 UUID per operation is the simplest choice, and it is what the examples below generate.

The key alone is not enough. The API also stores a fingerprint of the request, made of four things:

  1. the method;
  2. the path, lowercase and without a trailing slash;
  3. the query, the part after the ?;
  4. the body, compared by content and not by text, so the order of the JSON fields does not matter.

If the key arrives again with the same fingerprint, it is a repeat: the stored response comes back. If the fingerprint is different, it is not a repeat, it is a mistake, and the answer is 422 idempotency_key_reused.

CodeStatusWhenWhat to do
idempotency_key_reused422Same key, different requestGenerate a new key for this operation
idempotency_in_progress409The first request with that key is still runningWait a few seconds and repeat the same request, with the same key

On the 409, do not generate a new key. The first call may still finish well, and a new key would run the operation a second time, which is exactly what this mechanism exists to avoid.

If the processing fails, the key is released right away, and the next attempt with it runs for real. The reservation also has a deadline: if the process dies midway, it expires in a few minutes and the key accepts attempts again, instead of staying stuck for a day.

LimitValue
How long the response stays stored24 hours
Maximum size of the stored record256 KB
Maximum size of the request body1 MB

The 24 hours count from the first call. After that, the same key with the same request runs again, because there is nothing stored left to return.

The 256 KB ceiling is about the stored response, not about the request. A response larger than that is not stored in full: the API remembers that the operation already ran, and the repeat gets a 409 instead of a replay. The effect happened exactly once, which is what matters, but you need to read the resource to know how it ended up.

The stored response is valid for the access at the time it was produced. If the key, or the member it acts as, lost a permission since then, the repeat does not return that body: without the route permission the answer is 403, and with the route still allowed but the access changed (for example, the key lost finance:read and the stored sale carried the finance entries) the answer is 409 conflict. In both cases nothing runs again; read the resource to see how it ended up.

The 1 MB ceiling is about the request body and applies to every call, with or without Idempotency-Key. Above it the answer is 413 request_too_large.

Two different API keys have separate idempotency spaces. The same Idempotency-Key value on two keys is two operations, not a repeat, and neither sees the stored response of the other.

That is good on one side: two systems of the same company can generate the value their own way without agreeing on anything.

The same goes for revoking a key and creating another: it is a new key, it is a new space.

Terminal window
KEY=$(uuidgen | tr 'A-Z' 'a-z')
echo "$KEY"

uuidgen ships with macOS and with most Linux distributions. Where it does not exist, cat /proc/sys/kernel/random/uuid works.

  • Errors: the whole catalog, with both idempotency codes.
  • Authentication: how key rotation works, which is the trap above.