Skip to content
← All posts
· Delta1 Labs ReliabilityKeyright

Idempotency keys for retry-safe license operations

A license write that times out leaves you blind: did the seat get consumed, or not? Retry and you risk double-charging a credit pool or burning a seat twice; give up and you may have sold a license you never delivered. This is the lost-response problem every networked licensing client hits, and the fix is an idempotency key that turns an at-least-once transport into exactly-once effects. Here is how activation, metering, issuance and deactivation stay correct under retries against your self-hosted Keyright instance.

Most licensing clients are written as if the network either works or visibly fails. It does neither. The failure that actually costs you money is the one you cannot see: a request that reached the issuing service, did its work — consumed a seat, debited a credit pool, minted a license — and then lost its response on the way home. A dropped TLS connection, a reverse-proxy read timeout, a client process that restarted mid-call: all of them leave the caller holding a timeout with no way to know whether the operation happened.

Retry it and you might consume the seat twice, or debit the same render job’s minutes again, or issue a second license for one order. Give up and you might have charged a customer for a license you never delivered. This is the lost-response problem, and it is intrinsic to any operation that changes state over an unreliable transport. The fix is not a better network. It is an idempotency key: a client-chosen identifier that lets the server recognise a retry as the same logical operation and replay the original outcome instead of performing it a second time.

The ambiguity, drawn

Consider a metering draw against your self-hosted Keyright instance. The client posts the draw, the service debits the pool and commits, and then the response never arrives.

Client Issuing service POST /v1/meter · Idempotency-Key: job-8842 debit 12 · store result@key 200 OK (lost) ✕ timeout — retry POST /v1/meter · Idempotency-Key: job-8842 key seen · replay, no 2nd debit 200 OK · same MeterResult At-least-once delivery, exactly-once effect.

The second request is byte-for-byte the first one, including its Idempotency-Key. Because the server recorded the outcome under that key when it first committed the debit, it does not debit again — it returns the stored MeterResult. The transport delivered the request twice; the pool moved once. That is the whole contract: at-least-once delivery, exactly-once effect.

Activation is already idempotent — by its own inputs

Before reaching for keys everywhere, know what you get for free. Keyright activation is idempotent per (license key, machine fingerprint). Re-activating the same key on the same machine returns the existing activation and its lease; it does not consume a second seat. The inputs are the dedup identity, so the SDK can retry a timed-out ActivateAsync on its own without any extra header.

var client = KeyrightClient.Initialize(new KeyrightOptions
{
    Product         = "acme-app",
    PublicKeyBase64 = config["Keyright:PublicKey"],   // verify-only, not a secret
    ServiceUrl      = config["Keyright:ServiceUrl"],   // your self-hosted instance
});

// Safe to retry: the (key, fingerprint) pair is the natural idempotency identity.
LicenseInfo info = await Retry.OnTransient(() => client.ActivateAsync(userKey));

if (!info.IsValid)
    return Gate.Deny(info.Reason);   // fail closed on a genuine refusal, not on a timeout

The important discipline here is distinguishing a refused activation (seats exhausted, key revoked — a real LicenseInfo you must honour and not retry) from a transport failure (timeout, 503 — retry). We will make that split precise below.

The operations that need an explicit key

Not every write is self-keyed. Three common ones are deliberately not idempotent by their inputs, because two identical-looking requests are usually meant to be two separate effects:

  • Metering draws — two MeterAsync("render-minutes", 12) calls normally debit 24. The retry of a single draw must be told it is a retry.
  • Issuance from fulfillment automation — your webhook handler mints a license when an order is paid. A redelivered webhook must not mint a second one.
  • Deactivation — releasing a seat you already released should be a no-op, not an error.

For these, you attach an Idempotency-Key header (the SDK surfaces it as a parameter where relevant). The key is a stable identifier for the business operation, chosen once and reused for every retry of that operation:

// The render job's id IS the natural key — one draw per job, ever.
MeterResult draw = await Retry.OnTransient(
    () => client.MeterAsync(pool: "render-minutes", amount: job.Minutes, idempotencyKey: job.Id));

if (!draw.Allowed)
    return Gate.Deny("Render minutes exhausted. Top up to continue.");

For issuance driven by your own automation, the operation is an HTTP POST to the admin API and you set the header yourself. Use the order id so a redelivered payment event is harmless:

POST /admin/licenses HTTP/1.1
Host: licensing.example.com
Authorization: Bearer {admin_token}
Idempotency-Key: order-2026-5591
Content-Type: application/json

{
  "product": "acme-app",
  "edition": "pro",
  "seats": 5,
  "customer": "cust_8e21",
  "entitlements": ["export", "api"]
}

A second POST carrying Idempotency-Key: order-2026-5591 returns the same license that the first call created — same id, same key material — with no second license minted. Your fulfillment handler becomes safe to call as many times as the upstream commerce system redelivers the event.

Choosing the key correctly

The key has exactly two jobs, and both are easy to get wrong:

  1. Same across retries of one operation. If you generate a fresh Guid.NewGuid() on each attempt, every retry looks like a new operation and the mechanism does nothing. Derive the key from the business entity (job.Id, order.Id), or generate a GUID once, persist it next to the operation, and reuse it on every retry — including retries after a process restart.
  2. Different across genuinely distinct operations. If a customer legitimately runs two render jobs, each needs its own key, or you will collapse two real charges into one. The business id gives you this naturally; a shared constant does not.

A useful rule: the key should be as durable as the operation’s intent. If the intent survives a crash — you still owe that customer a license — the key must survive the crash too. That means it belongs in your own store (the order row, the job record), written before the first attempt.

What the server enforces, and why you should care

A correct idempotency implementation does more than cache a response. Two behaviours matter to you as the caller:

The request fingerprint must match. The server records a hash of the request body alongside the key. If a second request arrives with the same Idempotency-Key but a different body — say, seats: 5 the first time and seats: 50 the second — that is a bug on your side, not a retry, and the service rejects it with 409 Conflict rather than silently honouring either. Treat a 409 here as “you reused a key for a different operation” and fix the call site; never paper over it by rotating the key.

Concurrent duplicates are serialised. If two copies of the same request are in flight at once (a client retried before the first attempt’s response was lost, and both land), the service lets the first complete and answers the second from the stored result — or, if the first is still running, returns a retriable signal so the duplicate backs off rather than executing in parallel. You do not have to coordinate this across your own processes; the key does it.

A retry policy that only retries what is safe

Idempotency keys make retries safe; they do not decide when to retry. Pair them with a policy that retries transport failures and leaves business refusals alone. With Polly:

var transient = new ResiliencePipelineBuilder()
    .AddRetry(new RetryStrategyOptions
    {
        MaxRetryAttempts = 4,
        BackoffType      = DelayBackoffType.Exponential,
        UseJitter        = true,                       // spread retries, avoid a thundering herd
        ShouldHandle     = new PredicateBuilder()
            .Handle<HttpRequestException>()            // connection dropped
            .Handle<TaskCanceledException>()           // timeout
            .HandleResult<HttpResponseMessage>(r =>
                   r.StatusCode == HttpStatusCode.ServiceUnavailable   // 503
                || r.StatusCode == HttpStatusCode.GatewayTimeout       // 504
                || (int)r.StatusCode == 429),                          // throttled: back off
    })
    .Build();

The line to hold is this: retry 429, 503, 504 and raw connection failures; do not retry 400, 401, 403, 404, 409 or a valid-but-refused LicenseInfo. The first group are “the operation may or may not have happened” — exactly where the idempotency key protects you. The second group are definite answers; retrying them just hammers the service with a request that will keep being refused. A 409 from an idempotency endpoint is special, as noted: it means key reused with a different body, which is a code defect, not something to retry.

Finally, keep the gate fail-closed on refusals and fail-operational on transport errors you have exhausted retries for. A metering draw that cannot reach the service after four tries should fall back to the cached lease and its grace window — not deny a paying customer because of a blip — while a draw the service actively refused (pool empty) stays denied. The idempotency key is what makes the retries in between harmless.

Checklist

  • Activation and re-validation: rely on (key, fingerprint) idempotency; retry timeouts directly.
  • Metering, issuance, deactivation: attach an Idempotency-Key derived from the business id (job.Id, order.Id).
  • Choose the key once, persist it with the operation, reuse it on every retry — including after a restart.
  • Never generate a fresh key per attempt; never reuse a settled key for a different operation.
  • Retry only 429/503/504/connection errors; honour 4xx and refused LicenseInfo as final.
  • Treat 409 Conflict from an idempotency endpoint as a same-key/different-body bug at the call site.
  • Keep retries well inside the server’s key-retention window.

None of this shows up on the happy path, which is exactly why it has to be designed in rather than discovered. The transport will deliver some of your license writes more than once; with an idempotency key on every state-changing call, your seats, your credit pools and your issued licenses move exactly once anyway.

Try Nebula.NET

Harden your .NET code in minutes — start with the free edition.