Side effects and idempotency

Goal

Identify which operations in your system are dangerous to repeat and make them idempotent with the right technique for each case.

The problem

A user clicks “Subscribe to Pro”. The request reaches the server and the charge goes through on Stripe, but the response gets lost on the way back (airport Wi-Fi). The app shows “network error, please try again”. The user tries again and is charged twice.

The billing code was correct. It just didn’t account for a fact of networks: any operation can run more than once.

Side effects and idempotency

A side effect is any change to the world outside the function: writing to the database, charging a card, sending an email, publishing to a queue.

An operation is idempotent when running it once or ten times leaves the system in the same state.

Idempotent	Not idempotent
UPDATE users SET plan = 'pro' WHERE id = 7	UPDATE users SET credits = credits + 100 WHERE id = 7
DELETE FROM links WHERE id = 42	INSERT INTO payments (...) VALUES (...)
PUT /links/42 with the full body	POST /payments
Setting an absolute value	Incrementing, appending, sending, charging

By the HTTP spec, GET, PUT and DELETE should be idempotent and POST doesn’t have to be. In practice, POST is exactly where the money lives, so it’s the one that needs protection most.

Note

Idempotent doesn’t mean “no effect”. The first run changes the state; the runs after it change nothing more.

Four techniques

1. Turn it into an absolute operation

The cheapest option. Instead of “add 100 credits”, say “order 981 granted 100 credits”. Prefer setting state over modifying it.

2. Natural key + ON CONFLICT

If the operation already has a unique identity (an event ID, the user + month pair of an invoice), add a constraint and let the database reject the repeat:

CREATE TABLE monthly_invoices (
  user_id  bigint NOT NULL,
  month    date   NOT NULL,
  amount   integer NOT NULL,
  PRIMARY KEY (user_id, month)
);

-- Running it twice produces a single invoice
INSERT INTO monthly_invoices (user_id, month, amount)
VALUES ($1, date_trunc('month', now()), $2)
ON CONFLICT (user_id, month) DO NOTHING;

3. Conditional state transition

For state machines (an order going pending → paid), the update only happens if the current state is the expected one. The second run finds no matching row:

const result = await db.query(
  `UPDATE orders SET status = 'paid', paid_at = now()
   WHERE id = $1 AND status = 'pending'
   RETURNING id`,
  [orderId],
);

if (result.rowCount === 0) {
  return; // already processed (or in another state): don't repeat the side effects
}
await grantProAccess(orderId); // only runs on the real transition

4. Idempotency key

When the operation has no natural identity (any POST /payments), the client generates a unique key per intent and sends it with every attempt. The server stores the result under that key, and if the same key comes back, it returns the saved response instead of running the operation again. This is the pattern the Stripe API uses.

sequenceDiagram
  participant C as Client
  participant S as Server
  participant DB as Postgres
  C->>S: POST, key 7f3a
  S->>DB: reserve 7f3a ✔
  S->>S: charge on Stripe
  S--xC: response lost
  C->>S: retry, key 7f3a
  S->>DB: reserve 7f3a → conflict
  DB-->>S: saved response
  S-->>C: 201 (no second charge)

In practice: a complete Idempotency-Key

CREATE TABLE idempotency_keys (
  user_id       bigint      NOT NULL,
  key           text        NOT NULL,
  request_hash  text        NOT NULL,
  status        text        NOT NULL CHECK (status IN ('processing', 'done')),
  response_code integer,
  response_body jsonb,
  created_at    timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (user_id, key)
);
-- Clean up old keys with a daily job (e.g. older than 24 h).

import { createHash } from 'node:crypto';

const hashBody = (body: unknown) =>
  createHash('sha256').update(JSON.stringify(body)).digest('hex');

app.post('/api/subscriptions', async (req, res) => {
  const key = req.header('Idempotency-Key');
  if (!key) return res.status(400).json({ error: 'The Idempotency-Key header is required' });

  const userId = req.user.id;
  const requestHash = hashBody(req.body);

  // 1. Reserve the key. The PRIMARY KEY guarantees only one request wins.
  const reserved = await db.query(
    `INSERT INTO idempotency_keys (user_id, key, request_hash, status)
     VALUES ($1, $2, $3, 'processing')
     ON CONFLICT DO NOTHING
     RETURNING key`,
    [userId, key, requestHash],
  );

  // 2. The key already existed: this is a retry.
  if (reserved.rowCount === 0) {
    const { rows } = await db.query(
      'SELECT request_hash, status, response_code, response_body FROM idempotency_keys WHERE user_id = $1 AND key = $2',
      [userId, key],
    );
    const saved = rows[0];
    if (saved.request_hash !== requestHash) {
      return res.status(422).json({ error: 'Key reused with a different payload' });
    }
    if (saved.status === 'processing') {
      return res.status(409).json({ error: 'Request still being processed' });
    }
    return res.status(saved.response_code).json(saved.response_body); // replay
  }

  // 3. First run: do the actual work.
  try {
    const subscription = await stripe.subscriptions.create(
      { customer: req.user.stripeCustomerId, items: [{ price: req.body.priceId }] },
      { idempotencyKey: key }, // pass the key along: Stripe deduplicates too
    );
    const body = { id: subscription.id, status: subscription.status };

    await db.query(
      `UPDATE idempotency_keys SET status = 'done', response_code = 201, response_body = $3
       WHERE user_id = $1 AND key = $2`,
      [userId, key, body],
    );
    return res.status(201).json(body);
  } catch (err) {
    // Failed before finishing: release the key so the client can try again.
    await db.query('DELETE FROM idempotency_keys WHERE user_id = $1 AND key = $2', [userId, key]);
    throw err;
  }
});

Three details make the difference:

• The key is per intent, not per request. The client generates crypto.randomUUID() when the user clicks “Subscribe” and reuses that same key on every retry of that action.
• The body hash stops a client bug from reusing the key for a different operation.
• The key is passed on to Stripe. If the process dies between the charge and the UPDATE, the next attempt reaches Stripe with the same key, and Stripe returns the existing charge instead of creating another.

// On the client: one key per click, reused across retries
const key = crypto.randomUUID();
await retry(() =>
  fetch('/api/subscriptions', {
    method: 'POST',
    headers: { 'content-type': 'application/json', 'Idempotency-Key': key },
    body: JSON.stringify({ priceId }),
  }),
);

Pitfalls

• Generating the key inside the retry function. Each attempt gets a new key, and the protection is gone.
• Keys kept only in memory or with a short TTL in Redis. If the key disappears before the last possible retry, the duplicate gets through.
• Idempotent in your database but not at the provider. Protecting the INSERT while charging the card twice solves nothing.
• What AI tends to generate: payment endpoints with no protection against repeats, and automatic retries via axios-retry or similar applied to POST as well.

Go deeper

Materials in English unless noted.

• Article
  Designing robust and predictable APIs with idempotencyStripe

  The article that popularized idempotency keys in payment APIs.

• Article
  Implementing Stripe-like Idempotency Keys in PostgresBrandur Leach

  A complete Postgres implementation with recovery points. The next step after this lesson.

• Docs
  Idempotent requestsStripe

  How the Stripe API handles repeated keys, different payloads and expiry.

• Spec
  The Idempotency-Key HTTP Header FieldIETF (draft)

  The proposal to standardize the header.