# Exactly-Once Is a Myth; Idempotency Is the Answer

> Why distributed systems deliver at-least-once, what 'exactly-once' really means in Kafka and Spark, and how idempotent writes give you correct results.

- Source: https://akshaythakare.com/notes/system-design/exactly-once-and-idempotency
- Topic: Data Engineering System Design
- Level: advanced
- Reading time: 2 min
- Tags: system-design, streaming, idempotency, interview
- Updated: 2026-10-11

## Delivery guarantees

| Guarantee | Meaning | Risk |
| --- | --- | --- |
| At-most-once | Each message processed 0 or 1 times | Data loss |
| At-least-once | Processed 1 or more times | Duplicates |
| Exactly-once | Effect applied once | Hard; only within boundaries |

Networks fail after a write succeeds but before the acknowledgement arrives. The sender cannot tell whether to retry, so it retries. That makes at-least-once the realistic default.

## What "exactly-once" products actually mean

- **Kafka transactions** give exactly-once from Kafka topic to Kafka topic.
- **Spark Structured Streaming** gives exactly-once into sinks that support it (checkpointed offsets plus idempotent or transactional sinks such as Delta).

Once data crosses into a system outside that boundary (an API, a database without transactions), the guarantee no longer holds.

## Effectively-once = at-least-once + idempotent writes

| Technique | Example |
| --- | --- |
| Upsert on a natural key | `MERGE ... ON order_id` |
| Deterministic batch ids + ledger | Replaying batch 42 returns "already published" |
| Partition overwrite | Rewrite `business_date=2026-01-05` completely |
| Dedup table of processed message ids | Skip ids already seen |

## Watch the side effects

Emails, payments and API calls cannot be rolled back. Give each one an **idempotency key**, so a retry is recognised and ignored.

> **Tip: Interview framing**
>
> "I assume at-least-once delivery and make every write idempotent, so a duplicate or a replay produces the same result."