Hi Pythonistas!
User clicks "Pay Now."
Request sent to server.
Server processes payment.
Charges the card.
Network hiccup.
Response never reaches user.
User sees loading spinner.
Clicks "Pay Now" again.
Server charges again.
User charged twice.
Same problem everywhere:
Send message → network fails → resend → duplicate message
Place order → timeout → retry → duplicate order
Create post → error → retry → duplicate post
Retries are necessary.
Duplicate operations are not.
Idempotency solves this.
What Is Idempotency?
An operation is idempotent
if performing it multiple times
produces the same result as performing it once.
Idempotent:
DELETE /users/123 → user deleted
DELETE /users/123 → already deleted, same result
DELETE /users/123 → same result, no errorNot idempotent:
POST /payments → charge $100
POST /payments → charge $100 again ❌
POST /payments → charge $100 again ❌Math analogy:
Idempotent: abs(abs(abs(-5))) = 5
same result no matter how many times
Not idempotent: 5 + 1 = 6 → 6 + 1 = 7
result changes each time
Do We Need Idempotency Everywhere?
No.
Only where retries can cause harm.
When you NEED it:
Money involved → duplicate = charged twice
Creates something → duplicate = two orders
Sends something → duplicate = two welcome emails
External API calls → duplicate = real world affected twice
Queue consumers → at-least-once delivery, message processed twice
When you DON'T need it:
Read operations → GET, naturally idempotent
PUT requests → replace entire resource, already idempotent
Low stakes writes → updating last_seen timestamp
Analytics logging → approximate is fine
Unique constraints → database handles it (email must be unique)
The simple test:
If this operation runs twice:
→ Does it cause harm?
YES → need idempotency
NO → probably fine without it
Charge $100 twice → harm ✅ → need idempotency
Update name twice → no harm → skip
Send email twice → harm ✅ → need idempotency
Read profile twice → no harm → skip
HTTP Methods and Idempotency
Method Idempotent? Notes
──────────────────────────────────────────
GET ✅ Yes read only
PUT ✅ Yes replace entire resource
DELETE ✅ Yes delete same thing = same result
POST ❌ No creates new resource each time
PATCH ❌ No* depends on implementation
Idempotency Keys
How do you make non-idempotent operations idempotent?
Idempotency keys.
Client generates a unique key per operation.
Sends it with the request.
Server uses it to detect duplicates.
First request:
POST /payments
Idempotency-Key: a8098c1a-f86e-11da-bd1a-00112444be1e
{amount: 100, card: "..."}
→ server processes payment
→ stores result with key
→ returns success
Second request (retry):
POST /payments
Idempotency-Key: a8098c1a-f86e-11da-bd1a-00112444be1e
{amount: 100, card: "..."}
→ server sees key exists
→ returns stored result
→ does NOT charge again ✅
Implementation
Client generates unique key
import uuid
def pay(amount, card):
# generate unique key for this payment attempt
idempotency_key = str(uuid.uuid4())
local_storage.set('payment_key', idempotency_key)
return requests.post(
'/api/payments',
json={'amount': amount, 'card': card},
headers={'Idempotency-Key': idempotency_key}
)
def retry_pay(amount, card):
# reuse SAME key on retry
idempotency_key = local_storage.get('payment_key')
return requests.post(
'/api/payments',
json={'amount': amount, 'card': card},
headers={'Idempotency-Key': idempotency_key}
)
Server checks and stores:
def process_payment(request):
idempotency_key = request.headers.get('Idempotency-Key')
if not idempotency_key:
return error('Idempotency-Key header required', 400)
# check if already processed
cached = redis.get(f'idempotency:{idempotency_key}')
if cached:
return json.loads(cached) # return stored result
# process payment
result = bank_api.charge(
amount=request.json['amount'],
card=request.json['card']
)
# store result
redis.set(
f'idempotency:{idempotency_key}',
json.dumps(result),
ex=86400 # keep 24 hours
)
return resultRace Condition Two Requests at Once
What if two retries arrive simultaneously?
Both check Redis.
Both see key doesn't exist.
Both process payment.
Duplicate charge.
Solution: atomic check-and-set.
def process_payment(request):
idempotency_key = request.headers.get('Idempotency-Key')
redis_key = f'idempotency:{idempotency_key}'
# NX = only set if not exists (atomic)
lock_acquired = redis.set(
redis_key,
'processing',
nx=True,
ex=86400
)
if not lock_acquired:
# already being processed
result = wait_for_result(redis_key)
return result
try:
result = bank_api.charge(
amount=request.json['amount'],
card=request.json['card']
)
redis.set(redis_key, json.dumps(result), ex=86400)
return result
except Exception as e:
# delete key on failure so client can retry
redis.delete(redis_key)
raise enx=True = atomic.
Only one request sets the key.
Others wait or get cached result.
Multiple Tables - Same Database
One user action often touches multiple tables.
Place an order:
→ INSERT into orders table
→ INSERT into order_items table
→ UPDATE inventory table
→ INSERT into payments table
If this fails halfway:
orders table → inserted ✅
order_items table → inserted ✅
inventory table → updated ✅
payments table → FAILED ❌
User retries.
Now:
orders table → duplicate order ❌
order_items table → duplicate items ❌
inventory table → double deducted ❌
Disaster.
Two things solve this together:
Database Transaction: All-or-nothing.
Multiple Services - Different Databases Harder.
Database transactions only work within one database.
Order Service → orders DB
Payment Service → payments DB
Inventory Service → inventory DB
Can't wrap all three in one transaction.
Solution 1 - Saga Pattern
Break into steps.
Each step has a compensating action.
Step 1: Create order → compensate: cancel order
Step 2: Reserve inventory → compensate: release inventory
Step 3: Charge payment → compensate: refund payment
Step 3 fails:
→ run Step 2 compensation: release inventory
→ run Step 1 compensation: cancel order
→ system back to original state
Saga state saved at each step.
Retry resumes from where it stopped.
Not from the beginning.
Solution 2 — Outbox + Per-Event Idempotency
Remember outbox from Post 16?
Each event gets its own idempotency key:
main key: order_abc123
inventory key: order_abc123:inventory
payment key: order_abc123:payment
Each service processes its event idempotently.
Practical Guide
Single table:
→ idempotency key + Redis check
Multiple tables, same database:
→ idempotency key + database transaction
Multiple services:
→ idempotency key + saga pattern
→ OR outbox pattern + per-event key
Idempotency Key Design
import uuid
# UUID v4 — random, globally unique
key = str(uuid.uuid4())
# or hash of operation details
key = hashlib.sha256(
f"{user_id}:{amount}:{card_last4}:{timestamp_rounded_to_minute}"
.encode()
).hexdigest()Who generates:
always the client. never the server.
client must generate same key for retries.
Mental Model
Idempotency → same operation, same result, always
Idempotency key → unique ID client generates per operation
Client generates → always, not server
Retry → reuse same key
NX in Redis → atomic, prevents race condition
Transaction → multi-table consistency
Saga → multi-service consistency with compensation
Outbox → per-event idempotency across services
Natural idempotent → SET, upsert, check-before-act
Queue consumer → must always be idempotent
Stripe → best real-world example
Need it: → money, orders, emails, external calls
Skip it: → reads, PUT, low stakes writes
What's Coming Next
Now your system handles:
failures gracefully (circuit breakers).
retries safely (idempotency).
But what about operations
that span multiple services
and must ALL succeed or ALL fail?
Distributed Transactions.
The saga pattern in depth.
Two-phase commit.
How real systems coordinate
across multiple databases.
See you in Post 20.
