Hi Pythonistas!
Your API is live.
Users are using it.
Everything fine.
Then one day:
Normal traffic: 100 requests/second
Suddenly: 50,000 requests/second
Could be:
- a popular blog post linked your API.
- a mobile app bug causing infinite retries.
- a bot scraping your data.
- a DDoS attack.
Your server handles 1,000 requests/second.
50,000 hits it.
Server crashes.
Everyone affected.
Including legitimate users.
Rate limiting prevents this.
What Is Rate Limiting?
Controlling how many requests
a client can make in a given time window.
Rule: max 100 requests per minute per user
Request 1-100 → allowed ✅
Request 101 → rejected ❌ HTTP 429 Too Many Requests
Protects your server.
Fair usage for all users.
Prevents abuse.
Where to Apply
IP level:
Max 1000 requests/minute per IP
Protects against bots.
But many users share one IP (offices, NAT).
Unfair to block entire office.
User level:
Max 100 requests/minute per user_id
More precise.
Authenticated users only.
API key level:
Free tier: 100 requests/day
Pro tier: 10,000 requests/day
Enterprise tier: unlimited
How Stripe, OpenAI, Twilio charge for usage.
Endpoint level:
POST /login: 5 requests/minute (prevent brute force)
GET /feed: 60 requests/minute (normal usage)
POST /upload: 10 requests/minute (expensive operation)
Different limits for different endpoints.
Expensive operations get stricter limits.
Algorithm 1 - Fixed Window Counter
Simplest.
Divide time into fixed windows.
Count requests in each window.
Window: 1 minute
Limit: 100 requests
12:00:00 - 12:01:00 → window 1
12:01:00 - 12:02:00 → window 2
Good: simple, memory efficient.
Bad: boundary attack.
99 requests at 12:00:59 (end of window 1)
99 requests at 12:01:01 (start of window 2)
Total: 198 requests in 2 seconds.
Both windows show < 100.
Limit bypassed.
Algorithm 2 - Sliding Window Log
Track timestamp of every request.
Count requests in last N seconds.
Good: accurate, no boundary attack.
Bad: high memory.
Store every request timestamp.
1 million users × 100 requests = 100 million timestamps.
Algorithm 3 - Sliding Window Counter
Hybrid of both.
Two windows. Weighted count.
Good: accurate, memory efficient, no boundary attack.
Bad: approximate (tiny margin of error).
Used by Cloudflare and many large systems.
Algorithm 4 - Token Bucket
Most popular.
Imagine a bucket of tokens.
Bucket capacity: 100 tokens
Refill rate: 10 tokens/second
Each request consumes 1 token.
Bucket refills continuously.
Start: 100 tokens
10 requests → 90 tokens
Refill +10 → 100 tokens
100 requests → 0 tokens
Next request → rejected (no tokens)
Good: allows bursts, smooth refill, natural feeling.
Bad: needs syncing across distributed servers.
Algorithm 5 - Leaky Bucket
Requests come in at any rate.
Processed at fixed rate.
Excess rejected.
Requests → [bucket/queue] → processed at fixed rate
overflow → rejected
Like a leaky bucket:
water pours in at any rate.
leaks out constantly.
overflow spills.
Good: smooth output, prevents bursts hitting server.
Bad: burst requests rejected, not good for bursty traffic.
Algorithms Compared
Algorithm Accuracy Memory Burst Complexity
───────────────────────────────────────────────────────────
Fixed Window Low Low Yes Simple
Sliding Window Log High High No Medium
Sliding Window Counter High Low No Medium
Token Bucket High Low Yes Medium
Leaky Bucket High Medium No Medium
Simple systems → Fixed Window
Accuracy needed → Sliding Window Counter
Burst traffic ok → Token Bucket
Smooth output → Leaky Bucket
Distributed Rate Limiting
Multiple servers.
Each tracking count independently:
User limit: 100 requests/minute
Server1: 50 requests
Server2: 50 requests
Server3: 50 requests
Total: 150 requests → limit bypassed
Solution: Redis as shared counter.
All servers increment same Redis key.
Accurate. Atomic. No race conditions.
Rate Limit Headers
Tell clients about their limits.
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 45
X-RateLimit-Reset: 1705312800
Retry-After: 30
Good clients use these headers.
Back off when remaining is low.
Retry after Reset time.
Implementation in Python Using slowapi
from fastapi import FastAPI
from slowapi import Limiter
from slowapi.util import get_remote_address
limiter = Limiter(key_func=get_remote_address)
app = FastAPI()
@app.get('/api/feed')
@limiter.limit('60/minute')
async def get_feed(request: Request):
return {'feed': [...]}
@app.post('/api/login')
@limiter.limit('5/minute')
async def login(request: Request):
return {'token': '...'}Using Redis directly:
def rate_limit(user_id, limit=100, window=60):
now = int(time.time())
window_key = now // window
key = f'rate:{user_id}:{window_key}'
pipe = r.pipeline()
pipe.incr(key)
pipe.expire(key, window)
results = pipe.execute()
count = results[0]
return {
'allowed': count <= limit,
'remaining': max(0, limit - count),
'reset': (window_key + 1) * window
}
def api_handler(request):
result = rate_limit(request.user_id)
if not result['allowed']:
return {
'error': 'Too many requests',
'retry_after': result['reset'] - time.time()
}, 429
return handle_request(request)Rate Limiting at Different Layers
Nginx
http {
limit_req_zone $binary_remote_addr
zone=api:10m rate=100r/m;
server {
location /api/ {
limit_req zone=api burst=20 nodelay;
limit_req_status 429;
}
}
}Blocks before request reaches your app.
Fast. Efficient.
API Gateway
plugins:
- name: rate-limiting
config:
minute: 100
hour: 1000
policy: redis
redis_host: redis.internalManaged. No code changes.
Cloudflare:
Rate limiting at CDN level.
Before request reaches your servers.
Best protection against DDoS.
Handling Rate Limited Responses
Exponential backoff
def api_call_with_retry(url, max_retries=5):
for attempt in range(max_retries):
response = requests.get(url)
if response.status_code == 429:
wait = (2 ** attempt) + random.uniform(0, 1)
print(f'Rate limited. Waiting {wait:.2f}s')
time.sleep(wait)
continue
return response
raise Exception('Max retries exceeded')Wait 1s, 2s, 4s, 8s, 16s.
Random jitter prevents stampede.(Remember Post 8A?)
Respect Retry-After header
if response.status_code == 429:
retry_after = int(response.headers.get('Retry-After', 60))
time.sleep(retry_after)Don't guess. Use the header.
Rate Limiting vs Throttling
Often confused.
Rate Limiting → hard stop
→ request rejected
→ HTTP 429
Throttling → slow down
→ request delayed
→ eventually processed
Use rate limiting for:
- API protection
- Fair usage enforcement
- Preventing abuse
Use throttling for:
- Internal services
- Batch processing
- Where slow is better than rejected
Mental Model
Rate limiting → control requests per time window
Fixed window → simple, boundary attack possible
Sliding window log → accurate, high memory
Sliding window counter → accurate, memory efficient
Token bucket → allows bursts, smooth refill
Leaky bucket → constant output, no bursts
Redis counter → distributed rate limiting
429 → HTTP status for rate limited
X-RateLimit-* → headers to inform clients
Retry-After → when client can retry
Exponential backoff → client retry strategy
Rate limiting → hard reject
Throttling → slow down, eventually process
What's Coming Next
Now your system handles too much traffic.
But what about when your own services fail?
Service A calls Service B.
Service B is slow.
Service A waits.
Threads pile up.
Service A crashes.
Now both are down.
Here comes Circuit Breakers How to stop one failure from taking down everything.
