Hi Pythonistas!
Phase 1 is done.You understand how systems talk to each other.
HTTP, TCP, DNS, IP, REST. Now imagine you've built something with all of this.
It works.100 users use it daily. Fast. Smooth. No complaints.Then one day your post goes viral.
10,000 users hit your site in an hour. Response times go from 100ms to 5 seconds. Then it crashes.
What do you do?
This is the first big decision every growing system faces. And it has exactly two answers.
What Is "Scaling"?
Scaling = handling more load.
More users. More requests. More data.
Your server has limits:
CPU → calculations per second
RAM → data held in memory
Disk I/O → read/write speed
Network → data sent/received
When traffic increases these limits get hit.
Scaling means: increase these limits.
Exactly two ways to do this.
Vertical Scaling (Scale Up)
Make the existing server bigger.
Before:
4 CPU cores, 8 GB RAM
After:
16 CPU cores, 64 GB RAM
Same server. More power.
The Analogy
Your server is a worker doing a job.
The worker is overwhelmed.
Vertical scaling = give that worker:
a bigger desk.
a faster computer.
more hands.
Same person. Much more capable now.
How It Works
On a cloud provider:
Current: t3.medium (2 vCPU, 4GB RAM)
Upgrade: t3.2xlarge (8 vCPU, 32GB RAM)
Few clicks.
Server restarts with new specs.
Same IP. Same setup. Just bigger.
Why It's Attractive
Simple.
No code changes.
No architecture changes.
Just bigger hardware.
No distributed system complexity.
One server. One database.
No syncing data between machines.
Everything in one place.
The Limit
There's a ceiling.
The biggest server AWS offers today:
~448 vCPUs
~24 TB RAM
Sounds huge.
But:
extremely expensive.
eventually hits the physical limit of hardware.
one machine = one point of failure.
If that one server goes down everything goes down.
No backup. No redundancy.
Your entire system depends on one box.
The Restaurant Analogy
You run a restaurant.
One chef. One kitchen.
Business grows.
Vertical scaling = hire a faster chef, buy better equipment, bigger kitchen.
Works for a while.
But there's a limit.
One kitchen can only serve so many tables.
No matter how fast the chef is.
Horizontal Scaling (Scale Out)
Add more servers.
Instead of one powerful server many smaller servers working together.
Before:
1 server (16 CPU, 64GB RAM)
After:
4 servers (4 CPU, 16GB RAM each)
Same total resources.
Different distribution.
The Analogy
Horizontal scaling = open more restaurant branches.
Branch 1. Branch 2. Branch 3.
Each handles some customers.
Total capacity = sum of all branches.
How It Works
Load Balancer
/ | \
Server1 Server2 Server3
Requests come in.
Load balancer distributes them across servers.
Request 1 → Server1
Request 2 → Server2
Request 3 → Server3
Request 4 → Server1
We'll go deep on load balancers in the next post.
For now just know they exist and distribute traffic.
Why It's Powerful
No upper limit (theoretically)
Need more capacity? Add another server.
Need even more? Add ten more.
AWS, Google, Netflix run thousands of servers.
Redundancy fault tolerance
Server1 crashes.
Load balancer notices.
Stops sending traffic to Server1.
Server2 and Server3 keep working.
Users barely notice.
One server going down doesn't take down your system.
Cost efficient at scale
10 small servers are often cheaper than 1 massive server.
Commodity hardware.
The Cost of Horizontal Scaling
This is where it gets hard.
Your application must support it.
If your app stores data on the server's local disk:
User uploads a file → saved on Server1's disk
Later, user requests that file
→ Load balancer sends them to Server2
→ Server2 doesn't have that file
→ 404 error
Your app needs to be stateless.
Any server should be able to handle any request.
This means:
no local file storage (use shared storage like S3).
no local sessions (use shared session store like Redis).
no local caching that other servers don't know about.
Database Becomes the Bottleneck
You can have 10 app servers.
But if they all hit ONE database
that database becomes the new limit.
We'll solve this in upcoming posts.
Replication. Sharding.
More Moving Parts
1 server → 1 thing to monitor, deploy, debug
10 servers → 10 things to monitor, deploy, debug
More complexity.
More things that can go wrong.
Need proper tooling monitoring, logging, deployment automation.
We'll cover this in Phase 6.
Side by Side
| Vertical | Horizontal | |
| How | Bigger machine | More machines |
| Complexity | Low | High |
| Limit | Hardware ceiling | Theoretically unlimited |
| Single point | Yes one server |
No distributed of failure |
| Downtime | Usually restart | Zero downtime |
| Cost | Expensive at top | Efficient at scale |
| App changes | None | Must be stateless |
| Database | Still one DB | Becomes bottleneck |
The Real World Approach
Nobody picks purely one or the other.
Step 1 - Start with vertical scaling
1 server. Traffic grows.
Upgrade the server.
Simple. Fast. No code changes.
Step 2 - Hit the ceiling
Either:
server is maxed out.
or one server going down = total outage.
Step 3 - Go horizontal
Add a second server.
Add a load balancer.
Make your app stateless.
Step 4 - Scale the database separately
App servers scale horizontally easily.
Database scaling is harder.
Different techniques needed.
Coming in Phase 2 - replication, sharding.
You Can Do Both
Horizontal scaling doesn't mean tiny servers.
3 servers × (16 CPU, 64GB RAM each)
That's horizontal scaling (multiple servers).
AND each server is vertically scaled (powerful machines).
Real systems combine both.
Total capacity = (size per server) × (number of servers)
You grow in either dimension.
Auto Scaling
Modern cloud systems take this further.
Auto Scaling Groups.
Low traffic (night) → 2 servers running
Normal traffic (day) → 5 servers running
Traffic spike (sale) → 20 servers running
Spike ends → back down to 5
Servers added and removed automatically.
Based on real-time metrics:
CPU usage > 70% → add a server
CPU usage < 30% → remove a server
You pay only for what you use.
Only possible with horizontal scaling.
Vertical scaling can't do this on demand mid-spike.
Some cloud providers allow live resizing but limited, often needs restart.
Real World Examples
Early stage startup:
1 server. Vertical scaling.
App + database on same machine.
Works fine for thousands of users.
Growing startup:
App servers: horizontal (3-5 servers behind load balancer)
Database: vertical (one powerful database server)
Most common setup.
Large scale (Netflix, Uber, Instagram):
Thousands of servers.
Horizontal everything.
Auto scaling based on traffic.
Database: sharded across many machines.
Connecting What You Know
Remember the layers from Post 4?
Internet
└── Load Balancer ← new concept, next post
└── App Server 1
└── App Server 2
└── App Server 3
└── Database
Horizontal scaling is why load balancers exist.
Multiple servers need something to distribute traffic between them.
Also remember stateless vs stateful from our last conversation?
Stateless → easy to scale horizontally
Stateful → hard to scale horizontally
This is why REST APIs (stateless) scale so easily.
Any server handles any request.
Add more servers. Done.
Mental Model
Vertical scaling → bigger machine, simple, has a ceiling
Horizontal scaling → more machines, complex, no ceiling
Load balancer → distributes traffic across servers
Stateless app → required for horizontal scaling
Single point of failure → risk of vertical-only scaling
Auto scaling → add/remove servers based on load
Database bottleneck → app scales easily, DB doesn't (yet)
What Changed for Me
Before this:
I thought scaling meant "buy a bigger server."
After this:
I realized scaling is a decision tree.
Vertical first because it's simple.
Horizontal when you hit limits because it's necessary.
And the moment you go horizontal
your application's design changes forever.
Stateless becomes not a nice-to-have.
It becomes a requirement.
What's Coming Next
Now you know:
one server isn't enough.
You need many.
But how do requests know which server to go to?
Load Balancing.
The traffic cop of your system.
Algorithms, types, and how Nginx does it.
See you in Post 7.