SPECIFICATION · CONCEPT BRIEFDWG · 03-SCALE-WITH-LOAD-BALANCER

Scale Out

When one server isn't enough, add more — and a load balancer to spread traffic.

§01Vertical vs horizontal scaling

Vertical = a bigger machine (more CPU/RAM). Horizontal = more machines. Vertical hits a ceiling fast and is a single point of failure. Horizontal scales further and survives individual failures, but needs something in front to spread requests across the copies.

§02Load balancers

A load balancer accepts every client request and forwards it to one of the backend servers. The most common policy is round-robin: server 1, server 2, server 3, server 1, ... It can also pick the least-loaded server, or hash on a key for stickiness.

⚑ NOTE

Effective capacity

With N identical servers behind an LB, your steady-state capacity is roughly N × per-server capacity — minus a small overhead for the LB itself.

⚑ CHEATSHEET · QUICK REFERENCE

More traffic → more servers behind a load balancer.
An LB has high capacity (~500) and low latency (~1 tick). It's almost never your bottleneck.

▸ THE EXERCISE

One server can't keep up. Place a load balancer in front of two servers so traffic is spread across them. The database is shared.

▸ START EXERCISE BACK TO DRAWING SET