Load balancing and scale
Everything I write assumes several instances behind a load balancer. Stateless services, health checks, a CDN in front of anything static: scaling becomes raising the instance count.
Decisions and why
- Stateless services
- The token carries who the user is and shared state lives in PostgreSQL and Redis. Any instance serves any request, no sticky sessions.
- Health checks on the load balancer
- A sick instance leaves the target group on its own and ECS starts another. Nobody gets paged for it.
- CDN in front
- Static and prerendered content is served at the edge (this whole site works that way). The API only gets what needs an API.
- Queues for spikes
- Heavy work goes to a queue and consumers; a spike becomes seconds of delay, not 503s.
requestresponse
Pick a scenario above to watch the message travel through the architecture.