Load Balancing and Types of Load Balancers Explained

A clear guide to the types of load balancers in system design: server-side, DNS-based, and client-side. Learn how they chain together, L4 versus L7, the common algorithms, and sticky sessions.

1. Introduction

One server takes you only so far. Traffic grows, and most teams answer by running several copies of the same service. We call that horizontal scaling, and it fixes the capacity problem well.

But it raises a fresh question right away. With five servers instead of one, something must decide which server takes each request.

That something is a load balancer. Think of a busy restaurant with several chefs. All that cooking power still needs a host at the door who seats each guest at a free table.

Load balancing spreads incoming traffic across many servers so no single machine drowns. It sounds simple on the surface. Yet the details shape almost every large system you will ever design.

You do not need Amazon-sized traffic to hit this. A modest app can outgrow one instance the day it goes viral. The moment a second server appears, load balancing stops being optional.

1.1 What This Article Covers

We start with why this matters, then name every place a routing decision can happen. Here is the plan:

• Why one address in front of many servers changes everything
• The three types of load balancers, and where each one sits
• How those three chain together on a single real request
• Layer 4 versus Layer 7, and what each one can see
• Seven algorithms for picking a server, with a code example
• Sticky sessions, why they work, and why they hurt later
• Keeping the load balancer itself from becoming the weak point
• Six traps, a hands-on walkthrough, and interview questions

2. Why Load Balancing Matters

Three separate wins come out of this one component. Each one alone would justify the effort.

2.1 One Address, Many Servers

Picture a website running on three servers behind one public address. Users neither know nor care that three exist. They type a URL and expect a page.

Somebody has to catch that request and hand it to one of the three. Without that layer, you would hand every user a different address per server. That defeats the whole point of scaling out.

Load balancing hides all of it. Users see a single address, while the work quietly spreads across many machines. Horizontal scaling gives you the servers, and load balancing makes them behave like one service.

2.2 Staying Up When a Server Dies

There is a resilience angle too. When a load balancer sees that a server is sick, it simply stops sending traffic there.

Users never spot the failure. Their requests land on a healthy server instead, and the page loads as usual.

This is why interviewers love the topic. It touches scalability, availability, and performance all at once.

2.3 The Cost Angle

Money enters the picture as well, though people often skip it. Without load balancing, teams buy one giant server to leave headroom for spikes.

That machine then sits mostly idle outside peak hours. You pay for the peak every hour of every day.

Several right-sized servers work out cheaper. You scale the fleet up and down with demand, and you stop paying for silence at 3 a.m.

3. What Is a Load Balancer?

A load balancer is anything that decides which backend server handles a request. The client does not get to choose.

That decision can happen in three genuinely different places. Two separate questions hide here, so keep them apart:

• Where does the routing decision happen? Sections 3.1, 3.3, and 3.4 answer that
• How is a given balancer built? Hardware, software, or cloud, which section 3.2 covers

3.1 Server-Side Load Balancing

Most people picture this model by default. A dedicated component sits between clients and a group of backend servers.

It catches every incoming request, picks a backend, and forwards the request there. The response usually travels back the same way, so all traffic flows through this one component.

Our restaurant host fits here exactly. The host greets each guest, checks which tables are free, and seats people so no chef gets buried.

3.2 How Server-Side Balancers Are Built

Server-side balancers come in three flavours. All three sit in the request path and forward traffic, so treat them as one model built three ways:

• Hardware balancers are dedicated physical appliances, common in older data centres
• Software balancers such as NGINX and HAProxy run as ordinary processes, and they dominate today
• Cloud-managed balancers, such as the AWS Application Load Balancer and Network Load Balancer, are software balancers that the provider runs and scales for you

Whichever you pick, the job stays identical. Sit between clients and servers, choose a healthy backend, forward the request.

Choosing that backend still needs a rule. Send requests in rotation, or pick the least busy server. Those rules are the algorithms in section 6.

3.3 DNS-Based Load Balancing

DNS-based balancing works much earlier in the journey. It acts before any connection opens, and before a server-side balancer ever sees the request.

When a client looks up a domain name, the DNS server can hand different clients different IP addresses for the same domain. Round-robin DNS rotates through a list. GeoDNS returns the data centre nearest the caller.

This gives DNS a very different character. Nothing proxies the traffic. DNS hands out an address and then steps out of the picture.

Two limits matter. DNS routes to a whole region, never to one server. And it reacts slowly to failure, because browsers, operating systems, and resolvers all cache the answer for the record’s time-to-live.

Lowering that time-to-live speeds up failover. It also makes every client re-query far more often, which piles load onto your DNS infrastructure.

Despite the limits, DNS often forms the very first layer in a global system. It picks the region, then a balancer inside that region takes over.

3.4 Client-Side Load Balancing

Client-side balancing flips the model. No shared box sits in the middle at all.

Instead, the caller keeps its own list of available instances and picks one directly. A service registry such as Eureka or Consul usually supplies that list.

Removing the middle box removes a network hop. The caller talks straight to an instance, which cuts latency and drops one component that could fail.

You pay for that in complexity. Now the caller must discover instances and choose among them, so the client gets a little smarter and a little heavier.

3.5 The Word Client Trips People Up

A warning about the name, because it catches almost everyone. In web development, “client” usually means the browser or the user’s device.

So client-side load balancing sounds like the browser choosing a server. That is not what it means, and browsers essentially never do this.

The word is relative. It simply means whoever makes a given request.

Most of the time that caller is another backend service. If an Order Service calls an Inventory Service, then Order is the client for that call, even though Order is itself a server in your data centre.

The caller need not live in the backend at all. Two more examples show the same idea outside the data centre:

• A mobile app can ship with a list of API endpoints and pick among them, failing over when one runs slow. Here the client is a phone, yet it balances load
• gRPC builds client-side balancing right into its libraries. A desktop tool or a partner SDK can resolve one service name to several addresses and spread calls itself

What ties these together is trust. Every one of these callers is code you wrote and shipped, whether a backend service, your own app, or your own SDK.

3.6 Why Browsers Stay Out of It

A browser is the one caller that never does this, and good reasons explain why:

• It would need the private addresses of all your internal instances, which leaks your topology
• Talking to your service registry from the open internet widens the attack surface
• You would ship routing logic into untrusted code that you do not control

So the browser always talks to one stable public endpoint. A server-side balancer handles it from there, and client-side balancing stays reserved for callers you trust.

3.7 Client-Side Balancing in Spring

Spring Cloud LoadBalancer is the standard tool here. One microservice calls another by a logical name, and the library resolves that name to a healthy instance.

// An Order Service calling an Inventory Service by name.
// Here the Order Service is the "client", not any browser.
@Service
public class InventoryClient {

    private final WebClient webClient;

    public InventoryClient(WebClient.Builder builder) {
        // "inventory-service" resolves via the service registry,
        // then Spring Cloud LoadBalancer picks a healthy instance.
        this.webClient = builder.baseUrl("http://inventory-service").build();
    }

    public Mono<Integer> getStockLevel(String sku) {
        return webClient.get()
                .uri("/stock/{sku}", sku)
                .retrieve()
                .bodyToMono(Integer.class);
    }
}

One caveat before you assume this is the only way. Internal traffic has other options:

• Kubernetes routes through a Service object, which behaves like a built-in server-side balancer
• A service mesh such as Istio or Linkerd moves the logic into a sidecar proxy beside each service

No single approach wins outright. The trade-off runs between simplicity and control, and most modern systems mix an edge balancer with one of these internal mechanisms.

4. How the Three Types Work Together

These three are not rivals. You do not pick one and drop the rest.

In a big system they work as a team, each handling a different step of the journey. Once you see them chain up, the topic stops feeling like a pile of tricks.

Two walkthroughs make the chain concrete. The first follows a browser, where all three appear in their usual forms. The second follows a mobile app, so you do not leave thinking client-side always means service-to-service.

4.1 A Web Request from a Browser

A user in London opens a shopping site. The company runs it in two regions, one in Europe and one in North America. Here is the path, step by step:

1. DNS-based balancing goes first. The browser looks up the address, GeoDNS notices a European visitor, and it returns the European IP. This picks a region and nothing smaller
2. Server-side balancing goes second. The browser connects, and an edge balancer in Europe reads the request, sees a storefront page, and forwards it to one healthy storefront server
3. Client-side balancing goes third. That storefront server needs stock numbers, so it calls the inventory service. Now the storefront is the client, and it picks a healthy instance itself

Each step makes a narrower choice than the one before. DNS picks a region, the edge picks a public-facing server, the internal step picks one instance.

No single step could cover the others. DNS cannot read a URL path. The internal balancer has no clue which region the user sits in. They only work as a chain.

4.2 A Request from a Mobile App

Now change one thing. The user opens the company’s own native app instead of a browser.

That difference matters. A mobile app is trusted code the company wrote and shipped, so it can take on a job no browser could.

Say the backend exposes two or three public gateway addresses, one per region. The company bakes that short list into the app:

1. Client-side balancing happens first, on the phone. The app picks a gateway from its built-in list, often the fastest one last time, and retries the next address when one goes quiet
2. Server-side balancing happens next. That gateway address points at a regional balancer, which forwards the call to one of many healthy API servers
3. Client-side balancing happens again inside the backend, exactly as the storefront did before

Step 1 carries the lesson. A caller holding a list and choosing from it is client-side balancing, whether that caller runs on a phone or in a rack.

So client-side does not mean inside the backend. It means the caller chooses, wherever the caller happens to run.

4.3 What Happens When a Layer Fails

Seeing the chain makes failure easy to reason about. Each layer fails its own way and recovers on its own clock:

• When a backend instance dies, the layer above stops sending to it. A server-side balancer drops it after failed health checks, and a client-side balancer skips it in the list. Recovery takes seconds
• When a whole region’s edge balancer dies, DNS steers users to the other region. Caching makes this slower, often minutes, which is the price of working at the DNS layer
• When DNS itself struggles, little sits below to catch the fall. That is why providers run heavy redundancy, and why some large systems buy two DNS providers

A pattern emerges. Lower, finer layers recover fast but cover small failures, while higher, coarser layers recover slowly and cover big ones.

Good designs lean on each layer for what it handles best. Expecting any single layer to catch everything is how outages happen.

4.4 The Three Types at a Glance

Sketching this chain early in an interview pays off. It signals that you see load balancing as stages rather than one magic box.

5. L4 vs L7 Load Balancing

Now for the distinction that comes up in almost every interview.

5.1 Where L4 and L7 Fit

First, connect this back to the three types. L4 versus L7 is not a fourth type sitting beside them.

It is a property of server-side balancers only. The label tells you which network layer the in-path balancer inspects.

DNS balancing happens before a connection exists, so it has no L4 or L7 to speak of. Client-side balancing keeps the decision inside the caller rather than a separate box.

5.2 Layer 4 Load Balancing

A Layer 4 balancer works at the transport layer. It sees IP addresses and TCP or UDP ports, nothing more.

Whether the connection carries HTTP, a database protocol, or something exotic makes no difference to it. It forwards packets to a chosen backend and holds the connection open.

Doing so little keeps it blazing fast and cheap on CPU. Reach for L4 when you want raw speed and the routing choice ignores content. The AWS Network Load Balancer and IPVS are common examples.

5.3 Layer 7 Load Balancing

A Layer 7 balancer works at the application layer. For web traffic it reads the whole HTTP request: the path, the headers, the cookies, even the body.

That opens the door to content-aware routing. Send every /api/orders request to the order service, and every /api/users request to the user service.

It can do more besides. Terminating SSL, rewriting headers, and shifting a slice of traffic to a new version for an A/B test all live here. NGINX, HAProxy in L7 mode, the AWS Application Load Balancer, and Spring Cloud Gateway all fit.

Intelligence costs CPU. Parsing HTTP takes real work compared to blindly forwarding packets, so an L7 balancer handles fewer requests per core.

Most microservice architectures still choose L7 anyway. Path-based routing and SSL termination usually earn back that small cost.

5.4 L4 vs L7 at a Glance

A rule of thumb for interviews: any mention of routing by URL path, hostname, or header means Layer 7. Spreading raw TCP connections as fast as possible means Layer 4 is plenty.

5.5 Using Both Together

Large systems often refuse to choose. A common pattern puts an L4 balancer at the very edge, since it swallows huge volumes of raw traffic cheaply.

Behind it sit L7 balancers doing the smart, content-aware routing to individual services.

Cloud product names mirror this split. You will often find an AWS Network Load Balancer in front of a fleet of Application Load Balancers, or in front of a Kubernetes ingress controller that handles the L7 work.

Recognising the two-tier pattern shows depth. It proves you see L4 and L7 as partners rather than competitors.

6. Load Balancing Algorithms

A balancer still needs a rule for choosing among healthy backends. Here is the tour.

6.1 The Common Algorithms

• Round robin walks the server list in a fixed rotation. Simple, and it works when servers match in capacity and requests take similar time
• Weighted round robin adds a weight per server, so a beefier machine takes proportionally more traffic
• Least connections hands the next request to whoever has the fewest open connections. Handy when request durations vary wildly, as with web sockets
• Least response time extends that idea by watching how fast each server has replied lately. Good for latency-sensitive services
• IP hash maps the caller’s IP to a server, giving simple affinity without cookies
• Consistent hashing places servers and keys on a ring, so adding or removing a server reshuffles only a slice of traffic. The same idea powers distributed caches and sharded databases
• Random just picks one at will. That sounds naive, yet at volume it spreads load nearly as evenly as round robin, and it needs no shared state between balancer instances

6.2 Health Checks Decide Who Is Eligible

Every algorithm above chooses only from servers currently marked healthy. The algorithm and the health check work as a pair.

Health checks decide who is eligible. The algorithm decides which eligible server gets the next request.

So a perfectly tuned algorithm cannot save you. If health checks miss a dead server, the algorithm keeps feeding it traffic.

6.3 Picking an Algorithm

6.4 Round Robin in Code

A tiny round-robin selector shows the core idea. Keep a list and a pointer, then advance the pointer per request.

import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;

public class RoundRobinBalancer {

    private final List<String> servers;
    private final AtomicInteger index = new AtomicInteger(0);

    public RoundRobinBalancer(List<String> servers) {
        this.servers = List.copyOf(servers);
    }

    public String nextServer() {
        // floorMod keeps this correct after the counter overflows
        // past Integer.MAX_VALUE and turns negative.
        int i = Math.floorMod(index.getAndIncrement(), servers.size());
        return servers.get(i);
    }
}

Note the floorMod call. A plain % goes negative once the counter wraps past its maximum, and then get() throws.

A production balancer wraps this same loop in health checks, weights, and careful thread safety.

7. Sticky Sessions and Session Affinity

Load balancing assumes any server can serve any request. That assumption cracks the moment a server keeps something in memory about one user.

7.1 The Problem in Practice

A user logs in, and the server stores their session in local memory. Their next click lands on a different server.

That second server never saw the login. So it treats the user as a stranger and bounces them to the login page.

Shopping carts break the same way. Mixing stateful servers with load balancing produces this classic bug.

7.2 Sticky Sessions as a Fix

Sticky sessions, also called session affinity, pin a client to one backend for the life of their session. Two mechanisms dominate:

• Cookie-based affinity has the balancer set a cookie naming the server that took the first request. Later requests carry that cookie, and the balancer reads it to route them back
• IP-based affinity maps the caller’s IP to a server, with no cookie involved. It breaks when the IP changes mid-session, which phones do every time they leave Wi-Fi

7.3 The Trade-off

Sticky sessions drag back the exact problem horizontal scaling removed. One specific user now depends on one specific server.

When that server crashes, every user pinned to it loses their session. Load skews too, since a handful of very active users can bury one machine while others idle.

Treat stickiness as a short-term patch rather than a design choice. It works, but it quietly reintroduces a single point of failure per user.

7.4 Sticky Sessions Fight Autoscaling

Here is the part people discover the hard way. Affinity only forms as new sessions begin.

So a fresh server joins the fleet holding zero pinned users. Existing users stay glued to the older, already-busy machines.

Your new capacity helps far less than expected, at exactly the moment you need it most. Removing a server is just as awkward, because everyone pinned to it must migrate or lose their session outright.

8. Externalizing Session State in Spring Boot

The better answer removes the need for affinity entirely.

8.1 Keep Every Server Stateless

Rather than parking session data on one server, put it in a shared store every instance can read.

That keeps servers stateless. Statelessness is precisely what lets horizontal scaling and load balancing cooperate.

Any instance can then serve any request. Add servers, kill servers, replace servers, and nobody logged in ever notices.

8.2 Spring Session with Redis

In Spring Boot, the standard tool is Spring Session backed by Redis. Your code still reads and writes sessions through the normal Servlet API.

Underneath, Spring Session parks that data in Redis instead of local memory. Adding it is mostly configuration:

// build.gradle
implementation 'org.springframework.session:spring-session-data-redis'
implementation 'org.springframework.boot:spring-boot-starter-data-redis'

Then point the app at Redis:

# application.properties (Spring Boot 3.x and later)
spring.session.store-type=redis
spring.data.redis.host=localhost
spring.data.redis.port=6379

# On Spring Boot 2.x these two keys were named
# spring.redis.host and spring.redis.port

Watch those property names. Spring Boot 3.0 moved the Redis keys under spring.data.redis, and old tutorials still show the 2.x spelling.

Once this runs, a user can log in through one server and land anywhere next. Every instance reads the same Redis session, so nothing breaks and no stickiness is needed.

Remember the contrast. Sticky sessions treat the symptom by pinning users, while externalized state cures the cause by deleting server-side memory.

8.3 What It Costs

This is not free. Every session read becomes a network call rather than a memory lookup.

Inside one data centre that usually costs well under a millisecond. Most apps never feel it.

Redis also becomes a critical dependency. Deploy it with its own replication and failover, or you have simply moved the single point of failure somewhere new.

Almost always, that trade is worth taking. Interchangeable servers buy you more than a fraction of a millisecond ever costs.

9. Load Balancer High Availability

One single point of failure still hides in this design.

9.1 The Last Single Point of Failure

It is the load balancer itself. When every request flows through one instance and that instance dies, your healthy backend fleet might as well be switched off.

Production systems run balancers in a pair or a cluster. An active-passive setup with a floating IP is common, where the address moves to the standby when the primary drops.

Cloud-managed balancers handle this for you across availability zones. Reaching four or five nines demands redundancy at every layer, and this layer is no exception.

Put bluntly, a single balancer can never be more available than the single server it replaced.

9.2 Health Checks Make It Self-Healing

Health checks turn all of this into a self-healing system. The balancer probes each backend on a timer, and traffic stops flowing to anything that starts failing.

Spring Boot Actuator plugs straight into that. Expose the health endpoint and point your balancer at it:

# application.properties
management.endpoints.web.exposure.include=health
management.endpoint.health.probes.enabled=true

# The balancer then polls:
#   GET /actuator/health
# A healthy instance answers:
#   {"status":"UP"}

Tuning matters more than people expect. Probe too slowly and dead servers keep taking traffic. Probe too aggressively and a briefly slow server flaps in and out of rotation.

9.3 DNS Failover and Anycast

Two more mechanisms reinforce redundancy at global scale:

• DNS failover watches a balancer from outside your network and rewrites DNS records when it goes quiet. Caching makes it react slowly, in minutes rather than seconds
• Anycast routing advertises one IP address from many physical locations and lets network routing deliver traffic to the nearest healthy site. That dodges DNS caching delays completely, which is why large content delivery networks lean on it

Few application teams build either from scratch. Just know they exist, and ask one question whenever a design shows a load balancer: what happens if that exact box disappears?

10. Common Mistakes and Pitfalls

A handful of misunderstandings surface again and again, in production incidents and interview rooms alike.

10.1 Six Traps to Avoid

• Assuming L7 features cost nothing. Path routing and SSL termination burn CPU, so do not default to L7 when raw throughput rules
• Grabbing sticky sessions too fast. They fix today’s bug and quietly restore a single point of failure
• Forgetting the balancer can die. A highly available fleet behind one balancer instance is not highly available
• Leaving health checks untuned. Slow checks feed dead servers, and twitchy checks flap healthy ones out of rotation
• Treating DNS as instant. Clients and resolvers cache records, so a DNS failover can take minutes to finish spreading
• Switching algorithms with no documented reason. Per-service or per-environment drift makes performance bugs miserable to reproduce

10.2 What Interviewers Listen For

When an interview reaches load balancing, name Layer 4 or Layer 7 out loud and justify the pick. Multiple microservices behind different URL paths is an L7 requirement, so say so.

Expect a session follow-up whenever your design has logged-in users. Naming sticky sessions works as a first answer, and naming externalized storage as the long-term fix shows real depth.

Close by noting that the balancer needs redundancy too. Many candidates design a beautiful stateless fleet and leave one balancer as the weak point.

Mention client-side balancing for internal calls as well. Naming both models, and explaining when each fits, usually signals senior-level thinking here.

11. A Practical Walkthrough

Let us tie the ideas into one small, concrete setup you could run today.

11.1 The Setup

Say we run three copies of a Spring Boot storefront. Each listens on port 8080 on its own private address.

NGINX sits in front as our server-side Layer 7 balancer. Our goals are ordinary ones:

• Spread traffic by least connections, since page renders vary in cost
• Pull a failing instance out of rotation automatically
• Route the /api/orders path to a different service entirely

11.2 The NGINX Config

upstream storefront {
    least_conn;
    server 10.0.1.11:8080 max_fails=3 fail_timeout=10s;
    server 10.0.1.12:8080 max_fails=3 fail_timeout=10s;
    server 10.0.1.13:8080 max_fails=3 fail_timeout=10s;
}

upstream orders {
    server 10.0.2.21:8080;
    server 10.0.2.22:8080;
}

server {
    listen 80;

    location /api/orders/ {
        proxy_pass http://orders;          # L7 path-based routing
    }

    location / {
        proxy_pass http://storefront;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

Read the pieces one at a time. The least_conn line picks our algorithm from section 6.

Each max_fails and fail_timeout pair defines a passive health check. Three failures inside ten seconds, and NGINX benches that server for ten seconds.

The two location blocks are Layer 7 at work. Only a balancer that reads the URL path could split traffic this way, which is exactly what section 5.3 described.

One accuracy note worth knowing. Open-source NGINX ships passive health checks only, as shown here. Active probing against /actuator/health requires NGINX Plus, HAProxy, or a cloud balancer.

11.3 Watching a Server Fail

Now kill the instance at 10.0.1.12 and watch what unfolds:

1. The next few requests routed there fail or time out, and NGINX counts those failures
2. After three within the window, NGINX marks that server down and skips it
3. Remaining traffic splits across the other two by least connections, and users see nothing
4. Ten seconds later NGINX tries the server again, restoring it if it answers

Notice what did not happen. Nobody got logged out, because sessions live in Redis rather than local memory.

Had we relied on sticky sessions instead, every user pinned to that instance would have lost their cart. That single difference is the whole argument of sections 7 and 8.

[IMAGE PLACEHOLDER: diagram of the request chain, showing GeoDNS choosing a region, an edge L7 balancer choosing a storefront server, and the storefront calling inventory via client-side balancing]

12. Practical Takeaways and Key Terms

12.1 Habits to Carry Forward

• Choose L4 for raw speed, and L7 for path routing, SSL termination, or header rules
• Match the algorithm to the traffic: round robin for uniform work, least connections for variable work
• Prefer externalized sessions over stickiness, because stateless servers scale cleanly
• Run the balancer itself redundantly, so it never becomes the sole weak point
• Wire health checks in properly, then tune the thresholds rather than accepting defaults
• Use client-side balancing for internal calls when you want to skip an extra hop
• Whenever you draw a balancer, ask aloud what breaks if that exact box vanishes

12.2 Key Terms Recap

13. Interview Questions

14. Conclusion

Let us wrap up what we covered. Load balancing turns a pile of separate servers into one reliable service.

Three types share the work. DNS picks a region, a server-side balancer picks a server inside it, and client-side balancing picks an instance for service-to-service calls.

Among server-side balancers, the layer decides what you can see. Layer 4 gives you speed, Layer 7 gives you insight into the request, and most systems end up running both in tiers.

Sticky sessions fix an urgent problem and hand you a fragile one. Externalizing state into Redis is the durable answer, since it keeps every instance interchangeable.

One last habit to carry with you. Whenever you draw a load balancer, ask immediately whether that box is redundant. A system is only as available as its weakest single point of failure, and the balancer is the easiest one to forget.

Further Reading

• Scaling and Data Basics in System Design
• Back-of-the-Envelope Estimation and System Design Basics
• Consistent Hashing in System Design
• Caching Strategies in System Design
• Database Replication in System Design
• CAP Theorem and PACELC
• URL Shortener System Design