Consensus in Distributed Systems: Leader Election and Raft Made Simple

Consensus in distributed systems made simple: learn leader election, terms, quorum, log replication, and Raft intuition with no Paxos proofs and no code.

1. Introduction

Most backend systems start life on a single machine. One server holds the data, answers every request, and life is simple. Then traffic grows, and one machine is no longer safe. So you add more machines, and suddenly a hard question appears. If three or five servers all hold a copy of the same data, how do they agree on what is true? That single question is what consensus in distributed systems is all about.

This article tackles one of the most feared interview topics in plain language. We will skip the heavy mathematics. Instead, we will build a clear mental picture of how a group of servers picks a leader, keeps their data in sync, and keeps working even when some of them crash.

We will focus on Raft, a consensus algorithm that was designed for one reason above all: to be easy to understand. Along the way, we will touch on Paxos, but only lightly. The goal here is intuition, not proofs. By the end, you should be able to explain leader election and log replication to a colleague over coffee, and handle the topic calmly in a design interview.

2. Why Consensus Matters

Let us start with the “why” before the “how”. A distributed system runs on many machines, and any of them can fail at any moment. Networks drop messages. Disks die. A server freezes for a few seconds and then wakes up, confused about what it missed. Despite all this chaos, users expect one consistent answer.

Think about a simple bank balance. Suppose five servers each store your account. You withdraw money, and the request reaches three servers but not the other two. Now the system holds two different truths. Which one is correct? Without a rule for agreement, your balance becomes a matter of luck. Consensus is the rule that stops this mess.

2.1 The Replicated State Machine Idea

Here is the mental model that engineers use. Imagine every server runs the same program, and that program only changes when it applies commands from a list. If every server applies the exact same commands in the exact same order, every server ends up in the exact same state. This idea is called a replicated state machine.

The shared list of commands is called a log. So the real job becomes smaller and clearer. You do not need the servers to agree on the final state directly. You only need them to agree on the order of entries in the log. Get the log right, and matching state follows for free.

2.2 A Simple Picture

Picture a small team writing in a shared notebook. If everyone scribbles at once, the notebook turns into noise. So the team picks one person to hold the pen. Others suggest what to write, but only the pen-holder actually writes, and everyone copies the same lines in the same order. That pen-holder is the leader, and the notebook is the log. This tiny idea sits at the heart of Raft.

3. What Problem Does Consensus Actually Solve?

Consensus means getting a group of servers to agree on a value, even when some servers fail or messages arrive late. That sounds abstract, so let us pin it down with the properties people actually care about.

• Agreement: all healthy servers decide on the same value, never two different ones.
• Validity: the value they agree on was actually proposed by someone, not invented out of thin air.
• Termination: the group eventually reaches a decision instead of arguing forever.

3.1 Agreement, Not Just Copying

Copying data is easy. Agreeing is the hard part. A leader can shout an update to four followers, but what if two of them never hear it? Later, a different server might try to lead and tell a conflicting story. A good consensus protocol makes sure that once a decision is final, nobody can quietly rewrite it.

3.2 The Majority, or Quorum

Raft leans on one beautifully simple trick: the majority vote. A majority, also called a quorum, means more than half of the servers. In a cluster of five, a majority is three. Any decision needs the blessing of a majority before it counts as real.

Why does a majority work so well? Because two different majorities must always share at least one member. If one majority agreed to decision A and another tried to agree to decision B, the shared member would catch the clash and block it. That overlap quietly guarantees there is only one truth.

3.3 Why an Odd Number of Nodes

You will often see clusters of three, five, or seven servers. The odd number is deliberate. A cluster of 2f plus 1 servers can keep working while f of them are down. So five servers tolerate two failures, and three servers tolerate one. An even number wastes a machine, since four servers also tolerate only one failure, just like three.

4. A Quick Word on Paxos

No consensus article is complete without naming Paxos. Paxos was the original, famous consensus algorithm, and for years it was the default answer in textbooks. It is proven correct and it works. However, it carries a reputation for being painfully hard to understand.

Many engineers read the Paxos paper, nod along, and still cannot explain how to build a working system from it. The core idea splits into pieces that feel disconnected, and real systems need extra machinery that the basic description leaves out. We will not walk through its proofs here, and honestly, you rarely need them on the job.

This gap is exactly why Raft was created. Diego Ongaro and John Ousterhout wrote a paper with a bold title: “In Search of an Understandable Consensus Algorithm.” Their main design goal was understandability itself. They wanted an algorithm that a normal engineer could read once, picture clearly, and implement correctly. That focus is why we spend the rest of this article on Raft.

5. Raft in a Nutshell

Raft takes one scary problem and splits it into three smaller problems. This split is the real genius of the design, because each piece is easy to reason about on its own.

• Leader election: the cluster picks exactly one leader to be in charge.
• Log replication: the leader accepts commands and copies them to the followers.
• Safety: the rules that make sure a committed entry is never lost or changed.

Everything in Raft flows in one direction, from the leader to the followers. There is no confusing back-and-forth where any server can change the shared log. This strong-leader style keeps the mental model clean. First we elect a boss, then the boss keeps everyone in sync, and a few safety rules protect us when the boss changes.

6. Server States: Follower, Candidate, Leader

At any moment, every server in a Raft cluster sits in one of three roles. Understanding these three roles is half the battle, so let us go through each one slowly.

6.1 Follower

A follower is passive and calm. It does not start anything on its own. Instead, it waits for messages from the leader and replies to them. When a server first boots up, it begins as a follower. In normal times, almost every server is a follower, quietly copying whatever the leader sends.

6.2 Candidate

A candidate is a server that wants the top job. If a follower stops hearing from the leader, it suspects the leader has died. So it promotes itself to candidate and asks the others to vote for it. This role is temporary, a short phase that exists only during an election.

6.3 Leader

The leader runs the show. It handles all client requests, writes new entries to its log, and pushes those entries out to the followers. A healthy cluster has exactly one leader at a time. To keep its position, the leader sends out steady signals so that followers know it is still alive and in charge.

7. Terms: Raft’s Logical Clock

Before we watch an election, we need one more idea: the term. Raft chops time into numbered periods called terms. Term 1, term 2, term 3, and so on. Each term begins with an election, and each term has at most one leader.

A term acts like a simple clock, but it counts events, not seconds. Every server remembers the current term number. Whenever servers talk to each other, they include this number in their messages. That small habit does a lot of useful work.

• If a server sees a higher term than its own, it knows its information is stale and updates itself.
• If a message carries an older term, the receiver rejects it as out of date.
• A deposed leader that wakes up late quickly learns a newer term exists and steps down.

One detail surprises beginners. A term can end with no leader at all. If an election fails to produce a winner, that term simply has no leader, and the cluster moves on to the next term and tries again. Terms are cheap, so burning one on a failed election is not a problem.

8. Leader Election Step by Step

Now for the part everyone asks about in interviews. Leader election is how a leaderless cluster, or one whose leader just crashed, picks a new boss. Let us walk through it as a short story.

8.1 When an Election Starts

Each follower runs a countdown timer called the election timeout. Every heartbeat from the leader resets this timer. As long as heartbeats keep arriving, the follower stays happy and does nothing. But if the timer runs out with no word from the leader, the follower assumes the worst and acts.

Follower election-timeout logic (pseudocode)

on each tick:
    if time_since_last_heartbeat > election_timeout:
        current_term = current_term + 1   // move to a new term
        state = CANDIDATE
        vote_for(self)                    // candidate votes for itself
        send RequestVote to all other servers

8.2 Casting and Counting Votes

The candidate increments the term, votes for itself, and sends a RequestVote message to every other server. Each server may vote only once per term, on a first-come, first-served basis. When a server receives a vote request, it says yes if it has not already voted for someone else in that term.

The candidate then simply counts replies. The moment it collects votes from a majority of the cluster, it wins. There is no arguing and no ranking of candidates. A plain majority is enough to settle the matter.

8.3 The Three Possible Outcomes

An election can end in exactly three ways, and a good candidate must handle all of them.

• It wins: it gathers a majority of votes, becomes the leader, and immediately starts sending heartbeats.
• It loses: another server wins first and sends a heartbeat, so this candidate steps back down to follower.
• No winner: the timeout passes with no majority, so the candidate bumps the term and tries a fresh election.

8.4 Split Votes and Randomized Timeouts

That third outcome has a name: the split vote. Suppose two followers time out at the same instant. Both become candidates, both ask for votes, and the cluster splits its support. Neither reaches a majority, so nobody wins, and the term is wasted.

Raft solves this with a clever, low-tech trick. Each server picks its election timeout randomly from a range, often between 150 and 300 milliseconds. Because the timers differ, one follower almost always wakes up first. It grabs votes before the others even start, and the split vote rarely happens twice in a row.

9. Heartbeats: How a Leader Stays in Charge

Once elected, a leader must defend its position. It does this with heartbeats. A heartbeat is just an AppendEntries message that carries no new log data. The leader sends these pings to every follower at a steady, rapid interval.

Each heartbeat resets the followers’ election timers. So as long as the leader stays alive and the network works, no follower ever times out, and no new election begins. The heartbeat interval is much shorter than the election timeout, which gives the leader plenty of room to prove it is alive before anyone panics.

Notice the neat design choice here. One message type, AppendEntries, does two jobs. It carries real log entries when there is work to do, and it serves as a heartbeat when there is nothing new. Fewer message types mean fewer bugs, which fits Raft’s whole spirit of simplicity.

10. Log Replication: Keeping Everyone in Sync

Electing a leader is only half the story. The real purpose of the cluster is to agree on a growing list of commands. This is log replication, and the leader drives the whole process.

10.1 The Happy Path

Here is what happens when things go well. A client sends a command to the leader, such as “set x to 5”. The leader does not apply it right away. Instead, it follows a careful sequence that protects the data.

• The leader appends the command to its own log as a new entry.
• It sends that entry to all followers using AppendEntries messages, in parallel.
• Each follower writes the entry to its own log and replies with an acknowledgement.
• Once a majority has stored the entry, the leader marks it as committed.
• The leader applies the command to its state machine and replies to the client.

Each log entry holds three small things: an index, the term in which it was created, and the actual command. The index gives the entry its position in the list. The term stamps it with a time period. Together, these let any two servers compare logs and spot differences instantly.

10.2 What Committed Really Means

The word committed is the key to the whole design. An entry is committed once the leader has safely stored it on a majority of servers. That majority is the point of no return. After it, the entry is permanent, and even a leader crash cannot erase it.

Commit rule (pseudocode)

entry is committed when:
    the entry is stored on a MAJORITY of servers
    AND it was created in the leader's current term

only committed entries are applied to the state machine

This is why a client only gets a success reply after a commit. The system never confirms a write it might later lose. If the leader dies one millisecond after committing, the data still lives on a majority, and the next leader will have it. That promise is what makes the system trustworthy.

10.3 Followers That Fall Behind

Networks are messy, so followers often miss some entries. Maybe a follower was slow, or restarted, or briefly lost its connection. The leader handles this quietly. It tracks how far each follower has progressed and keeps retrying older entries until that follower catches up.

If a follower’s log ever conflicts with the leader’s, the leader wins. The follower throws away its mismatched entries and copies the leader’s version instead. This sounds harsh, but it keeps the rule dead simple. There is one source of truth, the leader’s log, and everyone eventually matches it.

11. Safety: Why a Stale Node Cannot Lead

We now reach the subtle part, but do not worry, the intuition is friendly. Imagine a leader commits an entry and then crashes. What if a server that never received that entry becomes the next leader? It would lead with a hole in its history, and the committed entry would vanish. Raft must stop this, and it does, with one clever voting rule.

11.1 The Up-to-Date Log Rule

When a candidate asks for a vote, it includes details about its own log. Each voter then checks a simple condition before saying yes. A server grants its vote only if the candidate’s log is at least as up to date as its own.

“Up to date” has a precise meaning here, and it is easy to remember. The voter compares the term of the last log entry first. A higher term wins. If the terms match, the longer log, meaning the one with the higher index, wins. A candidate that is behind simply cannot collect a majority.

11.2 Leader Completeness

Now connect the two ideas. A committed entry lives on a majority of servers. To win an election, a candidate also needs a majority of votes. Those two majorities must overlap in at least one server, and that server holds the committed entry. It will refuse to vote for anyone missing that entry.

So the winner always has every committed entry already. This guarantee has a name, leader completeness, and it is the quiet hero of Raft. Because of it, a committed entry can never be lost, no matter how many leaders come and go. You do not need the formal proof to trust the picture.

12. A Beginner Walkthrough: Leader Crash and Recovery

Let us tie everything together with one concrete story. Say we have five servers, named S1 through S5, running happily in term 4. S1 is the leader, and the other four are followers.

First, the normal flow. A client asks to store a value. S1 appends it to its log and sends it to the four followers. Three of them reply quickly, so S1 now has itself plus three copies, a majority of five. S1 commits the entry, applies it, and tells the client “done”. So far, so good.

Now disaster strikes. S1 crashes and goes silent. The four followers wait for heartbeats, but none arrive. Their election timers tick down, and thanks to randomization, S3 times out first. It jumps to term 5, votes for itself, and asks the others for support.

Here is where safety shines. S3 had copied that last committed entry, so its log is current. The other followers check S3’s log, find it up to date, and happily vote yes. S3 collects a majority, becomes the new leader for term 5, and starts sending heartbeats. The cluster recovers in a fraction of a second.

Finally, S1 wakes up and still thinks it is the leader of term 4. It tries to send an old heartbeat. But every server now lives in term 5, so they reject the stale message and tell S1 the new term. S1 realises it is outdated, steps down to follower, and syncs up. No data was lost, and no two leaders ever ran at once.

13. Where Raft Is Used in the Real World

Raft is not just a classroom idea. It quietly powers a lot of the infrastructure that modern systems depend on. Knowing a few real examples makes the concept stick, and it also impresses interviewers who like practical grounding.

• etcd, the key-value store behind Kubernetes, uses Raft to keep cluster state consistent.
• Consul, a service discovery and configuration tool, relies on Raft for its server agreement.
• CockroachDB and TiKV, two distributed databases, use Raft to replicate data across nodes.
• Modern Kafka uses a Raft-based mode to manage its own metadata without a separate coordinator.

Notice the common thread. Each of these systems must hold some critical state that cannot afford to disagree, such as configuration, cluster membership, or metadata. Raft gives them a dependable, well-understood way to keep that state safe across failures. When you need a group of servers to agree without drama, Raft is usually the tool people reach for.

14. Common Mistakes, Edge Cases, and Interview Insights

Beginners tend to trip over the same few points. Spotting them early will save you both bugs and interview stumbles.

• Confusing leadership with correctness. The leader drives progress, but the majority quorum is what actually makes a decision safe.
• Forgetting the odd-node rule. An even cluster size wastes a machine and gives no extra failure tolerance.
• Thinking a write is safe once the leader has it. A write is only safe after a majority stores it and the leader commits it.
• Assuming a term always has a leader. A split vote can leave a term empty, and that is perfectly normal.
• Ignoring the up-to-date log rule. Without it, a stale server could win and erase committed data.
• Setting timeouts carelessly. Too short, and you trigger needless elections; too long, and recovery after a crash feels slow.

14.1 Interview Insights

System design interviews reward clear stories over memorised definitions. When consensus comes up, start with the problem. Explain that many servers must agree despite failures, and that Raft solves this by electing a leader and replicating a log.

Next, structure your answer around Raft’s three pieces: leader election, log replication, and safety. This framing alone signals real understanding. If asked to go deeper, describe the election timeout, the majority vote, and the role of heartbeats, in that order.

Finally, be ready for the “why not Paxos” question. Keep it gracious and short. Say that Paxos is correct but famously hard to understand, while Raft was designed for clarity, which makes it easier to build and operate. That answer shows maturity without starting a holy war.

15. Practical Takeaways and Key Terms

Let us turn all this theory into a few habits you can carry into real design work.

• Reach for an odd cluster size, usually three or five, to balance safety and cost.
• Remember that every safe decision rests on a majority, never on a single server.
• Treat the leader as a convenience for ordering, not as a single point of truth.
• Tune election and heartbeat timeouts with care, since they shape both stability and recovery speed.
• When you need agreement across machines, lean on a proven tool like etcd rather than rolling your own.

15.1 Key Terms Recap

16. FAQ’s on consensus in distributed systems

17. Conclusion

Consensus feels intimidating from the outside, but the core idea is calm and simple. A group of servers must agree even when some of them fail. Raft reaches that goal by electing one leader, replicating a shared log, and guarding a few safety rules. Break the problem into those three pieces, and the fog clears.

Keep the key pictures in mind. Terms act as a clock, a majority makes every decision real, randomized timeouts settle elections, and the up-to-date log rule protects committed data. None of this needs heavy mathematics. It needs a clear model and a bit of practice.

Carry this intuition into your next design discussion. When someone asks how your system stays consistent across machines, you can now answer with a story instead of a shrug. Master these basics, and the deeper distributed systems topics ahead will feel far less scary.

18. Further Reading

• CAP Theorem and PACELC in System Design
• Database Replication in System Design: Leader-Follower, Sync vs Async
• Distributed Locking Explained: Redis Redlock and ZooKeeper
• The Raft Consensus Algorithm (official site with visual demo)
• In Search of an Understandable Consensus Algorithm (the Raft paper)