Caching Strategies in System Design: Patterns and Eviction Policies
-
Last Updated: July 18, 2026
-
By: javahandson
-
Series
Learn Java in a easy way
Caching strategies in system design explained: cache-aside, write-through and write-behind patterns, plus LRU, LFU and TTL eviction, with clear examples.
Your API feels fast on your laptop. Then it goes live. Suddenly the same endpoint takes 400 milliseconds, and your database CPU sits at 90 percent all day. Nothing in your code changed. Only the traffic did.
This is where caching strategies enter the picture. A cache is a small, fast store that sits in front of a slow one. You keep a copy of hot data close to the application, so most requests never touch the database at all. Done well, caching turns a struggling service into a calm one. Done badly, it serves stale data and hides bugs for weeks.
This article covers the first half of caching: the read and write patterns you will actually use, and the eviction policies that decide what gets thrown out when memory runs low. We will keep the language simple and back every idea with small, language-neutral examples in pseudocode, so the ideas carry over to any stack. Distributed caching, cache internals, and invalidation at scale come later, in part two.
By the end, you will know when to reach for cache-aside, when write-through earns its keep, why write-behind is both powerful and dangerous, and how LRU, LFU, and friends actually behave under real traffic.
Remember the latency numbers. Reading from memory takes around 100 nanoseconds. A disk seek costs roughly 10 milliseconds. That is a gap of about 100,000 times. A round trip to a database in the same datacenter costs a few hundred microseconds at best, and far more once the query itself runs.
So caching is not a small tweak. It moves work across a whole order of magnitude. Take a product page that runs six queries and takes 120 ms. Cache the finished result in memory, and the same page answers in under 2 ms. That is 60 times faster, and you changed no business logic at all.
There is a second benefit people forget. A cache protects the database. Most systems are read-heavy, often by 100 to 1. Now do the sums on 100,000 reads per second:
That is the difference between one database and a sharded cluster. The cache did not just make things faster. It changed the shape of the whole system.
Every caching decision comes down to two simple questions. First, how does data get into the cache and back out to the database? That is the caching pattern. Second, what do we remove when the cache is full? That is the eviction policy. Everything else is detail.

Before patterns, learn the vocabulary. It shows up in every design discussion and every interview.
Hit ratio is the number to watch. It tells you whether the cache is doing its job. Below 70 percent, something is usually wrong: your keys are too specific, your TTL is too short, or your cache is too small for the working set.
The maths makes it real. Say a hit costs 1 ms and a miss costs 100 ms. Work out the average wait:
Look at what happened. The cache did not get slower. Every hit still took 1 ms. It just stopped covering enough of the traffic, and the misses dragged the average up five times.
// A tiny hit-ratio tracker you can wrap around any lookup (pseudocode).
hits = 0
misses = 0
function recordHit(): hits = hits + 1
function recordMiss(): misses = misses + 1
function hitRatio():
total = hits + misses
if total == 0:
return 0
return hits / totalIn production you would not hand-roll this. Most cache libraries and cache servers expose hit statistics already. But the shape of the calculation is worth understanding, because you will be asked to reason about it.
A caching pattern answers one question: who is responsible for keeping the cache and the database in step? Three patterns cover almost everything you will build. Each one moves that responsibility somewhere different.
| Pattern | Who writes to the cache? | Who writes to the DB? |
|---|---|---|
| Cache-aside | Application, after a miss | Application, directly |
| Write-through | Cache layer, on every write | Cache layer, synchronously |
| Write-behind | Cache layer, on every write | Cache layer, later and async |
There are read-side variants too, such as read-through, where the cache itself fetches from the database on a miss. We will touch on that as we go, since it pairs naturally with write-through.
Cache-aside is the pattern you will meet first, and probably the one you will use most. It is sometimes called lazy loading, because data only enters the cache when somebody asks for it.
The flow is simple. The application checks the cache. If the data is there, return it. If not, read from the database, put a copy in the cache, and return it. The cache never talks to the database. The application coordinates everything.

// Cache-aside read path (pseudocode).
function getProduct(id):
key = "product:" + id
# 1. Look in the cache first.
cached = cache.get(key)
if cached is not null:
return cached # cache HIT
# 2. MISS: go to the database.
product = database.findProduct(id)
if product is null:
throw NotFound(id)
# 3. Populate the cache for next time, with a TTL.
cache.set(key, product, ttl = 10 minutes)
return productNotice the TTL on the write. Never cache without one unless you have a very good reason. A TTL is your safety net. If invalidation logic breaks somewhere, stale entries still expire on their own instead of living forever.
Concepts stick better with a real sequence. Say product 42 is a popular phone, and the TTL is 10 minutes. Three users load its page.
So one slow read paid for thousands of fast ones. If 5,000 people viewed that page in those ten minutes, the database served 1 query instead of 5,000. That single number is why cache-aside is worth the effort.
Reads are the easy half. Writes are where teams get into trouble. When product data changes, the cached copy becomes wrong. You have two choices: update the cache, or delete the key.
Delete the key. This is the safer default, and it has a name: write-invalidate. The next read will simply miss, fetch fresh data, and repopulate. Updating the cache in place looks tempting, but it opens a race condition. Two concurrent writers can interleave, and the cache ends up holding the older value while the database holds the newer one.
// Cache-aside write path (pseudocode).
function updateProduct(id, changes):
# 1. Write to the database first.
saved = database.updateProduct(id, changes)
# 2. Delete the key, do NOT update it in place.
# The next read will repopulate from the database.
cache.delete("product:" + id)
return savedOrder matters here too. Write to the database first, then delete the cache key. If you delete first and the database write then fails, you have evicted a perfectly good entry for nothing. Worse, a concurrent read could repopulate the cache from the old row in the gap between your delete and your write.
| Interview Insight: The Thundering Herd Interviewers love this follow-up. Ask: what happens when a very popular key expires? Every request misses at the same instant, and thousands of threads hit the database with the identical query. That is a thundering herd, also called a cache stampede. Two answers score well. First, use a lock or a single-flight guard, so only one thread rebuilds the entry while the rest wait. Second, add jitter to your TTLs, so a thousand keys written together do not all expire in the same second. Mentioning jitter unprompted signals real production experience. It is a small detail that only bites you once you have been on call. |
Write-through flips the responsibility. In cache-aside, your app talks to both stores and keeps them in step by hand. In write-through, the cache sits in the write path. Your app does not write to the database at all. It writes to the cache, and the cache forwards that write to the database before saying “done”.
Put the two side by side and the difference jumps out.
CACHE-ASIDE (write):
App ---- write ----> Database
App ---- delete ---> Cache (throw the stale copy away)
The app talks to both. It is the coordinator.
WRITE-THROUGH:
App ---- write ----> Cache ---- write ----> Database
<---- ok --------
<---- ok --------
The app talks to one thing. The cache handles the database itself.The name is literal. A write passes through the cache on its way to the database. Think of the cache not as a side copy you maintain separately, but as a station the write travels through.
That one detail explains everything else about the pattern. Because the cache wrote the value on its way past, the cache now holds the newest data. Always. It can never be stale, because there is no moment where the database has something the cache does not.
So the whole point of write-through is this: the cache and the database can never disagree.
Write-through usually travels with read-through. On a miss, the cache loads from the database itself, so your app never queries the database directly. Together they hide the database completely behind the cache layer.

Follow a real sequence. A user changes their display name to “Suraj K”.
| Time | What happens | Cost |
|---|---|---|
| 12:00:00.000 | App calls saveUser(user) | — |
| 12:00:00.001 | Cache writes to the database, then waits | 20 ms |
| 12:00:00.021 | Database confirms the write | — |
| 12:00:00.022 | Cache stores the object under users:7 | 1 ms |
| 12:00:00.023 | Cache returns ok to the app | — |
Total: 21 ms. The user waited for both stores, not one.
Now they open their profile one second later. The cache already holds users:7, so it is a HIT. That costs 1 ms and it shows “Suraj K”. Correct, and fast.
Run that exact same story under cache-aside and watch what goes wrong.
Both patterns gave the right answer. But cache-aside threw the data away and then immediately had to fetch it back. Write-through kept it, because it was already holding it.
That is the whole case for write-through. When a read follows the write closely, the cache-aside delete is wasteful. The write-through keep is not.
Nothing is free, and write-through charges you twice.
First, every write is slower. You now wait for two systems instead of one. Our 20 ms database write became a 21 ms save. Under cache-aside the write is just 20 ms, and the delete is fire-and-forget.
Second, and this is the bigger one, it caches things nobody reads. Write-through caches everything written, whether it is ever read back or not.
Picture an audit log. You write 5 million rows a day and almost nobody reads them. Under write-through, all 5 million land in your cache. They fill memory and evict the product data people actually are asking for. Your cache is now working against you.
Ask yourself one question. After this write, will someone read this soon?
Pair it with a TTL anyway, so any unread entries eventually drain out on their own.
In pseudocode, the pattern is just two functions. A read that fills the cache on a miss, and a write that updates both stores together.
// Write-through (with read-through) in pseudocode.
# READ-THROUGH: on a miss, load from the DB and cache it.
function getUser(id):
key = "user:" + id
cached = cache.get(key)
if cached is not null:
return cached # HIT: skip the database
user = database.findUser(id) # MISS: load it
cache.set(key, user, ttl = 30 minutes)
return user
# WRITE-THROUGH: write to the DB, then refresh the cache.
function saveUser(user):
database.saveUser(user) # 1. database first, and wait
cache.set("user:" + user.id, user, ttl = 30 minutes) # 2. refresh
return user
# On delete, remove from both stores.
function deleteUser(id):
database.deleteUser(id)
cache.delete("user:" + id)One distinction trips people up here, so it is worth stating plainly. Most cache libraries give you two different behaviours, and mixing them up quietly breaks things.
So the read path checks the cache first and only falls through on a miss. The write path never short-circuits. It always touches the database, then updates the cache. Getting these backwards is a classic bug: a read that always queries, or a write that sometimes never persists.
Write-behind takes the last step and makes it asynchronous. The application writes to the cache, and the cache says “done” immediately. The database write happens later, in the background, often batched with other writes.
Speed is the payoff here. Your write latency becomes cache latency, which might be under a millisecond. Better still, the cache can batch a hundred updates into one database round trip, and it can collapse repeated writes to the same key into a single final write.
Think about a view counter on a video. Ten thousand views per minute means ten thousand database updates under write-through. Under write-behind, you increment a number in memory and flush once every few seconds. The database sees a handful of writes instead of thousands.
// A simplified write-behind buffer for view counts (pseudocode).
# An in-memory map of videoId -> pending count.
pending = {} # shared, thread-safe map
# FAST PATH: the caller never waits for the database.
function recordView(videoId):
pending[videoId] = pending.get(videoId, 0) + 1
return # done in under 1 ms
# Runs on a timer every 5 seconds, in the background.
function flush():
# Take a snapshot and clear the buffer.
batch = pending.drain() # atomically remove all entries
for (videoId, count) in batch:
if count > 0:
database.incrementViews(videoId, count) # ONE write per videoNumbers make this pattern obvious. A video is trending and gets 10,000 views in one minute. The flush runs every 5 seconds.
Under write-through, every single view is a database write. That is 10,000 UPDATE statements in 60 seconds, roughly 167 per second, all fighting for a lock on the same row.
Under write-behind, the story changes completely.
So the database load dropped by more than 800 times, and the users got faster responses too. The counter is briefly behind the truth, by at most 5 seconds. For a view count, nobody notices and nobody cares.
Write-behind has one serious weakness, and you should say it out loud before anyone asks. If the cache node dies before the flush, those writes are gone. There is no database copy to fall back on, because the database never saw them.
That is why the pattern fits view counts, likes, and analytics events. Losing five seconds of view counts is annoying. Losing five seconds of payment records is a lawsuit. Never put money or orders behind an unreplicated write-behind buffer.
Real systems reduce the risk rather than remove it. They replicate the buffer, or they persist pending writes to an append-only log first, or they shorten the flush interval. Each of those choices trades some of the speed back for durability.
| Interview Insight: Choosing a Pattern Out Loud Do not name a pattern and stop. Interviewers want the reasoning, and the reasoning is always about the read-to-write ratio and the cost of stale or lost data. A strong answer sounds like this. “Product catalogue is read-heavy and tolerates a few seconds of staleness, so I would use cache-aside with a short TTL. User sessions are read right after every write, so write-through fits better. View counters are write-heavy and cheap to lose, so write-behind with a five-second flush saves the database.” That single paragraph shows you can map data characteristics onto a pattern. It matters more than knowing any specific cache product. |
Here is the whole picture in one place. Keep this table in your head for design discussions.
| Aspect | Cache-Aside | Write-Through | Write-Behind |
|---|---|---|---|
| Write latency | Database speed | Database plus cache | Cache speed only |
| Consistency | Eventually consistent | Strong between the two | Weakest; DB lags |
| Risk of data loss | None | None | Real, if the cache dies |
| Cold start | Slow, misses at first | Slow for unwritten data | Slow for unwritten data |
| Cache pollution | Low; only read data | High; caches all writes | High; caches all writes |
| Complexity | Low, but in your code | Moderate | High |
| Best for | Read-heavy catalogues | Read-after-write data | Write-heavy counters |
One last note. These patterns combine. A common production setup is cache-aside for reads plus write-behind for counters, sitting on the same Redis cluster. Nobody says you must pick one for the entire system.
Caches are small on purpose. Memory costs far more than disk, so your cache holds a fraction of your data. Sooner or later it fills up, and something must go. The rule that picks the victim is the eviction policy.
The goal of every policy is the same: keep the entries that will be read again, and drop the ones that will not. Since none of them can see the future, they all guess using the past. They just guess differently.

LRU evicts whatever has gone untouched the longest. The idea behind it is called temporal locality. In plain words: if you read something a moment ago, you will probably read it again soon.
That guess holds up well in practice. Think about the items that stay hot in a real system.
In all three cases, recent equals popular. LRU captures that for free, without counting anything. So it is the default in most cache libraries, the common choice for cache servers, and the sensible starting point for most application caches.
Say the cache holds 3 entries and is already full with A, B and C. Here is what happens as requests arrive. The list on the right shows most recent first.
| Step | Action | Cache after the step | Evicted |
|---|---|---|---|
| 1 | Start (full) | C, B, A | — |
| 2 | read A | A, C, B | — |
| 3 | read C | C, A, B | — |
| 4 | write D (cache full) | D, C, A | B |
| 5 | read B | B, D, C | A |
Follow step 4. B was the coldest, because nobody had touched it since the start. So B loses its place to D. Then in step 5, B comes back, and now A is the coldest one. Notice that a read is not free: every read reshuffles the order and changes who dies next.
An LRU cache needs to do two things fast: look up a value by key, and know which entry is the coldest. A hash map gives you the first. A doubly linked list, ordered from coldest to hottest, gives you the second. Put them together and both operations are constant time.
The idea is simple once you picture it. The map finds an entry instantly. Ordering lives in the list, which remembers when each entry was last used. Every time you touch an entry, you move it to the hot end. When the cache is full, you drop whatever sits at the cold end.
// LRU cache: a hash map plus a doubly linked list (pseudocode).
#
# map: key -> node (for O(1) lookup)
# list: coldest <-> ... <-> hottest (for O(1) eviction)
function get(key):
if key not in map:
return null # MISS
node = map[key]
list.moveToHot(node) # mark it as recently used
return node.value # HIT
function put(key, value):
if key in map:
node = map[key]
node.value = value
list.moveToHot(node)
return
if map.size == capacity: # cache is full
coldest = list.removeColdest() # evict least recently used
map.remove(coldest.key)
node = list.addAtHot(key, value) # insert as most recently used
map[key] = nodeRead the two operations slowly, because the moves are the whole point.
Notice that a get is not read-only here. Every read reshuffles the order and changes who dies next. That surprises people the first time they see it, but it is exactly what “recently used” means.
In real code you almost never hand-build an LRU. Two easier paths exist, and both are worth knowing.
Whichever route you take, watch out for concurrency. A cache is hit by many requests at once, so the structure underneath must be thread safe. Hand-rolled maps usually are not. Good cache libraries handle the locking for you, which is one more reason to prefer them over your own code.
LRU has one classic failure, and it is worth knowing before it bites you.
The job did its work, but it wiped your cache on the way through. Real caches fight this with a small admission window, which forces a new entry to prove it will be reused before it can push out an established one. The better cache libraries do this automatically, as we will see shortly.
LFU counts accesses and evicts the entry with the lowest count. The two policies ask different questions:
That difference matters when traffic is skewed. Picture a shop with one bestseller and a thousand slow movers. The bestseller is read 500 times an hour. The rest are read once a month.
But LFU has a memory problem. Think about last year Black Friday deal. It was read a million times back then, so its count is huge. Nobody has opened it since. Yet it sits in your cache forever, because no new entry can ever out-count it.
The fix is to forget slowly. Two ways work:
Most real caches do this for you. Some servers offer an LFU policy that decays counts over time, and some libraries use a windowed variant such as TinyLFU under the hood. Either way, the lesson is the same: raw LFU without decay is a trap.
// A typical cache configuration (pseudocode).
cache = newCache()
.maxSize(10_000) # evict once it holds 10,000 entries
.expireAfterWrite(10 minutes) # and expire each entry by TTL
.recordStats() # so you can read the hit ratio
.build()
# Good libraries do not use plain LRU or plain LFU. They combine
# them: a small recency window in front of a frequency-based main
# region. You get LRU's handling of recent items and LFU's
# resistance to one-off scans, without tuning either by hand.FIFO evicts the oldest entry by insertion time, and it completely ignores how often you read it. It is the simplest policy to build, and it is almost always the wrong choice for a read cache.
The problem is easy to see. Your homepage banner was cached when the app started. It is read 1,000 times a second. It was also the first thing inserted. So FIFO evicts it first, and it does not care about those 1,000 reads at all.
FIFO throws away exactly what you wanted to keep.
Still, it has honest uses. Queues, buffers, and anything where order genuinely defines value. Just do not reach for it to cache products.
TTL is not really an eviction policy. It is an expiry rule that runs alongside one. Each entry gets a lifespan, and once that lifespan ends, the entry is invalid whether or not the cache is full.
Use a TTL everywhere. It is the cheapest insurance you can buy, because it puts a limit on how wrong your cache can get. Say a new developer adds a write path and forgets to delete the cache key. That is a real bug. But with a 10 minute TTL, the stale data lasts 10 minutes instead of forever.
Think of it this way. Invalidation is your plan. TTL is what saves you when the plan has a hole in it.
Random eviction sounds silly, but it is real. Redis offers it, and it costs almost nothing to run because there is no bookkeeping. Under uniform access patterns it performs nearly as well as LRU. Under skewed patterns, which is most real traffic, it performs noticeably worse.
| Policy | Evicts | Best for | Weakness |
|---|---|---|---|
| LRU | The least recently used entry | General purpose, time-clustered reads | Scans wipe out hot data |
| LFU | The least frequently used entry | Skewed traffic with clear favourites | Old popularity lingers |
| FIFO | The oldest inserted entry | Queues and buffers | Ignores access entirely |
| Random | Any entry, at random | Cheap, uniform access | Poor on skewed traffic |
| TTL | Anything past its lifespan | Bounding staleness | Not a capacity policy |
Most cache servers make this a single configuration setting. You pick the policy by name, and common choices include an LRU mode, an LFU mode, a variant that only evicts keys carrying a TTL, and a mode that refuses to evict at all and instead rejects new writes. That last pair is handy when some entries must never disappear on their own.
| Interview Insight: Picking an Eviction Policy The expected answer is LRU, and it is usually right. What separates a strong candidate is naming the failure case unprompted. Say this: “I would start with LRU, since most access patterns cluster in time. But if there is a nightly batch job scanning the whole table, LRU gets wiped by the scan. Then I would move to a cache that combines recency and frequency, which resists exactly that kind of scan.” If they push on LFU, mention count decay. An LFU cache without decay preserves last year is winners forever, and that is the flaw the follow-up question is hunting for. |
Patterns and policies are independent choices, and that is easy to miss. You can run cache-aside with LRU, or cache-aside with LFU, or write-through with either. The pattern decides how data flows. The policy decides what survives when memory runs out.
Here is how the two decisions play out on a real product catalogue. Reads dominate at roughly 100 to 1. A price change can take a few seconds to appear. Losing the cache entirely must not break checkout.
Now change one requirement. Say the checkout flow reads a cart immediately after writing it. Cache-aside would work, but a miss right after a write is wasteful and occasionally stale. Write-through fits better, because the write and the cache update happen together.
That is the whole skill. Read the data characteristics, then pick. Nobody memorises a matrix. They ask two questions about the traffic and derive the answer each time.
These are the ones that show up in real code reviews and real incidents. Each one has a simple fix.
A few habits will carry you through most caching work, whatever your stack.
| Term | Meaning in One Line |
|---|---|
| Cache hit | The data was found in the cache |
| Cache miss | The data was absent, so the database was queried |
| Hit ratio | Hits divided by total lookups; the health metric of a cache |
| Cache-aside | The application loads and invalidates the cache itself |
| Read-through | The cache loads from the database on a miss |
| Write-through | The cache writes to the database synchronously |
| Write-behind | The cache writes to the database later, asynchronously |
| Eviction policy | The rule that picks which entry to remove when full |
| LRU | Evicts the entry untouched for the longest time |
| LFU | Evicts the entry with the fewest accesses |
| TTL | A lifespan after which an entry expires |
| Thundering herd | Many requests missing on the same key at once |
Caching strategies come down to two decisions, and both are simpler than they first look. The pattern decides who keeps the cache and the database in agreement. The policy decides what gets sacrificed when memory runs out.
Cache-aside puts you in control and fails gracefully, which is why it is the sensible default. Write-through buys consistency with latency. Write-behind buys speed with risk, and that risk is only acceptable for data you can afford to lose. On the eviction side, LRU is the honest starting point, LFU handles skew, and TTL is the safety net you should never skip.
Try it on your next project. Add a cache to one hot read path, set a TTL with jitter, turn on statistics, and watch the hit ratio for a week. You will learn more from that one number than from any article, including this one. Part two picks up where this leaves off: distributed caching, running a cache at scale across many machines, and the genuinely hard problem of invalidation across many nodes.