Caching Strategies in System Design: Patterns and Eviction Policies

  • Last Updated: July 18, 2026
  • By: javahandson
  • Series
img

Caching Strategies in System Design: Patterns and Eviction Policies

Caching strategies in system design explained: cache-aside, write-through and write-behind patterns, plus LRU, LFU and TTL eviction, with clear examples.

1. Introduction

Your API feels fast on your laptop. Then it goes live. Suddenly the same endpoint takes 400 milliseconds, and your database CPU sits at 90 percent all day. Nothing in your code changed. Only the traffic did.

This is where caching strategies enter the picture. A cache is a small, fast store that sits in front of a slow one. You keep a copy of hot data close to the application, so most requests never touch the database at all. Done well, caching turns a struggling service into a calm one. Done badly, it serves stale data and hides bugs for weeks.

This article covers the first half of caching: the read and write patterns you will actually use, and the eviction policies that decide what gets thrown out when memory runs low. We will keep the language simple and back every idea with small, language-neutral examples in pseudocode, so the ideas carry over to any stack. Distributed caching, cache internals, and invalidation at scale come later, in part two.

By the end, you will know when to reach for cache-aside, when write-through earns its keep, why write-behind is both powerful and dangerous, and how LRU, LFU, and friends actually behave under real traffic.

2. Why Caching Matters So Much

Remember the latency numbers. Reading from memory takes around 100 nanoseconds. A disk seek costs roughly 10 milliseconds. That is a gap of about 100,000 times. A round trip to a database in the same datacenter costs a few hundred microseconds at best, and far more once the query itself runs.

So caching is not a small tweak. It moves work across a whole order of magnitude. Take a product page that runs six queries and takes 120 ms. Cache the finished result in memory, and the same page answers in under 2 ms. That is 60 times faster, and you changed no business logic at all.

There is a second benefit people forget. A cache protects the database. Most systems are read-heavy, often by 100 to 1. Now do the sums on 100,000 reads per second:

  • With no cache, the database handles all 100,000.
  • With a cache at a 95 percent hit ratio, it handles 5,000.

That is the difference between one database and a sharded cluster. The cache did not just make things faster. It changed the shape of the whole system.

2.1 The Two Questions Every Cache Must Answer

Every caching decision comes down to two simple questions. First, how does data get into the cache and back out to the database? That is the caching pattern. Second, what do we remove when the cache is full? That is the eviction policy. Everything else is detail.

Diagram showing a client request hitting the cache first, with a cache hit returning immediately and a cache miss falling through to the database

3. Cache Hits, Misses, and Hit Ratio

Before patterns, learn the vocabulary. It shows up in every design discussion and every interview.

  • Cache hit — the data was in the cache. The request is fast, and the database stays untouched.
  • Cache miss — the data was not there. Someone must fetch it from the database, which costs time.
  • Hit ratio — hits divided by total lookups. A cache with 900 hits out of 1,000 lookups has a 90 percent hit ratio.

Hit ratio is the number to watch. It tells you whether the cache is doing its job. Below 70 percent, something is usually wrong: your keys are too specific, your TTL is too short, or your cache is too small for the working set.

The maths makes it real. Say a hit costs 1 ms and a miss costs 100 ms. Work out the average wait:

  • At 90 percent hits: (0.9 x 1 ms) + (0.1 x 100 ms) = about 11 ms.
  • At 50 percent hits: (0.5 x 1 ms) + (0.5 x 100 ms) = about 50 ms.

Look at what happened. The cache did not get slower. Every hit still took 1 ms. It just stopped covering enough of the traffic, and the misses dragged the average up five times.

// A tiny hit-ratio tracker you can wrap around any lookup (pseudocode).
 
hits   = 0
misses = 0
 
function recordHit():   hits   = hits + 1
function recordMiss():  misses = misses + 1
 
function hitRatio():
    total = hits + misses
    if total == 0:
        return 0
    return hits / total

In production you would not hand-roll this. Most cache libraries and cache servers expose hit statistics already. But the shape of the calculation is worth understanding, because you will be asked to reason about it.

4. Caching Patterns: How Data Moves

A caching pattern answers one question: who is responsible for keeping the cache and the database in step? Three patterns cover almost everything you will build. Each one moves that responsibility somewhere different.

PatternWho writes to the cache?Who writes to the DB?
Cache-asideApplication, after a missApplication, directly
Write-throughCache layer, on every writeCache layer, synchronously
Write-behindCache layer, on every writeCache layer, later and async

There are read-side variants too, such as read-through, where the cache itself fetches from the database on a miss. We will touch on that as we go, since it pairs naturally with write-through.

4.1 Cache-Aside (Lazy Loading)

Cache-aside is the pattern you will meet first, and probably the one you will use most. It is sometimes called lazy loading, because data only enters the cache when somebody asks for it.

The flow is simple. The application checks the cache. If the data is there, return it. If not, read from the database, put a copy in the cache, and return it. The cache never talks to the database. The application coordinates everything.

Sequence diagram of cache-aside showing a read path with cache lookup, database fallback and cache population, plus a write path that updates the database and deletes the cache key
// Cache-aside read path (pseudocode).
 
function getProduct(id):
    key = "product:" + id
 
    # 1. Look in the cache first.
    cached = cache.get(key)
    if cached is not null:
        return cached                 # cache HIT
 
    # 2. MISS: go to the database.
    product = database.findProduct(id)
    if product is null:
        throw NotFound(id)
 
    # 3. Populate the cache for next time, with a TTL.
    cache.set(key, product, ttl = 10 minutes)
    return product

Notice the TTL on the write. Never cache without one unless you have a very good reason. A TTL is your safety net. If invalidation logic breaks somewhere, stale entries still expire on their own instead of living forever.

4.1.1 A Worked Example: Three Users Hit the Same Product

Concepts stick better with a real sequence. Say product 42 is a popular phone, and the TTL is 10 minutes. Three users load its page.

  • User 1 at 10:00:00 — the cache is empty, so this is a MISS. The app queries the database, waits 100 ms, stores the result under product:42, and returns. This user waited 100 ms.
  • User 2 at 10:00:03 — the key is there, so this is a HIT. The database is never touched. This user waited 1 ms.
  • User 3 at 10:04:00 — still a HIT. The entry has 6 minutes of TTL left.
  • At 10:10:00 the TTL expires and the entry vanishes on its own.
  • User 4 at 10:10:01 — MISS again. The app reloads from the database and the cycle repeats.

So one slow read paid for thousands of fast ones. If 5,000 people viewed that page in those ten minutes, the database served 1 query instead of 5,000. That single number is why cache-aside is worth the effort.

4.1.2 Writes in Cache-Aside

Reads are the easy half. Writes are where teams get into trouble. When product data changes, the cached copy becomes wrong. You have two choices: update the cache, or delete the key.

Delete the key. This is the safer default, and it has a name: write-invalidate. The next read will simply miss, fetch fresh data, and repopulate. Updating the cache in place looks tempting, but it opens a race condition. Two concurrent writers can interleave, and the cache ends up holding the older value while the database holds the newer one.

// Cache-aside write path (pseudocode).
 
function updateProduct(id, changes):
    # 1. Write to the database first.
    saved = database.updateProduct(id, changes)
 
    # 2. Delete the key, do NOT update it in place.
    #    The next read will repopulate from the database.
    cache.delete("product:" + id)
    return saved

Order matters here too. Write to the database first, then delete the cache key. If you delete first and the database write then fails, you have evicted a perfectly good entry for nothing. Worse, a concurrent read could repopulate the cache from the old row in the gap between your delete and your write.

4.1.3 Where Cache-Aside Shines and Where It Hurts

  • Good: only requested data gets cached, so memory holds exactly what people actually use.
  • Good: if the cache goes down, the application still works. It just runs slower.
  • Good: the pattern is easy to reason about, and it is explicit in your code.
  • Bad: every first read is a miss, so cold starts feel slow.
  • Bad: cache and database can drift, because two separate systems are being updated by hand.
  • Bad: the invalidation logic lives in your service code, and it is easy to forget one write path.
Interview Insight: The Thundering Herd
Interviewers love this follow-up. Ask: what happens when a very popular key expires? Every request misses at the same instant, and thousands of threads hit the database with the identical query. That is a thundering herd, also called a cache stampede.
Two answers score well. First, use a lock or a single-flight guard, so only one thread rebuilds the entry while the rest wait. Second, add jitter to your TTLs, so a thousand keys written together do not all expire in the same second.
Mentioning jitter unprompted signals real production experience. It is a small detail that only bites you once you have been on call.

4.2 Write-Through

Write-through flips the responsibility. In cache-aside, your app talks to both stores and keeps them in step by hand. In write-through, the cache sits in the write path. Your app does not write to the database at all. It writes to the cache, and the cache forwards that write to the database before saying “done”.

Put the two side by side and the difference jumps out.

CACHE-ASIDE (write):
 
  App  ---- write ----> Database
  App  ---- delete ---> Cache        (throw the stale copy away)
 
  The app talks to both. It is the coordinator.
 
 
WRITE-THROUGH:
 
  App  ---- write ----> Cache  ---- write ----> Database
                              <---- ok --------
       <---- ok --------
 
  The app talks to one thing. The cache handles the database itself.

4.2.1 Why It Is Called “Through”

The name is literal. A write passes through the cache on its way to the database. Think of the cache not as a side copy you maintain separately, but as a station the write travels through.

That one detail explains everything else about the pattern. Because the cache wrote the value on its way past, the cache now holds the newest data. Always. It can never be stale, because there is no moment where the database has something the cache does not.

So the whole point of write-through is this: the cache and the database can never disagree.

Write-through usually travels with read-through. On a miss, the cache loads from the database itself, so your app never queries the database directly. Together they hide the database completely behind the cache layer.

Side-by-side diagram comparing write-through, where the cache writes to the database synchronously, with write-behind, where writes are queued and flushed asynchronously

4.2.2 A Worked Example: Editing a Profile

Follow a real sequence. A user changes their display name to “Suraj K”.

TimeWhat happensCost
12:00:00.000App calls saveUser(user)
12:00:00.001Cache writes to the database, then waits20 ms
12:00:00.021Database confirms the write
12:00:00.022Cache stores the object under users:71 ms
12:00:00.023Cache returns ok to the app

Total: 21 ms. The user waited for both stores, not one.

Now they open their profile one second later. The cache already holds users:7, so it is a HIT. That costs 1 ms and it shows “Suraj K”. Correct, and fast.

4.2.3 The Contrast That Makes It Click

Run that exact same story under cache-aside and watch what goes wrong.

  • The app writes to the database, which costs 20 ms.
  • The app then deletes users:7, because that is what cache-aside does on a write.
  • One second later, the profile page reads users:7 and MISSES. You just deleted it.
  • So it goes to the database anyway. That is another 100 ms the user did not need to wait.

Both patterns gave the right answer. But cache-aside threw the data away and then immediately had to fetch it back. Write-through kept it, because it was already holding it.

That is the whole case for write-through. When a read follows the write closely, the cache-aside delete is wasteful. The write-through keep is not.

4.2.4 The Two Costs

Nothing is free, and write-through charges you twice.

First, every write is slower. You now wait for two systems instead of one. Our 20 ms database write became a 21 ms save. Under cache-aside the write is just 20 ms, and the delete is fire-and-forget.

Second, and this is the bigger one, it caches things nobody reads. Write-through caches everything written, whether it is ever read back or not.

Picture an audit log. You write 5 million rows a day and almost nobody reads them. Under write-through, all 5 million land in your cache. They fill memory and evict the product data people actually are asking for. Your cache is now working against you.

4.2.5 The Rule of Thumb

Ask yourself one question. After this write, will someone read this soon?

  • Yes — use write-through. Profiles, shopping carts, session data, settings.
  • No — do not. Audit logs, event streams, telemetry, anything written once and read never.

Pair it with a TTL anyway, so any unread entries eventually drain out on their own.

4.2.6 Write-Through in Code

In pseudocode, the pattern is just two functions. A read that fills the cache on a miss, and a write that updates both stores together.

// Write-through (with read-through) in pseudocode.
 
# READ-THROUGH: on a miss, load from the DB and cache it.
function getUser(id):
    key = "user:" + id
    cached = cache.get(key)
    if cached is not null:
        return cached                 # HIT: skip the database
    user = database.findUser(id)      # MISS: load it
    cache.set(key, user, ttl = 30 minutes)
    return user
 
# WRITE-THROUGH: write to the DB, then refresh the cache.
function saveUser(user):
    database.saveUser(user)           # 1. database first, and wait
    cache.set("user:" + user.id, user, ttl = 30 minutes)  # 2. refresh
    return user
 
# On delete, remove from both stores.
function deleteUser(id):
    database.deleteUser(id)
    cache.delete("user:" + id)

One distinction trips people up here, so it is worth stating plainly. Most cache libraries give you two different behaviours, and mixing them up quietly breaks things.

  • A read helper should skip the database on a hit. If it always ran the query, the cache would do nothing.
  • A write helper should always run the write and then refresh the cache. Skipping the write would lose the update.

So the read path checks the cache first and only falls through on a miss. The write path never short-circuits. It always touches the database, then updates the cache. Getting these backwards is a classic bug: a read that always queries, or a write that sometimes never persists.

4.3 Write-Behind (Write-Back)

Write-behind takes the last step and makes it asynchronous. The application writes to the cache, and the cache says “done” immediately. The database write happens later, in the background, often batched with other writes.

Speed is the payoff here. Your write latency becomes cache latency, which might be under a millisecond. Better still, the cache can batch a hundred updates into one database round trip, and it can collapse repeated writes to the same key into a single final write.

Think about a view counter on a video. Ten thousand views per minute means ten thousand database updates under write-through. Under write-behind, you increment a number in memory and flush once every few seconds. The database sees a handful of writes instead of thousands.

// A simplified write-behind buffer for view counts (pseudocode).
 
# An in-memory map of videoId -> pending count.
pending = {}      # shared, thread-safe map
 
# FAST PATH: the caller never waits for the database.
function recordView(videoId):
    pending[videoId] = pending.get(videoId, 0) + 1
    return                              # done in under 1 ms
 
# Runs on a timer every 5 seconds, in the background.
function flush():
    # Take a snapshot and clear the buffer.
    batch = pending.drain()             # atomically remove all entries
    for (videoId, count) in batch:
        if count > 0:
            database.incrementViews(videoId, count)  # ONE write per video

4.3.1 A Worked Example: A Video Getting 10,000 Views

Numbers make this pattern obvious. A video is trending and gets 10,000 views in one minute. The flush runs every 5 seconds.

Under write-through, every single view is a database write. That is 10,000 UPDATE statements in 60 seconds, roughly 167 per second, all fighting for a lock on the same row.

Under write-behind, the story changes completely.

  • Each view just increments a number in memory. The user waits under 1 ms.
  • Views pile up in the buffer for 5 seconds. About 833 of them.
  • The scheduler wakes up and issues one UPDATE that adds 833 at once.
  • Over the full minute that is 12 database writes instead of 10,000.

So the database load dropped by more than 800 times, and the users got faster responses too. The counter is briefly behind the truth, by at most 5 seconds. For a view count, nobody notices and nobody cares.

4.3.2 The Risk You Must Name

Write-behind has one serious weakness, and you should say it out loud before anyone asks. If the cache node dies before the flush, those writes are gone. There is no database copy to fall back on, because the database never saw them.

That is why the pattern fits view counts, likes, and analytics events. Losing five seconds of view counts is annoying. Losing five seconds of payment records is a lawsuit. Never put money or orders behind an unreplicated write-behind buffer.

Real systems reduce the risk rather than remove it. They replicate the buffer, or they persist pending writes to an append-only log first, or they shorten the flush interval. Each of those choices trades some of the speed back for durability.

Interview Insight: Choosing a Pattern Out Loud
Do not name a pattern and stop. Interviewers want the reasoning, and the reasoning is always about the read-to-write ratio and the cost of stale or lost data.
A strong answer sounds like this. “Product catalogue is read-heavy and tolerates a few seconds of staleness, so I would use cache-aside with a short TTL. User sessions are read right after every write, so write-through fits better. View counters are write-heavy and cheap to lose, so write-behind with a five-second flush saves the database.”
That single paragraph shows you can map data characteristics onto a pattern. It matters more than knowing any specific cache product.

4.4 Comparing the Three Patterns

Here is the whole picture in one place. Keep this table in your head for design discussions.

AspectCache-AsideWrite-ThroughWrite-Behind
Write latencyDatabase speedDatabase plus cacheCache speed only
ConsistencyEventually consistentStrong between the twoWeakest; DB lags
Risk of data lossNoneNoneReal, if the cache dies
Cold startSlow, misses at firstSlow for unwritten dataSlow for unwritten data
Cache pollutionLow; only read dataHigh; caches all writesHigh; caches all writes
ComplexityLow, but in your codeModerateHigh
Best forRead-heavy cataloguesRead-after-write dataWrite-heavy counters

One last note. These patterns combine. A common production setup is cache-aside for reads plus write-behind for counters, sitting on the same Redis cluster. Nobody says you must pick one for the entire system.

5. Eviction Policies: What to Throw Away

Caches are small on purpose. Memory costs far more than disk, so your cache holds a fraction of your data. Sooner or later it fills up, and something must go. The rule that picks the victim is the eviction policy.

The goal of every policy is the same: keep the entries that will be read again, and drop the ones that will not. Since none of them can see the future, they all guess using the past. They just guess differently.

5.1 LRU — Least Recently Used

LRU evicts whatever has gone untouched the longest. The idea behind it is called temporal locality. In plain words: if you read something a moment ago, you will probably read it again soon.

That guess holds up well in practice. Think about the items that stay hot in a real system.

  • News articles. Today story gets read a million times. Last Tuesday story gets read twice.
  • Trending products. A phone launches and everyone views the same page for a week.
  • Active sessions. A logged-in user hits your API every few seconds, then goes quiet at night.

In all three cases, recent equals popular. LRU captures that for free, without counting anything. So it is the default in most cache libraries, the common choice for cache servers, and the sensible starting point for most application caches.

5.1.1 A Worked Example

Say the cache holds 3 entries and is already full with A, B and C. Here is what happens as requests arrive. The list on the right shows most recent first.

StepActionCache after the stepEvicted
1Start (full)C, B, A
2read AA, C, B
3read CC, A, B
4write D (cache full)D, C, AB
5read BB, D, CA

Follow step 4. B was the coldest, because nobody had touched it since the start. So B loses its place to D. Then in step 5, B comes back, and now A is the coldest one. Notice that a read is not free: every read reshuffles the order and changes who dies next.

5.1.2 How to Build One

An LRU cache needs to do two things fast: look up a value by key, and know which entry is the coldest. A hash map gives you the first. A doubly linked list, ordered from coldest to hottest, gives you the second. Put them together and both operations are constant time.

The idea is simple once you picture it. The map finds an entry instantly. Ordering lives in the list, which remembers when each entry was last used. Every time you touch an entry, you move it to the hot end. When the cache is full, you drop whatever sits at the cold end.

// LRU cache: a hash map plus a doubly linked list (pseudocode).
#
#   map:  key -> node          (for O(1) lookup)
#   list: coldest <-> ... <-> hottest   (for O(1) eviction)
 
function get(key):
    if key not in map:
        return null                     # MISS
    node = map[key]
    list.moveToHot(node)                # mark it as recently used
    return node.value                   # HIT
 
function put(key, value):
    if key in map:
        node = map[key]
        node.value = value
        list.moveToHot(node)
        return
 
    if map.size == capacity:            # cache is full
        coldest = list.removeColdest()  # evict least recently used
        map.remove(coldest.key)
 
    node = list.addAtHot(key, value)    # insert as most recently used
    map[key] = node

Read the two operations slowly, because the moves are the whole point.

  • On get, you find the node through the map, then slide it to the hot end. So the coldest end always holds the true least-recently-used entry.
  • On put with a full cache, you remove the node at the cold end and delete its key from the map. Then you add the new entry at the hot end.

Notice that a get is not read-only here. Every read reshuffles the order and changes who dies next. That surprises people the first time they see it, but it is exactly what “recently used” means.

5.1.3 You Rarely Write This Yourself

In real code you almost never hand-build an LRU. Two easier paths exist, and both are worth knowing.

  • Many languages ship an ordered map or a ready-made LRU structure, where you flip a single option to reorder entries on access instead of on insert. That one flag turns an insertion-ordered map into an LRU.
  • A cache library will do it properly, with size limits, statistics, and thread safety already handled. This is the option to reach for in production.

Whichever route you take, watch out for concurrency. A cache is hit by many requests at once, so the structure underneath must be thread safe. Hand-rolled maps usually are not. Good cache libraries handle the locking for you, which is one more reason to prefer them over your own code.

5.1.3 Where LRU Falls Over

LRU has one classic failure, and it is worth knowing before it bites you.

  • A nightly report scans a million rows, one time each.
  • Every one of those reads looks recent to the cache.
  • So each row pushes out a genuinely hot entry.
  • By morning your cache is full of rows nobody will ever ask for again.

The job did its work, but it wiped your cache on the way through. Real caches fight this with a small admission window, which forces a new entry to prove it will be reused before it can push out an established one. The better cache libraries do this automatically, as we will see shortly.

5.2 LFU — Least Frequently Used

LFU counts accesses and evicts the entry with the lowest count. The two policies ask different questions:

  • LRU asks: when did you last read this?
  • LFU asks: how often do you read this at all?

That difference matters when traffic is skewed. Picture a shop with one bestseller and a thousand slow movers. The bestseller is read 500 times an hour. The rest are read once a month.

  • Under LRU, a burst of odd requests can push the bestseller out. It was not read in the last few seconds, so it looks cold.
  • Under LFU, its count is 500 and theirs is 1. Nothing can dislodge it.

But LFU has a memory problem. Think about last year Black Friday deal. It was read a million times back then, so its count is huge. Nobody has opened it since. Yet it sits in your cache forever, because no new entry can ever out-count it.

The fix is to forget slowly. Two ways work:

  • Decay the counts over time, so old popularity fades.
  • Count within a sliding window, so only recent reads matter.

Most real caches do this for you. Some servers offer an LFU policy that decays counts over time, and some libraries use a windowed variant such as TinyLFU under the hood. Either way, the lesson is the same: raw LFU without decay is a trap.

// A typical cache configuration (pseudocode).
 
cache = newCache()
    .maxSize(10_000)              # evict once it holds 10,000 entries
    .expireAfterWrite(10 minutes) # and expire each entry by TTL
    .recordStats()               # so you can read the hit ratio
    .build()
 
# Good libraries do not use plain LRU or plain LFU. They combine
# them: a small recency window in front of a frequency-based main
# region. You get LRU's handling of recent items and LFU's
# resistance to one-off scans, without tuning either by hand.

5.3 FIFO — First In, First Out

FIFO evicts the oldest entry by insertion time, and it completely ignores how often you read it. It is the simplest policy to build, and it is almost always the wrong choice for a read cache.

The problem is easy to see. Your homepage banner was cached when the app started. It is read 1,000 times a second. It was also the first thing inserted. So FIFO evicts it first, and it does not care about those 1,000 reads at all.

FIFO throws away exactly what you wanted to keep.

Still, it has honest uses. Queues, buffers, and anything where order genuinely defines value. Just do not reach for it to cache products.

5.4 TTL and Random Eviction

TTL is not really an eviction policy. It is an expiry rule that runs alongside one. Each entry gets a lifespan, and once that lifespan ends, the entry is invalid whether or not the cache is full.

Use a TTL everywhere. It is the cheapest insurance you can buy, because it puts a limit on how wrong your cache can get. Say a new developer adds a write path and forgets to delete the cache key. That is a real bug. But with a 10 minute TTL, the stale data lasts 10 minutes instead of forever.

Think of it this way. Invalidation is your plan. TTL is what saves you when the plan has a hole in it.

Random eviction sounds silly, but it is real. Redis offers it, and it costs almost nothing to run because there is no bookkeeping. Under uniform access patterns it performs nearly as well as LRU. Under skewed patterns, which is most real traffic, it performs noticeably worse.

PolicyEvictsBest forWeakness
LRUThe least recently used entryGeneral purpose, time-clustered readsScans wipe out hot data
LFUThe least frequently used entrySkewed traffic with clear favouritesOld popularity lingers
FIFOThe oldest inserted entryQueues and buffersIgnores access entirely
RandomAny entry, at randomCheap, uniform accessPoor on skewed traffic
TTLAnything past its lifespanBounding stalenessNot a capacity policy

Most cache servers make this a single configuration setting. You pick the policy by name, and common choices include an LRU mode, an LFU mode, a variant that only evicts keys carrying a TTL, and a mode that refuses to evict at all and instead rejects new writes. That last pair is handy when some entries must never disappear on their own.

Interview Insight: Picking an Eviction Policy
The expected answer is LRU, and it is usually right. What separates a strong candidate is naming the failure case unprompted.
Say this: “I would start with LRU, since most access patterns cluster in time. But if there is a nightly batch job scanning the whole table, LRU gets wiped by the scan. Then I would move to a cache that combines recency and frequency, which resists exactly that kind of scan.”
If they push on LFU, mention count decay. An LFU cache without decay preserves last year is winners forever, and that is the flaw the follow-up question is hunting for.

6. Bringing Patterns and Policies Together

Patterns and policies are independent choices, and that is easy to miss. You can run cache-aside with LRU, or cache-aside with LFU, or write-through with either. The pattern decides how data flows. The policy decides what survives when memory runs out.

Here is how the two decisions play out on a real product catalogue. Reads dominate at roughly 100 to 1. A price change can take a few seconds to appear. Losing the cache entirely must not break checkout.

  • Pattern: cache-aside, because the cache must never be a hard dependency.
  • Write path: update the database, then delete the key.
  • TTL: 10 minutes, with a random jitter of up to 60 seconds to avoid synchronised expiry.
  • Policy: LRU, or a recency-plus-frequency cache if a nightly scan exists.
  • Size: enough to hold the hot 20 percent of the catalogue, based on your estimate.

Now change one requirement. Say the checkout flow reads a cart immediately after writing it. Cache-aside would work, but a miss right after a write is wasteful and occasionally stale. Write-through fits better, because the write and the cache update happen together.

That is the whole skill. Read the data characteristics, then pick. Nobody memorises a matrix. They ask two questions about the traffic and derive the answer each time.

7. Common Mistakes

These are the ones that show up in real code reviews and real incidents. Each one has a simple fix.

  • Caching without a TTL. Sooner or later an invalidation path gets missed, and a stale entry lives forever. A TTL bounds the blast radius.
  • Updating the cache on write instead of deleting the key. Two concurrent writers interleave, and the cache keeps the older value. Delete instead; the next read fixes it.
  • Deleting the cache before the database write. If the write fails or a read sneaks in first, you cache the old row again.
  • Giving every key the same TTL. They all expire together, and the database takes the full hit at once. Add jitter.
  • Caching everything by reflex. Write-heavy, rarely-read data just pollutes memory and evicts entries you actually needed.
  • Ignoring the hit ratio. A cache running at 40 percent adds latency, cost, and staleness while barely helping. Measure it, then fix the keys or the size.
  • Making the cache a hard dependency. If a Redis outage takes down your service, you built a database, not a cache.
  • Caching per-user data under a shared key. It is rare, it is embarrassing, and it leaks one user data to another.

8. Practical Takeaways

A few habits will carry you through most caching work, whatever your stack.

  • Start with cache-aside. It is simple, it degrades gracefully, and it covers the majority of read-heavy cases.
  • Always set a TTL, and always add jitter to it.
  • Write to the database first, then delete the cache key. Never the reverse.
  • Track the hit ratio from day one. Most cache libraries and servers expose it for free.
  • Reach for write-through only when reads follow writes closely.
  • Reach for write-behind only when the data is cheap to lose.
  • Default to LRU, and move to a recency-plus-frequency policy when scans hurt you.
  • Size the cache from an estimate of the hot working set, not from a guess.

8.1 Key Terms Recap

TermMeaning in One Line
Cache hitThe data was found in the cache
Cache missThe data was absent, so the database was queried
Hit ratioHits divided by total lookups; the health metric of a cache
Cache-asideThe application loads and invalidates the cache itself
Read-throughThe cache loads from the database on a miss
Write-throughThe cache writes to the database synchronously
Write-behindThe cache writes to the database later, asynchronously
Eviction policyThe rule that picks which entry to remove when full
LRUEvicts the entry untouched for the longest time
LFUEvicts the entry with the fewest accesses
TTLA lifespan after which an entry expires
Thundering herdMany requests missing on the same key at once

9. Conclusion

Caching strategies come down to two decisions, and both are simpler than they first look. The pattern decides who keeps the cache and the database in agreement. The policy decides what gets sacrificed when memory runs out.

Cache-aside puts you in control and fails gracefully, which is why it is the sensible default. Write-through buys consistency with latency. Write-behind buys speed with risk, and that risk is only acceptable for data you can afford to lose. On the eviction side, LRU is the honest starting point, LFU handles skew, and TTL is the safety net you should never skip.

Try it on your next project. Add a cache to one hot read path, set a TTL with jitter, turn on statistics, and watch the hit ratio for a week. You will learn more from that one number than from any article, including this one. Part two picks up where this leaves off: distributed caching, running a cache at scale across many machines, and the genuinely hard problem of invalidation across many nodes.

Further Reading

Leave a Comment