Caching Strategies

Theory

Caching stores frequently accessed data in a fast layer (memory, CDN edge) to reduce latency and backend load. The fundamental trade-off is consistency vs speed — cached data may be stale until invalidated or expired.

Cache-aside (lazy loading): app checks cache first; on miss, reads DB, writes to cache, returns. App owns invalidation on writes. Most common pattern — simple but requires careful TTL and invalidation logic to avoid stale reads.

Write-through: app writes to cache and DB synchronously — cache always warm but write latency includes cache write. Write-behind (write-back): app writes to cache, async flush to DB — highest write throughput but risk of data loss on cache failure before flush.

Write-around: writes go directly to DB, bypassing cache — avoids flooding cache with write-heavy data never read again. Good for write-once-read-rarely workloads like audit logs.

TTL (time-to-live) bounds staleness. Add jitter (random ±10%) to TTLs so keys don't all expire simultaneously. Shorter TTL for frequently changing data; longer for static reference data. No TTL means manual invalidation only — dangerous if forgotten.

Cache stampede: many requests miss the same hot key simultaneously, all hammer the DB. Mitigations: mutex lock (only one thread recomputes), probabilistic early expiration (refresh before TTL ends), request coalescing (singleflight pattern), or pre-warming on deploy.

Redis supports rich data structures (strings, hashes, sorted sets), persistence (RDB/AOF), pub/sub, and Lua scripting — use as cache + session store + rate limiter. Memcached is simpler (strings only), multithreaded, no persistence — pure ephemeral cache at massive scale.

Eviction policies: LRU (Least Recently Used) evicts oldest accessed keys — good general default. LFU (Least Frequently Used) keeps hot keys longer — better for skewed access patterns. Redis also offers allkeys-lru, volatile-lru, allkeys-lfu. Monitor evicted_keys metric — high eviction means undersized cache.

Architecture Diagram

Client request
         |
    Cache lookup
    /         \
  HIT        MISS
   |           |
 return    Origin / DB
           + populate cache

Examples

import redis, json

r = redis.Redis(host="localhost", port=6379, decode_responses=True)

def get_user(user_id: int) -> dict:
    key = f"user:{user_id}"
    cached = r.get(key)
    if cached:
        return json.loads(cached)

    user = db.query("SELECT * FROM users WHERE id = %s", user_id)
    if user:
        r.setex(key, 300, json.dumps(user))  # TTL 5 min + jitter in prod
    return user

def update_user(user_id: int, data: dict):
    db.execute("UPDATE users SET ... WHERE id = %s", user_id, data)
    r.delete(f"user:{user_id}")  # invalidate on write

def get_with_lock(key: str, loader, ttl=300):
    val = r.get(key)
    if val:
        return json.loads(val)

    lock_key = f"lock:{key}"
    if r.set(lock_key, "1", nx=True, ex=10):  # acquire lock
        try:
            val = r.get(key)  # double-check
            if val:
                return json.loads(val)
            data = loader()
            r.setex(key, ttl, json.dumps(data))
            return data
        finally:
            r.delete(lock_key)
    else:
        time.sleep(0.05)  # wait for winner
        return get_with_lock(key, loader, ttl)

# redis.conf
maxmemory 2gb
maxmemory-policy allkeys-lru
appendonly yes

# CLI checks
redis-cli INFO stats | grep evicted_keys
redis-cli --latency-history
redis-cli MEMORY USAGE user:12345

Interview Questions

Best Practices

• Cache only idempotent, reconstructable data — never cache auth tokens or user-specific secrets in shared layers without encryption.
• Add jitter to TTLs (TTL + random(0, 60s)) so hot keys do not all expire at once.
• Use cache-aside for most read-heavy paths — simpler invalidation than write-through when DB is source of truth.
• Monitor hit ratio and miss latency — a cache with 40% hits may still add latency if misses are slower than direct DB.
• Size eviction policy to workload: LRU for general traffic, LFU for skewed hot-key patterns.
• Invalidate on write explicitly — TTL alone causes stale reads during the expiry window.

Common Mistakes

• Caching without a stampede guard — one expiry triggers thousands of concurrent DB queries and melts the database.
• Using infinite TTL on frequently updated data — users see stale prices, permissions, or inventory for hours.
• Treating Redis as durable storage — cache loss should be recoverable; persist critical state in the DB.
• Caching large blobs (multi-MB JSON) — evicts useful keys and increases network overhead; cache pointers or slices instead.
• No cache warming after deploy — cold start sends 100% traffic to DB until cache fills.

Trade-off Analysis

Caching Strategies improves cache-aside, cdn, eviction, stampede prevention. but adds operational and cognitive complexity — justify with load and team size.

Favor simplicity until metrics (p99 latency, error rate, cost) prove the pattern necessary.

Every redundancy layer trades capital/operational cost for availability — align with explicit SLO targets.

Document accepted inconsistency windows and recovery behavior before production cutover.

Cheat Sheet

Practical Exercises