Caching strategies

Introduction

In this article, we will learn about popular Caching strategies used today.
It should be noted that applying cache is inherently a trade-off, it helps improve system performance but increases complexity, while also requiring acceptance that data retrieved by users might not be the latest.
When using cache, you need to answer the question of which entity will be responsible for writing to the cache and when to write it.

Below are popular Cache Patterns.

The app manages the cache itself, reading from cache first and upon a cache miss, it queries the Database and then writes back to the cache.
This is the most popular type and even if the cache fails, the app remains functional.
For each cache miss, it always takes 3 processing steps including Reading cache -> Querying Database -> Writing cache.
The first read is always slow (cold start) because there is no cache initially.
Used when reading frequently, writing rarely and accepting stale data within the TTL (Time To Live).

When data modification operations occur, they interact directly with the Database without writing to the cache. Cache is only created when someone reads.
Does not waste RAM on written data that no one reads again.
The drawback is that the first read after a write is always a cache miss.
Typically combined with Cache-Aside for practical use.

When data changes occur, it writes to both the cache and the Database to keep both in sync before returning OK to the client.
The advantage is that the Cache is always in sync with the Database. No separate invalidation mechanism is required.
However, it will make all data write operations slower and may cache data that no one reads, leading to cache pollution.

After writing to the cache, it returns the result immediately to the client without waiting to write to the Database.
Afterwards, a worker flushes data in batches to the Database.
This is the fastest write pattern because it aggregates multiple data items to write in a single batch, significantly reducing Database load.
It is also the only pattern where data loss can occur if the cache expires before writing to the Database.
Should not be used for critical data where errors are unacceptable (like balance or order), but should only be used for frequently changing data where small errors are acceptable (like counters, view counts, or metrics).

Similar to Cache-Aside, but the App only fetches data from the cache rather than querying the Database and the cache itself acts as a provider to automatically check if data is available before querying the Database.
Keeps app code clean, keeping cache loading logic in one place.
Because it requires an additional supporting cache provider, it can become a single point of failure.

Used to solve the biggest issue with Cache, which is avoiding Cache Stampede / Thundering Herd that occurs when a critical key expires and thousands of simultaneous requests directly access the Database to retrieve data.
With this mechanism, it refreshes the cache before it expires (based on TTL), relying on predictions of which keys are about to be read.
Should not refresh all keys that are about to expire (which would waste resources on rarely viewed data). Instead, a Key is only Refresh-Ahead when it satisfies 2 conditions:
- The remaining time (TTL) falls within the Refresh Window, such as TTL = 10 minutes then warning zone = 20% TTL (which is under 2 minutes).
- There is an active real-world Request accessing that Key:
  - If the Key is in the warning zone AND a User has just performed a GET on it
  - Then the App will immediately trigger fetching new data in the background.
With this approach, users almost never encounter a cache miss.
Used effectively when hot key sets (frequently used keys) are known in advance.

Happy coding!

See more articles here.