How to Build an LRU Cache in Less Than 10 Minutes and 100 Lines of Code
By: Sunny Beatteay
Tags: Web Development, Programming, Python, Software Development, Coding Interviews

How to Build an LRU Cache

For the purposes of this article, I will be using Python to implement the LRU cache. It’s succinct, easy to read, and a lot of people know it. However, if Python isn’t your thing or you’re curious about how to implement it in other languages, you can check out my examples repository.

Requirements

Before we start building our cache, we have to understand what the requirements are. The first will be the API. Which methods do we need to implement?

While production quality caches are feature rich, we’ll keep it simple. You’ll only need to create two methods:

get(key)

set(key, value)

But that isn’t all. The LRU cache itself has a number of requirements:

When the max size of the cache is reached, remove the least recently used key. Whenever a key is fetched or updated, it becomes the most recently used. Both get and set operations must complete in O(1) time complexity (meaning that no matter how large the cache is, it takes the same amount of time to complete). When fetching a key that doesn’t exist, return a null value.

With these requirements in mind, we can get to work.

Photo by ConvertKit on Unsplash

Data structure

The first question that we need to answer is, which data structure should back our LRU cache? After all, a cache is not a data structure itself.

Since the cache uses get and set and needs to operate in O(1) time, you might’ve thought of a hash map or dictionary. That would indeed fulfill some requirements. But what about removing the LRU key? A dictionary doesn’t allow us to know which is the oldest key.

We could use a time stamp as part of the dictionary value and update it whenever the key is fetched. This would tell us which key-value pair is the oldest. The problem is, we would need to loop through all the dictionary entries to check which is the oldest — breaking our O(1) requirement.

So what’s the answer?

This is where I should let you in on a secret. We’re actually going to need two data structures: one for fetching the values (dictionary/hash map) and one for keeping the items sorted by frequency of use.

The second data structure

So what should the second data structure be? If you thought of an array, you’re getting close.

We can use an array to keep the items sorted. And each key in the dictionary can reference the index of a value in the array. Whenever a key is fetched, we can move that value to the front of the array, pushing the rest of the items back, and update the index in the dictionary.

However, there’s still a small issue. Under the hood, arrays are a continuous row of data cells. If we need to move values to the front of the array and push the rest down, that operation will get slower as the array increases in size. In other words, inserting items at the beginning of an array takes O(N) time.

Image Credit: Author

But we are close. We just need a data structure with the same sorting benefits of an array that can also get, set, and delete data in O(1) time. And the data structure that fits all of those requirements is a doubly linked list (DLL).

A doubly linked list is the same as a linked list, except each node in the list has an additional reference to the previous node as well as the next node.

Each key in the dictionary will reference a node in our linked list. This way, we’ll be able to easily fetch data, as well as update and delete it.