Advanced Streamlit Caching. Caching = Better User Experience | by Rahul Agarwal | Towards Data Science

It is straightforward now how to create a web app using Streamlit, but there are a lot of things that it doesn’t allow you to do yet. One of the major issues I faced recently was around caching when I was trying to use a News API to create an analytical news dashboard in Streamlit.
The problem was that I was hitting the News API regularly and was reaching the free API limits. Also, running the news API every time the user refreshed the app was getting pretty slow.
A solution to that would have been caching the API data. But when I used @st.cache decorator, the page never refreshed as the parameters to the API call remained the same.
And that is where I came to understand the limitations of Streamlit when it comes to caching.
So, in a nutshell, What I wanted was a way not to hit the API every time the page refreshes. At the same time, since I was getting the News data, I also wanted to hit the API at every 5-minute interval.
And, here is the hacky way I accomplished that.
Update:
In the new release notes for 0.57.0 which just came out yesterday, streamlit has made updates to st.cache. One notable change to this release is the “ability to set expiration options for cached functions by setting the max_entries and ttl arguments”. From the documentation:
max_entries (int or None) — The maximum number of entries to keep in the cache, or None for an unbounded cache. (When a new entry is added to a full cache, the oldest cached entry will be removed.) The default is None.
ttl (float or None) — The maximum number of seconds to keep an entry in the cache, or None if cache entries should not expire. The default is None.
Two use cases where this might help would be:
If you’re serving your app and don’t want the cache to grow forever.
If you have a cached function that reads live data from a URL and should clear every few hours to fetch the latest data
So now what you need to do is just:
@st.cache(ttl=60*5,max_entries=20)def hit_news_api(country, n):    # hit_api
It is really unbelievable how fast things change.
Before I begin, Here is a tutorial for streamlit if you are not able to follow this post.
towardsdatascience.com
When we mark a function with Streamlit’s cache decorator @st.cache, whenever the function is called streamlit checks the input parameters that you called the function with. What happens in the backend is when a function is decorated with @st.cache streamlit keeps all the states of a function in the memory.
For example, if we have a simple streamlit app like below:
import streamlit as stimport timedef expensive_computation(a, b):    time.sleep(2)  # 👈 This makes the function take 2s to run    return a * ba = 2b = 21res = expensive_computation(a, b)st.write("Result:", res)
When we refresh the app, we will notice that expensive_computation(a, b) is re-executed every time the app runs. This isn’t a great experience for the user.
Now if we add the @st.cache decorator:
import streamlit as stimport time@st.cache  # 👈 Added thisdef expensive_computation(a, b):    time.sleep(2)  # This makes the function take 2s to run    return a * ba = 2b = 21res = expensive_computation(a, b)st.write("Result:", res)
Since the results are now cached for the parameters a and b the expensive_computation doesn’t run again every time the user refreshes the page. And that results in a far better user experience.
So what if I wanted to execute a function if the page refreshes after a set time period? i.e., it has been some time since the cached function was last run, and the results need to be updated.
To understand this, let us create a basic example. In this example, our function takes a parameter n and a parameter country and returns the top n news item from that country.
In this minimal example, it is akin to returning a list of n random numbers. We want to keep showing this list to our users till the next 5 minutes. If the user refreshes the page after 5 minutes, we want the list to be refreshed. This is quite similar to hitting a news API.
So our basic app code looks like:
import streamlit as stimport randomdef hit_news_api(country, n): st.write("Cache Miss for n:",n) return [random.randint(0,1000) for i in range(n)]results = hit_news_api("USA", 5)st.write(results)
And the app looks like:
Now every time, I refresh the page, the function hit_news_api gets called again, and that means we will hit our API limits as our number of users increase. So how do we solve this problem?
We can use caching. Right. So let us see what happens if we cache the hit_news_api function.
import streamlit as stimport random@st.cachedef hit_news_api(country, n): st.write("Cache Miss for n:",n) return [random.randint(0,1000) for i in range(n)]results = hit_news_api("USA", 5)st.write(results)
The problem now is that our dashboard never calls the news API after hitting it once if the API parameters remain the same. And we see static results for an eternity for the same set of params. What should we do?
We can use this hack — Pass a time-dependent dummy parameter to the hit_news_api function call that changes value every five minutes/ten minutes. The code to do this is:
Here the truncate_time function helps us to truncate the current datetime to the nearest lowest divisor for that minute. For Example, given dt_time=’2020–03–27 02:32:19.684443' and period = 5 the function returns ‘2020–03–27 02:30:00.0’.
See how we call the hit_news_api function now with the truncated time. Now, whenever the hit_news_api function is called from the time between dt_time=’2020–03–27 02:30:00.0' and dt_time=’2020–03–27 02:34:59.99999' the parameters to the function call remains the same, and hence we don’t hit the API again and again. Exactly what we wanted!
This is all well and good, but can you spot a problem with this approach? The cache size is going to grow over time. And that means we need to clear the cache at fixed intervals manually. How do we automate the cache clearing process?
It is a pretty simple hack. We can use this below code at the top of our code to clear the whole cache at a period of 1 day.
from streamlit import cachingfrom datetime import datedef cache_clear_dt(dummy):   clear_dt = date.today()   return clear_dtif cache_clear_dt("dummy")<date.today():   caching.clear_cache()
This is how the whole app looks with all the above functions.
Streamlit has democratized the whole process to create apps, and I couldn’t recommend it more.
Yet, there are some things for which we need to go about in a roundabout way. And there are still a lot of things that I will love to have in this awesome platform. I have been in talks with the Streamlit team over the new functionality that they are going to introduce, and I will try to keep you updated on the same.
You can find the full code for the final app here at my Github repo.
If you want to learn about the best strategies for creating Visualizations, I would like to call out an excellent course about Data Visualization and applied plotting from the University of Michigan, which is a part of a pretty good Data Science Specialization with Python in itself. Do check it out.
Thanks for the read. I am going to be writing more beginner-friendly posts in the future too. Follow me up at Medium or Subscribe to my blog to be informed about them. As always, I welcome feedback and constructive criticism and can be reached on Twitter @mlwhiz.
Also, a small disclaimer — There might be some affiliate links in this post to relevant resources, as sharing knowledge is never a bad idea.