Heroku and Streamlit Under the Hood
By: Jake Miller Brooks
Tags: Data Science, Heroku, Streamlit, Procfile, Shell

Heroku and Streamlit Under the Hood

You’ve probably heard of streamlit. It’s one of the best ways out there to quickly build attractive, user-friendly data apps. If you haven’t heard of streamlit, you should definitely go check it out, you can find a quick-start guide here.

If you have heard of streamlit, and you’ve started playing around with it a bit, then sooner or later you’ve probably wanted to share your app outside of your local machine. I’ll go a step further and say that since much of the beauty of streamlit lies in its ability to make components that typically lie in the general realm of the developer rather than the data scientist, accessible to anyone who can work through a bit of python. It’s incredibly user friendly.

Heroku is one of the best ways to deploy an app with streamlit without a whole lot of overhead needed to get up and running. There are numerous guides to getting your app online with this combination: see here, and here for a couple great examples.

If you start reading these guides, you’ll find that many of them have something in common. They almost all ask you to define a Procfile, and write a shell script called setup.sh, that looks something like this:

Sample “setup.sh” text (Image by Author)

I have yet to find one that breaks down the contents of this script, but my interest was piqued, so I figured I’d share what I found. Let’s start with the first clue: the very first line. If you know your way around the terminal a bit, you probably already know where this is going, but if not, a good place to start on commands like mkdir is the man page. Here’s what happens if you type:

man mkdir

into your terminal.

The Text of The man Page for “mkdir” with the p flag included (Image by Author)

So in plain English, the first line of our setup.sh script will create a directory in the home directory called “.streamlit”, making any intermediate directories (shouldn’t be necessary), as necessary. If you try

ls ~

and don’t find “.streamlit” in the output, just remember that you can use “-a” to show any hidden directories like so:

ls -a ~

I did say this was a clue though, so… What is the clue here? We just made something called a “dotfile,” which is how you talk about configuration files if you’re super hip (try searching github or youtube for dotfile if this is new for you, it’s a brave new world out there). Okay, so we made sure this configuration file for streamlit exists (actually the slash at the end makes it a directory, because streamlit has a few things that live here), what next? Maybe streamlit has something about this in the docs?

It sure does! Here’s the link to the relevant bits, labeled, appropriately, “streamlit configuration.” The first thing we’ve done here is to echo some text into a file called credentials.toml, but if you search streamlit docs for this “credentials.toml” bit, you won’t find anything except maybe on the forum pages. So what exactly did we do here? There’s another clue that lies in the context in which this script is being run — via the Procfile. The contents of the procfile look like this:

web: sh setup.sh && streamlit run your-app.py

So what is this “web” bit? Luckily Heroku has some documentation to help us out with that. The page on the Procfile states:

A Heroku app’s web process type is special: it’s the only process type that can receive external HTTP traffic from Heroku’s routers. If your app includes a web server, you should declare it as your app’s web process.

So our app has a web server. We’ve declared it as our web process, and the setup.sh bit is to do some, well, setup work before running our app. The setup work is largely to make sure that the image we build for the web server interacts with streamlit in a way that allows streamlit to touch the components it needs to serve our app. The last three clues can all be found on the streamlit documentation page for configuration referenced above. We create a config file (the format is a toml), and we set the variables in the [server] section as follows:

[server]

headless = true

From the docs we read:

# If false, will attempt to open a browser window on start.

Reasonable enough: we just want our app to be served, there’s no need for a browser to pop up anywhere on the serving end.

enableCORS = false

This one is pretty technical. Mozilla has a good page explaining the underlying technology (Cross-Origin Resource/Request Sharing), but it’s beyond the scope of this post.

port = $PORT

From the docs:

# The port where the server will listen for browser connections.

$PORT is an environment variable which gives access to the port Heroku is using (frequently not 8501, which is the default value for this variable).

The “

” sequences are echoed as new lines, so each variable gets its own line in the resulting config.toml, which is where streamlit reads its configuration settings.

So what is all of this shell stuff doing? Pretty simple really, the setup.sh file allows Heroku and streamlit to get on the same page. Go make some apps!