Creating an Interactive Datetime Filter with Pandas and Streamlit
By: M Khorasani
Tags: Cloud Computing, Data Visualization, Time Series Analysis, Dashboard, Data Science

Creating an Interactive Datetime Filter with Pandas and Streamlit

Visual by author.

Introduction

Perhaps the most proliferated type of data that we grapple with on a daily basis is timeseries data. Basically, anything that is indexed using date, time or both can be considered as a timeseries dataset. And more often than not, you may require to filter your timeseries data with, well, date and time themselves. Filtering your data frame based on any other form of index is a rather trivial task; the same cannot be stated about datetime however, especially when the date and time are quoted in different columns. Even after you manage to filter them, it is another task applying it to your data frame and instantaneously visualizing it.

Luckily we have Pandas and Streamlit to assist us in this regard in order to create and visualize interactive datetime filters. I assume that most of us are more than familiar with Pandas and possibly use it routinely in our data lives, but I suspect that many are unacquainted with Streamlit given that it is the new kid round the block. Regardless I am going to tender a quick introduction to both lest anyone asks.

Pandas

Pandas is arguably the most agile, efficient, flexible, robust, resilient and user friendly binding when it comes to wrestling with data in Python. And in case you think I threw in far too many hyperboles into that previous sentence, then you have greatly underestimated Pandas. This mighty toolkit, gives you the ability to manipulate, mutate, transform and not least visualize data in frames, all with a couple of lines of code. In this application we will use Pandas to read/write our data from/into a CSV files and to resize our data frames based on selected start and end dates/times.

Streamlit

Streamlit, as characterized by its founders themselves, is a pure Python API that allows you to create machine learning applications. Wrong. It is actually a lot more than that. Streamlit is a web framework, a quasi-port forwarding proxy server and a frontend UI library all mixed into one bundle of goodness. Simply put you can develop and deploy countless web apps (or local apps) for a whole slew of purposes. For our application, we will be utilizing Streamlit to render an interactive sliding filter for our timeseries data that will also be visualized instantaneously.

Packages

Without further ado, let's go ahead and insert the stack of packages we’ll be using.

And in case you need to install any of the above packages, please proceed by using ‘pip install’ in Anaconda prompt.

pip install streamlit

Dataset

We will be using this randomly generated dataset, that has a column for the date, time and value as shown below.

Image by author.

The date is formatted as follows:

YYYYMMDD

While the time is formatted as:

HHMM

You can format your datetime with any other formatting that suits your needs, but you will have to make sure that you declare it in your script as explained in the proceeding section.

Datetime Filter

In order to implement our filter, we will use the following function that takes as arguments — message and df which correspond to the message displayed by the slider widget and the raw data frame that needs to be filtered.

Initially, we will invoke the Streamlit slider widget which is documented as follows.

streamlit.slider(label, min_value, max_value, value, step) Parameters label (str or None) — A short label explaining to the user what this slider is for. min_value (a supported type or None) — The minimum permitted value. Defaults to 0 if the value is an int, 0.0 if a float, value — timedelta(days=14) if a date/datetime, time.min if a time max_value (a supported type or None) — The maximum permitted value. Defaults to 100 if the value is an int, 1.0 if a float, value + timedelta(days=14) if a date/datetime, time.max if a time value (a supported type or a tuple/list of supported types or None) — The value of the slider when it first renders. If a tuple/list of two values is passed here, then a range slider with those lower and upper bounds is rendered. For example, if set to (1, 10) the slider will have a selectable range between 1 and 10. Defaults to min_value. step (int/float/timedelta or None) — The stepping interval. Defaults to 1 if the value is an int, 0.01 if a float, timedelta(days=1) if a date/datetime, timedelta(minutes=15) if a time (or if max_value — min_value < 1 day)

Please note that our slider will return two values, i.e. the start datatime and end datetime values. Therefore we must declare the initial value of the slider using an array as:

[0,len(df)-1]

And we must equate the widget to two variables as shown below, i.e. the start and end datetime indices that will be used to filter the data frame:

slider_1, slider_2 = st.slider('%s' % (message),0,len(df)-1,[0,len(df)-1],1)

Subsequently, we need to remove any trailing decimal places from our start/end time column and add leading zeroes in case the time is less than a whole hour, i.e. 12:00AM quoted as 0, as shown below:

while len(str(df.iloc[slider_1][1]).replace('.0','')) < 4:

df.iloc[slider_1,1] = '0' + str(df.iloc[slider_1][1]).replace('.0','')

Then we need to append our date to time and parse our datetime in format that is comprehensible by using the datetime.strptime binding in Python as shown below:

start_date = datetime.datetime.strptime(str(df.iloc[slider_1][0]).replace('.0','') + str(df.iloc[slider_1][1]).replace('.0',''),'%Y%m%d%H%M%S')

To display our selected datetimes we can use the strftime function to reformat the start/end as follows:

start_date = start_date.strftime('%d %b %Y, %I:%M%p')

In order to use other datetime formatting’s please refer to this article. Finally, we will display the selected datetimes and will apply the filtered indices to our dataset as shown below:

st.info('Start: **%s** End: **%s**' % (start_date,end_date)) filtered_df = df.iloc[slider_1:slider_2+1][:].reset_index(drop=True)

Download CSV

You may find it convenient to download your filtered data frame as a CSV file. If so please use the following function to create a downloadable file in your Streamlit app.

This function’s arguments — name and df correspond to the name of the downloadable file and data frame that needs to be converted to a CSV file respectively.

Streamlit App

Finally, we can bind everything together in the form of a Streamlit application that will render the datetime filter, data frame and a line chart that will all be updated instantaneously when we move our sliders.

You can run your final app, by typing the following commands in Anaconda prompt. First, change your root directory to where your source code is saved:

cd C:/Users/...

Then type the following to run your app:

streamlit run file_name.py

Results

And there you have it, an interactive dashboard that allows you to visually filter your timeseries data and visualize it at the same time!

Photo by author.

If you want to learn more about data visualization and Python, then feel free to check out the following (affiliate linked) courses:

Data Visualization with Python:

Python for Everybody Specialization:

The source code for this tutorial can be found in the following GitHub repository.

In addition, feel free to subscribe to Medium and explore more of my tutorials here.