Explore any data with a custom, interactive web app: Data science with sports
By: Jp Hwang
Tags: Technology, Data Visualization, Data Science, Programming, Python

Most good data projects start with the analyst doing something to get a feel for the data that they are dealing with.

They might hack together a Jupyter notebook to look at data summaries, first few rows of data and matplotlib charts. Some might look through the data as an Excel sheet and fidget with pivot tables. The ones truly one with the data might even prefer to stare directly at the raw table of data.

None of these are ideal solutions. Some of these solutions might be only suitable for the masochistic among us. So what is a person to do?

For me, I prefer to build a web app for data exploration.

There’s something about the ability to slice, group, filter, and most importantly — see the data, that helps me to understand it and help me to formulate questions and hypotheses that I want answered in the .

It allows me to interact with the data visually.

My preferred toolkit of choice for this task these days is Plotly and Streamlit. I’ve written enough about Plotly over the last while — I think it’s the best data visualisation package out there for Python. But Streamlit has really changed the way I work. Because it is so terse, it takes almost no extra effort to turn my plots and comments in a python script into a web app with interactivity as I tinker. (FYI — I wrote a comparison between Dash and Streamlit here)

I prefer to build a web app for data exploration

So in this article, I’d like to share with a simple example building a data exploration app with these tools.

Now, for a data project – we need data, and here I will be using stats from the NBA. Learning programming can be dry, so using something relatable like sports data helps me to stay engaged; and hopefully it will for you too.

(Don’t worry if you don’t follow the NBA as the focus is on the data science and programming!)

Before we get started

To follow along, install a few packages — plotly , streamlit and pandas . Install each (in your virtual environment) with a simple pip install [PACKAGE_NAME] .

The code for this article is on my GitHub repo here, so you can download/copy/fork away to your heart’s content.

The script is called data_explorer_app.py — so you can run it from the shell with:

streamlit run data_explorer_app.py

Oh, this is the first in a set of data science / data analysis articles that I plan to write about using NBA data. It’ll all go to that repo, so keep your eyes peeled!

If you are following along, import the key libraries with:

import pandas as pd

import plotly.express as px

import streamlit as st

And we are ready to go.

Data Deep Diving

Streamlit-ing

We use Streamlit here, as it is designed to help us build data apps quickly. So what we are going to build is a Streamlit app that will then run locally. (For more information — you can check out my Dash v Streamlit article here.)

If you’ve never used Streamlit, this is all you need to build bare-bones app:

import streamlit as st

st.write("Hello, world!")

Save this as app.py , and then execute it with a shell command streamlit run app.py :

Look, ma, it’s a web app!

And you have a functioning web app! Building a streamlit app is that easy. Even more amazingly, though, building a useful app isn’t much harder.

Oh, by the way, you don’t need to stop and restart the server every time the script is changed. Whenever the underlying script file is updated, you will see a button pop-up on the top right corner like so:

Look for this prompt to refresh the app

Just keep the script running, and hit Rerun here every time you want to see the latest version at work.

Ready? Okay, let’s go!

Raw data exploration

What I like to do initially it to look at the entire raw dataset. As a first step, we load the data from a CSV file:

df = pd.read_csv("data/player_per_game.csv", index_col=0).reset_index(drop=True)

Once the data has been loaded, simply typing st.write(df) creates a dynamic, interactive table of the entire dataframe.

Explore the entire dataset as an interactive table

And the various statistics for columns can be similarly plotted with st.write(df.describe()) .

Two dynamic tables in two lines of code

I know you can plot a table in Jupyter notebooks, but the difference is in the interactivity. For one, tables rendered with Streamlit are sortable by columns. And as you will see later, you can incorporate filters and other dynamic elements that aren’t as easy to incorporate in notebooks — which is where the real power comes in.

Now we are ready to start adding a few charts to our app.

Distribution visualisations

Statistical visualisation of individual variables are extremely useful, to an extent that I think it’s an indispensable tool above and beyond looking at the raw data.

We will begin the analysis by visualising the data by one variable, with an interactive histogram. A histogram can be constructed with Plotly like so:

hist_fig = px.histogram(df, x=hist_x, nbins=hist_bins)

Traditionally, we would have to manually adjust the x and nbins variables to see what happens, or create a huge wall of histograms from various permutations of these variables. Instead, let’s see how they can be taken in as inputs to interactively investigate the data.

The histogram will analyse data from one column of the pandas dataframe. Let’s render it as a drop-down box by calling the st.selectbox() module. We can just grab a list of the columns as df.columns , and additionally we provide a default choice, which we get the column number of using df.columns.get_loc() method. Putting it together, we get:

hist_x = st.selectbox("Histogram variable", options=df.columns, index=df.columns.get_loc("mp_per_g"))

Then, a slider can be called with the st.slider() module for the user to select the number of bins in the histogram. The module can be customised a minimum/maximum/default and increment parameters as you see below.

hist_bins = st.slider(label="Histogram bins", min_value=5, max_value=50, value=25, step=1)

These parameters can then be combined to produce the figure:

hist_fig = px.histogram(df, x=hist_x, nbins=hist_bins, title="Histogram of " + hist_x,

template="plotly_white")

st.write(hist_fig)

Putting it together with a little heading st.header(“Histogram”) , we get:

Histogram portion of the app

I recommend taking a second here to explore the data. For example, take a look at different stats like rebounds per game:

Histogram of rebounds per game

Or positions:

Histogram of positions

The interactivity makes for easier, dynamic, active exploration of the data.

You might have noticed in this last graph that the histogram categories are not in any sort of sensible order. This is due to the fact that this is a categorical variable. So without a provided order, Plotly is (I think) plotting these categories based on the order that it starts to encounter each category for the first time. So, let’s make one last change to would fix that.

Since Plotly allows for a category_orders parameter, we could pass a sorted order of positions. But then it wouldn’t be relevant for any of the other parameters. Instead, what we can do is to isolate the column based on the chosen input value, and pass them on by sorting them alphabetically like so:

df[hist_x].sort_values().unique()

All together, we get:

hist_cats = df[hist_x].sort_values().values

hist_fig = px.histogram(df, x=hist_x, nbins=hist_bins, title="Histogram of " + hist_x,

template="plotly_white", category_orders={hist_x: hist_cats})

Histogram of positions — alphabetically sorted

This way, any categorical (or ordinal) variables would be presented in order

Now we can go another step and categorise our data with boxplots. Boxplots do a similar job as histograms in that they show distributions, but they are really best at showing how those distributions changed according to another variable.

So, the boxplot portion of our app is going to include two pulldown menus like below.

box_x = st.selectbox("Boxplot variable", options=df.columns, index=df.columns.get_loc("pts_per_g"))

box_cat = st.selectbox("Categorical variable", ["pos_simple", "age", "season"], 0)

And it’s just a matter of passing those two inputs to Plotly to build a figure:

box_fig = px.box(df, x=box_cat, y=box_x, title="Box plot of " + box_x, template="plotly_white", category_orders={"pos_simple": ["PG", "SG", "SF", "PF", "C"]})

st.write(box_fig)

Then… voila! You have an interactive box plot!

Interactive boxplot

You will notice here that I manually passed an order for my simplified positions column. The reason is that this order is a relatively arbitrary, basketball-specific order (from PG to C), not an alphabetical order. As much as I would like everything to be parametric, sometimes you do have to resort to manual specifications!

Correlations & filters

Another big thing to do in data visualisation, or exploratory data analysis is to understand correlations.

It can be for example handy for some manual feature engineering in data science, and it might actually point you towards an investigative direction that you may not have considered.

Let’s just stick to three dimensions in our scatter plot for now.

No, not in x, y and z directions. I am not a monster. I’ve got an example of one below — can you make sense of what’s going on?

I’m not a huge fan of 3-D Scatter plots (matplotlib)

Not for me, thanks.

Colour will be the third dimension here to represent data. I’ve left all columns available for the the first two columns, and just a limited selection for colours — but you can really do whatever you want.

corr_x = st.selectbox("Correlation - X variable", options=df.columns, index=df.columns.get_loc("fg3a_per_g"))

corr_y = st.selectbox("Correlation - Y variable", options=df.columns, index=df.columns.get_loc("efg_pct"))

corr_col = st.radio("Correlation - color variable", options=["age", "season", "pos_simple"], index=1)

Correlate away!

And the chart can be constructed as follows:

fig = px.scatter(df, x=corr_x, y=corr_y, template="plotly_white", color=corr_col, hover_data=['name', 'pos', 'age', 'season'], color_continuous_scale=px.colors.sequential.OrRd)

So tell me — are they correlated?

But this chart is not ideal. For one because the data is dominated by outliers. See the lonely dots on the top left? Those folks with effective FG% of 1.5 are not some gods of basketball, but it’s a side effect of extremely small sample sizes.

So what can we do? Let’s put a filter into the data.

I’m going to put in two interactive portions here, one to choose the filter parameter, and the other to put the value in. As I don’t know what the parameter is here, I will simply take an empty text box that will take numbers as inputs.

corr_filt = st.selectbox("Filter variable", options=df.columns, index=df.columns.get_loc("fg3a_per_g"))

min_filt = st.number_input("Minimum value", value=6, min_value=0)

Using these values, I can filter the dataframe like so:

tmp_df = df[df[corr_filt] > min_filt]

And then pass the temporary dataframe tmp_df into the figure instead of the original dataframe, we get:

Correlations between efficiency and number of shots (filtered for high-volume shooters)

This chart could be used to take a look at correlations between various stats. For example, to see that great 3 pt shooters are also typically great free throw shooters:

Free throw accuracy vs 3pt shot accuracy

Or that great rebounders tend to be shot blockers as well. It’s also interesting that the game has changed so that no modern players average many blocks per game.

Rebounds per game vs blocks per game

Plotting rebounds and assists, they show something of an inverse correlation, and are quite nicely stratified according to position here.

Assists per game vs rebounds per game

Already we can see quite a lot of trends and correlations from our app. Lastly, let’s create some heatmaps to view general correlations between sets of columns of data.

Generalised correlations with heatmaps

Scatter plots are useful for seeing individual data points, but sometimes it’s good to just visualise datasets such that we can immediately see which columns might be well correlated, not correlated, or inversely correlated.

Heatmaps are perfect for this job, by setting it up to visualise what are called correlation matrices.

Since a heatmap is best at visualising correlations between sets of input categories, let’s use an input that will take multiple categories. As a result, st.multiselect() is the module of choice here, and df.corr() is all we need to create the correlation matrix.

The combined code is:

hmap_params = st.multiselect("Select parameters to include on heatmap", options=list(df.columns), default=[p for p in df.columns if "fg" in p])

hmap_fig = px.imshow(df[hmap_params].corr())

st.write(hmap_fig)

And we get:

Assists per game vs rebounds per game

It’s so clear which of these columns are positively correlated or not correlated. and I also suggest playing with different colour scales / swatches for extra fun!

That’s it for today — I hope that was interesting. For my money, it’s hard to beat interactive apps like this for exploration, and the power of Plotly and Streamlit make it so easy to build these customised apps for my purpose.

And keep in mind that what I have suggested here are just basic suggestions, and what I am sure that you could build something far more useful for your own purpose and to your preference. I look forward to seeing them all!