Turn Your Pandas Code into a Web App
Tags: python, webdev, datascience, pandas
Published: 2023-03-16 15:17:51

# Give a front-end to your data insights

Pandas is one of the most popular data science libraries for analysing and manipulating data. But what if you want to take the data that you have manipulated with Pandas and present it to other people in a visual manner? With Anvil, you can take a local script or a Jupyter notebook and turn it into an interactive dashboard — all for free and entirely in Python.

In this tutorial, we’re going to build a dashboard app using the [Netflix Movies and TV Shows dataset](https://www.kaggle.com/shivamb/netflix-shows) by Shivam Bansal. You can also download the data directly [here](https://anvil-website-static.s3.eu-west-2.amazonaws.com/learn/tutorials/pandas-uplink-tutorial/netflix_titles.csv).

We’ll use Pandas to manipulate our data in a local script or a notebook, Plotly to visualize it, and the [Anvil Uplink](https://anvil.works/beta-docs/uplink?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial) to connect it to an app and turn it into an interactive dashboard.

These are the steps that we will cover:

1. We’ll start by [creating our Anvil app](#step-1-create-your-anvil-app).
2. Then, we’ll [design our dashboard](#step-2-design-your-dashboard) using Anvil components.
3. After that, we’ll [clean and prepare the data](#step-3-get-the-data-into-shape) for plotting.
4. We will then [connect our script or notebook to our app](#step-4-making-the-connection) via the Uplink.
5. Using Plotly, we will [build our plots](#step-5-build-the-plots).
6. Finally, we will [add some finishing touches](#step-6-finishing-touches) to our dashboard.

Let’s get started!

> If you don’t have a local Python environment or you want to work entirely in the cloud, [here’s a version of this tutorial entirely in Anvil](https://anvil.works/learn/tutorials/web-app-with-pandas?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial).

---

## Step 1: Create your Anvil app

Go to the Anvil Editor, click on “Blank App”, and choose “Rally”. We just want an empty app, so we’ll delete the current Form1 and then add a new Blank Panel form:

![Creating a new form](https://anvil-website-static.s3.eu-west-2.amazonaws.com/learn/tutorials/pandas-uplink-tutorial/replace-form.gif)

Now let’s [rename our app](https://anvil.works/beta-docs/client/titles-and-logos?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial). Click on the app’s name, on the top left corner of the screen. This takes us to the General Settings page, where we can modify our app’s name.

## Step 2: Design your dashboard

We will build our user interface by adding [drag-and-drop components](https://anvil.works/beta-docs/client/components?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial) from the [Toolbox](https://anvil.works/beta-docs/editor#toolbox?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial). First, add a [ColumnPanel](https://anvil.works/beta-docs/client/components/containers#columnpanel?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial), where we will fit all our other components. Then, let’s add three [Plots](https://anvil.works/beta-docs/client/components/plots?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial) into the ColumnPanel - we can adjust their sizes by clicking and dragging their edges (use Ctrl+Drag for finer adjustments).

![Dragging and dropping components](https://anvil-website-static.s3.eu-west-2.amazonaws.com/learn/tutorials/pandas-uplink-tutorial/dragging-plots.gif)

We’ll also drop a [Label](https://anvil.works/beta-docs/client/components/basic#label?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial) into our app and place it above the plots. Select your Label and, in the [Properties Panel](https://anvil.works/beta-docs/editor#properties-panel?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial) on the right, change the text to “Netflix Content Dashboard”. Set the font_size to 32 and, if you want, you can [customize the font](https://anvil.works/beta-docs/how-to/customising-the-font?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial).

![Editing component properties](https://anvil-website-static.s3.eu-west-2.amazonaws.com/learn/tutorials/pandas-uplink-tutorial/dragging-label.gif)

Finally, we’ll set the `spacing_above` property of all the components to `large` (you can find this under the Layout section of the Properties Panel).

## Step 3: Get the data into shape

It’s time to clean our data and get it ready to display. Create a file in your favorite development enviroment – open up a new Jupyter notebook, or create a new script in your IDE of choice. All the code included in this tutorial will work with both options.

First, import Pandas at the top. Then, write the `csv_to_df` function, which will fetch the data in from our CSV file and convert it into a Pandas DataFrame.

```python
import pandas

def csv_to_df(f):
    return pd.read_csv(f, index_col=0)
```

Before we can plot our data, we need to clean it and prepare it for plotting. We’ll do all the necessary transformations to our data inside a function that we’ll call `prepare_netflix_data`. In it, we’ll first call our `csv_to_df` function to fetch our data.

```python
def prepare_netflix_data():
    netflix_df = csv_to_df('netflix_titles.csv')
```

Our figures will only use data from the `type`, `country` and `date_added` columns, so we’ll slice our DataFrame using the [`loc` property](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.loc.html).

```python
netflix_df = netflix_df.loc[:,['type', 'country', 'date_added']]
```

There are also some missing values that we’ll have to deal with, so that we don’t include missing data in our plots. Pandas has [a few different ways](https://pandas.pydata.org/docs/user_guide/missing_data.html#missing-data) to handle missing values, but in this case we’ll just use [`dropna`](https://pandas.pydata.org/docs/user_guide/missing_data.html#dropping-axis-labels-with-missing-data-dropna) to get rid of them:

```python
netflix_df = netflix_df.dropna(subset=['country'])
```

The `country` column contains the production country of each movie or TV Show in Netflix. However, some of them have more than one country listed. For simplicity, we’re going to assume the first one mentioned is the most important one, and we’ll ignore the others. We’ll also create a separate DataFrame that only contains the value counts for this column, sorted by country – this will be useful later on when we input the data into our plots.

```python
netflix_df['country'] = [countries[0] for
    countries in netflix_df['country'].str.split(',')]
country_counts = pd.DataFrame(   netflix_df['country'].value_counts().rename_axis('countries').reset_index(name='counts')).sort_values(by=['countries'])
```

Our `date_added` variable currently contains only strings, which is not a very easy format to work with when we want to plot something in chronological order. Because of this, we’ll convert it into datetime format using [`to_datetime`](https://pandas.pydata.org/docs/reference/api/pandas.to_datetime.html?highlight=to_datetime), which will allow us to easily order the data by year later on.

```python
netflix_df['date_added'] = pd.to_datetime(netflix_df['date_added'])
```

We’ll return our transformed dataframe and the `country_counts` variable that we just created. This is how the full function should look by the end:

```python
def prepare_netflix_data():
    netflix_df = csv_to_df('netflix')
    netflix_df = netflix_df.loc[:,['type', 'country', 'date_added']]
    netflix_df = netflix_df.dropna()
    netflix_df['country'] = [countries[0] for countries in netflix_df['country'].str.split(',')]
    country_counts = pd.DataFrame(netflix_df['country'].value_counts().rename_axis('countries').reset_index(name='counts')).sort_values(by=['countries'])
    netflix_df['date_added'] = pd.to_datetime(netflix_df['date_added'])
    return netflix_df, country_counts
```

Finally, we can print the output of `netflix_df` and `country_counts`, to make sure everything looks right.

```python
print(prepare_netflix_data())
```

This is what the output should look like:

![Our code output](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ruvkegf2urknu0ygizmd.png)

Now that we’re happy with how our data looks, we can build our plots but, before that, we’ll have to connect our local script to our Anvil app.

## Step 4: Making the connection

Right now, our code is running on our computer or in our Jupyter notebook, but we will want the plots we build to display on our app and run on the web browser. To be able to do this, we will have to connect our local code to our Anvil app and make the functions we’ve written in our local code callable from the app. In this way, the local code acts as the server, while the app runs all the client-side code in the web browser.

> This is called _client-server_ programming, and it’s how every web app works. Want to learn more? [Check out our client-server explainer article](https://anvil.works/articles/client-vs-server?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial).

To connect our local script to our app, let’s first enable the [Anvil Uplink](https://anvil.works/beta-docs/uplink?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial). In the Editor, click the + button in the Sidebar Menu, select **Uplink** and click on `+ Enable server Uplink`.

![The Uplink dialog](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tgtlwrb6vv90rixrsp2u.png)


After this, we need to `pip install` the Uplink library on our local machine (use `!pip install instead` if working on a Jupyter notebook):

```
pip install anvil-uplink
```

Now we’ll add the following lines at the top of our local script (you can copy them from your Uplink window dialog):

```python
import anvil.server

anvil.server.connect("<your-uplink-key-here>")
```

If you’re using a local script, we will also add [`anvil.server.wait_forever()`](https://anvil.works/beta-docs/uplink/setting_up#connecting?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial) at the very bottom of our script, which will keep our script running to allow our app to call functions in it. All the rest of the code we write going forward will go above this line. This is not necessary for Jupyter notebooks.

```python
anvil.server.wait_forever()
```

> When developing in a notebook, your `anvil.server.callable` functions will only remain accessible while your notebook kernel is active. Many hosted notebook environments, such as Google Colab, will shutdown kernels that have been idle for some time. You may be able to increase this timeout by having a cell continually running - `while True: sleep(1)` will do, and this is exactly what `anvil.server.wait_forever()` does. Even then, hosted kernels are unlikely to run forever, and for production deployments you should see our [Deploy a Google Colab Notebook with Docker guide](https://anvil.works/learn/tutorials/google-colab-web-service?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial).

With this, our local script is connected to our app. This means it can now do anything we can do in an Anvil [Server Module](https://anvil.works/beta-docs/server?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial).

## Step 5: Build the plots

We now need to take our processed data and turn it into the plots that we will want to display. Our data is in a `pandas` DataFrame right now, which means that we require `pandas` to read it. Because of this, we can’t send the data directly to our app because `pandas` is not available to the web browser, or client (learn more about why certain packages are not available in the client in our [Client vs Server code in Anvil explainer](https://anvil.works/articles/client-vs-server?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial)).

However, we can send already built plots instead. Because of this, we’ll first build our figures in our script or notebook, and then we’ll return the plots to our app to display them.

Our dashboard will contain three figures:

- A **map** showing the number of films per production country
- A content type **pie chart**
- A **line chart** of content added through time

Now, we need to create them. We’ll first import `plotly.graph_objects`, then write the `create_plots` function. Before anything else, we’ll call our `prepare_netflix_data` function to fetch our transformed data. After that, we’ll first create and return our map plot, using [Plotly’s Scattergeo](https://plotly.com/python-api-reference/generated/plotly.graph_objects.Scattergeo.html#plotly.graph_objects.Scattergeo). Remember to add this code above the `anvil.server.wait_forever()` call if working on a local script.

```python
import plotly.graph_objects as go

@anvil.server.callable
def create_plots():
    netflix_df, country_counts = prepare_netflix_data()

    fig1 = go.Figure(
    go.Scattergeo(
        locations=sorted(netflix_df['country'].unique().tolist()),
        locationmode='country names',
        text = country_counts['counts'],
        marker= dict(size= country_counts['counts'], sizemode = 'area')))

    return fig1
```

We made this function available from the client side of our app by decorating it as [`@anvil.server.callable`](https://anvil.works/beta-docs/server#calling-server-functions-from-client-code?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial), which means we can access our function’s output from our Anvil app in order to display our figures. We’ll do this by using `anvil.server.call` inside the `__init__` method in our Form1 code to call our function. Then, we’ll asign the output to our plot’s [`data` property](https://anvil.works/beta-docs/api/anvil#Plot_attributes?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial).

```python
class Form1(Form1Template):
def **init**(self, **properties): # Set Form properties and Data Bindings.
  self.init_components(**properties)

      fig1 = anvil.server.call('create_plots')

      self.plot_1.data = fig1
```

We can check that everything is working like we want it to by running our local script or notebook and our app. This is how it looks so far:

![A partial dashboard](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eooauoeevynrah17uzz4.png)

Let’s now do the same with our other two plots. We’ll add a pie chart and a line chart to our `create_plots` function:

```python
@anvil.server.callable
def create_plots():
    netflix_df, country_counts = prepare_netflix_data()

    fig1 = go.Scattergeo(
        locations=sorted(netflix_df['country'].unique().tolist()),
        locationmode='country names',
        text = country_counts['counts'],
        marker= dict(
        size= country_counts['counts'],
        line_width = 0,
        sizeref = 2,
        sizemode = 'area',
        reversescale = True
        ))

    fig2 = go.Pie(
        labels=netflix_df['type'],
        values=netflix_df['type'].value_counts(),
        hole=.4,
        textposition= 'inside',
        textinfo='percent+label'
        )

    fig3 = go.Scatter(
        x=netflix_df['date_added'].dt.year.value_counts().sort_index().index,
        y=netflix_df['date_added'].dt.year.value_counts().sort_index()
        )

    return fig1, fig2, fig3
```

And we’ll call them from the client in the same way we did our previous plot:

```python
class Form1(Form1Template):
def **init**(self, **properties): # Set Form properties and Data Bindings.
    self.init_components(**properties)

        fig1, fig2, fig3 = anvil.server.call('create_plots')

        self.plot_1.data = fig1
        self.plot_2.data = fig2
        self.plot_3.data = fig3

```
With that, we have a functional dashboard:

![A dashboard with no custom styling](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2vxqapu84c7z23d1w5a7.png)<figcaption>How our dashboard looks so far.</figcaption>

## Step 6: Finishing touches

With everything set up, we can move onto customizing our plots. Plotly supports pre-configured templates that describe plot styling, and Anvil provides [pre-built templates](https://anvil.works/beta-docs/client/components/plots#configuring-plots?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial) to match our [Rally and M3 themes](https://anvil.works/blog/announcing-new-themes?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial). We can apply these through the [`templates` property](https://anvil.works/beta-docs/client/components/plots#configuring-plots?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial).

For this dashboard, I’m going to choose the `rally` plot theme, which will match the colour scheme and styling of our app. I will set it as the default plot theme for the app by adding this line of code to the `__init__` method:

```python
class Form1(Form1Template):
def **init**(self, **properties): # Set Form properties and Data Bindings.
    self.init_components(**properties)

        Plot.templates.default = "rally"
```

This is how the plots look now:

![A dashboard with some custom styling](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zmxh97lvd6r7v02hlkof.png)<figcaption>Our dashboard styled with the Rally plots theme.</figcaption>

Finally, we’ll change our plots’ margins and font by using their `layout` property. We’ll also add titles to each one:

```python
fig1, fig2, fig3 = anvil.server.call('create_plots')

self.plot_1.data = fig1
self.plot_1.layout.margin = dict(t=60, b=30, l=0, r=0)
self.plot_1.layout.font.family='Raleway'
self.plot_1.layout.title = 'Production countries'

self.plot_2.data = fig2
self.plot_2.layout.margin = dict(t=60, b=30, l=10, r=10)
self.plot_2.layout.font.family='Raleway'
self.plot_2.layout.title = 'Content breakdown by type'

self.plot_3.data = fig3
self.plot_3.layout.margin = dict(t=60, b=40, l=50, r=50)
self.plot_3.layout.font.family='Raleway'
self.plot_3.layout.title = 'Content added over time'
```

After these changes, our dashboard is ready to be shared! All you need to do now is [publish your app](https://anvil.works/beta-docs/deployment-new-ide/quickstart#publish-your-app?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial) to make it available to the world.

![The final dashboard](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mb5a5e9akt48tjlev4y4.png)<figcaption>The final result.</figcaption>

---

## Want to turn off your computer?

Now that we’ve finished developing our dashboard, you might want to want to stop relying on your local script. You can upload the data set to Anvil, and run our data processing code in an Anvil [Server Module](https://anvil.works/docs/server?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial), so our dashboard will still work if we stop our script or turn off our computer.

If you want to learn how to do this, check out this tutorial:

[**Build a Web App with Pandas -->**](https://anvil.works/learn/tutorials/web-app-with-pandas?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial)

---

## Clone the App

If you want to check out the source code for our Netflix dashboard, click on the links below and clone the app, download the local script and access the Jupyter notebook on Colab:

[**Open in Anvil**](https://anvil.works/build#clone:R3HTVPTFCS7DNSDD%3dUR6CHBYV74JF5E75WCUJJ34I)

[**Download the script**](https://anvil-website-static.s3.eu-west-2.amazonaws.com/learn/tutorials/pandas-uplink-tutorial/pandas-uplink-tutorial.py)

[**Access the Notebook**](https://colab.research.google.com/drive/1cyojdyZ-sPjxtfJXU7T2sprbLd653GGi?usp=sharing)

---

## New to Anvil?

If you’re new here, welcome! [Anvil](https://anvil.works/?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial) is a platform for building full-stack web apps with nothing but Python. No need to wrestle with JS, HTML, CSS, Python, SQL and all their frameworks – just **build it all in Python**.

Yes – Python that [runs in the browser](https://anvil.works/docs/client/python?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial). Python that [runs on the server](https://anvil.works/docs/server?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial). Python that [builds your UI](https://anvil.works/docs/client?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial). A d[rag-and-drop UI editor](https://anvil.works/docs/client/ui?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial). We even have a built-in [Python database](https://anvil.works/docs/data-tables?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial), in case you don’t have your own.

Why not have a play with the app builder? **It’s free!** Click here to get started:

[**Get building -->**](https://anvil.works/build?utm_source=crosspost:dev.to:/learn/tutorials/pandas-uplink-tutorial)