Interactive Exploratory Data Analysis that Generates Python | by Roman Orac | Towards Data Science

Exploratory Data Analysis (EDA) is one of the first steps in the Data Science process — usually, it follows the data extraction. EDA helps us to get familiar with the data before we proceed with modeling or decide to repeat the extraction step.
EDA helps Data Scientists to:
get familiar with the data
find bugs in the data extraction process
decide if the data needs cleaning
decide what to do with the missing values if there are any
visualize data distributions, etc.
With Exploratory Data Analysis we get the “feel” for the data
By reading this article you’ll get:
A practical example of EDA in JupyterLab on Avocado dataset
A code snippet of each interactive operation
An efficient way to share your analysis with your colleagues
In case you’ve missed my previous articles about this topic, see Mito — A Spreadsheet that Generates Python.
In this article, we’ll analyze the avocado dataset - one of the more popular datasets on Kaggle — and show a few basic operations in the Exploratory Data Analysis.
The avocado dataset contains historical data on avocado prices and sales volume in multiple US markets. You can download it from Kaggle. The original data stems from the HASS AVOCADO BOARD. The Kaggle dataset is licensed under the Open Database License (ODbL).
We’ll analyze this dataset with the Mito extension for JupyterLab, which enables users to complete their EDA interactively inside JupyterLab, and have each edit generate the equivalent Python in the code cell below.
First, we need to load the Mitosheet interactive environment in Jupyter Lab with this function:
# make sure you have Mito installed before running this functionimport mitosheetmitosheet.sheet()
These functions call an empty Mitosheet into the Jupyter Environment
To load the data into the sheet, we can either pass in an existing Dataframe as an argument to the Mitosheet call:
mitosheet.sheet(df)
Or we can click the Import button in the toolbar and select a file from our local file system.
When we select “avocado.csv” in the import menu, Mito will automatically turn the CSV file into a Dataframe and generate the equivalent Python in the code cell below:
# Imported /Users/Downloads/avocado.csvavocado_csv = pd.read_csv(r’/Users/Downloads/avocado.csv’)
The CSV file already has an index column, which appears in the Mitosheet as “Unnamed: 0”, so the first thing we want to do is delete this column, which we can do by clicking the delete column button in the toolbar.
This will generate the equivalent Python in the code cell below. The code is also auto-documented:
# Deleted column Unnamed: 0 from avocado_csvavocado_csv.drop('Unnamed: 0', axis=1, inplace=True)
Let’s say we are interested in the different regions of avocados in the dataset.
We can select the column menu for the “Region” column and go to the values tab.
In the values tab, we can see:
all the unique values in the column,
the frequency of each of those values,
and what percent of the total entries each value represents.
From this distribution, it seems the avocados are equally distributed by region.
We have decided that we want to only look at rows where the avocado’s region is “Albany.”
We can select the Filter/Sort tab in the column menu and set a filter where the “region” column is exactly “Albany.”
The generated code for this filter looks like this:
# Filtered region in avocado_csvavocado_csv = avocado_csv[avocado_csv['region'] == 'Albany']
Now that we have filtered the dataset, we want to explore the values we are left within the “Date” column.
First, let’s use the add column button in the toolbar.
Next, let’s add a formula to this column that parses the month value from each of the date values in the “Date” column.
=MONTH(Date)
Once we have retrieved the month values in a new column, we can use a pivot table to count how many entries there are for each month and compare the aggregated counts.
All we need to do is click the “Pivot” button in the toolbar and configure the table from the side menu that appears.
This pivot table operation will generate the following Python code snippet:
# Pivoted avocado_csv into df3unused_columns = avocado_csv.columns.difference(set(['Month']).union(set([])).union(set({'Month'})))tmp_df = avocado_csv.drop(unused_columns, axis=1)pivot_table = tmp_df.pivot_table(    index=['Month'],    values=['Month'],    aggfunc={'Month': ['count']})
In the column menu, we can also apply a sort by the “Month count” column to see the list of counts for each month in descending order.
Here we can see that for the filtered Albany entries, January has the most entries and June and September have the least entries.
Finally, we can visualize the analysis by clicking the “Graph’ button in the toolbar. We can configure it in the side menu.
You can download this visualization as PNG so that can share it with your colleagues.
We can also copy the equivalent code for any visualization by clicking the “Copy Graph Code” button.
The outputted code for this graph is:
# Import plotly and create a figureimport plotly.graph_objects as gofig = go.Figure()# Add the bar chart traces to the graphfor column_header in ['Month']:    fig.add_trace(        go.Bar(             x=df3[column_header],            y=df3['Month count'],            name=column_header        )    )# Update the title and stacking mode of the graph# See Plotly documentation for customizations: https://plotly.com/python/reference/bar/fig.update_layout(    xaxis_title='Month',    yaxis_title='Month count',    title='Month, Month count bar chart',    barmode='group',)fig.show(renderer="iframe")
If we run the above code, the same chart that appears in the Mitosheet will appear below the code cell as an output.
We went through a few simple operations in the EDA process. While this is not a comprehensive EDA guide, it shows how we can avoid typing code for the simple data wrangling operations in the initial EDA.
Generated code is useful for sharing it with colleagues and for inexperienced Data Scientists to learn “the pandas way” of doing analysis.
Mito allows for powerful EDA processes to happen interactively within a JupyterLab Ecosystem.
Here are the links to learn more about Mito and the link to the Avocado Dataset.
If you enjoy reading these stories, why not become a Medium paying member? It is $5 per month, and you will get unlimited access to 10000s of stories and writers. If you sign up using my link, I will earn a small commission.