Accelerating Data Exploration. Using data describe for exploratory... | by Himanshu Sharma | Towards Data Science

Exploratory data analysis is one of the important initial steps for creating a better understanding of a dataset and the underlying data. It helps us in understanding what the data is all about, what are the different features of the dataset, understanding the association between different data points, etc.
Exploration of data beforehand gives us an edge while creating a Machine learning or deep learning model. It not only helps in understanding the data correlation but also helps in analyzing the statistical properties and any hidden pattern in the dataset.
EDA generally consumes a lot of time and effort because we need to create different bars and graphs to visualize and analyze the data. What if I tell you that you can perform EDA in significantly less time and effort? Yes, it is possible and we can create different plots in single lines of code.
Data Describe is an open-source python library that can be used to easily understand and analyze the dataset by creating different visualizations.
In this article, we will use Data Describe to perform EDA easily and effortlessly.
Let’s get started...
We will start by installing a Data Describe using pip. The command given below will do that.
!pip install data_describe
In this step, we will import the required libraries and functions to create a machine learning model and dashboard. We will be using the IRIS dataset for this article so we will be importing that from sklearn. The link for the dataset is given below:
scikit-learn.org
import pandas as pdfrom sklearn.datasets import load_irisdat = load_iris()df = pd.DataFrame(dat['data'], columns=dat['feature_names'])df['outcome'] = dat['target']df.head()
Now we will start with the Data Exploration. We will start by analyzing the statistical summary and then creating different graphs and plots.
a. Statistical Summary
dd.data_summary(df)
Here we can analyze the statistical summary which shows the Median, Mode, Quartiles, etc of the dataset.
b. Heatmap
dd.data_heatmap(df)
c. Correlation Matrix
dd.correlation_matrix(df)
The correlation matrix helps in understanding the correlation between the data points.
d. Distribution Plot
The Distribution Plot shows the distribution of the dataset, which can help us in understanding the skewness of the dataset. We will be using the ipywidget python library to plot the distribution plot of all the data points.
from IPython.display import displayfor col in df.columns:     display(dd.distribution(df,                                 plot_all=True).plot_distribution(col))
e. Scatter Plot
dd.scatter_plots(df, plot_mode='matrix')
f. Cluster Plot
Cluster Plot helps in visualizing the clusters of the target variable.
dd.cluster(df)
g. Feature Importance
This is one of the most important plots which helps us to identify the most important features in the dataset. We will be using the RanbdomForest Classifier for feature importance.
These are some of the plots and graphs that we can create using data Describe for analyzing and visualizing the dataset. Go ahead try this with different datasets and create different plots easily and effortlessly. In case you find any difficulty please let me know in the response section.
This article is in collaboration with Piyush Ingale.
Thanks for reading! If you want to get in touch with me, feel free to reach me at hmix13@gmail.com or my LinkedIn Profile. You can view my Github profile for different data science projects and packages tutorials. Also, feel free to explore my profile and read different articles I have written related to Data Science.