4 Libraries that can perform EDA in one line of python code | by Satyam Kumar | Towards Data Science

Exploratory data analysis (EDA) is an approach to analyze the data and find patterns, visual insights, etc. that the data set is having, before proceeding to model. One spends a lot of time doing EDA to get a better understanding of data, that can be minimized by using auto visualizations tools such as Pandas-profiling, Sweetviz, Autoviz, or D-Tale
EDA involves a lot of steps including some statistical tests, visualization of data using different kinds of plots, and many more. Some of the steps of EDA are discussed below:
Data Quality Check: Can be done using pandas library functions like describe(), info(), dtypes(), etc. It is used to find several features, its datatypes, duplicate values, missing value, etc.
Statistical Test: Some statistical tests like Pearson correlation, Spearman correlation, Kendall test, etc are done to get a correlation between the features. It can be implemented in python using the stats library.
Quantitative Test: Some quantitative test is used to find the spread of numerical features, count of categorical features. It can be implemented in python using the functions of the pandas library.
Visualization: Feature visualization is very essential to get an understanding of the data. Graphical techniques like bar plots, pie charts are used to get an understanding of categorical features, whereas scatter plots, histograms are used for numerical features.
To perform the above-mentioned tasks we need to type several lines of code. Here auto-visualization library comes into the play, which can perform all these tasks using just 1 line of code. Some of these auto-visualization tools we will discuss in this article:
Pandas-Profiling
Sweetviz
Autoviz
D-Tale
The dataset used for exploratory data analysis using the pandas-profiling library is the Titanic dataset downloaded from Kaggle.
Pandas profiling is an open-source python library that automates the EDA process and creates a detailed report. Pandas Profiling can be used easily for large datasets as it is blazingly fast and creates reports in a few seconds.
You can install pandas-profiling using PyPl:
pip install pandas-profiling
GitHub repository for pandas profiling.
The pandas-profiling library generates a report having:
An overview of the dataset
Variable properties
Interaction of variables
Correlation of variables
Missing values
Sample data
Sweetviz is an open-source python auto-visualization library that generates a report, exploring the data with the help of high-density plots. It not only automates the EDA but is also used for comparing datasets and drawing inferences from it. A comparison of two datasets can be done by treating one as training and the other as testing.
You can install Sweetviz using PyPl:
pip install sweetviz
GitHub repository for Sweetviz.
The Sweetviz library generates a report having:
An overview of the dataset
Variable properties
Categorical associations
Numerical associations
Most frequent, smallest, largest values for numerical features
Autoviz is an open-source python auto visualization library that mainly focuses on visualizing the relationship of the data by generating different types of plot.
You can install Autoviz using PyPl:
pip install autoviz
GitHub repository for Autoviz.
The Autoviz library generates a report having:
An overview of the dataset
Pairwise scatter plot of continuous variables
Distribution of categorical variables
Heatmaps of continuous variables
Average numerical variable by each categorical variable
D-Tale is an open-source python auto-visualization library. It is one of the best auto data-visualization libraries. D-Tale helps you to get a detailed EDA of the data. It also has a feature of code export, for every plot or analysis in the report.
You can install D-Tale using PyPl:
pip install dtale
GitHub repository for D-Tale.
The dtale library generates a report having:
An overview of the dataset
Custom filters
Correlation, Charts, and Heatmaps
Highlight datatypes, missing values, ranges
Code export
I prefer to do my EDA with self-defined functions using several python libraries. The above-discussed libraries should be used to speed up your work.
For beginners, it is good to start doing EDA using the pandas' library and writing python code before trying these libraries, as it is more important to be equipped with the fundamental knowledge and programming practices.
The best data auto-visualization amongst the above discussed is the DTale library, as it reports with detailed EDA, custom filters, and code export. Code export is the main highlight of this library that makes it better than others.
[1] Towards Data Science (Aug 30, 2020): EDA with 1 line of python code
Loved the article? Become a Medium member to continue learning without limits. I’ll receive a small portion of your membership fee if you use the following link, with no extra cost to you.
satyam-kumar.medium.com
Thank You for Reading