Visualization is Important for EDA, Why?
By: Ezgi Gumusbas
Tags: Seaborn, Matplotlib, Data Science, Data Visualization, Exploratory Data Analysis

Data Visualization

Visualization is Important for EDA, Why?

Importance of visuals for data scientists

Photo by Lukas Blazek on Unsplash

When we firstly look at the data in the pandas dataframe, we generally can understand columns’ characteristics. With the simple codes, we can see types of columns (.info), and other statistical numbers such as min, max, median, mean (.describe), etc. But, one of the most important points for EDA is to determine outliers and understand the trend of target and other features. And, it is super easy to see them with visualizations. In this blog, I will give some examples to show what we can define with simple and quick visualizations.

Outliers:

When I was working on the Tanzanian water well dataset, I realized that some of the locations in the data were out of Tanzania. The graph above helps me to describe this problem. It is super fast and easy to visualize the longitude and latitude of the water wells with a scatter plot and understand if the data has outliers or wrong values for corresponding columns.

Distribution of Features:

The graph above shows the distribution of the feature (water quantity of the well) according to the target. We can see the same results in the dataframe also, but when I visualize the distribution, it helps to see the overall picture easily and save my time to figure out the insights of data. Seaborn’s count plot is super useful for this type of graph.

Maps:

This map shows the location of the water wells. Each color represents the status of the wells as functional, non-functional, and functional but needs repair. We can do this mapping using the Folium library in Python. We can see clusters easily and learn more about the distribution of the wells according to status. Another good feature is that Folium is an interactive library, we can zoom in. For example, we can look closer to selected regions which has large clusters, even street by street. As another approach, we can choose only one label of the target which we want to focus more. For instance, the below graph shows the functional but needs repair water wells distribution;

Easy Comparison:

When we visualize the data, we can see lots of important things in the same graph. For example, I can compare weekly holiday sales according to store types in the same graph above, even we can see average lines that the red line shows the average holiday sales and the green line represents the average non-holiday sales. This is just an example, we can add many indicators to the same graphs for comparison.

Another good example of an easy comparison below;

This graph shows the weekly sales of the store. We have 52 weeks to compare, so we can sort them in the dataframe and look the list up to down. We can calculate differences between them, take averages, or do advanced statistical analysis with this dataframe. But, for the first insight, visualizations help us to see everything in the same place and give the opportunity to a quick comparison.

Last example for comparison;

Box plots are super useful to see the minimum, first quartile, median, third quartile, and maximum values and compare different labels according to these statistical values.

Peak points of Time Series:

When we look at the stock prices or sales data of the stores, we have to focus on extreme changes such as peak points or lowest points. The graph above shows the average weekly sale of the retail store. When we visualize this data, it is obviously seen that the last months of the year have a different distribution than other months. It is not easy to see this distribution in the data frame.

There are many other reasons to use visualization techniques for EDA. For example, word clouds are useful visuals where the words are shown in varying sizes depending on how often they appear in your data. Like word clouds, there are many other useful visualization techniques to see the features of the data and it is obvious that the first step before the modeling is EDA because we cannot model the data well if we do not have some strong insights about data.