Find the Patterns of a Dataset by Visualizing Frequency Distribution
By: Zubair Hossian
Tags: Getting Started, Data Visualization, Data Science, Statistical Analysis, Machine Learning

Road map…

Why graphical representation of frequency distribution is needed?

Barplot for frequency distribution

Pie Charts for frequency distribution

Histogram for frequency distribution

Skewed Frequency Distribution

Symmetrical Distribution of frequency

Let’s start our journey…

What is Data Visualization?

Data visualization is the practice of translating information into a visual context, such as a map or graph, to make data easier for the human brain to understand and pull insights from. The main goal of data visualization is to find a pattern from data. Data visualization is one of the parts of Data Science. After collecting data we have to modify and modelling the data. At last, it should be visualized for final conclusions.

Why graphical representation of frequency distribution is needed??

To understand it deeply, you must have prior knowledge of Frequency Distribution. To know what is a frequency distribution, how to develop a frequency distribution table. You can read our previous article based on frequency distribution.

In our previous article, we may have a look at how to create a frequency distribution table from raw data. We may face another problem when we try to find patterns from the data. Do you guess how to analyze a frequency table to find patterns from a dataset? If you want to find a pattern from a frequency table, you have to look up the frequency of each unique value or class interval. Just by looking up the frequency of each unique value, can you find any pattern? No, you have to compare the frequencies of each value at the same time. It will be easy for a few unique values or class intervals, or when the frequency values are less and easier to compare. But we will get puzzled if we try to compare the frequency for a large number of unique values. We can solve this problem by visualizing the data. Graphs make it much easier to scan and compare frequencies, providing a single picture of the entire distribution of variables. Because they are easy to grasp and also eye-catching. Sometimes you have to represent data in front of a non-technical audience. Graphs are better way of representation if we need to present our findings to a non-technical audience. In this article, We’ll discuss three kinds of graphs to represent the distribution table:

1.Bar Plots

2. Pie Charts

3. Histograms

We are considering a dataset for better demonstration

Through this article, we are using the wnba.csv dataset. The Women’s National Basketball Association (WNBA) is the professional basketball league in the USA. It has currently composed twelve teams. In our dataset, we have stats from all games of season 2016–2017. The dataset has 143 rows and 32 columns. The overview of a dataset is given below.

Barplot for frequency distribution

A bar plot or bar chart is a graph, representing the category of data with rectangular bars where lengths and heights are proportional to the values they represent.

Barplot was used for variables measured on an ordinal or nominal scale. Example of bar plot to show the population change in Edge from 1801–1961.

Photo by author

How to generate bar plot

To generate a bar plot we need two sets of values.

(i)One set containing a list of unique values.

(ii) Another set containing frequency for each unique value.

We can get two sets of values easily from a frequency table. To get a frequency table, we can use the pandas Series.value_counts() method. Then we have to use the Series. plot.bar() method. We can do that for the Pos (Player Position) variable in our dataset.

We can use Series.value_counts().plot.bar() to create bar plot just using one line code.

wnba[‘Pos’].value_counts().plot.bar()

Output:

photo by author

The Series. lot.bar() method generates vertical bar blot where frequencies are in the y-axis and each unique value in the x-axis. Sometimes, you need to generate a horizontal bar plot. To generate a horizontal bar plot you can use the Series.plot.barh() method.

wnba[‘Pos’].value_counts().plot.barh()

Output:

photo by author

How to Customize Bar Plot

The Series.value_counts().plot.bar() method has several parameters.

(i)x- to set a label for the x-axis.

(ii)y- to set a label for the y-axis.

(iii)color — to set a custom color.

(iv) rot — to rotate the tricks label of the x-axis.

(v) and so on.

By changing those parameters, you can decorate your bar plot as your need.

To know details, you can read the documentation of barplot.

Pie Charts for frequency distribution

A pie chart is a type of chart that visually displays data in a circular graph. Circular, spheres, and angular representations of data are mostly used graphs to represent real-world data. The shape of a pie chart is circular where the pie represents the whole data and the slice of the pie represents the parts of the data and records it discretely.

It can assume as a regular pie. Let us look at the following example that represents the ingredients used to prepare butter cake.

Photo by author

The whole pie represents a value of 100. The various colors represent the ingredients used to prepare the cake. We can understand division just by seeing the slice color.

Why do we use the pie chart?

The main advantage of a pie chart over a bar plot is that it gives a better sense for relative frequencies in the distribution, or proportional. Looking at the bar plot we just see which categories are more or less numerous but we can’t tell the proportion.

Just by seeing the pie chart, we can demonstrate the proportion of each category.

How to generate Pie Chart

We can generate pie charts using the Series.plot.pie() method. We can apply the method on the ‘Pos’ column in our dataset.

`wnba[‘Pos’].value_counts().plot.pie()

Output:

How to Customize Pie Chart

In most cases, we want that percentage of each slice should be displayed inside the slice. We can easily be doing it by setting the autopct parameter. This parameter accepts python strings and we can use ‘%.1f%%’ to have the percentage displayed with a precision of one decimal. In the below, we break down the strings.

photo by author

We can show below how it looks for ‘Pos’ variable with a precision value of 2.

wnba[‘Pos’].value_counts().plot.pie(figsize = (6,6), autopct = ‘%.2f%%’)

Output:

Histogram for frequency distribution

A frequency distribution shows how often each different value in a set of data occurs. The histogram is the most widely used technique to visualize frequency distribution. It looks like a bar plot, but there is a significant difference.

Why do we use the Histogram?

To describe the distribution in elaborate ways when properties of the variable are measured in interval and ratio scales. Let’s try to understand the fact with a real-world example.

Let’s use the ‘PTS’ (total points ) which is measured on a ratio scale.

print(wnba[‘PTS’].describe())

Output:

photo by author

If we try to analyze the above graph, we find that 75% of variables are distributed within a relatively narrow interval. The remaining 25% are distributed in relatively large intervals. To visualize the fact, we have to draw a graph that immediately shows the distribution.

How to generate Histogram

We can generate a histogram using the Series.plot.hist() method. We can apply the method on the ‘PTS’(total points) column in our dataset.

wnba[‘PTS’].plot.hist()

Output:

Photo by author

How to Customize Histogram

Sometimes, you have to create a fixed number of the bar such as 2, 5, 10, 20, 30. What will happen behind the scene if you want to create 5 bars in the histogram for a ‘PTS’ variable? The following steps will help to do your job.

(I) Generate a grouped frequency distribution table for the ‘PTS’ variable with 5 class intervals.

(ii) For each class interval, plot a bar with height corresponding to the frequency of the interval.

print(wnba[‘PTS’].value_counts(bins = 5).sort_index())

Output:

photo by author

Here, the bins parameter denotes the number of class intervals.

Now, we try to represent the data with the help of a histogram.

wnba[‘PTS’].plot.hist(bins=5)

Output:

photo by author

You can also use the arange() function from numpy to generate the values for plotting the frequency of a specific range and interval, use grid parameter to understand bars clearly, use rot parameter for better readability.

from numpy import arange

wnba[‘PTS’].plot.hist(grid = True, xticks = arange(2,585,58.2), rot = 30)

Output:

photo by author

If you analyze the graph, you can find the patterns you wanted to see. We can see that 75% values between 2–277 and the remaining values distributed between 277–584. So the graph is left-sided (skewedly distributed), we will know about it in the letter part of the article.

Skewed Frequency Distribution

A skewed distribution is where one tail is longer than another. These distributions are sometimes called asymmetric or asymmetrical distributions. Two types of skewed Distribution

(i)right-skewed distribution

If the tail points to the right, then the distribution is said to be right skewed.

(ii) left-skewed distribution

If the tail points to the left, then the distribution is said to be left-skewed.

Photo by author

Symmetrical Distribution

A distribution is said to be symmetrical when the distribution on either side of the mean is a mirror image of the other.

A very common symmetrical distribution is one where the values are high in the middle and gradually decrease in frequency toward both ends of the histogram. This pattern is also called Normal Distribution.