Find the pattern of dataset by visualizing frequency distribution | Towards Data Science

“Visualization gives you answers to questions you didn’t know you had.” — Ben Schneiderman
Data science is defined as the art of interpreting data and getting useful information out of it. At the same time, Data visualization involves the representation of data. Though they are from completely different entities, the context is same. Data Science is not a single process, method, or workflow. We can bound in a way that data visualization is the subset of Data Science.
The best example of data science on our day-to-day basis is E-commerce recommendation for a user while shopping. The machine is learning about the user’s activities regularly and manipulates it thus by giving the best recommendation based on your interests and choice of shopping. It is very hard to analyze raw data. Data scientists need to visualize the manipulates data and analyze it to provide the best choices for users. And this is where data visualization comes into the picture.
There are several statistical and mathematical approaches by which we can get the insight information about a dataset. Among them, frequency distribution is the most easiest but important technique.
Why graphical representation of frequency distribution is needed?
Barplot for frequency distribution
Pie Charts for frequency distribution
Histogram for frequency distribution
Skewed Frequency Distribution
Symmetrical Distribution of frequency
Let’s start our journey...
Data visualization is the practice of translating information into a visual context, such as a map or graph, to make data easier for the human brain to understand and pull insights from. The main goal of data visualization is to find a pattern from data. Data visualization is one of the parts of Data Science. After collecting data we have to modify and modelling the data. At last, it should be visualized for final conclusions.
To understand it deeply, you must have prior knowledge of Frequency Distribution. To know what is a frequency distribution, how to develop a frequency distribution table. You can read our previous article based on frequency distribution.
towardsdatascience.com
In our previous article, we may have a look at how to create a frequency distribution table from raw data. We may face another problem when we try to find patterns from the data. Do you guess how to analyze a frequency table to find patterns from a dataset? If you want to find a pattern from a frequency table, you have to look up the frequency of each unique value or class interval. Just by looking up the frequency of each unique value, can you find any pattern? No, you have to compare the frequencies of each value at the same time. It will be easy for a few unique values or class intervals, or when the frequency values are less and easier to compare. But we will get puzzled if we try to compare the frequency for a large number of unique values. We can solve this problem by visualizing the data. Graphs make it much easier to scan and compare frequencies, providing a single picture of the entire distribution of variables. Because they are easy to grasp and also eye-catching. Sometimes you have to represent data in front of a non-technical audience. Graphs are better way of representation if we need to present our findings to a non-technical audience. In this article, We’ll discuss three kinds of graphs to represent the distribution table:
1.Bar Plots
2. Pie Charts
3. Histograms
We are considering a dataset for better demonstration
Through this article, we are using the wnba.csv dataset. The Women’s National Basketball Association (WNBA) is the professional basketball league in the USA. It has currently composed twelve teams. In our dataset, we have stats from all games of season 2016–2017. The dataset has 143 rows and 32 columns. The overview of a dataset is given below.
A bar plot or bar chart is a graph, representing the category of data with rectangular bars where lengths and heights are proportional to the values they represent.
Barplot was used for variables measured on an ordinal or nominal scale. Example of bar plot to show the population change in Edge from 1801–1961.
How to generate bar plot
To generate a bar plot we need two sets of values.
(i)One set containing a list of unique values.
(ii) Another set containing frequency for each unique value.
We can get two sets of values easily from a frequency table. To get a frequency table, we can use the pandas Series.value_counts() method. Then we have to use the Series. plot.bar() method. We can do that for the Pos (Player Position) variable in our dataset.
We can use Series.value_counts().plot.bar() to create bar plot just using one line code.
wnba[‘Pos’].value_counts().plot.bar()
Output:
The Series. lot.bar() method generates vertical bar blot where frequencies are in the y-axis and each unique value in the x-axis. Sometimes, you need to generate a horizontal bar plot. To generate a horizontal bar plot you can use the Series.plot.barh() method.
wnba[‘Pos’].value_counts().plot.barh()
Output:
How to Customize Bar Plot
The Series.value_counts().plot.bar() method has several parameters.
(i)x- to set a label for the x-axis.
(ii)y- to set a label for the y-axis.
(iii)color — to set a custom color.
(iv) rot — to rotate the tricks label of the x-axis.
(v) and so on.
By changing those parameters, you can decorate your bar plot as your need.
To know details, you can read the documentation of barplot.
A pie chart is a type of chart that visually displays data in a circular graph. Circular, spheres, and angular representations of data are mostly used graphs to represent real-world data. The shape of a pie chart is circular where the pie represents the whole data and the slice of the pie represents the parts of the data and records it discretely.
It can assume as a regular pie. Let us look at the following example that represents the ingredients used to prepare butter cake.
The whole pie represents a value of 100. The various colors represent the ingredients used to prepare the cake. We can understand division just by seeing the slice color.
Why do we use the pie chart?
The main advantage of a pie chart over a bar plot is that it gives a better sense for relative frequencies in the distribution, or proportional. Looking at the bar plot we just see which categories are more or less numerous but we can’t tell the proportion.
Just by seeing the pie chart, we can demonstrate the proportion of each category.
How to generate Pie Chart
We can generate pie charts using the Series.plot.pie() method. We can apply the method on the ‘Pos’ column in our dataset.
`wnba[‘Pos’].value_counts().plot.pie()
Output:
How to Customize Pie Chart
In most cases, we want that percentage of each slice should be displayed inside the slice. We can easily be doing it by setting the autopct parameter. This parameter accepts python strings and we can use ‘%.1f%%’ to have the percentage displayed with a precision of one decimal. In the below, we break down the strings.
We can show below how it looks for ‘Pos’ variable with a precision value of 2.
wnba[‘Pos’].value_counts().plot.pie(figsize = (6,6), autopct = ‘%.2f%%’)
Output:
A frequency distribution shows how often each different value in a set of data occurs. The histogram is the most widely used technique to visualize frequency distribution. It looks like a bar plot, but there is a significant difference.
Why do we use the Histogram?
To describe the distribution in elaborate ways when properties of the variable are measured in interval and ratio scales. Let’s try to understand the fact with a real-world example.
Let’s use the ‘PTS’ (total points ) which is measured on a ratio scale.
print(wnba[‘PTS’].describe())
Output:
If we try to analyze the above graph, we find that 75% of variables are distributed within a relatively narrow interval. The remaining 25% are distributed in relatively large intervals. To visualize the fact, we have to draw a graph that immediately shows the distribution.
How to generate Histogram
We can generate a histogram using the Series.plot.hist() method. We can apply the method on the ‘PTS’(total points) column in our dataset.
wnba[‘PTS’].plot.hist()
Output:
How to Customize Histogram
Sometimes, you have to create a fixed number of the bar such as 2, 5, 10, 20, 30. What will happen behind the scene if you want to create 5 bars in the histogram for a ‘PTS’variable? The following steps will help to do your job.
(I) Generate a grouped frequency distribution table for the ‘PTS’ variable with 5 class intervals.
(ii) For each class interval, plot a bar with height corresponding to the frequency of the interval.
print(wnba[‘PTS’].value_counts(bins = 5).sort_index())
Output:
Here, the bins parameter denotes the number of class intervals.
Now, we try to represent the data with the help of a histogram.
wnba[‘PTS’].plot.hist(bins=5)
Output:
You can also use the arange() function from numpy to generate the values for plotting the frequency of a specific range and interval, use grid parameter to understand bars clearly, use rot parameter for better readability.
from numpy import arangewnba[‘PTS’].plot.hist(grid = True, xticks = arange(2,585,58.2), rot = 30)
Output:
If you analyze the graph, you can find the patterns you wanted to see. We can see that 75% values between 2–277 and the remaining values distributed between 277–584. So the graph is left-sided (skewedly distributed), we will know about it in the letter part of the article.
A skewed distribution is where one tail is longer than another. These distributions are sometimes called asymmetric or asymmetrical distributions. Two types of skewed Distribution
(i)right-skewed distribution
If the tail points to the right, then the distribution is said to be right skewed.
(ii) left-skewed distribution
If the tail points to the left, then the distribution is said to be left-skewed.
A distribution is said to be symmetrical when the distribution on either side of the mean is a mirror image of the other.
A very common symmetrical distribution is one where the values are high in the middle and gradually decrease in frequency toward both ends of the histogram. This pattern is also called Normal Distribution.
Throughout the article, we try to learn about the technique of visualizing frequency distribution. However, when variables are measured in a nominal or ordinal scale, we should use a bar or pie chart. But if the variables are measured on an interval or ratio scale then we have to use the histogram.
If you are a data science enthusiast, please stay connected with me. I will come back shortly with another interesting article. Don’t forget to put some claps, if you like the article.
Previous series of articles about the basics of data science
towardsdatascience.com
towardsdatascience.com
Interesting article which will help you to know how to embed the interactive dataset with articles
towardsdatascience.com
If you enjoy the article, follow me on medium for more.
Connect me on LinkedIn for collaboration.