All the Basics of Frequency Distribution | Towards Data Science

Data plays a key role in every organization because it helps business leaders to make suitable decisions based on facts, statistical numbers, and trends. Due to this growing scope of data, data science came into the picture which is a multidisciplinary field. In data science, data analysis is the most vital part. To understand data clearly we have to know the knowledge of Frequency Distribution of statistics.
The main purpose of the data analysis is to gain information from data so that we can take better decisions for our system, organization, or any problem. What’s going on in your mind?. We can easily analyze data just by looking in a table format. Yeah! we can when the dataset is small. What if for a large dataset!!! Imagine you have a dataset of 1000 rows and 50 columns. Can you analyze this dataset just by looking? In order to analyze this type of large dataset, at first, we have to simplify it. Frequency distribution is one of the important techniques to analyze the data.
Table Of Contents:
What is Frequency Distribution ?Figure Out Frequency Table Using Real World Example.Frequency Table For Ordinal, Interval, or Ratio Scales Variable.What is Relative frequency and percentage frequency.How to Make Grouped Frequency Distribution Table.Frequency Distribution of a Continuous Variable.Final Words.
What is Frequency Distribution ?
Figure Out Frequency Table Using Real World Example.
Frequency Table For Ordinal, Interval, or Ratio Scales Variable.
What is Relative frequency and percentage frequency.
How to Make Grouped Frequency Distribution Table.
Frequency Distribution of a Continuous Variable.
Final Words.
What is Frequency Distribution ?
A frequency distribution is a representation, either in a graphical or tabular format, that displays the number of observations within a given interval or categories. Sometimes, it is also called the Frequency Distribution table.
Let’s have look at the table. It has two columns. One column records the unique variable’s name. Another column records the number of observations or occurrences of each unique value.
Through this article, we are using wnba.csv dataset. It has 143 rows and 32 columns. A complete overview of the dataset is given below. Feel free to observe the dataset.
It’s time to try something new. We are going to create a frequency table using python. we can use Series.value_counts() method. And we will try to make an frequency table of pos(position) column in our dataset.
import pandas as pdwnba = pd.read_csv(‘wnba.csv’)freq_dis_pos = wnba[‘Pos’].value_counts()freq_dis_pos
Output:
We can also get the frequency table of other columns using the same type of code. But keep in mind that it will only work fine for the categorical variables.
In our output, we see that values are in descending order. This order helps us to know that which has the maximum value of frequency. This order helps us if we have a nominal variable case. If your variable is measured in ordinal, interval, or ratio scales, it becomes more difficult to analyze. In order to understand the variable, you can see our previous post on variable.
towardsdatascience.com
In summary, this table will help you to find the variable type.
Now we try to find the frequency table of height column.
freq_dis_height = wnba[‘Height’].value_counts()freq_dis_height
Output:
Sometimes, you are in trouble. To solve this problem you might have to sort the table with respect to index value .Then you can use Series.sort_index() method.
freq_dis_height = wnba[‘Height’].value_counts().sort_index()freq_dis_height
Output:
Sometimes you need to transform data in ascending order. To represent data in ascending order, we can set the ascending parameter false.
freq_dis_height =wnba[‘Height’].value_counts().sort_index(ascending= False)freq_dis_height
Output:
Can you make the above table on an ordinal scale? Yeah, you can. At first, divide the whole range of height into some intervals. For example, in the given dataset the height ranges from 165 cm to 206 cm. Suppose, total range is 45 cm. Now, divide it into 5 categories each with a 9 cm interval. And give some categorial name to each interval like tall, short or 1st, 2nd, etc. Then calculate the frequency.
What is Relative frequency and percentage frequency?
The above discussed frequency is known as absolute frequencies of a certain variable.
(i)Relative Frequency:
The relative frequency of a particular observation or class interval is found by dividing the frequency(f) by the number of observation(n).
Relative frequency = frequency ÷ number of observations
(ii)Percentage Frequency:
The percentage frequency is found by multiplying 100 to the relative frequency.
Percentage frequency = relative frequency X 100
In pandas library, we can compute all the proportions at once by dividing each frequency with the total number of instances. An example is shown below with the Women’s National Basketball Association (wnba.csv) dataset.
wnba[‘Age’].value_counts() / len(wnba))
But it’s slightly been faster by setting the Series.value_counts() normalize value became True. Then simply the output multiply with 100.
percentages_pos = wnba[‘Age’].value_counts(normalize=True).sort_index() * 100percentages_pos
Output:
This percentage will help us to find the important information as per our need.
Percentiles rank
The percentile rank of a score is the percentage of score in its distribution and lower than it. To find percentiles rank, We can use a library called scipy.stats.percentileofscore in python.
If we want to find the percentiles rank of index 25. We just write the code as below.
from scipy.stats import percentileofscorepercentile_of_25 = percentileofscore(wnba[‘Age’], 25, kind = ‘weak’)percentile_of_25
Output:
40.55944055944056
You are very much surprised to know that we can easily find percentiles just write one line code. Pandas Series.describe() method help us to find percentiles.
persecntiles = wnba[‘Age’].describe()
Output:
We are not interested in the value of the first three rows. The 25th,50th, and 75th are returned by default, the scores have divided the distribution into four equal parts. Also known as quartiles. The first quartile (also called the lower quartile) is 24 (note that 24 is also the 25th percentile). That means 25% of the total data are within 0 to 24 years. The second quartile (also called the middle quartile) is 27 (note that 27 is also the 50th percentile). And the third quartile (also called the upper quartile) is 30 (note that 30 is also the 75th percentile).
We may be interested to find the percentiles for percentages other than 25%, 50%, or 75%. For that, we can use the percentiles parameter of pandas Series.describe().This method requires to pass the percentages as we want in between 0 and 1.
persecntiles = wnba[‘Age’].describe(percentiles = [.1, .15, .33, .5, .592, .85, .9])persecntiles
Output:
How to Make Grouped Frequency Distribution Table?
Sometimes frequency distribution tables are not well organised. Then we have to find grouped frequency distribution table. We set the limit of the grouped value by just changing the bins parameter in the pandas series.value_counts() method.
grouped_freq = wnba[‘Age’].value_counts(bins = 5).sort_index()grouped_freq
Output:
Sometimes, this interval will not give better output. In order to get better output, we have to create custom interval. Pandas give us the oppurtunity to create custom range.
ntervals = pd.interval_range(start = 0, end = 600, freq = 100)intervals
Output:
IntervalIndex([(0, 100], (100, 200], (200, 300], (300, 400], (400, 500], (500, 600]], closed=’right’, dtype=’interval[int64]’)
Here , we have to provide three parameter. Start parameter gives the starting point of our range. End parameter gives end point of the custom range and Freq value gives the number of value in each frequency.
Frequency Distribution of a Continuous Variable
For a continuous variable, if we take a class for each distinct value of the variable, the number of classes will become unduly large, thus defeating the purpose of tabulation.
When continuous variables are used in tables, their values are often grouped into categories. Here, We can use the concept of the interval which we learn previously.
Lastly,
Throughout the article, we have to learn how to organize the data by using the Frequency Distribution Table. We have to know that how powerful the frequency distribution table is! The frequency Distribution table helps us to understand the data deeply. However, it’s time to know, how to visualize this organize data. To know about it, please stay with me. I will come back with the necessary visualize techniques soon.
Previous series of articles about the basics of data science
towardsdatascience.com
towardsdatascience.com
Interesting article which will help you to know how to embed the interactive dataset with articles
towardsdatascience.com
If you enjoy the article, follow me on medium for more.
Connect me on LinkedIn for collaboration.