Descriptive and Inferential Statistics for Data Science
By: Vishal Sharma
Tags: Machine Learning, Data Science, AI, Math, Mathematics

MATHS FOR DATA SCIENCE

Descriptive and Inferential Statistics for Data Science

Inferential and Descriptive Statistics!

Photo by Dhru J on Unsplash

Statistics is mainly divided into two domains- descriptive and inferential. While one is used to describing a dataset summary, other one is used for bringing out meaningful numbers of a population from a sample.

In the series of “Maths for Data Science” articles, I have talked about Probability, Regression, Normal Distribution, and other important subjects. But, this article is going to be on the lighter side of knowledge. Some key concepts I will cover are of descriptive statistics including mean, median, standard deviation, etc. Along with it, the power of inferential statistics will also be covered.

Descriptive Statistics

Before diving into descriptive statistics, let’s talk about data first.

Data is the new oil — Clive Humby

You may think of data as numbers in a spreadsheet but data can come in many forms — from the text, spreadsheets, images to databases. In this modern era, data is an important asset to utilize a new way of the world. No matter what field you are in — insurance, banking, or healthcare, data is utilized to make better decisions and accomplish goals.

There is mainly two type of data — Quantitative and Categorical.

Quantitative data takes on numeric values that allow us to perform mathematical operations (like the number of students in a class).

Quantitative- Continuous and Discrete

Continuous data can be split into smaller and smaller units like height. We can split height into centimeters, meters, millimeters, feet, inches, and many more units. Discrete data only takes on countable values like number of students in a class.

Categorical is used to label a group or set of items (like gender — male, female, transgender).

Categorical — Nominal and Ordinal!

Ordinal data take on a ranked ordering like Excellent, Good, Poor. And, Nominal data do not have an order or ranking like gender of a person.

Time for descriptive stats introduction! A descriptive statistic as the name suggests describes a dataset quantitatively.

Use the summary features of a dataset to answer questions!

Quantitative data can be having four aspects from which we can describe our dataset.

Measures of Center (Mean, Median, Mode) Measures of Spread The Shape of the data. Outliers

Measures of Center

Mean: Mean is the total sum of values divided by total number of values.

Median: The middle number in the list from lowest to highest.

Mode: Mode is the value that appears the most or the most frequent number in your dataset.

Measures of Spread

Measures of Spread are used to provide us an idea of how spread out our data is from one another.

Before jumping into the methods, let’s first understand the five number summary:

Minimum: The smallest number in the dataset. Q1​: The value such that 25% of the data falls below. Q2​: The value such that 50% of the data falls below. Q3​: The value such that 75% of the data falls below. Maximum: The largest value in the dataset.

Now, let’s look at some of the most common methods to check the data spread.

Range: It is the difference between maximum and minimum value of the list or your data.

Interquartile Range (IQR): The interquartile range is calculated as the difference between Q3​ and Q1.

Variance: The variance is the average squared difference of each observation from the mean.

Standard Deviation: Average distance of each observation from the mean. Also, The standard deviation is the square root of the variance

The shape of the Data

Using a histogram is the best way to visualize the spread of data. Looking at it, we can identify the shape of our data — right-skewed, left-skewed, or symmetric.

Symmetric: Mean equals Median

Right-Skewed: Mean greater than Median

Left-Skewed: Mean less than Median

Outliers

Outliers are the points that fall very far from the rest of the data points. This generally affects measures like the mean and standard deviation.

There can be the following aspects related to them:

Typo error Understanding why they exist Use 5 points summary to understand the outliers

You can use these formulas to check that a given number is an outlier or not.

Inferential Statistics

Now that we have described our data, it is time to draw conclusion on a larger population using the collected dataset.

For inferential statistics, we must be clear about a few terms:

Sample: a subset of the population.

Population: our entire group of interest.

Parameter: numeric summary about a population.

Statistic: numeric summary about a sample.

For example, we send our ad catalog to 10,000 individuals. And, 5000 of them replied. In these 5000 individuals, 75% were interested in our catalog ad. But, we wanted to find out how many individuals from the whole population are interested in our ad?

Population= 10,000 Sample=5000 Statistic= 75% Parameter= Proportion of the population that is interested in ad

So, we have to draw a conclusion for all of the 10,000 individuals from the responses we have got i.e only 5000. Draw a conclusion from these 5000 and apply it to the bigger population!

Binomial Distribution, Normal Distribution, Confidence Intervals, Regression, and Central Limit Theorem are some indexes to inferential statistics.

I have made separate articles on them!

Summary

I have talked about Descriptive and Inferential statistics in this article:

Descriptive Statistics Inferential Statistics

Peace!

In Plain English

Enjoyed this article? If so, get more similar content by subscribing to our YouTube channel!