How and why to Standardize your data: A python tutorial
By: Serafeim Loukas
Tags: Normalization, Sklearn, Standardization, Machine Learning, Scikit Learn

How and why to Standardize your data: A python tutorial

In this post I explain why and how to apply Standardization using scikit-learn in Python

Figure taken from: https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html. Left subplot: the unscaled data. Right subplot: the transformed data.

Hi there.

This is my first Medium post. I am an electrical & computer engineer currently finishing my PhD studies in the biomedical engineering and computational neuroscience field. I have been working on machine learning problems for the past 4 years. A very common question that I see all around the web is how to standardize and why to do so, the data before fitting a machine learning model.

How does scikit-learn’s StandardScaler work ?

The first question that comes to one’s mind is:

Why to standardize in the first place?

Why to standardize before fitting a ML model?

Well, the idea is simple. Variables that are measured at different scales do not contribute equally to the model fitting & model learned function and might end up creating a bias. Thus, to deal with this potential problem feature-wise standardized (μ=0, σ=1) is usually used prior to model fitting.

To do that using scikit-learn , we first need to construct an input array X containing the features and samples with X.shape being [number_of_samples, number_of_features] .

Keep in mind that all scikit-learn machine learning (ML) functions expect as input an numpy array X with that shape i.e. the rows are the samples and the columns are the features/variables. Having said that, let’s assume that we have a matrix X where each row/line is a sample/observation and each column is a variable/feature.

Note: Tree-based models are usually not dependent on scaling, but non-tree models models such as SVM, LDA etc. are often hugely dependent on it.

Core of method

The main idea is to normalize/standardize i.e. μ = 0 and σ = 1 your features/variables/columns of X , individually, before applying any machine learning model. Thus, StandardScaler() will normalize the features i.e. each column of X, INDIVIDUALLY so that each column/feature/variable will have μ = 0 and σ = 1 .

The mathematical formulation of the standardization procedure. Image generated by the author.

Working Python code example:

from sklearn.preprocessing import StandardScaler

import numpy as np



# 4 samples/observations and 2 variables/features

X = np.array([[0, 0], [1, 0], [0, 1], [1, 1]]) # the scaler object (model)

scaler = StandardScaler() # fit and transform the data

scaled_data = scaler.fit_transform(X)



print(X)

[[0, 0],

[1, 0],

[0, 1],

[1, 1]])



print(scaled_data)

[[-1. -1.]

[ 1. -1.]

[-1. 1.]

[ 1. 1.]]

Verify that the mean of each feature (column) is 0:

scaled_data.mean(axis = 0)

array([0., 0.])

Verify that the std of each feature (column) is 1:

scaled_data.std(axis = 0)

array([1., 1.])

The effect of the transform in a visual example

Figure taken from: https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html. Left subplot: the unscaled data. Right subplot: the transformed data.

Summary

StandardScaler removes the mean and scales each feature/variable to unit variance. This operation is performed feature-wise in an independent way.

removes the mean and scales each feature/variable to unit variance. This operation is performed in an way. StandardScaler can be influenced by outliers (if they exist in the dataset) since it involves the estimation of the empirical mean and standard deviation of each feature.

How to deal with outliers

Manual way (not recommended): Visually inspect the data and remove outliers using outlier removal statistical methods such as the Interquartile Range (IQR) threshold method.

Recommended way: Use the RobustScaler that will just scale the features but in this case using statistics that are robust to outliers. This scaler removes the median and scales the data according to the quantile range (defaults to IQR: Interquartile Range). The IQR is the range between the 1st quartile (25th quantile) and the 3rd quartile (75th quantile).

That’s all for today! Hope you liked this first post! Next story coming next week. Stay tuned & safe.

Stay tuned & support me

If you liked and found this article useful, follow me and applaud my story to support me!

References

[1] https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html

[2] https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.RobustScaler.html

Get in touch with me