How and why to Standardize your data: A python tutorial | Towards Data Science

Hi there.
This is my first Medium post. I am an electrical & computer engineer currently finishing my PhD studies in the biomedical engineering and computational neuroscience field. I have been working on machine learning problems for the past 4 years. A very common question that I see all around the web is how to standardize and why to do so, the data before fitting a machine learning model.
How does scikit-learn’s StandardScaler work ?
The first question that comes to one’s mind is:
Why to standardize in the first place?
Well, the idea is simple. Variables that are measured at different scales do not contribute equally to the model fitting & model learned function and might end up creating a bias. Thus, to deal with this potential problem feature-wise standardized (μ=0, σ=1) is usually used prior to model fitting.
To do that using scikit-learn, we first need to construct an input array X containing the features and samples with X.shape being[number_of_samples, number_of_features] .
Keep in mind that all scikit-learn machine learning (ML) functions expect as input an numpy array X with that shape i.e. the rows are the samples and the columns are the features/variables. Having said that, let’s assume that we have a matrix X where each row/line is a sample/observation and each column is a variable/feature.
Note: Tree-based models are usually not dependent on scaling, but non-tree models models such as SVM, LDA etc. are often hugely dependent on it.
The main idea is to normalize/standardize i.e. μ = 0 and σ = 1 your features/variables/columns of X, individually, before applying any machine learning model. Thus, StandardScaler() will normalize the features i.e. each column of X, INDIVIDUALLY so that each column/feature/variable will have μ = 0 and σ = 1.
from sklearn.preprocessing import StandardScalerimport numpy as np# 4 samples/observations and 2 variables/featuresX = np.array([[0, 0], [1, 0], [0, 1], [1, 1]])# the scaler object (model)scaler = StandardScaler()# fit and transform the datascaled_data = scaler.fit_transform(X) print(X)[[0, 0], [1, 0], [0, 1], [1, 1]])print(scaled_data)[[-1. -1.] [ 1. -1.] [-1.  1.] [ 1.  1.]]
Verify that the mean of each feature (column) is 0:
scaled_data.mean(axis = 0)array([0., 0.])
Verify that the std of each feature (column) is 1:
scaled_data.std(axis = 0)array([1., 1.])
StandardScaler removes the mean and scales each feature/variable to unit variance. This operation is performed feature-wise in an independent way.
StandardScaler can be influenced by outliers (if they exist in the dataset) since it involves the estimation of the empirical mean and standard deviation of each feature.
Manual way (not recommended): Visually inspect the data and remove outliers using outlier removal statistical methods such as the Interquartile Range (IQR) threshold method.
Recommended way: Use the RobustScaler that will just scale the features but in this case using statistics that are robust to outliers. This scaler removes the median and scales the data according to the quantile range (defaults to IQR: Interquartile Range). The IQR is the range between the 1st quartile (25th quantile) and the 3rd quartile (75th quantile).
That’s all for today! Hope you liked this first post! Next story coming next week. Stay tuned & safe.
If you liked and found this article useful, follow me and applaud my story to support me!
[1] https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html
[2] https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.RobustScaler.html
LinkedIn: https://www.linkedin.com/in/serafeim-loukas/
ResearchGate: https://www.researchgate.net/profile/Serafeim_Loukas
EPFL profile: https://people.epfl.ch/serafeim.loukas
Stack Overflow: https://stackoverflow.com/users/5025009/seralouk