Fast Feature Engineering in Python: Tabular Data
By: Sayar Banerjee
Tags: Data Science, Machine Learning, Feature Engineering

Tabular data is the most common type of data that Data Science practitioners work with. In this format, data is arranged in the form of rows and columns.

Let’s see some common problems faced with tabular data and how to solve them.

🔢 Handling Numerical Data

Let's create a synthetic dataset using the sklearn.datasets module having 10000 rows and 25 features.

Contents of X_df

Scaling, Standardizing, and Normalizing

When training linear models such as Linear/Logistic Regression and SVM, scaling your dataset can have a huge impact on model performance.

Because these models are optimized using distance metrics, scaling ensures that features having larger values do not dominate the outcome of the model fit.

In our first method of scaling, we will use the MinMaxScaler provided by sklearn to scale each feature to be between the values -1 and 1.

Contents of X_df

The MinMaxScaler accepts a parameter called feature_range where the user can specify the desired range. It’s considered a best practice to keep the ranges to be either -1 to 1 or 0 to 1 depending on whether your feature has positive values or not.

Under the hood, this is what is being calculated:

where the difference of an element in a feature and the min value of the feature is divided by the range of the feature.

Another form of scaling is called standardization where every feature is transformed to have a mean of 0 and standard deviation of 1. This is very useful in order to normally distribute our features.

The StandardScaler module can be used to standardize our features:

Contents of X_df_standardized

Standardization works as follows:

where each individual element of a feature is subtracted by the mean of the feature set and divided by the standard deviation.

So far, we have looked at scaling methods that operate at a feature level. Methods such as unit length normalization enable scaling across observations as well.

This type of normalization is useful when the data has a lot of equivalent features such as text data because each word is treated as a feature.

The Normalizer scales every observation to have a unit norm length of 1.

Contents of X_df_normalized

Here is the formula for normalization:

The denominator is the l2 norm of the observation which is the square root of the sum of squares of the observation’s value in each feature.

Creating Polynomial Features

Polynomial features might be needed in two situations:

When you suspect that features have a non-linear relationship with the target variable When you suspect that features are dependent on one another (interaction).

(10000, 350)

We can clearly see that our initial data which had only 25 features now has 350! 😮

Note: Building polynomial features will lead to feature explosion so be sure to use good feature selection practices.

Transform Features

Scikit Learn also makes it very easy to create custom transformers to apply to one or more features. All you have to do is define a function and pass it to an instance of the FunctionTransformer .

If you are working with Pandas Dataframes, instead of using FunctionTransformer you can use the .apply method available with the data frame to transform your data.

Binning

Binning is a method used to treat numerical variables like categorical ones. Binning involves breaking up our numerical data into discrete bins.

Numpy has a method called .digitize which helps create bins with ease:

If you are working with DataFrames, you can also use the .cut method provided by Pandas to create bins: