A thumbnail of Feature Engineering
By: Mahitha Singirikonda
Tags: Feature Engineering, Feature Scaling, Eda, Data Science, Machine Learning

You can’t build a great building on a weak foundation. You must have a solid foundation if you’re going to have a strong super structure.

-Gordon B.Hinckley

Feature Engineering is the crucial module in the life cycle of a Data science project.

For instance, you are tasked with a gardening project and all you were provided is an uneven ,weed filled backyard and bunch of stuff from retail construction store like wood blocks, soil, concrete raw material .To proceed ,you need to make a note of necessary steps like what is the purpose of the project, requirements to be considered and things you need to do in order to make the process fast, easy and effective. We also might need to preprocess certain materials to serve the purpose.This collection of all needed materials and making them usable for building the project is called engineering and when it is done for features in a Data science project is called Feature Engineering.

Raw data can be haphazard. It is a good practice to be able to answer all the above questions after Features are Engineered.

The major objective of Feature Engineering is identifying, preprocessing or modifying the features which contribute to the effective model. To achieve this there are multiple steps depending on the data.

1.EDA:

Before starting any Machine learning project the data is messy and is not suitable for modelling.

In order to prepare the data for modelling we have to explore the data and find relevant insights.

Exploratory Data Analysis is exploring the data set for data types(categorical or numerical),missing values, outliers, cleaning the data like checking for format, headers etc.

Dtale, Pandas profiling, Sweetviz, Autoviz, Dataprep are libraries which can automate all the EDA along with plots.

2.Handling the missing values :

This can be done in many ways like imputing with mean, median, mode, using imputation algorithms like:

KNN

LLS impute which uses regression

MICE(Multiple imputations by chained equations)

CART(Classification and Regression trees)

GMC impute(Gaussian mixture clustering imputation)

CMVE(Collateral missing value imputation)

AMVI(Ameliorative missing value imputation)

ABBA(Adaptive Bicluster based approach imputation)

The above are a few algorithms used for imputing missing values. The missing values can be replaced using a hybrid of any of the above algorithms. The imputed dataset can be evaluated with either internal or external validation. Hence the reliability and accuracy of our imputation technique can be measured.

3.Handling imbalanced Dataset:

Imbalance data leads to biased models which is not ideal. Hence we need a way to handle Imbalanced data. This can be approached in following ways:

Resampling

More training data

Using K-fold

Cluster the abundant class

Changing performance metrics

We also can customize a way to handle imbalanced data sets.

4.Outliers:

Removing outliers results in a statistically significant increase in the accuracy of the model. Hence this is an important step to be performed. Anomaly detection can be achieved by the following ways:

Using scatter plots

Box Plots

Z-score

Using IQR(inter quartile range)

Isolation forest

Cluster analysis

Ensemble techniques

5.Feature scaling:

Raw data can have values in any range. It is atypical to be able to analyze features with the drastic range changes If the data is in different units of measurement , it is important to bring them to the same scale. Hence the data has to be transformed in limited ranges or same units which in turn can achieve accurate models. The Feature Transformation can be done in the following ways:

Standard scaler

Min Max scaler(0–1)

Gaussian Transformations like

logarithmic transformation

reciprocal transformation

square root transformation

exponential transformation

box cox transformation

6.Handling Categorical data:

The categorical data handles useful information but a machine cannot calculate when used as it is, these variables should be modified to numerical so that the calculations can be performed and we can derive more insights from the data. The categorical variables are classified into ordinal and nominal where ordinal categorical variables can be ranked. Some of the ways to convert the categorical data are as follows:

One hot encoding

Label encoding

Binary encoding

Replacing

Backward difference encoding

Count or frequency encoding

Conclusion:

Feature Engineering is the crucial step where we are trying to create a bond to be able to understand the data clearly and extract all possible information the data is trying to say to us. This task takes much of a Data scientist’s time. If the base is strong the building will be strong and hence if Feature Engineering is done properly we can achieve an effective model.