Dimensionality Reduction Techniques
By: Vineet Maheshwari
Tags: Machine Learning, Data Science, Dimensionality Reduction, Feature Engineering

Source : HPC

Now, what is this dimensionality reduction? Seems a bit complicated. But with the increasing amount of data on a daily basis, it is getting quite complicated to keep track of it. A tremendous amount of data is being generated from different social networking sites, educational websites, and it is very difficult to visualize that data. Therefore these techniques are used.

In cases, where there is huge number of variables, it is quite difficult to visualize, draw inferences for that dataset. So these techniques try to take out a subset from that dataset, which can capture a normal an amount of information laid out by the original set of variables.

So if we are having a dataset of X dimensions, we can convert it into a subset of Y dimensions. This is called dimensionality reduction.

Why do we need these techniques?

If the dimensions are reduced, the space required to store the data is also reduced.

It helps us in visualizing the data.

It helps us to take care of multicollinearity by removing those features that are of no use.

Computation time can be reduced thus increasing the speed of our model.

List of Dimensionality Reduction Techniques:

Random Forest Principal Component Analysis (we will discuss about it) Missing Value Ratio Forward Feature Selection Backward Feature Elimination Low Variance Filter High Correlation Filter

Principal Component Analysis (PCA):

This technique extracts some amount of features from the original dataset. These extracted features/variables are called Principal Components. The main motive behind this technique is to extract a low dimensional set of features from a high dimensional dataset. And it is more useful when we are dealing with 3 or high dimensional data.

If we want to perform this technique, then the matrix must contain numeric and standardize data.

PCA is a linear combination of original variables.

The first principal component should explain the maximum variance in the dataset.

And the remaining variance is being explained by the second principal component. These components are uncorrelated, and the cycle goes on.

Source : Medium

Z² = Φ¹²X¹ + Φ²²X² + Φ³²X³ + .... + Φp2Xp -> This is the equation by which we can calculate Principal component.

We will talk about the implementation part in next posts.

Let’s have a look towards some feature engineering.

Source : Amazon

Success of machine learning as actually a success in engineering the features that a learner can understand. It is the process of transforming the raw data into features which can represent an underlying more problem in more accurate way, resulting in improved accuracy of our model.

Steps involved in solving any machine learning problem are:

Collecting data.

Pre-processing data.

Feature Engineering.

Defining our model.

Predicting output.

Feature engineering can be anything like, introducing a new feature that can point out some useful information and can increase the accuracy of our model. Or it can be an introduction of a new feature by the combination of two different features.

Sometimes, removing an unwanted feature from our model also contributes to feature engineering, because that model was the reason of the downfall of our model.

Steps involved in Feature Engineering :

Deleting features.

Creating new features.

Checking from the start if all the features are working in a fine way.

Running our model again and again with different test cases, to check on the accuracy of our model with the extracted features.

That’s all folks!

HAPPY LEARNING !!!