Importance of Dimensionality Reduction!!
Tags: Statistics, Machine Learning, Data Analytics, Data Science

In Machine Learning and Statistics,in order to build a good performing model we try to pass on those features in the dataset that are significant to one another.

In other words,Dimensionality reduction is the process of reducing the number of random variables under consideration, by obtaining a set of principal variables.

Problem with high dimensional data?

It can mean high computational cost to perform learning. It often leads to over-fitting when learning a model, which means that the model will perform well on the training data but poorly on test data. Data are rarely randomly distributed in high-dimensions and are highly correlated, often with spurious correlations. The distances between a nearest and farthest data point can become equidistant in high dimensions, that can hamper the accuracy of some distance-based analysis tools.

Why do we need Dimensionality Reduction?

Dimensionality reduction helps with these problems, while trying to preserve most of the relevant information in the data needed to learn accurate, predictive models.

There are often too many factors on the basis of which the final prediction is done. These factors are basically variables called features.

The higher the number of features, the harder it gets to visualize the training set and then work on it.

Sometimes, most of these features are correlated, and hence redundant. This is where dimensionality reduction algorithms come into play.

Importance of Dimensionality reduction?

It reduces the time and storage space required.

It helps Remove multi-collinearity which improves the interpretation of the parameters of the machine learning model.

It becomes easier to visualize the data when reduced to very low dimensions such as 2D or 3D.

It avoids the curse of dimensionality.

It removes irrelevant features from the data, Because having irrelevant features in the data can decrease the accuracy of the models and make your model learn based on irrelevant features.

In machine learning, “Dimensionality” simply refers to the number of features/variables in your dataset. When the number of features/variables is very large relative to the number of observations in your dataset, certain algorithms struggle to train effective models. This is called the “Curse of Dimensionality.”

Dimensionality reduction can be divided into:

Feature Selection-

Feature selection is a method introduced in the machine learning in order to remove less significant features from the data,so that the model is trained only on the features that contribute most to the prediction(dependent) variable.

Types of Feature Selection methods are:

* Filter

* Wrapper

* Embedded Feature Extraction-

Feature extraction is a process of dimensionality reduction by which an initial set of raw data is reduced to more manageable groups for processing.

This process is generally done with image and text data, where only the important feature is extracted and sent ahead for processing instead of taking the whole data.

Feature Extraction can be Unsupervised (PCA) or supervised (LDA).

We will see about each methods and techniques in the upcoming articles!Stay tuned.

Happy Learning!!!!!