Automate your Feature Selection Workflow in one line of Python code | by Satyam Kumar | Towards Data Science

More training data in terms of the number of instances leads to a better data science model, but this does not hold true for the number of features. A real-world dataset has a lot of features, some of them are useful to train a robust data science model, others are redundant features that can impact the performance of the model.
Feature selection is an important element of a data science model development workflow. Selecting all the possible combinations of features is a polynomial solution. There are various feature selection techniques and hacks that data scientists use to remove the redundant features.
Read this article, to know 7 feature selection techniques.
In this article, we will focus on automating the feature selection workflow using an open-source Python package — Featurewiz.
Featurewiz is an open-source library used for creating and selecting the best features from the dataset, which can be further used to train a robust data science model. Featurewiz also comes up with feature engineering capabilities. It can create hundreds of new features with just a click of code. Featurewiz API has a parameter ‘feature_engg’, that can be set to ‘interactions’, ‘group by’, and ‘target’, and it will create hundreds of features in one go.
Feature Engineering or creating new features is not only the capabilities of featurewiz. It can reduce the number of features, and select the best set of features to train a robust model.
Featurewiz uses two algorithms to select the best features from the dataset.
SULOV
Recursive XGBoost
SULOV stands for Searching for Uncorrelated List of Variables, very similar to the mRMR algorithm. Steps SULOV algorithms follows are:
Compute all the pairs of correlated variables that exceed a threshold.Compute MIS (Mutual Information Score) with respect to the target variable.Compare each pair of correlated variables, and remove the features with low MIS.The remaining features have high MIS and low correlation.
Compute all the pairs of correlated variables that exceed a threshold.
Compute MIS (Mutual Information Score) with respect to the target variable.
Compare each pair of correlated variables, and remove the features with low MIS.
The remaining features have high MIS and low correlation.
After SULOV algorithms select the best set of features having high MIS and log correlation, the repetitive XGBoost algorithm is used to compute the best features among the remaining variables. The steps are as follows:
Build a dataset for the remaining set of features and split them into train and validation.Compute the top 10 features on the train using validation.Repeat steps 1 and 2 with a different set of features each time.Combine all sets of 10 features and de-duplicate them, this will result in the best set of features.
Build a dataset for the remaining set of features and split them into train and validation.
Compute the top 10 features on the train using validation.
Repeat steps 1 and 2 with a different set of features each time.
Combine all sets of 10 features and de-duplicate them, this will result in the best set of features.
Featurewiz uses the above two discussed algorithms to find the best set of features, that can be further used to train a robust machine learning model.
Featurewiz can be installed using Pypl
pip install featurewiz
After installation, featurewiz can be imported:
from featurewiz import featurewiz
Now the developer needs to write only one line of code, to get the best set of features from the dataset.
out1, out2 = featurewiz(dataname, target, corr_limit=0.7, verbose=0,   sep=",", header=0, test_data="", feature_engg="", category_encoders="")
Featurewiz can not only handle datasets with one target variable but can also handle datasets with multi-label target variables. The data frame returned from featurewiz contains the best set of features and is ready to use for model training. Developers don’t need to specify the types of problem, if it's a regression or classification, features can decide it automatically.
In this article, we have discussed an open-source library featurewiz, that automates the feature selection of the dataset. Along with feature selection, featurewiz also has the capabilities to perform feature engineering and generate hundreds of features, just with a click of code.Featurewiz using two algorithms (SULOV, and recursive XGBoost) to select the best set of features. Featurewiz speeds up the workflow of a data scientist by doing the entire feature selection just in a click of a single line of code. A data scientist can use several feature selection techniques to filter out the best of features, you can read 7 of these feature selection techniques in the below-mentioned article.
towardsdatascience.com
[1] Featurewiz GitHub repo: https://github.com/AutoViML/featurewiz
Thank You for Reading