5 Techniques to work with Imbalanced Data in Machine Learning | by Satyam Kumar | Towards Data Science

For classification tasks, one may encounter situations where the target class label is un-equally distributed across various classes. Such conditions are termed as an Imbalanced target class. Modeling an imbalanced dataset is a major challenge faced by data scientists, as due to the presence of an imbalance in the data the model becomes biased towards the majority class prediction.
Hence, handling the imbalance in the dataset is essential prior to model training. There are various things to keep in mind while working with imbalanced data. In this article, we will discuss various techniques to handle class imbalance to train a robust and well-fit machine learning model.
Checklist:1) Upsampling Minority Class2) Downsampling Majority Class3) Generate Synthetic Data4) Combine Upsampling & Downsampling Techniques 5) Balanced Class Weight
Before processing to discuss the 5 above-mentioned techniques, let’s focus on choosing the right metric for an Imbalanced dataset task. Choosing an incorrect metric such as Accuracy seems to perform well but actually is biased towards the majority class label. The alternate choice of performance metrics can be:
AUC-ROC Score
Precision, Recall, and F1-score
Confusion Matrix for visualization of TP, FP, FN, TN
After choosing the right metric for your case study, one can employ various techniques to handle the imbalance in the dataset.
Upsampling or Oversampling refers to the technique to create artificial or duplicate data points or of the minority class sample to balance the class label. There are various oversampling techniques that can be used to create artificial data points.
Read the below mentioned article to get better understanding of 7 such oversampling techniques:
towardsdatascience.com
Downsampling or Undersampling refers to remove or reduce the majority of class samples to balance the class label. There are various undersampling techniques implemented in the imblearn package including:
Random Under Sampling
Tomek Links
NearMiss Sampling
ENN (Edited Nearest Neighbours)
and many more.
Follow the imblearn documentation to get implementation of each of the above-mentioned techniques:
imbalanced-learn.org
Undersampling techniques are not recommended as it removes the majority class data points. Generating synthetic data points of minority samples is a type of oversampling technique. The idea is to generate synthetic data points of minority class samples in the nearby region or neighborhood of minority class samples.
SMOTE (Synthetic Minority Over-Sampling Technique) is a popular synthetic data generation oversampling technique that utilizes the k-nearest neighbor algorithm to create synthetic data.
There are various variations of SMOTE including:
SMOTENC: SMOTE variant for continuous and categorical features.
SMOTEN: SMOTE variant for data with only categorical features.
Borderline SMOTE: New Synthetic samples will be generated using the borderline samples.
SVMSMOTE: Use an SVM algorithm to detect samples to use for generating new synthetic samples.
KMeansSMOTE: Over-sample using k-Means clustering prior to oversample using SMOTE.
Adaptive Synthetic (ADASYN): Similar to SMOTE but it generates a different number of samples depending on an estimate of the local distribution of the class to be oversampled.
Follow Imblearn documentation for the implementation of above-discussed SMOTE techniques:
Undersampling techniques is not recommended as it removes the majority class data points. Oversampling techniques are often considered better than undersampling techniques. The idea is to combine the undersampling and oversampling techniques to create a robust balanced dataset fit for model training.
The idea is to first use an oversampling technique to create duplicate and artificial data points and use undersampling techniques to remove noise or unnecessary generated data points.
Imblearn library comes with the implementation of combined Oversampling and Undersampling techniques such as:
Smote-Tomek: Smote (Oversampler) combined with TomekLinks (Undersampler).
Smote-ENN: Smote (Oversampler) combined with ENN (Undersampler).
Follow Imblearn documentation for the implementation of Smote-Tomek and Smote-ENN techniques.
The undersampling technique removes the majority class data points which results in data loss, whereas upsampling creates artificial data points of the minority class. During the training of machine learning, one can use class_weight parameter to handle the imbalance in the dataset.
Scikit-learn comes with the class_weight parameters for all the machine learning algorithms.
class weight — balanced: The class weight is inversely proportional to class frequencies in the input data.
Computation Formula:n_samples / (n_classes * np.bincount(y))
{class_label: weight}: Let’s say, target class labels are 0 and 1. Passing input as class_weight={0:2, 1:1} means class 0 has weight 2 and class 1 has weight 1.
The presence of a class imbalance in the data can be a major challenge while training a robust model. The 5 above-discussed techniques can be used to handle the class imbalance prior to train a machine learning model.
One can also employ cost-sensitive learning or penalize the algorithms that increase the cost of classification of majority classes. Also, Decision Trees and Random Forest should be preferred over other machine learning algorithms as they tend to perform well on imbalanced data.
[1.] Imblearn Documentation: https://imbalanced-learn.org/stable/index.html
Loved the article? Become a Medium member to continue learning without limits. I’ll receive a small portion of your membership fee if you use the following link, with no extra cost to you.
satyam-kumar.medium.com
Thank You for Reading