Upgrade your Image Classifier with Balanced data
By: Nishna Ajmal
Tags: Machine Learning, Visualization, Image Classification, Data Cleaning, Towards Data Science

Upgrade your Image Classifier with Balanced data

How preprocessing & stratification can do wonders to image classifier performance

FER

This article walks you through the data munging techniques to refine the machine learning models for image classifiers, particularly facial emotion recognition. The R notebook containing the source code is available in the GitHub link here

In this project the ‘facial emotion recognition’ dataset, from the Kaggle competition Challenges in Representation Learning: Facial Expression Recognition Challenge is used. The data consists of 27000 training images (48 x 48 pixels of faces almost centred so that all the faces extend over equal spaces in the image ) and a test set containing 9000 images. This is a classification task is to categorize each image to one of seven classes (0=Angry, 1=Disgust, 2=Fear, 3=Happy, 4=Sad, 5=Surprise, 6=Neutral).

I tried several algorithms including Nearest Neighbour, Bayesian Classifiers, Decision Trees and Neural Networks to build machine learning model for the task. But, none of them was returning success rate above 30%. That left me wondering what went wrong and how to approach this task differently. Recently I came across an article by one of the top performers of the Plant Seedlings Classification Challenge, that suggests the readers deal with the imbalanced class problem before classifying the images.

In this article, I would explain how I built a better Image classifier with decision tree algorithm using clean balanced data and visualizations.

Step1: Explore the data

On visualizing the distribution of each class in the dataset, we understand that the dataset is heavily imbalanced.

A dataset is described as imbalanced if the classes are not represented approximately equal. [1]

Distribution of classes

There are more than 7000 ‘Happy ’images whereas the number of ‘Disgust’ images is less than 500. This would result in a biased classifier that predicts most of the images as majority class.

Popular algorithms like Decision Tree and Logistic Regression have a bias towards majority classes in the dataset [2]

To understand the complexity of the algorithm, I applied the technique t-Distributed Stochastic Neighbor Embedding (t-SNE), which is considered the best approach to visualize high dimensional data.

t-SNE is a technique for the visualization of similarity data that is capable of retaining the local structure of the data while also revealing some important global structure such as clusters at multiple scales [3] t-SNE that visualizes high-dimensional data by giving each data point a location in a two or three-dimensional map[3]

The tSNE visualization of the original dataset shows us little insight into the class structure of the data.

t-SNE Original Dataset

Step 2: Partitioning

Next step is to partition the training dataset to train set & validation set. The validation set would be used for tuning and the final model will be evaluated using the test dataset

Step 3: Creating a benchmark classifier

The benchmark provides a basis for evaluating and comparing algorithms for learning from examples. [4]

If we create a benchmark classifier at this stage, the performance would be less than 27 %.

Benchmark Classifier Results

Step 4: Data Preprocessing

On Visualizing a random instance, it’s evident that the image is horizontally aligned and some amount of noise is also present as background.

Preprocessing methods normalises the image acquired from uncontrolled environments. [5]

Therefore all the images are first rotated & cropped as shown in the fig below.

Fig: Sample Image (i) Original (ii) Rotated (iii) Cropped

The instances are then centred and scaled to the range {0.8, 1.2}. The distribution of values in each pixel before and after the transformation can be plotted as.

Distribution of values in each pixel

Step 5: Feature extraction

Well, this is a high dimensional dataset with 2304 features. To reduce the dimensions, we can use the most widely used technique PCA.

Principal component analysis (PCA) is a statistical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables (entities each of which takes on various numerical values) into a set of values of linearly uncorrelated variables called principal components. [6]

PCA Output

From the PCA summary its clear that the first 25 principal components hold more than 80% of the data.

PCA Summary

The tSNE visualization after PCA is strikingly better, but most of the classes are mapped somewhat proximate and not categorized with clarity.

t-SNE PC 1–25

The same preprocessing steps and the orthogonal rotation from PCA are applied to the validation data and test data

Step 5: Stratification

In this step, we will deal with the imbalanced class problem in the train data.

SMOTE is an oversampling approach in which the minority class is over-sampled by creating “synthetic” examples rather than by over-sampling with replacement [1]

A combination of the synthetic oversampling of minority class (SMOTE)and random undersampling of the majority class, will give a nice balanced train data which can then be used to build a classifier that performs better. This is evident from the tSNE visualization of the new train data below

t-SNE Balanced Data

Step 6: Classification

We build a classifier using the prepared train data. From the Confusion matrix of the model, we could see that the overall accuracy has improved and is nearly 60 %.

Confusion matrix — Classifier trained using the processed data

Step 7: Hyper parameter tuning

By tweaking the complexity parameter (cp) we can reduce the relative error as shown in the plot below

Variation in Relative error with complexity parameter

Step 8: Build the final model

The last step is to build the final model with the tuned parameter (cp value with minimum Xval-Relative Error) and evaluate it using the test data.

The confusion matrix stats say the classifier performance has improved dramatically and the overall accuracy is greater than 90 %. Other performance metrics such as sensitivity, specificity etc too is better.

Confusion Matrix — Final model

Other Performance metrics comparison for various classes

Conclusion

The preprocessing, dimensionality reduction & balancing the dataset improved the performance of the decision tree classifier remarkably from 27% to 91%. Visualizing the high dimensional data using tSNE can give you a better idea of the distribution of classes, which can help significantly in preprocessing the data that can be used to build a better classifier.

References

[1] N. Chawla, K. Bowyer, L. Hall and W. Kegelmeyer, SMOTE: Synthetic Minority Over-sampling Technique (2002), Journal of Artificial Intelligence Research, 16, pp.321–357.

[2] Guest Blog, How to handle Imbalanced Classification Problems in machine learning (2017), [Blog] Analytics Vidhya

[3] L. van der Maaten and G.Hinton, Visualizing Data using t-SNE (2008), Journal of Machine Learning Research, 9, pp.2579–2605.

[4] Z. ZHENG, A benchmark for Classifier Learning (1993), In Australian Joint Conference on Artificial Intelligence. World Scientific, pp.281–286.

[5] M.G. Sumithra and P. Rajeswari, A Survey: Pre Processing techniques for Facial Expression Recognition (2015), International Journal on Applications of Information and Communication Engineering Volume 1: Issue 1: January 2015, Pages:47–51, 1(1), pp.47–51.

[6] En.wikipedia.org, Principal component analysis (2018), [online] Wikipedia