# ML-Based Pain Recognition Model Using Mixup Data Augmentation
## 2 Literature Review
The authors of [11] conducted experiments on ImageNet-2012, CIFAR-10, CIFAR-100, and speech data, showing that mixup improves the generalization of state-of-the-art neural network architectures. It was also found that mixup reduces the memorization of corrupt labels; additionally, mixup increases the robustness to adversarial examples as well as stabilizing the training of generative adversarial networks. While mixup, proposed by the authors of [11], works by linearly interpolating inputs and their corresponding labels, it was shown by the authors in [20] that the samples can also be mixed in a nonlinear fashion; the authors of [20] named this technique \"nonlinear mixup\". Unlike linear mixup where the input and output label pairs share the same, linear, scalar mixing policy, the nonlinear mixup approach embraces a nonlinear interpolation policy for both the input and their corresponding labels, and the mixing policy for the labels is adaptively learned based on the mixed input. Experiments have been conducted on benchmark sentence classification datasets and it has been indicated that nonlinear mixup plays a vital role in improving the performance of the classification model. The study conducted by the authors of [21] demonstrated the effectiveness of mixup data augmentation in natural language processing (NLP). The mixup technique is used extensively in the field of healthcare, especially for medical image classification. In [22], a technique called \"balanced-mixup\" was proposed for highly imbalanced medical image classification. In this approach, \"balanced-mixup\" simultaneously performs regular and balanced sampling of the training data. The resulting two sets of samples are then mixed up to create a more balanced training distribution from which a neural network can effectively learn without heavily underfitting the minority classes. The authors of [23] examined the effectiveness of mixup in enhancing the performance of a COVID-19 image classification model. They conducted a comparative analysis between traditional methods and those methods that incorporated mixup. Their findings consistently indicated that the models using mixup augmentation surpassed those relying solely on conventional methods, underscoring the superiority of mixup in enhancing classification accuracy. Furthermore, numerous mixup techniques are widely used. The authors in [24] introduced an extension of the original mixup method, called manifold mixup. Manifold mixup is a regularization technique for deep neural networks designed to improve robustness and generalization. While mixup performs interpolations on the input data, manifold mixup performs semantic interpolations on hidden layer representations within the neural network. In this technique, more diverse and abstract representations are generated because the mixing is carried out at different levels in the network. This approach smoothens the decision boundaries at multiple-layer representation levels, leading to flatter class representations with fewer directions of variance. The authors in [25] introduced yet another augmentation strategy called CutMix. CutMix cuts and pastes patches from one image onto another. The labels are then proportionally mixed according to the area of the patches, retaining more of the training data's information. Mixup creates synthetic training examples by linearly interpolating random input training samples and their labels, which is proven to improve model accuracy. However, mixup might lead to what is known as \"manifold intrusion\", which is nothing but a type of underfitting where the mixed labels conflict with those of the original training samples mainly due to poorly tuned mixing parameters. To overcome this drawback, a technique called AdaMixup was introduced by the authors in [26]. AdaMixup overcomes this limitation of manifold intrusion by automatically learning the optimal mixing policies through an additional network and objective function, which then enhances the regularization.