# Uncertainty Quantification for MLP-Mixer Using Bayesian Deep Learning
###### Abstract  
Convolutional neural networks (CNNs) have become a popular choice for various image classification applications. However, the multi-layer perceptron mixer (MLP-Mixer) architecture has been proposed as a promising alternative, particularly for large datasets. Despite its advantages in handling large datasets and models, MLP-Mixer models have limitations when dealing with small datasets. This study aimed to quantify and evaluate the uncertainty associated with MLP-Mixer models for small datasets using Bayesian deep learning (BDL) methods to quantify uncertainty and compare the results to existing CNN models. In particular, we examined the use of variational inference and Monte Carlo dropout methods. The results indicated that BDL can improve the performance of MLP-Mixer models by 9.2 to 17.4% in term of accuracy across different mixer models. On the other hand, the results suggest that CNN models tend to have limited improvement or even decreased performance in some cases when using BDL. These findings suggest that BDL is a promising approach to improve the performance of MLP-Mixer models, especially for small datasets. uncertainty quantification; Bayesian deep learning; MLP-Mixer; variational inference (VI); MC-dropout +
Footnote †: journal:  
images into patches, referred to as tokens, which then transform into simple MLPs using matrix multiplication. The use of two layers of mixers, token mixing, and channel mixing, enabled inter- and intra-interaction between patches. While MLP-Mixers have shown state-of-the-art performance and perform well on large datasets, they still lag behind CNN models on small datasets and have a larger number of parameters [9; 10]. To address these issues, researchers have proposed various versions of the MLP-Mixer model, such as Sparse-MLP, S2-MLP, As-MLP, and Hire-MLP [10]. Despite these challenges, the key advantages of the MLP-Mixer are its simplicity, flexibility, and scalability, which make it an attractive alternative to CNNs for vision tasks. In the field of deep learning for computer vision, the performance of various architectures, such as MLP-Mixers and CNNs, is typically evaluated using measurements such as sensitivity, specificity, accuracy, precision, recall, receiver operating characteristics (ROCs), area under the ROC curve (AUC), F1 score, and other similar metrics [11]. Although these metrics have been successful in differentiating between good and bad models, they do not provide any information about the confidence of a particular model. Measuring the confidence of a model is crucial, especially in domains such as medical diagnosis, where errors can have serious consequences. Thus, it is essential to estimate the uncertainty of both the model and the data. One method for uncertainty quantification is through the Bayesian deep learning (BDL) models [12], which treats the weights of the model as random variables. BDL incorporates prior information about the distribution of the weights and updates it with new data to estimate the uncertainty associated with the model's predictions. BDL is a type of deep learning that utilizes Bayes' theorem to calculate the posterior distribution by incorporating prior distribution and likelihood of the data as input components [13]. This approach is useful in measuring different uncertainties related to the data and architecture used [13]. In BDL, various statistical distributions can be utilized to represent the prior and posterior distributions; however, the normal and Bernoulli distributions are the most prevalent when it comes to image data [14]. To generate output, the Bayesian approach obtains samples from the posterior distribution. Markov chain Monte Carlo (MCMC) is one of the most well-known exact sampling methods for posterior distribution, however, it is impractical to scale up for larger datasets, such as images, and can be computationally expensive [14]. To address this issue, other approximation methods such as variational inference (VI) [15] and Monte Carlo dropout (MC-Dropout) [16] are used.