# Uncertainty Quantification for MLP-Mixer Using Bayesian Deep Learning
## 3 Experiments and Results  
To evaluate the effectiveness of the proposed method, an empirical study was conducted using five deep learning models based on MLP-Mixers and CNN architectures. The models were trained and tested on two different image datasets to assess their performance in image classification tasks. Two versions of each model were tested: one with MC-Dropout and the other with VI. The models were trained for varying numbers of epochs with early stopping criteria. However, most of the models lasted between 30 and 80 epochs, with the main difference being that MC-Dropout models tended to stop earlier than VI models in general. The first model utilized in this study was the MLP-Mixer model, which employed a unique approach to image classification. In this model, the input images were divided into 32 patches, and a custom MLP-Mixer was designed for this study. The custom MLP-Mixer had approximately 30 million trainable parameters for both the MC-Dropout and VI versions of the model. Unlike many other models, the MLP-Mixer was not pre-trained on any dataset, as it was developed specifically for this study. Furthermore, it was built entirely from dense layers, without incorporating any CNN layers. This was a deliberate design choice, as the researchers sought to investigate the potential of dense layer networks for image classification tasks. The second model employed in this study was a custom CNN model composed of four CNN layers to extract features from the input images. The CNN layers consist of multiple filters that perform feature extraction operations such as detecting edges, textures, and shapes. The extracted features were then fed into three dense layers, including an output layer with four/three classes for the two given datasets. In this output layer, a softmax activation function was used to produce a probability distribution over the four classes. Notably, the custom CNN contained around 38 million trainable parameters for both MC-Dropout and VI models. The third model used in the experiments was the ResNet152, which is a widely used and popular model for image classification. This model consists of 152 convolutional layers and was pre-trained on the large-scale ImageNet dataset. The ResNet152 was modified for the classification task in this study by adding three dense layers, with the final layer being an output layer that employed a softmax activation function to generate a probability distribution over the classes. The utilized ResNet had around 60 million trainable parameters for both MC-Dropout and VI models. The fourth model utilized in the experiments is based on the Inception V3 model, which is a pre-trained model that has previously been trained on the ImageNet dataset. This model is well-known for its remarkable ability to detect and classify objects in images, making it a suitable choice for this study. The Inception V3 model has around 42 layers, and three dense layers were added to the top of the pre-trained model. The output layer consisted of four/three classes for the utilized datasets, and a softmax activation function was employed to produce a probability distribution over these classes. Both MC-Dropout and VI models of the Inception V3 model had approximately 23 million parameters. The last model used in our experiments is based on the DenseNet121 architecture, which is an efficient CNN architecture designed to mitigate the problem of vanishing gradients and improve the flow of gradients. This architecture consists of dense blocks, where each dense block contains multiple layers connected by direct connections, allowing the model to use the features learned by previous layers and learn more complex representations. The DenseNet121 is pre-trained on the ImageNet dataset and has a total of 120 convolution layers. Three dense layers, including an output layer with four/three classes, were added on top of the model, and a softmax activation function was used to produce a probability distribution over the four classes.