# EFCMF: A Multimodal Robustness Enhancement Framework for Fine-Grained Recognition
## 3 Experiment
### Experiment Results
#### 3.2.2 Performance of the Model under Different Hyperparameters  
In this study, the two hyperparameters \(p\) and \(q\) were performed 36 experiments on each dataset at intervals of 0.2 from 0 to 1. In order to investigate the effects of hyperparameters \(p\) and \(q\) on the model's robustness and accuracy, the experimental results are plotted in this study as heat maps shown in Figures 2-4. Figure 2 shows the model's accuracy for different values of \(p\) and \(q\). The most special cases are at \(p=1\), \(q=0\), and \(q=0.2\). The reason is that for \(p=1\), the model only has data from the visual or language modality for each input sample. When \(q<0.2\), the probability of missing data of visual modality is higher, and the model is mostly training the extraction module of language modality at this time. \(q<0.2\) makes the model use only the information of linguistic modality for fine-grained recognition, and therefore the accuracy is lower. Figure 3: Heat map of accuracy of model facing modal missing for different values of \(p\) and \(q\). (**a**) Birds. (**b**) Flowers. Figure 2: Heat map of model accuracy for different values of \(p\) and \(q\). (**a**) Birds. (**b**) Flowers. Figure 3 shows the heat map of the accuracy of the model in the face of the missing modality. It can be seen that the accuracy of the model increases with the increase of the modal deactivation probability \(p\) and the decrease of the visual modality integrity probability \(q\) and stays at a low level when facing the missing visual modality. The above phenomenon is related to the degree of training in the language modality feature extraction module. The higher the training degree of the linguistic modality feature extraction module, the better the model can face the visual modality missing. Figure 4: Heat map of the accuracy of the model against the adversarial examples attack with different values of \(p\) and \(q\). (**a**) Birds. (**b**) Flowers. Tables 6 and 7 show that the model's accuracy mostly stays the same when \(p\) is less than 0.8. The accuracy of the model drops substantially when \(p=1\). The reason is that with \(p=1\), the model only uses the data of the visual modality or the data of the linguistic modality to update the weights each time, especially in \(q=0\). The model only uses the features of the linguistic modality for training, and the model degenerates into a recognition model of the linguistic modality, so the accuracy drops considerably. In the face of the adversarial example attack, the model can achieve the maximum number of accuracy maxima in the face of the adversarial sample attack at \(p=0.8\). This phenomenon is because a higher \(p\)-value can better ensure the independence between two features, whose standard features can represent more information in an integrated way and, therefore, can achieve higher accuracy. In contrast, at \(p=0\), the model is not trained with random modal deactivation. Its average accuracy will be lower than the two maximum value points and higher than the minimum points. The results show that the parameter \(p\) can improve in robustness only by using the correct value. Otherwise, it may not only fail to improve the robustness but may also decrease it. In the face of modality missing, it shows that both \(p=0\) and \(p=1\) correspond to poor average accuracy. These extreme cases represent training only the verbal modal feature extraction module and the visual modal feature extraction module, respectively, leading to model failure. (2) Analysis of the effect of the visual modal integrity probability parameter \(q\) on the model  
By averaging Figures 2-4 on the vertical axis, we can obtain the average accuracy of the visual modal integrity probability \(q\) in each case, as shown in Figure 6. There is an maximum point at \(q=0.4\) for both datasets, achieving the highest accuracy under most adversarial sample attacks. Equation (14) shows the reason for this point.