# An Effective Med-VQA Method Using a Transformer with Weights Fusion of Multiple Fine-Tuned Models
## 3 Methodology
### SOUPS Fusion Method
The greedy soup model with batch size 16 outperforms the best-value model with the same 16 batch size by 1.67%, 1.27%, 1.67%, and 1.51% for accuracy, weighted average precision, weighted average recall, and weighted average F1-recall, respectively. Figure 13: The model optimization using different batch sizes and learning rate \(=1.0\times 10^{-4}\). The lighter line in the graph represents the validation accuracy, whereas the bold line is the smoothed validation accuracy. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Accuracy** & **Weights Avg Precision** & **Weights Avg Recall** & **Weights Avg F1-Score** \\ \hline bv-32-\(1.0\times 10^{-4}\) & 84.97 & 85.98 & 84.97 & 85.15 \\ bv-16-\(1.0\times 10^{-4}\) & 85.74 & 86.40 & 85.74 & 85.94 \\ gs-32-\(1.0\times 10^{-4}\) & 86.80 & 87.29 & 86.80 & 86.93 \\ gs-16-\(1.0\times 10^{-4}\) & 87.41 & 87.67 & 87.41 & 87.45 \\ \hline \hline \end{tabular}
\end{table} 
Table 14: A comparison between the best-value model and the greedy soup model in different batch size values. Figure 14: The best-value model with batch size = 16 and learning rate = \(1.0\times 10^{-4}\). \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Answer** & **Precision** & **Recall** & **F1-Score** & **Samples No.** \\ \hline
0 & 0.9231 & 0.7347 & 0.8182 & 49 \\
1 & 0.3077 & 0.5333 & 0.3902 & 15 \\
2 & 0.9091 & 0.8955 & 0.9023 & 67 \\ no & 0.8952 & 0.8538 & 0.8740 & 650 \\ yes & 0.8304 & 0.8774 & 0.8532 & 530 \\ accuracy & & & 0.8574 & 1311 \\ macro avg & 0.7731 & 0.7790 & 0.7676 & 1311 \\ weighted avg & 0.8640 & 0.8574 & 0.8594 & 1311 \\ \hline \hline \end{tabular}
\end{table} 
Table 13: The result of the the best-value model with batch size = 16 and lr = \(1.0\times 10^{-4}\). chart of the 12 models' validation accuracy. Tables 14-16 refer to the best-value-based and greedy-soup-based models with bv-batch-lr and gs-batch-lr, respectively, where batch and lr are replaced with batch size and learning rate values. For example, the model gs-128-4 is the model that is fine-tuned with the greedy soup method and training in batch size 128 and a learning rate of \(1.0\times 10^{-4}\). \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{**Accuracy**} & **Avg. Macro Precision** & **Avg. Macro Recall** & **Avg. Macro Precision** & **Weighted Avg. Recall** & **Weighted Avg. F1-Score** \\ \hline bv-32-1.0 \(\times 10^{-4}\) & 0.8497 & 0.8138 & **0.8388** & **0.8123** & 0.8598 & 0.8497 & 0.8515 \\ bv-16-1.0 \(\times 10^{-4}\) & 0.8574 & 0.7731 & 0.7790 & 0.7676 & 0.8640 & 0.8574 & 0.8594 \\ gs-32-1.0 \(\times 10^{-4}\) & 0.868 & 0.7997 & 0.8108 & 0.7999 & 0.8729 & 0.8680 & 0.8693 \\ gs-32-9.0 \(\times 10^{-5}\) & 0.8581 & 0.7787 & 0.7751 & 0.7762 & 0.8604 & 0.8581 & 0.8582 \\ gs-32-8.0 \(\times 10^{-5}\) & 0.8604 & 0.7735 & 0.7773 & 0.7725 & 0.8637 & 0.8604 & 0.8614 \\ gs-32-2.0 \(\times 10^{-4}\) & 0.8497 & 0.7853 & 0.7768 & 0.7803 & 0.8497 & 0.8497 & 0.8495 \\ gs-32-3.0 \(\times 10^{-4}\) & 0.6667 & 0.5961 & 0.6258 & 0.5937 & 0.6925 & 0.6667 & 0.6581 \\ gs-32-1.0 \(\times 10^{-3}\) & 0.6461 & 0.5714 & 0.6073 & 0.5767 & 0.6743 & 0.6461 & 0.6346 \\ gs-32-1.0 \(\times 10^{-5}\) & 0.8574 & **0.8352** & 0.7590 & 0.7697 & 0.8577 & 0.8574 & 0.8557 \\ gs-16-1.0 \(\times 10^{-4}\) & **0.8741** & 0.7924 & 0.8002 & 0.7939 & **0.8767** & **0.8741** & **0.8745** \\ gs-64-1.0 \(\times 10^{-4}\) & 0.8345 & 0.7940 & 0.7897 & 0.7875 & 0.8442 & 0.8345 & 0.8350 \\ gs-128-1.0 \(\times 10^{-4}\) & 0.8413 & 0.7750 & 0.7684 & 0.7652 & 0.8515 & 0.8413 & 0.8422 \\ \hline \hline \end{tabular}
\end{table} 
Table 15: The result of the model with different batch sizes and learning rates. Figure 15: The comparison between best-value model and greedy soup model with different batch size (16 and 32). [PERSON] et al.