# An Effective Med-VQA Method Using a Transformer with Weights Fusion of Multiple Fine-Tuned Models
## 3 Methodology
### SOUPS Fusion Method
It is calculated using the equation: \[F1=\frac{2\times\text{precision}\times\text{recall}}{\text{precision}+\text{ recall}}\] (24)
* **Macro-averaged precision:** The macro-averaged precision is calculated by taking the average of the precision scores for each class. The equation for the macro-averaged precision can be expressed as: \[\text{Macro Avg Precision}=\frac{1}{n}\sum_{i=1}^{n}\frac{TP_{i}}{TP_{i}+FP_{i}}\] (25) where \(n\), \(TP_{i}\), and \(FP_{i}\) are the total number of classes, the number of true positives for class \(i\), and class \(i\), and the number of false positives for class \(i\), respectively. * **Macro-averaged recall:** The macro-averaged recall is calculated by taking the average of the recall scores for each class. InaTeX, the equation for the macro-averaged recall can be expressed as: \[\text{Macro Avg Recall}=\frac{1}{n}\sum_{i=1}^{n}\frac{TP_{i}}{TP_{i}+FN_{i}}\] (26) where \(n\), \(TP_{i}\), and \(FN_{i}\) are the total number of classes, true positives for class \(i\), and false negatives for class \(i\), respectively. * **Macro-averaged F1-score:** The macro-averaged F1-score is calculated by taking the average of the F1-scores for each class. The equation for the macro-averaged F1-score can be expressed as : \[\text{Macro Avg F1-score}=\frac{1}{n}\sum_{i=1}^{n}\frac{2\times\text{ precision}_{i}\times\text{recall}_{i}}{\text{precision}_{i}+\text{recall}_{i}}\] (27) where \(n\) is the total number of classes, and \(\text{precision}_{i}\) and \(\text{recall}_{i}\) are the precision and recall values for class \(i\), respectively. * **Weighted-average precision:** The weighted-average precision is calculated by taking the weighted average of the precision scores for each class. The equation for the weighted-average precision can be expressed as : \[\text{Weighted Avg Precision}=\frac{\sum_{i=1}^{n}w_{i}\times\frac{TP_{i}}{TP_{i}+TP _{i}}}{\sum_{i=1}^{n}w_{i}}\] (28) where \(n\), \(TP_{i}\), \(FP_{i}\), and \(w_{i}\) are the total number of classes, the number of true positives for class \(i\), the number of false positives for class \(i\), and the weight for class \(i\), respectively. * **Weighted average recall:** The weighted average recall is calculated by taking the weighted average of the recall scores for each class. The equation for the weighted-average precision can be expressed as : Figure 7:0 (29) where \(n\), \(TP_{i}\), \(FN_{i}\), and \(w_{i}\) are the total number of classes, the number of true positives for class \(i\), the number of false negatives for class \(i\), and the weight for class \(i\), respectively. * **Weighted-average F1-score:** The weighted-average F1-score is calculated by taking the weighted average of the F1-scores for each class. InaTeX, the equation for the weighted-average F1-score can be expressed as : Figure 7:1 (30) where \(n\), \(\text{precision}_{i}\), \(\text{recall}_{i}\), and \(w_{i}\) are the total number of classes, the precision values for class \(i\), recall values for class \(i\), and the weight for class \(i\), respectively. ### Results and Analysis

In this paper, we proposed two models that are based on ELCTRA-base, SWIN-base, and MLP for textual, visual, and fusion VQA phases. The first model parameters were selected based on the best validation model performance in the training. The second model is a fusion of three of the first model with different hyper-parameters based on the greedy soup method. The proposed models are compared to the SOTA models proposed by [PERSON] et al [5]. [PERSON] et al. [5] designed a VQA model that utilized ResNet101, LSTM, and multi-glimpse attention for visual, textual, and fusion, respectively. They designed a consistency loss function that penalizes the model if it answers the related questions with inconsistent answers. They trained the model without attention, with attention, with attention with SQuINT [83] configuration, and with attention and consistency loss.