## 8 Discussion
(**b**) Continuation of learning with deep fine-tuning, where both the CNNs and the Transformer learn the dataset characteristics. Figure 12: Examples of exams (**a**–**c**), their medical reports and those generated by our method, and the evaluation metrics. Right foot references are in blue, left foot in red. Green text indicates exact matches; purple text shows omissions; pink text highlights the additions or discrepancies of the model. Figure 13: Additional examples of exams (**a**–**c**), their medical reports and those generated by our method, and the evaluation metrics. Right foot references are in blue, left foot in red. Green text indicates exact matches; purple text shows omissions; pink text highlights the additions or discrepancies of the model. In the final Grad-CAM in Figure 12c, we can see that the model received low scores, a BLEU-4 of 0.00, a METEOR of 0.10, and a ROUGE-L of 0.18, indicating a low correspondence with the actual medical reports. The physician performed a brief analysis, not detecting any specific findings, while the model conducted a more detailed analysis, identifying the presence of hallux valgus and a surgical pin that the physician did not mention. This example illustrates the difficulty in quantifying image-to-text results, as even when the model generates an accurate report, the use of different words and expressions can result in low metrics. Figure 13a presents the results of the model's analysis of X-ray images of the right and left feet, with visualization of important regions identified by the Grad-CAM technique. The reports generated by the model, when compared to medical reports, showed significant correspondence, as evidenced by a BLEU-4 score of 0.76, a METEOR score of 0.50, and a ROUGE-L score of 0.81. We note that the model was accurate in identifying normal bone and joint characteristics, as well as detecting hallux valgus in both feet. However, there was a discrepancy in identifying signs of osteoarthritis in the left foot, not mentioned by the model. This suggests that, while the model performs well overall, there is still room for improvement in detecting more subtle anomalies such as osteoarthritis. In Figure 13b, the results show that the model successfully captured various normal features of the right foot, with a BLEU-4 metric of 0.59, a METEOR metric of 0.41, and a ROUGE-L metric of 0.72. However, there were significant discrepancies in the left foot, where the model failed to identify the hallux valgus, signs of osteoarthritis, and the presence of a metal screw fixing the metatarsal to the proximal phalanx of the first toe. The analysis of the Grad-CAMs reveals that the model's attention is concentrated on significant parts of the images, indicating that the model recognizes relevant regions, but still fails to accurately identify certain anomalies in the left foot. This suggests that, although the model is correctly focusing on important areas, there may be a need for additional adjustments to improve sensitivity and accuracy in detecting anomalous features. Figure 13c shows a relatively low performance of the model, with a BLEU-4 metric of 0.08, a METEOR metric of 0.17, and a ROUGE-L metric of 0.31. The report generated by the model failed to identify signs of talonacular osteoarthritis and hallux valgus present in the medical report. A relevant observation is the blurred aspect of the images in this exam, with a whitish coloration outside the feet, which may have contributed to the unsatisfactory results of the model. The analysis of the Grad-CAMs reveals that the image has characteristics not common to the rest of the dataset, making it difficult for the model to focus on important features and negatively affecting performance. The BLEU, METEOR, and ROUGE-L scores achieved in this study are quantitative indicators of the correspondence between the reports generated by the model and the actual medical reports.