# ACapMed: Automatic Captioning for Medical Imaging
## 4 Results  
We investigated the accuracy of the LSTM model for generating captions. The model was trained for 100 epochs; the accuracy is reported on the test set. Figure 6 illustrates the training accuracy and loss where we can see that the model learned to generate the next words for each caption sequence over time. Indeed, the accuracy was improved over epochs and the loss decreased. Moreover, the BLEU score was calculated for the set of generated captions as demonstrated in Section 3.5. We reached a BLEU score of 42.28% for the captioning using beam search with index = 5. We illustrate the results of different experiments (regarding how each  
Figure 6: Performance of the LSTM language generation model. Left side: training accuracy; right side: training loss. Epoch # refers to the number of epochs. word in the caption is generated) using argmax and beam search with index = 3, 5, and 7. Better results were obtained with beam search and beam index = 5 while other indices gave similar values. Results are shown in Table 3, and are also compared to other state-of-the-art techniques on the same dataset. We can see that the model proposed by [24], a winner of the ImageCLEF challenge in 2021, gave the best BLEU result with a value of 51.00%. In [PERSON] et al. [24], the authors introduced a multi-label classification-based method, where each word in the caption was considered to be a class of the image. The authors then trained their model to predict high-ranking words (i.e., classes of the image) based on a statistical method and then used them to construct the caption. Our results outperformed those of [25], which employed a pattern-based captioning method by combining the medical concepts identified for the image and achieved a BLEU score of 25.70%. Captions were then created based on the characteristics of captions in the training and validation sets and depended marginally on the outcomes of the model, which used concept identification. Similarly, we outperformed the results of [26], where an encoder-decoder model was presented with images as input and an attention mechanism between the encoder and the decoder. The model was inspired by the famous 'Show, Attend, and Tell' [16] model, where the image encoder was changed into a ResNet-101 network. However, this model does not employ medical concepts for the captioning process. In order to further evaluate the performance of our model, we show some examples of images from the validation set and both their ground-truth and generated captions in Figures 7-9. We can see that our model was able to accurately generate captions in both upper sub-figures and the lower left sub-figure of Figure 7, where all words were correctly predicted (i.e., the caption 'left upper lobe mass' for the example was correctly generated by our model). Even the order of the words in the caption was correct. However, in the lower right sub-figure of Figure 7, although the wording appears to be relevant and correctly generated, their ordering was wrong and, hence, the caption was incorrect as well. Moreover, the model generated some wrong words (in red) as in Figure 8 (i.e., words such as 'heterogeneous' and 'crossing the' in the lower left sub-figure were wrongly inserted in the caption, affecting the quality and the correctness of the interpretation). The model missed some words (in red), such as in the upper right sub-figure (i.e.,'material' and 'Note the large amount of subcutaneous fat circumferentially compatible' were missing). The model was trained to generate one sentence for each image and that is why it was not able to generate a caption composed of more than one sentence, which could be the case for many long medical reports. Moreover, in some cases, the model generated completely wrong captions as in Figure 9. Likewise, it appears that some reports used sentences that were too long, which lowered the quality of the embedding generated by the used encoders, and thereby, led to wrong captions.