# Bi-LS-AttM: A Bidirectional LSTM and Attention Mechanism Model for Improving Image Captioning
## 4 Experiments
### Experimental Results on the Retrieval of Image-Sentence  
In assessing image-sentence retrieval, we primarily focused on retrieval scores. Table 7 presents the R@K and Medr scores from our model's image-sentence retrieval across various datasets. Generally, our model surpasses previous methodologies on most metrics, with a particularly strong performance on the MSCOCO dataset. Notably, the Bi-LS-AttM outstrips advanced models in both image-to-sentence and sentence-to-image retrieval tasks. However, some metrics reveal suboptimal performance; for instance, the Mind's Eye model [42], which effectively integrates image and text features, outperforms our model on the Flickr30K dataset. We posit that incorporating an adaptive attention mechanism could enhance efficiency in image-sentence retrieval tasks, a hypothesis we aim to investigate in future research. We conducted additional experiments to authenticate our model's performance in image-sentence retrieval tasks. Figure 7 presents examples from several retrieval experiments on the MSCOCO validation set. In each caption query, the model retrieves visually congruent images and captions, illustrating its proficiency in discerning the visual-textual association in image caption rankings. The upper dashed line represents image \begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{4}{c}{**Flickr30K**} & \multicolumn{4}{c}{**MSCOCO**} \\ \cline{2-11}  & **B-1** & **B-2** & **B-3** & **B-4** & **METEOR** & **B-1** & **B-2** & **B-3** & **B-4** & **METEOR** \\ \hline Baseline models (without attention) & 56.2 & 38.9 & 23.0 & 14.8 & 14.7 & 59.6 & 45.8 & 28.4 & 19.7 & 15.9 \\ Our model (Bi-LS-AttM) & **64.5** & **44.6** & **29.8** & **20.2** & **18.6** & **68.8** & **51.0** & **35.9** & **25.2** & **21.5** \\ \hline \hline \end{tabular} The results shown in bold type are the best values.  
\end{table} 
Table 6: Performance scores of the baseline model and our model across various metrics. Figure 6: **(a)** Comparison of Captioning Models on Flickr30K; **(b)** Comparison of Captioning Models on MSCOCO. retrieval predicated on keywords, while the lower one symbolizes sentence retrieval based on images. In the sentence-image retrieval task, we generated three images that mirror the keywords and sentences and then selected the matching image based on its similarity. For the image-sentence retrieval task, we produced three appropriate captions grounded on the im \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Dataset**} & \multirow{2}{*}{**Model**} & \multicolumn{4}{c}{**Image to Sentence**} & \multicolumn{4}{c}{**Sentence to Image**} \\ \cline{2-10}  & & **R@1** & **R@2** & **R@3** & **Medr** & **R@1** & **R@2** & **R@3** & **Medr** \\ \hline \multirow{5}{*}{Flickr30k} & Deep VS [41] & 22.2 & 48.2 & 61.4 & 4.8 & 15.2 & 37.7 & 50.5 & 9.2 \\  & m-RNN(AlexNet) [40] & 18.4 & 40.2 & 50.9 & 10.0 & 12.6 & 31.2 & 41.5 & 16.0 \\  & Mind’s Eye [42] & 18.5 & 45.7 & 58.1 & 7.0 & 16.6 & 42.5 & **58.9** & 8.0 \\  & Bi-LSTM [9] & 28.1 & 53.1 & 64.2 & 4.0 & 19.6 & 43.8 & 55.8 & 7.0 \\  & Deep Bi-LSTM [8] & 29.2 & **54.0** & 64.9 & 3.8 & 20.8 & 44.5 & 56.7 & 6.7 \\  & Our model (Bi-LSTM & & & & & & & & \\  & and attention mechanism) & **29.5** & 53.8 & **65.0** & **3.6** & **21.0** & **44.7** & 57.3 & **6.5** \\ \hline \multirow{5}{*}{MSCOCO} & Deep VS [41] & 16.5 & 39.2 & 52.0 & 9.0 & 10.7 & 29.6 & 42.2 & 14.0 \\  & m-RNN(AlexNet) [40] & 12.4 & 29.3 & 48.6 & 15.0 & 9.5 & 25.4 & 38.2 & 18.0 \\  & Mind’s Eye [42] & 12.8 & 35.6 & 50.1 & 11.0 & 11.6 & 33.7 & 48.5 & 10.0 \\  & Bi-LSTM [9] & 13.4 & 33.1 & 44.7 & 13.0 & 9.4 & 26.5 & 37.7 & 19.0 \\  & Deep Bi-LSTM [8] & 16.6 & 39.4 & 52.4 & 9.0 & 11.6 & 30.9 & 43.4 & 13.0 \\  & Our model (Bi-LSTM & & & & & & & \\  & and attention mechanism) & **17.2** & **40.0** & **53.6** & **8.0** & **12.2** & **35.6** & **49.8** & **11.0** \\ \hline \hline \end{tabular} The results shown in bold type are the best values.  
\end{table}