# Bi-LS-AttM: A Bidirectional LSTM and Attention Mechanism Model for Improving Image Captioning
## 4 Experiments
### Discussion  
**Effect of Bi-LS-AttM**: To gauge the influence of the Bi-LS-AttM, we compared the computational time of the Bi-LS-AttM model with the Bi-LSTM and deep Bi-LSTM models for caption creation and image-to-sentence retrieval tasks. Table 8 delineates the computational durations of these models for the respective tasks. We randomly selected 20 images from the Flickr30K validation set and assessed each model ten times for caption creation and image-to-sentence retrieval. The table provides the average time duration across the ten trials, excluding model initialization and training time. The caption generation time expenses encompass the extraction of image features, bidirectional caption content sampling, computation of the final caption result, and caption accuracy evaluation. Conversely, the retrieval time accounts for the computation of the image-to-sentence retrieval score, image and sentence query, and sorting operations in descending order. By employing the Fast R-CNN framework and fine-tuning the relevant parameters, our model demonstrates significant time savings in accomplishing the given task. From Table 8, we can see that our model saves about 36.5% and 26.3% of the time compared to the Bi-LSTM and deep Bi-LSTM models, respectively, in solving image captioning and image-sentence retrieval tasks. We have verified the efficiency of the Bi-LS-AttM. **Effect of image caption**: We used the Bi-LS-AttM model to generate real, accurate, and novel image descriptions. Figure 8 shows the comparison of the baseline model and our model in generating captions on the datasets. We evaluated generated captions from various perspectives. In some descriptions, the relationships between objects are well expressed (e.g., \"A hot dog and a red bottle of drink are on the table\"). In the example above, the objects 'hot dog' and 'table' are accurately identified, and the relationship between them is established. Finally, the image is described accurately and in a novel way (e.g., \"A boy dressed in black surfs the sea with a red surfboard.\"). However, the baseline model solely provides descriptive accounts of the images, lacking the generation of novel and expressive sentences to depict them. From the perspective of object detection, the object recognized in the baseline model is \"bread\" rather than the more accurate \"hot dog\". The results of the studies show our model has good efficiency. Our model can achieve a balance between performance and efficiency. **Examples of failed experiments**: Figure 9 depicts a notable number of anomalies arising from our experimental approach. It is pertinent to note that these inaccuracies primarily originate from the Flickr30K validation set, which we hypothesize may be due to the limited range and diversity present in the training dataset of this source. For instance, in the preceding images, our model exhibits imprecision in object identification (i.e., identifying \"white clothes\" when the man is not clothed). Another example demonstrates an illogical caption suggesting a man cycling on water. We surmise that these limitations could be ameliorated through improvements in our visual feature extraction aspect. Despite these anomalies, we see them as potential avenues for further research rather than setbacks. Nonetheless, it is crucial to emphasize that a substantial number of remaining cases were accurately represented, as exemplified in Figure 8. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Task** & **Bi-LSTM** & **Deep Bi-LSTM** & **Our Models** \\ \hline Caption creation & 1.04s & 1.07 s & **0.67 s** \\ Image-Sentence Retrieval & 5.78 s & 5.81 s & **4.28 s** \\ \hline \hline \end{tabular} The results shown in bold type are the best values.  
\end{table} 
Table 8: The cost of checking 10 images on Flickr30K. Figure 8: Examples of image captioning for the baseline and our model on the datasets. The captions generated by the baseline model are above, while the captions generated by our model are below.