# Image Caption Generation via Unified Retrieval and Generation-Based Method
## 4 Experiments
### Comparison with the State-of-the-Art Methods  
Our captioning model make comparison with several powerful state-of-the-art methods. The performance of these model are shows in Table 1. These models are the generation-based methods. VS-LSTM [42] replace the low-level visual features with semantic attributes. Bottom-Up and Top-Down Attention(Up-Down) [23] adopt the Faster R-CNN features as the encoder, and uses a two-layer LSTM as the decoder. ADP-ATT [6] learnt an adaptive attention model that automatically determined when to look and where to look for word generation. GCN-LSTM [43] used the Graph Convolutional Neural Network to exploit relationships of each two different entities in the image to perform image captioning. LSTM-A [44] integrated semantic attributes into the CNN plus LSTM framework for caption generation. SCST [8] makes use of attention on the grid of features and a one-layer LSTM as thedecoder. RFNet [45] exploited multiple CNNs as encoder and followed a recurrent fusion procedure to form representations to the decoder. UGRIC [11] is a model that incorporate the retrieval-based method and the generation-based method. For fair comparison, we evaluate our model on the MSCOCO \"Karpathy\" test split. In Table 1, B@1, B@4, M, R, C and S correspond to the BLEU@1, BLEU@4, METEOR, ROUGE, CIDEr and SPICE scores. All values are reported as percentage. We show the single-model experiment results of our approach along with several competitive methods. As can be observed from Table 1 that our approach is superior to the first six models in all the evaluation metrics. After the model trained by the self-critical loss, the experiment results is a little worse on BLEU-1 of the UGRIC model, but our model is superior in terms of the other five evaluation metrics. We can see UGRIC unified the retrieval and generation-based method got 123.5 CIDEr score which is inferior to ours. Although GCN-LSTM is a very strong baseline. Our model is superior in terms of all the evaluation metrics except for Bleu-1 after self-critical sequence training. It can be seen that our model got 128.0 CIDEr score which is higher than GCN-LSTM. This indicates the competitive of our model. Although our model is superior to the baselines. It is worth mentioning that our model is inferior to the models [26; 27; 28]. In Figure 3 we show several examples selected from the validation set of the description results generated by our approach. The first column is the input image, the second column is the corresponding retrieved similar image. We can see that the retrieved result is similar to the input image. The first row of the original picture is describing two men fishing on a boat with a dog. The retrieved similar image is describing a man with a dog on the boat without the fishing action which is different with the input image. Although they are different in some detailed place, these similar sequences still bring useful textual information(\"boat\", \"dog\"). The third row of the input image is a baby holding a toothbrush. The retrieved image and sentences are both visually and semantically similar to the input image. Our generated description is almost same with the ground truth. On the whole, the proposed method can generate fluency and semantic correct captions with help of the similar semantic information.