# Learn and Tell: Learning Priors for Image Caption Generation
## 4 Experiments
### Qualitative Analysis  
Figure 6 shows a few examples with images and captions generated by the comparison models compared to results generated with our proposed PANN. We derived the captions using the same setting and training method, i.e., SCST optimization [38]. From these examples, we found that these \"base\" models are less accurate for the image content, while our method can result in relatively better captions. More specifically, our proposed global attention method is superior in the following two aspects: (1) Our proposed PANN could help the language model correctly focus on the caption objects. For example, our proposed method helped the model focus on the \"flight\" in the first example and \"tv\" on the second example in Figure 6, while the other models chose to ignore them. (2) Our proposed PANN could help the language model count objects of the same kind more accurately. For example, there are two benches and two motorcycles in the third and fourth images in Figure 6, but the comparison models only gave one in their captions. However, our proposed model only tends to recognize concrete objects while ignoring relatively abstract concepts. For instance, in the fourth image, \begin{table}
\begin{tabular}{c c c c c c c c c c c c} \hline \hline \multicolumn{11}{c}{**Single Model**} \\ \hline \hline
**Model** & \multicolumn{4}{c}{**Cross-Entropy Loss**} & \multicolumn{4}{c}{**SCST Training (CIDEr-D Score Optimization)**} \\ \hline
**Metric** & **B@1** & **B@4** & **M** & **R** & **C** & **S** & **B@1** & **B@4** & **M** & **R** & **C** & **S** \\ \hline Up-Down [4] & 77.3 & 35.2 & 26.1 & 54.6 & 112.3 & 19.4 & 79.8 & 36.3 & 27.7 & 56.9 & 120.1 & 21.4 \\ RFNet [47] & 76.4 & 35.8 & 27.4 & 56.8 & 112.5 & 20.5 & 79.1 & 36.5 & 27.7 & 57.3 & 121.9 & 21.2 \\ GCN-LSTM [15] & 77.3 & 36.8 & 27.9 & 57.0 & 116.3 & 20.9 & 80.5 & 38.2 & 28.5 & 58.3 & 127.6 & 22.0 \\ Att2 all [7] & 77.5 & 34.6 & 26.1 & 55.3 & 110.5 & 19.3 & 77.3 & 34.2 & 26.7 & 56.9 & 114 & 19.8 \\ AoANet [11] & 77.4 & 37.2 & 28.4 & 57.5 & 119.8 & 21.3 & 80.2 & **38.9** & 29.2 & **58.8** & 129.8 & 22.4 \\ objRel [18] & **79.2** & 37.4 & 27.7 & 56.9 & 120.1 & 21.4 & 80.5 & 38.6 & **28.7** & 58.4 & 128.3 & 22.6 \\ Ours & 78.9 & **37.9** & **28.8** & **57.6** & **120.3** & **21.5** & **80.6** & 38.7 & **28.7** & 58.7 & **129.6** & **22.7** \\ \hline \multicolumn{11}{c}{Ensemble Method} \\ \hline Up-Down [4] & 77.6 & 35.7 & 26.7 & 54.9 & 112.4 & 19.6 & 80.0 & 36.8 & 27.8 & 57.2 & 122.3 & 21.5 \\ RFNet [47] & 77.4 & 37.0 & 27.9 & 57.3 & 116.3 & 20.8 & 80.4 & 37.9 & 28.3 & 58.3 & 125.7 & 21.7 \\ GCN-LSTM [15] & 77.4 & 37.1 & 28.1 & 57.2 & 117.1 & 21.1 & **80.9** & 38.3 & 28.6 & 58.5 & 128.7 & 22.1 \\ Att2 all [7] & 77.9 & 35.1 & 26.8 & 55.6 & 112.7 & 20.6 & 78.1 & 35.3 & 27.1 & 57.1 & 116.4 & 20.5 \\ objRel [18] & 80.1 & 37.8 & 28.1 & 57.0 & 122.3 & **21.6** & 80.7 & **38.8** & **28.9** & 58.7 & 128.4 & 22.6 \\ Ours & **80.6** & **38.2** & **28.8** & **57.8** & **122.8** & **21.6** & **80.9** & 38.7 & **28.9** & **58.8** & **129.8** & **22.9** \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparative analysis of existing state-of-the-art approaches. We compared the original models with the proposed PANN. Two types of results are presented, i.e., single model and an ensemble method. All results were obtained on MS-COCO’s “Karpathy” test split, where B@1, B@4, M, R, C, and S stand for BLEU-1, BLEU-4, METEOR, ROUGE-L, CIDEr-D, and SPICE scores, respectively. All values are reported as a percentage (%). our model could not give the season words \"autumn\" or \"fall\"; this limitation of understanding higher concepts will be our next research target in future work. In the Figure 7, we visualize the region proposal with the highest attention weight at each time step of the caption generation process. We selected the [4] as the \"base\" model and \"base\" + PANN as the comparison, and training processes were exactly the same using XE-loss, which has been discussed above.