# Controllable Image Captioning with Feature Refinement and Multilayer Fusion
## 4 Experiments
### Comparative Experiment
#### 4.5.1 Comparison on the Same Test Data  
A well-known advantage of controllable image captioning is the ability to generate diverse image captions through different control signals. Each control signal is produced in different ways, so some images or sentences will be removed. Hence, we compared IANR with the two latest controllable models, VSR [16] and SCT [11], which release codes, extract features, and pretrained models.  
For fair comparison, those models were compared on the common parts of the VSR and SCT test datasets. The common part of the MSCOCO Entities has 4678 images and 14,179 sentences. The common part of the Flickr30k Entities has 1000 images and 4982 sentences. The input feature sequence of \"SCT\" is the region feature corresponding to the words in the sentence. The \"SCT _w/o_ sequence\" generates sentences by predicting the sequence of the selected regions. \"VSR\" achieves a better score by specifying the ground verb and associated object node features. \"VSR _w/o_ verb\" removes the ground verb information and only uses the features of the ground truth region and relations.  
The quantitative results are shown in Table 4. It is obvious that the captions generated by IANR in two datasets have much higher accuracy and diversity (CIDEr 224 VS 165.66 in VSR, Self-CIDEr 50.05 VS 46.29 in VSR). \"SCT _w/o_ sequence\" obtained the worst results because it lacked the ground region sequences. Compared with VSR, IANR does not only need to know what the relationship is. This is more consistent with the application of image captioning. Even though there is less information in control signals, the metrics of accuracy and diversity are still higher than VSR.  
\begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**Model** & **B@4** & **M** & **R** & **C** & **S** \\ \hline \multicolumn{6}{l}{State-Of-The-Art Controllable Models} \\ MC [38] & **37.1** & 27.7 & 56.1 & 126.4 & 21.5 \\ ASG2 Caption [12] & 23 & 24.5 & 50.1 & 204.2 & 42.1 \\ SCT [11] & 20.9 & 24.4 & 52.5 & 193 & 45.3 \\ VSR [16] & 23.1 & 28.0 & 55.6 & 235.1 & 48.9 \\ LCLC [15] & 35 & **27.9** & 57 & 116.8 & 21.7 \\ PC [32] & 36.4 & – & **57.5** & 124 & 21.2 \\ Sub-GC [36] & 36.2 & 27.7 & 56.6 & 115.3 & 20.7 \\ POS [13] & 31.1 & 25.3 & 53 & 103.6 & 18.8 \\ ConCap [49] & 40.5 & 30.9 & - & 133.7 & 23.8 \\ TSG [50] & 38.2 & 28.2 & 59.1 & 132.8 & 22.0 \\ ASA [51] & 44.0 & 32.0 & - & 140.4 & 23.8 \\ FVC-MT [35] & 22.4 & 25.8 & 55 & **206.3** & **47.6** \\ \hline \multicolumn{6}{l}{State-Of-The-Art Uncontrollable Models} \\ CPTR [52] & **40** & 29.1 & **59.4** & 129.4 & – \\ CGVRG [53] & 38.9 & 28.8 & 58.7 & 129.6 & 22.3 \\ AoANet [22] & 38.9 & **29.2** & 58.8 & **129.8** & **22.4** \\ BUTD [21] & 36.3 & 27.7 & 56.9 & 120.1 & 21.4 \\ SCA-CNN [54] & 30.2 & 24.4 & 52.4 & 91.2 & – \\ SCST [18] & 31.9 & 25.5 & 54.3 & 106.3 & – \\ \hline IANR & 27.57 & 28.54 & 55.59 & **256.88** & **52.51** \\ \hline \hline \end{tabular}
\end{table}
Table 3: Comparisons with the state-of-the-art on the MSCOCO Entities dataset. B@4, M, R, C, and S stand for BLUE@4, METEOR, ROUGE-L, CIDEr, and SPICE, respectively. Bold for the best.