# Boosted Transformer for Image Captioning
## 4 Experiments
### Ablation Studies
#### 4.3.2 Influence of VGA  
From Table 2, we can see that with VGA (b), all the metrics obtained improvement to some extent. VGA_xth had the similar means as that in Table 1. VGA re-attended to all the positional sequence information with the guidance of visual information, achieving a more reasonable attention distribution. The overuse of VGA can result in overfitting, so we only applied VGA to the fourth layer, which can obtain the best performance according to the ablative experiments. Some visualizations are shown in Section 4.5, which is better to reflect the effectiveness of VGA. In order to validate how much visual information VGA involves, we got rid of the sequence-to-image attention module, which was the sole way to involve visual information. From the results in (c) and (d), VGA utilized the image information in another way and could work well with the sequence information, where (c) s2s-VGA means the decoder layer was composed of sequence self-attention and VGA. (d) s2s means the decoder layer only contained sequence self-attention without the guidance of visual features.  
\begin{table}
\begin{tabular}{l l c c c c c c c c} \hline \hline  & **Model** & **B@1** & **B@2** & **B@3** & **B@4** & **M** & **R** & **C** & **S** \\ \hline (a) & baseline & 80.2 & 64.9 & 50.5 & 38.4 & 28.6 & 58.2 & 128.1 & 22.6 \\ \hline \multirow{4}{*}{(b)} & VGA\_1 st & 80.8 & 65.5 & 51.1 & 39.1 & 29.0 & 58.8 & 130.3 & 22.9 \\  & VGA\_2 nd & 80.8 & 65.7 & 51.4 & 39.4 & 29.1 & 58.9 & 130.7 & 22.9 \\  & VGA\_3 rd & 81.0 & 65.7 & 51.3 & 39.3 & 29.2 & 59.1 & 130.8 & 23.1 \\  & VGA\_4 th & **81.1** & **65.9** & **51.4** & **39.4** & **29.3** & **58.9** & **131.0** & **23.1** \\  & VGA\_4 & 80.8 & 65.5 & 51.1 & 39.2 & 29.0 & 58.8 & 129.8 & 22.9 \\ \hline (c) & s2s-VGA & 79.5 & 63.9 & 49.6 & 37.8 & 28.4 & 57.9 & 124.4 & 21.8 \\ (d) & s2s & 44.5 & 24.8 & 14.1 & 8.1 & 13.3 & 34.8 & 11.0 & 4.1 \\ \hline \hline \end{tabular}
\end{table}
Table 2: The results of different ways of using the VGA module, evaluated on the COCO Karpathy test split. s2s, sequence self-attention.  
\begin{table}
\begin{tabular}{l l c c c c c c c c} \hline \hline  & **Model** & **B@1** & **B@2** & **B@3** & **B@4** & **M** & **R** & **C** & **S** \\ \hline (a) & baseline & 80.2 & 64.9 & 50.5 & 38.4 & 28.6 & 58.2 & 128.1 & 22.6 \\ \hline \multirow{4}{*}{(b)} & VGA\_1 st & 80.8 & 65.5 & 51.1 & 39.1 & 29.0 & 58.8 & 130.3 & 22.9 \\  & VGA\_2 nd & 80.8 & 65.7 & 51.4 & 39.4 & 29.1 & 58.9 & 130.7 & 22.9 \\  & VGA\_3 rd & 81.0 & 65.7 & 51.3 & 39.3 & 29.2 & 59.1 & 130.8 & 23.1 \\  & VGA\_4 th & **81.1** & **65.9** & **51.4** & **39.4** & **29.3** & **58.9** & **131.0** & **23.1** \\  & VGA\_4 & 80.8 & 65.5 & 51.1 & 39.2 & 29.0 & 58.8 & 129.8 & 22.9 \\ \hline (c) & s2s-VGA & 79.5 & 63.9 & 49.6 & 37.8 & 28.4 & 57.9 & 124.4 & 21.8 \\ (d) & s2s & 44.5 & 24.8 & 14.1 & 8.1 & 13.3 & 34.8 & 11.0 & 4.1 \\ \hline \hline \end{tabular}
\end{table}
Table 1: The results of different ways of using the CGA module, evaluated on the COCO Karpathy test split. B@N, BLUE; M, METEOR; R, ROUGEL; C, CIDEr; S, SPICE; CGA-c, Concept-Guided Attention with aligned concept representations; CGA-y, Concept-Guided Attention with concept embedding vectors; CGA_1 st-GS(vv), Concept-Guided Attention with two identical visual features; GS, Gated Sum; CC, Channel Connect.