# Parallel Image Captioning Using 2D Masked Convolution
## 5 Result and Discussion  
To show how well our model performed, we compared the result of the MSCOCO testing dataset against some of the popular traditional image caption models with LSTM as well as other non-attention methods, including those in [16; 17], using the above evaluation metrics. As shown in Table 1, our model obtained a better result than most of the LSTM models across all the evaluation metrics, while it still fell far behind the state-of-the-art result of the LSTM approaches in [8; 14]. However, from our understanding, the significant performance increase compared to the methods in [8; 14] resulted from a critical schedule sampling method as well as from REINFORCE algorithm [29] training, which reduced the variance of the gradient and increased the probability of sampling the correct caption. Thus, we believe that, by carefully fine-tuning the model, the results from our model would be improved and could reach those of the state-of-art LSTM approaches. Figure 5 shows the qualitative comparison between our generated caption and ground truth. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Model** & **BLEU-1** & **BLEU-2** & **BLEU-3** & **BLEU-4** & **METEOR** & **CIDEr** \\ \hline NIC [12] & 66.6 & 46.1 & 32.9 & 24.6 & 23.7 & 85.5 \\ Attention (Soft) [13] & 70.7 & 49.2 & 34.4 & 24.3 & 23.90 & - \\ Attention (Hard) [13] & 71.8 & 50.4 & 35.7 & 25.0 & 23.04 & - \\ SCST [14] & - & - & - & 33.3 & 26.3 & 111.4 \\ Up-Down Attention [8] & **79.8** & - & - & **36.3** & **27.7** & **120.1** \\ \hline ConvCap [16] & 71.1 & 53.8 & 39.4 & 28.7 & 24.4 & 91.2 \\ Caption Transformer [17] & **73.0** & **56.9** & **43.6** & **33.3** & - & **108.1** \\ \hline Baseline & 70.6 & 53.2 & 38.9 & 28.3 & 24.3 & 90.9 \\ Baseline + Positional Embedding & 71.3 & 54.1 & 40.3 & 30.1 & 24.8 & 94.3 \\ Baseline + PE + Residual Connection & 73.0 & 54.6 & 42.2 & 31.6 & 25.2 & 94.4 \\ Baseline + PE + RC + Attention & 73.1 & 55.1 & 42.4 & 32.5 & 25.7 & 96.2 \\ Baseline + PE + RC + A + BeamSearch (size = 3) & **73.6** & **56.6** & **44.1** & **33.2** & **25.9** & **99.4** \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The comparison of our model with other LSTM and non-LSTM models on the MSCOCO test dataset based on the Karpathy splitting method [24]. Figure 4: The top three sentences generated by the beam search algorithm. We also compared the result of our model to the non-LSTM approaches in [16; 17]. [PERSON] et al. [16] implemented their image captioning model with a technique similar to our approach, which replaces the LSTM cells with a masked convolution network. While the architecture in [16] is conceptually simpler and uses fewer parameters, the results with our model are much better, outperforming the other approach across all evaluation metrics. As for [PERSON] et al. [17], their approach to the image captioning model is much different from our model, which makes use of the multi-headed self-attention module in [18] to generate the word sequence without the presence of LSTM. With this architecture, the best-performing model generated caption results that are comparable to our model across most of the evaluation metrics, but performed better based on CIDEr and worse based on BLEU-1. The reason for the better performance in CIDEr may result from the model being trained and optimized based on CIDEr cross entropy. Figure 6 shows the training progression of our model. Figure 5: The qualitative comparison between the captions generated by our model and the ground truth (GT) captions randomly picked from the MSCOCO dataset. Another thing to notice is the performance increase in speed during the training process. Since our model uses a convolutional approach, we can feed our words as input to the model all at once, unlike the LSTM approach, which works in a sequential process, one word at a time. By feeding all the words at once, our model also benefits from parallel GPU computing, allowing us to use a large batch size for training and to reduce the training time per epoch by 35%.