# An Analysis of the Use of Feed-Forward Sub-Modules for Transformer-Based Image Captioning Tasks
## 5 Results Discussion  
The associated epoch-by-epoch graphs are shown in Figure 4. The evaluation metric scores are displayed in Tables 3 and 4 for the comparison models and the variants, respectively. Table 5 shows a sample of the captions generated by the model variants alongside the ground truth captions (GT). Additional information is displayed in the Appendices A.1 and A.2. Figure A1 in the Appendix A.1 shows additional graphs that highlight some specific comparisons that provide deeper insights into the performance of select model variants. In Appendix A.2, Tables A1 and A2 display a larger sample of captions generated by the linear models explored in this work. Figure 5: The configuration options available at each stage in the variants tested. The first stage (bottom) shows the formats available for the input features to the model. The second stage contains the options that were considered for learning the feature representation. The top shows the next stage depending on whether it is in the encoder or decoder section. As indicated, some options are only available to the encoder, decoder or to both. As can be seen in Table 4, FC4 and FC1 are by far the best models overall when the evaluation metric performance, simplicity and training time is taken into consideration. FC1, follows the \"normal\" configuration shown in Table 2 with 6 encoder-decoder stacks, whereas FC4 simply has a deep fully connected decoder input sub-module. The evaluation result metrics of the 6-layer stacked FC1 that are significantly higher than FC0, its 3-layer counterpart show the merits of increasing the number of layers. This is also clearly observed in the graphs in Figure 11a. Both FC4 and FC1 performed better than the model that had projected input image features into deep higher dimensional spaces (FC3, FC6 and FC7) or where a reconstruction of the most important input features (as in an autoencoder) was attempted (FC5). This is because the input features which are from a Faster R-CNN network are already compressed with the most salient features (regional proposals) highlighted and extracted as feature vectors, making any further such manipulation mostly counterproductive, actually leading to a degradation in performance. Compared to the state-of-the-art Oscar [22] model, which uses self-attention and pre-training (on 6.5 million image-text pairs), FC4 attains 94% (38.1 vs. 40.5), 95% (28.2 vs. 29.7), 90% (124.4 vs. 137.6) and 96% (21.8 vs. 22.8) of the Oscar-big model [22] score in the BLEU-4, METEOR, CIDER and SPICE metrics, respectively. Compared to the Oscar-large model (larger hidden size, see their paper for more details), FC4 attains 91% (38.1 vs. 41.7), 92% (28.2 vs. 30.6), 89% (124.4 vs. 140.0) and 89% (21.8 vs. 24.5) of the model's score in the BLEU-4, METEOR, CIDER and SPICE metrics, respectively. In terms of time required to train, the results of tests that were performed at different batch sizes as shown in Table 1, demonstrate that the fully connected feed-forward variant consistently required less time to train than the self-attention variant. From the same table, it can be deduced that most of the time is spent in encoder section of the model. For example, at a batch size of 50, 60.7% and 68.8% of the total time is spent in the encoder section for the fully connected compared to the self-attention variant, respectively. This is because that section involves reading to memory several gigabytes of image data compared to a few megabytes of caption data in the decoder section. The time taken by the FC4 model is also much less than the time taken by the self-attention based model. We surmise that this could be because of the much simpler architecture of the feed-forward model which leads to far fewer processing steps. In terms of inference time, in order to process and create captions for 5000 images, the feed-forward based model takes 2.1 min compared to 1.9 min for the self-attention based one.