# Dual-Modal Transformer with Enhanced Inter- and Intra-Modality Interactions for Image Captioning
## 2 Background  
In one of the previous work studies, the image captioning task had been carried out using the Flickr 30k dataset and the captioning system had been implemented using the long short-term memory (LSTM) network which sought to achieve a median rank (Med -R) of 5. An attention-based model has been used that combines both bottom-up and top-down strategies. Recurrent neural network (RNN) has been used that gives Bilingual Evaluation Understudy Score (BLEU)-2 and BLEU-3 scores of 50.4 and 35.7, respectively. One of the studied works proposed the methodology of merging systems that primarily takes into consideration partial captions, which are subsequently merged with image vectors to generate the desired captions, which also incorporates the RNN hidden state vector [13]. Visualising or seeing an image does not demand much effort, whereas, on the other hand, describing the scene in the image requires a huge effort from the human side. There has been a potential increase in the use of Flickr 8k and Flickr 30k datasets, which comprise about 8000 images and 30,000 images, respectively, purely justifying their naming convention [14]. The Flickr 30k dataset also depicts its advantageous behaviour in the domain of text grounding tasks, performing relatively well as compared to other conventional datasets by approximately 3.088%. Flickr and MS COCO simulate explosive growth in different deep learning tasks and their respective applications. Faster RCNN can also be incorporated to generate text descriptions about a specific image in the sequence of LSTM and RNN, which resulted in a BLEU-1 score of about 59.8 [15]. The Flickr 30k dataset has established its importance and advantage over the field of automatic image captioning (AIC), preferred to depict the most remarkable data out of images by finding the relationship among different objects present in the image [16]. The half and half bidirectional LSTM approach with CNN [17] has also depicted significant outcomes for picture extraction and captioning on Flicker complete datasets, thereby achieving a true positive ratio of 86% and a false positive ratio of 10%. Extending the image captioning from English to the Arabic language, the authors have proposed a deep recurrent neural network--long short-term memory RNN-LSTM model concatenated with CNN [18], which uses Google translated Arabic captions for training the model. Furthermore, despite having a small Arabic version of Flickr and MS COCO, the model can achieve a BLEU-1 score of 46.2. The authors improvised on the conventional transformer model and introduced the object relation to the transformer model. The proposed methodology shows a significant increase in results. The CIDE-rand BLUE-4 scores achieved were 128.3 and 38.6, respectively [19]. The authors in [20] proposed a fully attentive image captioning methodology. The proposed architecture uses a multi-layer encoder-decoder architecture for generating image regions and output sentences. The MS COCO dataset has been used for experimentation and achieved a BLUE-4 score of 39.7. [PERSON] et al. [21] proposed an attention-aligned transformer model for image captioning; image mask operation has been added to improvise on results. The proposed methodology achieved a BLUE-4 score with a value of 39.8 and CIDEr value of 133.9. [PERSON] et al. [22] proposed a hierarchical approach employing the GAN framework as well as RL optimisation. This three-step training strategy produces high-quality captions achieving a BLEU-1 score of 73.03. For conducting object detection in an image, an object detection mod called Faster R-CNN is used, which is trained on the COCO 2017 dataset [23]. This model consists of two major modules, the primer and one convolutional layer for proposed regions. The other module works as the detector by using these proposed regions to generate region proposals.