## 2 Related Work
### Image Captioning  
The structure of image captioning is mainly divided into three approaches. (1) The first is the template-based method [13; 14], which fills the sentences with object detection results. [PERSON] et al. [13] detect all candidate scenes, objects, and objects' attributes, then use the CRF algorithm to predict the correct three-element filling in the template as the captioning. The template-based methods require a manually designed syntactic template but have higher accuracy than the retrieval-based models. (2) The second is retrieval-based methods [15; 16], which retrieve similar images from the whole database and selectively combine the sentences of similar images to create image captioning. The quality of the caption depends on the similarity evaluation method and the quality of the database. The retrieval-based methods can generate sufficiently fluent sentences, but they heavily rely on the sentences in the database. Hence, it is difficult to generate a new corpus to describe new images. (3) The second is encoder-decoder-based methods, in which the encoder structure first converts the image into some vectors to represent the content of the image, and then, the decoder structure generates sentences word by word based on the vector. Early encoder-decoder methods [14; 17] encode images with soft attention to focus on different regions to generate a feature vector, and then decode the feature vector to sentences. [PERSON] et al. [18] incorporate spatial and channel-wise attention into CNN to encourage the model to focus on meaningful regions. Considering that the whole image contains a large amount of redundant information, to reduce redundant information, [PERSON] et al. [19] propose a bottom-up attention mechanism to provide region information, which comes from the detection network (Faster R-CNN [20]). The detection model trained on the large dataset not only removes redundant areas irrelevant to the description but also extracts better visual features. However, the region features a lack of relationship information. Some researchers [21; 22; 23] adopt the scene graph [24] to express objects and the semantic relationship between objects. Scene graphs bring richer information, but richer information contains more redundancy information, and the scene graph must be processed by graph convolution. These reasons limit the development of the methods based on the scene graph.  
In particular, with the success of transformers [25] in the field of NLP, many state-of-the-art models [26; 27; 28] adopt the transformer structure to design the encoder or decoder. [PERSON] et al. [26] incorporate the attention block into the encoder and decoder to measure the relevance between the attention result and the query. [PERSON] et al. [27] extend self-attention with additional \"slots\" to encode prior information. [PERSON] et al. [29] design an LRN that discovers the relationship between the object and the image regions. Apart from this, BERT, a pre-trained model composed of deep bidirectional transformers, achieved outstanding results. Many models [30; 31; 32] exploit BERT-like structures pre-trained on a large corpus to achieve state-of-the-art performance.  
Although many researchers have improved the accuracy of generated sentences through the attention-based model or transformer-based model, since the captioning model needs to be trained on the dataset, the number of images and the types of objects in the training set are limited. Hence, it is difficult to describe objects that are rarely seen or never seen in the training set.