# Text Augmentation Using BERT for Image Captioning
## 1 Introduction  
Image captioning is the task of automatically generating a textual description of an image [1]. The goal pursued by the researchers is to make these textual descriptions as similar as possible to how a human would describe an image. Systems of such generation capability can be used to help visually impaired people improve a human-computer interactions by introducing visual concepts to a computer and create better features for information retrieval using images [2; 3]. In recent years, most approaches to solving the image captioning task include using neural networks. Most of the used neural networks architectures are of an encoder-decoder type for example [4; 5; 6; 7; 8]. In such models, an image is first encoded to its hidden representation and then a textual description of this image is generated (decoded) based on this hidden representation. Convolutional neural networks, such as VGG [9] and ResNet [10], are most often used as encoders, because they have proven themselves in a variety of different computer vision tasks. Recurrent neural networks, such as RNN [11] or LSTM [12], are used as decoders due to their wide applicability for natural language processing tasks. However, the training and use of recurrent neural networks is quite challenging. To preserve the information from the previous steps RNN uses a hidden state which task is to summarize and store this information in a single vector. Nevertheless, this vector usually has a rather small dimension and, as a result, a limited ability to remember information. In addition, as the length of the sequence increases the complexity of training, the model for generating a sequence of words also increases. Such methods as attention introduced in [13] and transformers described in [14], which has proven their efficiency in solving a large set of NLP tasks, particularly sequence to sequence tasks,allow for dealing with mentioned problems for caption generation task. The attention mechanism helps RNN to attend areas of the image that are the most important on a next word generation step. This helps to improve the quality of generated captions and gives some interpretation of the generation process. The transformer has the self-attention and multi-head attention modules, which allow you to get rid of recurrence while not losing the quality of taking into account the context and history of previously generated words. Year after year, models for image captioning are becoming increasingly complicated. However, high-quality training of complex models (in particular neural networks) requires a sufficiently large amount of labeled data. Although such commonly used for image captioning dataset as MSCOCO [15] and Flickr30K [16] have a relatively large number of images in the training set, they still have a lack of different descriptions per one image. Each image has in average no more than five different captions. More than that some captions within the same image are quite similar and do not show all the diversity with which a person can describe an image. Therefore, many models has poor performance from the generated descriptions diversity point of view. Problems with an insufficient number of diverse answers that a person can give in response to a specific problem also exist for other vision-language tasks. Accordingly, for example, in the main datasets [17; 18; 19; 20; 21] for the visual question answering problem there are only a few correct answers for each question. However, the various answers that a person could give in reality are quite large. The similar problems can be encountered when solving the visual dialog task. The datasets [22; 23] for this task contain only a limited number of replicas. To tackle this problem, we applied text augmentation methods to image captions from a MSCOCO dataset. The dataset augmentation is widely used for boosting a performance and obtaining model stability in a many machine learning tasks. The various language model can be used for a text augmentation.