## 2 Related Works  
In the field of image captioning, research has been mainly focused on improving the accuracy of the captioning system. Recently, studies have been conducted to generate diverse captions. A traditional captioning system converges to generate a description suitable for multiple similar images after training, resulting in general and safe captions. As a result, the diversity of generated captions is reduced, often resulting in the same explanations for similar but different images.  
Diverse captioning was designed to generate different captions for different images and to generate novel sentences not included in the training set. To generate diverse captions, popular generative models such as VAEs [9] and generative adversarial networks (GANs) [14] were used in previous work. [PERSON] et al. [15] constructed a conditional GAN model that adds an image as a condition. The generator produced captions and the discriminator distinguished the captions generated from the natural ones. After learning the generator and discriminator, the researchers separated the evaluator and proposed the E-GAN metric, which is a natural sentence evaluation scale. [PERSON] et al. [13] also proposed the GAN model, which enabled back propagation in categorical distribution by selecting words in long short-term memory (LSTM) using Gumbel softmax. The criterion score in the discriminator included not only image-to-sentence distances but also sentence-to-sentence distances for distinguishing real from fake. This scheme induced large variation between sentences.  
In this paper, a system is proposed that generates multiple captions using VAE. Such VAEs have advantages, such as stable training and generating meaningful latent representations that enable us to derive the desired results. [PERSON] et al. [10] proposed a Conditional VAE model that takes image content such as objects, as conditions for generating sentences associated with them. Two priors, Gaussian mixture model (GMM) and average Gaussian (AG), were used to determine how latent space was modeled. Experimental results showed that diversity and accuracy were both improved in relation to conventional LSTM systems and that the model with AG was more diverse and controllable than the one with GMM. In addition to captioning, there were examples of applying VAE to visual question and answering (VQA) tasks [11] and to paragraph generation [12]. [PERSON] et al. [11] projected the hidden states of the last step of the encoder LSTMs to the latent space and succeeded in generating novel questions that were diverse and not in the training set. [PERSON] et al. [12] proposed a model for generating paragraphs using the Stanford image paragraph data set. The proposed VAE-based model consists of a topic generation network and a sentence generator network that generates each sentence. As in earlier work [11], the hidden state vector of the last step of the sentence generator network RNN model was also transformed to latent space. In previous methods, training was used to construct a latent distribution of sentence embedding information, and variation was added to sentence style when generating the captions.  
In the image captioning task, not only the sentence information, but also the image information plays an important role. However, this was not reflected in the previous works. In the work reported in this paper, we use this image information to generate multiple captions that describe various content in the image. There has been research on generating different descriptions from the training set, and methods of generating different captions in similar images. However, there are not many pieces of research on generating several different sentences for one image; therefore, there was no existing relevant evaluation standard or popular evaluation method. Hence, an evaluation method is proposed for the captioning of multiple images.