Language Resource Efficient Learning for Captioning (2021)

Abstract: Due to complex cognitive and inferential efforts involved in the manual generation of one caption per image/video input, the human annotation resources are very limited for captioning tasks. We define language resource efficient as reaching the same performance with fewer annotated captions per input. We first study the performance degradation of caption models in different language resource settings. Our analysis of caption models with SC loss shows that the performance degradation is caused by the increasingly noisy estimation of reward and baseline with fewer language resources. To mitigate this issue, we propose to reduce the variance of noise in the baseline by generalizing the single pairwise comparison in SC loss and using multiple generalized pairwise comparisons. The generalized pairwise comparison (GPC) measures the difference between the evaluation scores of two captions with respect to an input. Empirically, we show that the model trained with the proposed GPC loss is efficient on language resource and achieves similar performance with the state-of-the-art models on MSCOCO by using only half of the language resources. Furthermore, our model significantly outperforms the state-of-the-art models on a video caption dataset that has only one labeled caption per input in the training set.

## introduction generating natural language descriptions for images and videos @xcite @xcite is one of the core steps towards ultimate image and video understanding. however, the cost of collecting a caption dataset is nontrivial. actually, it is much higher than the cost of collecting a detection/classification dataset with the same number of images/videos, figure 1 : behavior of models in different language resource settings: "re-k" means k labeled captions per input are used in training and "full" means using all 5 labeled captions per input in training. xe loss is supervised learning @xcite ; sc loss is reinforcement learning @xcite ; gpc loss is the proposed method. the scale of a caption dataset can be defined by the product of the number of images/videos and the number of captions per input. thus there are two ways to reduce the labeling resources: reducing the number of images/videos (image/video resource) or reducing the number of captions per input (language resource). in this paper, we focus on "language resource efficient" that aims to reach the same performance with fewer language resources. as shown in figure 1 , with fewer number of captions provided per input from full setting to re-1 setting in training, the performance of the model degrades in general. slower degrading curve means that the corresponding model requires fewer number of captions per input to reach the performance of other models and therefore is more resource-efficient. in figure 1 , the performance degradation of models with xe loss trained by supervised learning is easy to understand as there are fewer supervisions with fewer captions per input. however, the performance degradation of models with sc loss trained by reinforcement learning is more complex. in sc loss, we sample one caption from the model for each input in each epoch of training. thus, the number of captions sampled from the model in training is irrelevant to the size of language resource. on the other hand, when calculating the reward in sc loss, we first calculate the evaluation score based on each groundtruth caption and then average the evaluation scores across multiple groundtruth captions of the same input, which is related to the size of language resource. similar things happen when we calculate the baseline value in the sc approach, which evaluates the score of the caption decoded greedily from the model. thus, the calculation of reward and baseline is affected by the size of language resource. as the evaluation score, e.g., cider @xcite , still has some gap from human judgement, we consider that both the reward and baseline in the sc model have noise. such noise is amplified when the language resource is small, which contributes to the performance degradation of models with sc loss. to solve this issue, we introduce generalized pairwise comparison (gpc) to reduce the noise in the baseline. gpc measures the difference between the evaluation scores of two captions with respect to an input. for example, neither "a white cat plays with a ball" nor "a dog plays with a ball" is a correct caption for the input image with a groundtruth caption "a brown cat plays with a brown ball". however, the former is much closer to the groundtruth. such subtlety comes from the fact that a caption is a complex object. we use the pairwise comparison to quantify the difference rather than the absolute value. we propose to combine multiple gpcs by comparing the sampled caption to multiple other captions, which results in gpc loss. the gpc loss can be decomposed into two items. the first item is the reward of the sampled caption, same as the reward in sc. the second item is the average evaluation score of multiple captions other than the sampled caption, which works as a new baseline with smaller noise in estimation. we show theoretically and empirically that gpc loss has a less noisy baseline compared to sc loss. experimental results show that our proposed gpc can achieve the same performance as the state-ofthe-art sc loss @xcite while using only half of the captions per input on mscoco @xcite . we also test gpc on a video caption dataset tgif @xcite , which has only one caption per video in the training set (low language resource), and achieve significant performance improvement over the state-of-the-art models on all metrics. in summary, the main contributions of this work are as follows: ## related work with the success of the encoder-decoder architecture in machine translation @xcite , researchers begin to apply the encoder-decoder architecture to directly generate image/video descriptions in an end-to-end way @xcite @xcite . convolutional neural networks are utilized as the encoder to encode visual contents as distributional vector representations, and recurrent neural networks are widely used as the decoder to produce natural and meaningful description sentences. the success of encoder-decoder architectures in the captioning task has attracted more research interest on this topic. many research works have been proposed to improve the basic architecture. for example, the spatial @xcite and temporal @xcite attention mechanisms have been proposed in image and video captioning respectively to dynamically select relevant visual content for generating future words. semantic concepts produced in object detection and action recognition tasks are also beneficial to caption generation and have been encoded into the decoder in different approaches @xcite . all these research works focus on improving the network architecture under the supervised problem formulation with cross-entropy loss. recently, researchers @xcite @xcite have proposed to use reinforcement learning to bridge the gap between training and testing in the captioning task, which considerably boosts the performance. dai and lin @xcite combine the contrast loss with the cross-entropy loss to generate more discriminating captions. luo et al. @xcite take a different approach to generate more discriminating captions by adding the contrastive loss as reward. all these works improve the caption performance by changing the loss functions. meanwhile, many caption datasets @xcite @xcite covering different media such as images, gifs and videos have been proposed to promote the captioning research. among them, mscoco is one of the largest one in both the number of instances and the number of instancecaption pairs. many of the caption datasets are either small in the number of instances such as msrvtt @xcite or the number of captions per instance such as tgif @xcite and @xcite due to the limited budget for collecting data. surprisingly, the huge cost of data collecting has been neglected and most research works are trained and evaluated on rich resource datasets such as mscoco. to the best of our knowledge, we are the first to take the labeling resource into account at the beginning to develop the model. ## reducing variance by generalized pairwise comparison and make the estimation of the prefect baseline b more accurate by generalized pairwise comparison (gpc). using re-1 as an example, we have only one groundtruth caption y i for input x i , which results in only one call of the evaluation score φ in the calculation of the baseline. gpc enables us to add more independent calls of φ even in the re-1 setting to reduce the noise in the baseline and still keeps the difference between the reward and the baseline meaningful, which is the merit of sc loss. where y i is the groundtruth caption used as the reference, and y, y cp could be any two captions, e.g., captions sampled from the model or captions associated with other input in the dataset. as illustrated in figure 2 , the generalized pairwise comparison measures the difference between the evaluation scores of φ(y, y i ) and φ(y cp , y i ) with the reference caption y i as follows: we show that sc is a special case of gpc if we substitute y and y cp by the sampled caption y s and the greedily decoded caption y g respectively: in gpc view, the meaning of r(y s ) -b is that how much better the sampled caption y s is compared to the greedily decoded caption y g on the evaluation score φ with the reference y i . actually, we could substitute y cp with any other caption in gpc and the corresponding meaning is the comparison of the sampled caption y s with any other caption, including the greedily decoded caption. furthermore, we could combine multiple generalized pairwise comparisons instead of only using single one. for m multiple gpcs, we average them and get: thus, comparing eq (12) with eq (8), we could reduce the noise in the baseline even in the re-1 setting (@xmath0. we leave making the estimation of the reward more accurate in the future work. ## conclusion in this paper, we propose the language resource efficient concept for captioning tasks in terms of the number of captions per input. our analysis shows that in captioning tasks, fewer captions per input lead to larger noise in estimating the reward and baseline for self-critical loss of reinforcement learning. we propose to reduce the noise in the baseline by multiple generalized pairwise comparisons, which results in the gpc loss. experimental results show that our proposed model is efficient on language resource and achieves similar performance with the state-of-the-art models by using only half of the captions per input. furthermore, the proposed model performs significantly better than the state-of-the-art models on a video caption dataset that has only one labeled caption per input.