Understanding Retrieval Robustness for Retrieval-augmented Image Captioning (2024)

Abstract: Recent advances in retrieval-augmented models for image captioning highlight the benefit of retrieving related captions for efficient, lightweight models with strong domain-transfer capabilities. While these models demonstrate the success of retrieval augmentation, retrieval models are still far from perfect in practice: the retrieved information can sometimes mislead the model, resulting in incorrect generation and worse performance. In this paper, we analyze the robustness of a retrieval-augmented captioning model SmallCap. Our analysis shows that the model is sensitive to tokens that appear in the majority of the retrieved captions, and the input attribution shows that those tokens are likely copied into the generated output. Given these findings, we propose to train the model by sampling retrieved captions from more diverse sets. This decreases the chance that the model learns to copy majority tokens, and improves both in-domain and cross-domain performance.

## introduction recent retrieval-augmented image captioning models have shown success in strong image captioning performance while reducing model parameters by retrieving related captions for a given image @xcite @xcite . these models use retrieved information as additional context besides the input image. however, similar to retrieval-augmented language models @xcite , image captioning models enhanced with retrieval can sometimes be misled by irrelevant information. for example, in figure 1 the captioning model is misled by the token "elephant" in the retrieved captions, and generates captions that do not match the given image. for retrieval-augmented language models, yoran et al. @xcite have studied the cases where retrieval misled the model prediction, and address this problem with a retrieval-robust llm by continuous an elephant is parked in front of a building with retrieval retrieved captions without retrieval with robust retrieval a truck parked in front of a restaurant a white truck parked in front of a building o a fake elephant is being driven on a truck by a palace o road sign for elephant & castle walworth pointing to the right o a sign in english and chinese points out castle road o a sign pointing to elephant &castle and walwort figure 1 : comparison of generated image captions that are predicted without retrieval, misled by retrieval, and predicted with a more retrieval-robust model. the retrieval-augmented model generates the token "elephant", which appears in 3/4 of the retrieved captions. to bridge this gap in the literature, we study the robustness of the smallcap retrieval-augmented captioning model @xcite . by the definition of retrieval robustness proposed in @xcite , retrieved context should boost model performance when relevant, and should not adversely affect it when irrelevant. we thoroughly examine the robustness of the model with regards to the order of the retrieved captions, and the relevance of the retrieved content. we also present a novel analysis of model behaviour based on majority voting, supported by input attribution and attention analyses to investigate how the retrieved tokens influence the model generation. and finally, inspired by @xcite , we propose to sample the retrieved captions from a larger list during training to prevent the model from overfitting to the top relevant captions. our evaluation shows improved model robustness and better out-of-domain generalization. the main findings of this paper are: 1) we study the robustness of an existing retrieval-augmented captioning model smallcap and find it is not robust to processing randomly retrieved content. recent efforts aim to enhance rag model robustness against misguided or hallucinated generations. one approach involves filtering retrieved content @xcite @xcite @xcite by applying or training an additional evaluator. another direction focuses on improving robustness during the training of the generation model itself. specifically, for retrieval-augmented question answering with large language models, @xcite propose continued training with a synthetic dataset that contains both relevant and irrelevant context, while @xcite suggests incorporating irrelevant documents. in retrieval-augmented translation, robustness is improved through shuffling retrieved translations @xcite , ensemble model decoding @xcite , and controlled interactions between source and retrieved translations @xcite . ## robustness of retrieval-augmented image captioning to evaluate the robustness of the smallcap retrieval-augmented caption model @xcite , we conduct controlled experiments and observe its resilience to changes in (1) the order of the retrieved captions and (2) the content relevance of the retrieved captions. ## majority tokens explain behavior to better understand how each token of the retrieved content relates to the observed sensitivity discussed in the previous section, we hypothesize that the model is driven by the presence of majority tokens. in other words, when the model is prompted with retrieved captions, we assume that the predicted tokens are influenced by the tokens that appear in the majority of the retrieved captions. to test this assumption, we propose a majority voting analysis, followed by input attribution, and an attention analysis of the model behavior. ## improving robustness to retrieval via sampling in order to improve the robustness of the model to potentially noisy captions, we propose to randomly sample the captions from a larger retrieval list for a given image, instead of training with only the top-k retrieved captions. in this manner, the model can learn from more diverse context that includes both top-and lower-ranked captions. ## conclusion and future work we studied the robustness of the state-of-theart retrieval-augmented image captioning model smallcap and provide an through analysis and explanation of how retrieved captions effect the final prediction. our exploration shows that small-cap is robust to the order of the retrieved captions, but it is sensitive to retrieval noise, which has implications for using retrieval-augmented models in new domains. with extensive input attribution analysis, we show that such sensitivity is due to majority tokens in the retrieved captions. we demonstrate a more retrieval robust model can be trained with sampling methods during training. we expect that our analysis can inspire better retrieval-robust captioning models in the field. in the future, we will investigate whether the majority voting behaviour is exploited in other retrieval-augmented captioning models. we hope to further explore if other techniques such as tokendropping or prefix-tuning would further improve retrieval robustness.