Multi-Modal Open-Domain Dialogue (2021)

Abstract: Recent work in open-domain conversational agents has demonstrated that significant improvements in humanness and user preference can be achieved via massive scaling in both pre-training data and model size (Adiwardana et al., 2020; Roller et al., 2020). However, if we want to build agents with human-like abilities, we must expand beyond handling just text. A particularly important topic is the ability to see images and communicate about what is perceived. With the goal of getting humans to engage in multi-modal dialogue, we investigate combining components from state-of-the-art open-domain dialogue agents with those from state-of-the-art vision models. We study incorporating different image fusion schemes and domain-adaptive pre-training and fine-tuning strategies, and show that our best resulting model outperforms strong existing models in multi-modal dialogue while simultaneously performing as well as its predecessor (text-only) BlenderBot (Roller et al., 2020) in text-based conversation. We additionally investigate and incorporate safety components in our final model, and show that such efforts do not diminish model performance with respect to human preference.

## introduction an important goal of artificial intelligence is the construction of open-domain conversational agents that can engage humans in discourse. indeed, the future of human interaction with ai is predicated on models that can exhibit a number of different conversational skills over the course of rich dialogue. much recent work has explored building and training dialogue agents that can blend such skills throughout natural conversation, with the ultimate goal of providing an interesting and engrossing experience for humans @xcite . coupled with the advancement of large-scale model training schemes, such models are becoming increasingly human-like and engaging @xcite @xcite . in order to better approach human-like ability, however, it is necessary that agents can converse with both textual and visual context, similarly to how humans interact in the real world; indeed, communication grounded in images is naturally engaging to humans @xcite . recent efforts have gone beyond classical, fact-based tasks such as image captioning or visual question answering @xcite to produce models that can respond and communicate about images in the flow of natural conversation @xcite . in this work, we explore the extension of largescale conversational agents to image-based dia-logue. we combine representations from imagebased models that have been trained on object detection tasks @xcite with representations from transformers with billions of parameters pre-trained on massive (text-only) dialogue datasets, to produce responses conditioned on both visual and textual context. to ensure that our model retains the ability to engage in regular, text-based conversation, we include in our training procedure multi-tasking with datasets expressly designed to instill conversational skills in the model @xcite . we find that our best resulting models are as proficient in text-only conversation as the current best reported dialogue models, with respect to both performance on the relevant datasets and human evaluations of preference. concatenating image feature embeddings to the input of our model's encoder leads to better performance than concatenating the embeddings to the encoder's output, and using spatially based image embeddings performs better than single-vector embeddings. simultaneously, our model significantly outperforms recent strong multi-modal dialogue models when in an image-dialogue regime; we measure several metrics via pairwise human judgments using acute-eval @xcite to show that our model is not only more preferred by humans but can also discuss and reference visual context throughout a conversation. see figure 1 for one sample cherrypicked conversation with our model, with random and lemon-picked conversations in figures 2 and 3 . one important avenue we explore with our best models is safety -that is, ensuring that our models are not offensive to their conversational partners. dialogue safety is indeed a well-studied, but still unsolved, research area @xcite @xcite @xcite @xcite ), yet we note that safety in the context of image-dialogue is relatively less explored. in this work we examine gender bias and toxicity of text generations in the context of various styles from the image-chat dataset @xcite . notably, after tuning the model to reduce toxicity and gender bias, we find that human preference for this model does not diminish. the training procedure and initial pre-trained model weights will be made publicly available to allow for fully reproducible results. visual dialogue/caption generation many tasks have been designed to measure the ability of a model to produce text in the context of images. specifically, coco captions @xcite and flickr30k @xcite require a model to produce a caption for a given image. a variety of sequence-to-sequence @xcite @xcite and retrievalbased @xcite @xcite models have been applied to these tasks, however they do not go beyond the one-turn text generation expected for captioning an image. other recent architectures have explored text generation @xcite in the context of the visual dialog @xcite task; however, this task is primarily used to measure the ability to answer questions about an image in the flow of a natural conversation, which differs somewhat from the open-domain dialogue task. further still, there have been recent forays into open-domain natural dialogue in the context of images, e.g. in the image-chat @xcite and image-grounded conversations @xcite tasks. again, retrieval-based @xcite and sequence-tosequence @xcite ) models have been used to conduct dialogue in this regime. ## model architectures the inputs to our models are visual and/or textual context, where applicable. we explore different ways to encode images, and we additionally compare ways of combining (fusing) the image and text representations before outputting a response. ## training details when training the model, we fix the weights of the pre-trained image encoders, except the linear projection to the transformer output dimension, and fine-tune all of the weights of the transformer encoder/decoder. ## conclusion in this work, we explored a necessary component of open-domain dialogue models preferred by humans: the ability to perceive and converse in the context of what is seen. we showed that we can match prior work in text-only dialogue in both automated metrics and preference/humanness metrics, and our best model surpasses existing models in multi-modal dialogue. finally, we demonstrated that we do not sacrifice human preference for our model by incorporating safety components into it.