An Empirical Study on the Generalization Power of Neural Representations Learned via Visual Guessing Games (2021)

Abstract: Guessing games are a prototypical instance of the “learning by interacting” paradigm. This work investigates how well an artificial agent can benefit from playing guessing games when later asked to perform on novel NLP downstream tasks such as Visual Question Answering (VQA). We propose two ways to exploit playing guessing games: 1) a supervised learning scenario in which the agent learns to mimic successful guessing games and 2) a novel way for an agent to play by itself, called Self-play via Iterated Experience Learning (SPIEL). We evaluate the ability of both procedures to generalise: an in-domain evaluation shows an increased accuracy (+7.79) compared with competitors on the evaluation suite CompGuessWhat?!; a transfer evaluation shows improved performance for VQA on the TDIUC dataset in terms of harmonic average accuracy (+5.31) thanks to more fine-grained object representations learned via SPIEL.

## background & related work learning a language requires interacting with both the environment and other agents @xcite . language games represent one common example of this @xcite , as seen by the important role of play in l1 child language acquisition @xcite as well as l2 learners @xcite . among the language games defined in the literature @xcite , guessing games represent the first step in a curriculum for language learning. for example, in guesswhat?! @xcite , two agents interact with each other: a questioner generates questions aimed at finding a hidden object in the scene and an oracle, aware of the target object, answers the questions supporting the questioner in playing the game. different from other language games @xcite , guessing games have a specific goal which represents a clear incentive for learning. in addition, they require that the questioner masters both natural language generation and understanding with a focus on object categories and attributes. for humans, concepts learned in this way are generic and generalisable to new tasks and domains where grounded reasoning is important @xcite . however, how well can ai agents generalise with concepts acquired from visual guessing games? the literature has not explored if representations built from self-play are transferable, focusing instead on large scale self-supervised learning. for instance, large scale image captioning datasets have been used to train multi-modal transformers @xcite @xcite . multi-task learning @xcite has been used to leverage the diversity of training signals provided combining datasets, but only for discriminative tasks. while some dialogue work @xcite aims to bootstrap a conversing agent from vqa datasets, most work on guesswhat?! @xcite @xcite has designed bespoke models for the task, ignoring the utility of this dataset for other vision+language tasks. we propose self-play as a mechanism for learning general grounded representations. we seed our approach with the guesswhat?! corpus of questions and objects, and demonstrate how to generalise to other downstream tasks. we propose two different strategies to exploit these data. first, a supervised learning phase is undertaken to learn a questioner and oracle model able to play guessing games. second, the trained agents can be used to play guessing games on images requiring only object annotations as supervision. we show that an agent trained on guesswhat?! dialogues can use self-play to adapt to new and harder tasks. specifically, we investigate models' gener-alisation performance and quality of the learned representations on the compguesswhat?! benchmark @xcite , a more extensive evaluation suite for guesswhat?!. furthermore, we study how the learned representation help solve vqa on the dataset tdiuc @xcite . we show overall comparable performance with stateof-the-art models and improvements for specific question types that require object attribute information to be answered correctly. ## methodology our proposed transfer/fine-tuning procedure requires a training set of guessing games d g from which we learn a questioner q and an oracle o via supervised learning. given a set of images i, it is possible to use the trained models q and o to run the self-play procedure for n epochs obtaining the model q n . finally, given a downstream task t and an associated dataset d t based on images from i, we use q n 's parameters as initialisation for the training procedure on d t . to apply this procedure, both the questioner and the oracle require a multi-modal encoder γ able to generate d-dimensional representations for the textual tokens h t , for the objects h o , as well as fusing the visual and textual modalities in a representation of the current context h c . after the self-play procedure, only the encoder γ of the model q n is used in the fine-tuning process on the downstream task t using the dataset d t . it is important to underline that the presented self-play procedure does not depend on a specific implementation of the multi-modal encoder γ. a possible implementation is presented in section 2.4 and it is used in the experimental evaluation of this paper. ## experimental evaluation to assess the generality of our learned representations, we include two evaluation paradigms: 1) indomain evaluation and 2) transfer evaluation. we evaluate several variants of our model: 1) vlp-sl: vlp-based model trained on guesswhat?! data using multi-task learning; 2) spiel-gs: vlp-sl model fine-tuned with our spiel procedure where the generator head uses only gold successful games (gs); 3) spiel-gm: same as 2) but both successful and failed gold games are used by the generator head. in both spiel variants, the guesser head is trained using failed and successful generated games because it is important for the guesser head to be exposed to both types of signal to learn a more robust policy. we decided to investigate the two variants spiel-gs and spiel-gm to get more insights about the effect that successful and failed games have on the generator head ability to produce effective dialogues. ## conclusions in this work, we verified that representations learned while playing guessing games can be transferred to other downstream tasks such as vqa. we presented two ways of learning from guessing games data namely multi-task learning and spiel. models using spiel performed better both on indomain evaluation on compguesswhat?! as well as on the transfer task tdiuc. our self-play pro- the current study showed how we can apply the spiel training procedure to a vqa dataset such as tdiuc. we believe that this work can be extended to other datasets because the spiel procedure only requires a set of images and associated object bounding boxes. these could be either gold or generated by a trained object detector therefore classifying guessing games as a holistic self-training procedure for multi-modal datasets.