On the Pitfalls of Visual Learning in Referential Games
Abstract
This paper focuses on the effect of game design and visual representations of real-world entities on emergent languages in referential games.
Strikingly, we find that the agents in such games learn to successfully communicate even when provided with visual features from a randomly initialized neural network.
Through a series of experiments, we highlight the agents’ inability to effectively utilize high-level features.
Using Gradient weighted-Class Activation Mapping, we verify that the agents often ’look’ at regions not related to entities.
Culminating with a positive result, we show how environmental pressure from agent population can nudge the learners into effectively capturing high-level visual features
Introduction
The ability to communicate through natural language is often seen as a long standing goal in AI.
The current prevalent paradigm in natural language learning leverages statistical patterns using large amounts of data.
Alternatively, the field of language emergence [2, 9, 29, 13] exploits language’s functional nature.
Recently, there has been a revival of interest in language emergence games by studying them through the lens of neural agents [20, 14, 21].
Efforts are made to understand the linguistic properties of emergent languages, such as grammar [30], compositionality [1], priors and biases [6, 11] and capacity and bandwidth [24].
Another line of work stresses upon the shortcomings of emergent language through neural agents, such as anti-efficient encoding [5] and the need for specialized environmental pressures [18, 25].
However, an area that is often ignored is the effect of visual representations on shaping the emergent language properties.
[3] demonstrate an alarming behaviour in emergent languages where agents rely on low-level visual features for communication.
In our work, we highlight the role of visual representations and game design in aggravating this behaviour.
Through empirical results, we show that the agents are ineffective at learning from high-level visual representations of images.
This is surprising since it has been common practice in emergent language literature to represent real-world images as either final layer activations of pre-trained deep ConvNets such as VGG-16 [28] (e.g. in [14, 20, 19, 12]) or using representations learned in-game [8, 25, 22].
Further, we reformulate Gradient-weighted Class Activation Mapping (Grad-CAM)
[26] to visualize where the agents are looking.
This is the first application of Grad-CAM in interpreting referential games.
Lastly, we conduct experiments in line with strong results obtained from language emergence in population of agents [17, 16, 23, 7].
We show that a simple populationbased Referential Game induces a regularization effect, nudging the agents to communicate through high-level visual features.
Our novel contributions are thus as follows:
1) We evaluate agents’ reliance on low-level features for communication using two synthetic testbeds in Emergent Communication.
We reformulate Grad-CAM for use in Multi-Agent Reinforcement Learning setting and show that the agents ’look’ at regions not related to entities.
36th Conference Processing Systems (NeurIPS 2022).
We demonstrate that environmental pressure through population of agents regularize agents to use high-level visual features for communication.
Referential Game
Two agents, a sender S and a receiver R play the game.
The sender is shown a target image t and K distracting images D = { d k } i = K k
=1 .
Consequently, sender must come up with a message m of length l with tokens from vocabulary V .
The receiver is shown the target image and distracting images in a shuffled order, along with the message m .
A payoff of 1 is provided to both the agents iff the receiver identifies the target correctly.
We consider the following agent and game variants:
Distractor Aware Sender :
Given the target image t and distracting images D , sender must come up with message m .
Distractor Agnostic Sender : Given only target image t , sender must come up with message m .
Same Target Game : Given message m and a shuffled set of images containing t and D , the receiver must identify the target correctly.
Different Target Game : Instead of target image t , the receiver is shown a different target image t ′ , which is conceptually same as t
We use K = 1 , l = 5 , |V| = 10 in all experiments.
Previous works have shown that even a small game size can be a challenging learning environment.
Crucially, keeping these hyperparameters fixed makes sure that there are no confounding factors when studying the effects of game variations.
2.1 Data
We generate two synthetic datasets 1 , SHAPES and BULLET, to represent images corresponding to different visual attributes.
Both datasets consist of 30 X 30 images with objects characterized by one of 3 colours, 2 sizes and 3 shapes.
SHAPES has 2D objects placed on a background with randomized location.
BULLET has 3D objects placed on a checkered plane with randomized location as well as camera viewpoint.
Unlike SHAPES where background is constant, BULLET has a checkered plane where pixel values change with changing camera orientation.
Thus, BULLET showcases more variation in pixel-level features as compared to SHAPES, while having the same high-level attribute set.
We argue that studying emergent languages using proposed datasets has an advantage over
ImageNet [10], as attribute information can readily be controlled.
Further, raw pixel inputs provide a much more realistic task [19], than using one-hot vectors of attributes, as has been done in many prior works [18, 4].
2.2 Learning and Architecture
We re-implement agent architectures from [14].
The sender and receiver both have one ConvNet visual module f s and f r and one LSTM language module L s and L r respectively.
The visual module maps input images to fixed dimension embedding.
Sender’s language module L s generates sequence of message tokens m with embeddings from f s as the initial hidden state.
The receiver’s language module L r receives message m and a message embedding is obtained by performing affine transformation g on last hidden state in L r .
Finally, the receiver compares the message embedding and visual embeddings from each of the images to point to a target.
We use Straight-Through
Gumbel-softmax (GS) relaxation [15], while sampling messages from sender’s language module.
This allows the game to be end-to-end differentiable which is crucial for Grad-CAM (Section 4.2).
The energy function for each image v , E ( v, m ) =
f r ( v ) T g
( L r ( m )) , can be used to define probability distribution over images as p ( v | m ) ∝ e E ( v,m ) .
Since our learning setup is similar to the one proposed by [14], we optimize the same loss function.
L = E m ∼ Π θ K − 1 (cid:80)
[Equation]
To study the effect of visual representations, we also include the following two learning variants:
In-game learning (I): The weights for both f r and f s are updated during learning in game play.
Pre-trained classifier (P): A visual module is pre-trained on the task of classifying entities.
Weights are saved and representations of input images are computed before gameplay.
f s and f r are not used.
[Table]
Evaluation Metrics
We evaluate the game variants on four metrics: (1) Communication Success: Measured as average reward obtained at convergence.
(2) Generalization Success: Measured as communication success on novel combinations of unseen concepts.
(3) Topographical Similarity: Measured as correlation between all-pair distances in message space and in input space and can be seen as language compositionality measure [20].
(4) Noise Accuracy: Measured as test time communication success over images containing only gaussian noise.
It quantifies agents’ reliance on pixel-level visual features [3].
Experiments 4.1 Effects from game variations
All combinations of proposed game variants are enumerated and run until convergence for 10 different seed values.
Evaluation metrics are reported in Table 1.
To develop a baseline for comparison, we also report results from games with a random visual module (R) .
In this setting, f s and f r are fully connected single layer networks with weights randomly initialized and frozen during training.
We obtain the following statistically significant observations from our experiments:
• High levels of communication success are reached in all the games ( > 90% ).
Surprisingly, experiments with Random Visual Module also achieve high communication success.
• Noise accuracy is lower in Distractor-Agnostic setting as compared to Distractor-Aware .
A plausible reason could be the absence of context for sender without distractors.
Since relative comparison is not possible, encoding high-level visual information would be incentivized.
Higher topographical similarity in Distractor-Agnostic also supports our argument.
• While Distractor-Aware setting has a high noise accuracy, using Different-Target lowers it as compared to Same-Target game.
This matches with the results obtained in [3].
• All Distractor-Aware Pre-Train experiments have performance comparable to baseline (random) experiment.
This is an alarming result as it suggests that agents gain no significant benefit in communication when provided with high-level visual information.
Statistical significance is calculated using t-test for comparison of means when comparing two groups.
All the above observations are significance with p-value < 0.05.
4.2 Where are the agents looking?
To qualitatively access whether the agents are looking at high-level visual attributes or at pixel-level differences, we reformulate Gradient-weighted Class Activation Mapping [26].
As the game is end-to-end differentiable, we can attribute the outcome of the game to specific regions in the inputs by generating a class discriminative localization map.
Let m be the message generated by sender and u ∈ t ∪D be the image chosen by the receiver.
The energy function E ( u, m ) represents the likelihood of u being the target image.
For any image v ∈ t ∪ D , for which we want to generate localization map H , we define A us and A ur as activations from final convolutional layers in sender’s and receiver’s visual modules.
We calculate the localization map for sender as α us = ∂E
( v,m ) ∂A
us , H us = ReLU ( α us
A us )
We repeat this process for receiver and for all the images in the sender’s and receiver’s input.
Figureshows that grad-CAM can accurately highlight regions of entities in targets and distractors.
However, in many cases, the agents "look" at background regions.
This is more prominent in BULLET dataset.
Interestingly, in some cases, agents look at negative space of the object.
This suggests that agents may communicate through negation of attributes (e.g. ’the object not in the bottom right corner’) as has been previously expressed in [8, 27].
4.3 Population based Referential Game
We extend the referential game setup to a population of agents.
Consider N agents, consisting of an equal number of senders and receivers.
At every game turn, a referential game is played between a random pair of sender and a receiver.
We argue that population as an environmental pressure should reduce the agents’ reliance on low-level features.
This is because high communication success from low-level features is a sign of conceptual pact between two agents.
However, when the game partners are continuously shuffled, low-level pacts would no longer always match.
Figure 2 shows that noise accuracy at convergence indeed reduces with larger population sizes.
Conclusion
Our work ascertains that agents in referential games often rely on low-level features for communication; however, some game settings are more susceptible to do so than others.
Thus, noise accuracy should always be reported in emergent language experiments.
Moreover, a high generalization accuracy can be misleading as it may result from pixel-level comparison rather than a compositional language.
Additionally, as we observe a prevalence of negation of attributes in emergent communication, there is a need to incorporate it in compositionality measures.
The multi-input, multi-output Grad-CAM setup through GS channel, presented in our work, can be adapted as an interpretability tool for numerous Multi-Agent RL scenarios such as visual dialog agents, embodied agents or in cooperative/competitive communication games.
Lastly, a promising direction for future work is the effect of population interaction on conceptual pacts among agents and how it evolves over time.