Emergent Communication with Attention
Authors: Ryokan Ri
Ryo Ueda
Jason Naradowsky
Venue: Submitted to ICLR 2023 (2023)
Keywords: emergent communication
attention mechanism
compositionality
interpretability

Abstract: To develop computational agents that can better communicate with others using their own emergent language, we endow the agents with an ability to focus their attention on particular concepts in the environment. Humans often understand a thing or scene as a composite of concepts and those concepts are further mapped onto words. We implement this intuition as attention mechanisms in Speaker and Listener agents in a referential game and show attention leads to more compositional and interpretable emergent language. We also demonstrate how attention helps us understand the learned communication protocol by investigating the attention weights associated with each message symbol and the alignment of attention weights between Speaker and Listener agents. Overall, our results suggest that attention is a promising mechanism for developing more human-like emergent language.

Under review as a conference paper at ICLR 2023
EMERGENT COMMUNICATION WITH ATTENTION
Anonymous authors
Paper under double-blind review
ABSTRACT
To develop computational agents that better communicates using their own emer- gent language, endow the agents with an ability focus their attention on particular concepts in the environment. Humans often understand a thing or scene a composite of concepts and those concepts are further mapped onto words. We implement this intuition as attention mechanisms in Speaker and Listener agents in referential game and show attention leads to more compositional and inter- pretable emergent language. We also demonstrate how attention helps us under- stand the learned communication protocol by investigating the attention weights associated with each message symbol and the alignment of attention weights be- tween Speaker and Listener agents. Overall, our results suggest that attention a promising mechanism for developing more human-like emergent language.
INTRODUCTION
We endow computational agents with an ability to focus their attention on a particular concept in the environment and communicate using emergent language. Emergent language refers to the commu- nication protocol based on discrete symbols developed by agents to solve a specific task (Nowak & Krakauer 1999 Lazaridou al., 2017). One important goal in the study of emergent language is clarifying conditions that lead to more compositional languages. Seeking compositionality provides insights into the origin of the compositional natures of human language and helps develop efficient communication protocols in multi-agent systems that are interpretable by humans.
Much recent work studies emergent language with generic deep agents (Lazaridou Baroni 2020) with minimum assumptions on the inductive bias of the model architecture. For example, a typical speaker agent encodes information into single fixed-length vector initialize the hidden state of a RNN decoder and generate symbols (Lazaridou et al., 2017, Mordatch & Abbeel 2018 Ren cct al. 2020). Only a few studies have explored some architectural variations (Stowik et al. (020) to improve the compositionality of emergent language, and there remains much to be discussed about the effects of the inductive bias provided by different architectures.
We posit that towards more buman--like emergent language we need to explore other modeling choices that reflect the human cognitive process. this study. we focus on the attention mecha- nism. Attention is one of the most successful neural network architectures (Bahdanau et al., 2015) Xu et al., 2013] Vaswani et al., (017) that have an analogy in psychology (Lindsay) 2020). The conceptual core of attention is an adaptive control of limited resources, and we hypothesize that this creates pressure for learning more compositional emergent languages. Compositionality entails a whole consisting of subparts. Attention allows the agents to dynamically highlight different sub- parts of an object when unndttiingunderrtaadinngg each symbol, which potentially results in clear associations between the object attributes and symbols.
Another reason to explore the attention mechanism is its interpretability. Emergent language is opti- mized for task success and the learned communication protocol often results in counter-intuitive and opaque encoding (Bouchacourt & Baroni 2018). Several metrics have been proposed to measure specific characteristics of emergent language (Brighton & Kirby 2006 Lowe et al. 2019) but these metrics provide rather holistic view of emergent language and do not tell us a fine finenerained view of what each symbol is meant for or understood as. Attention weights, on the other hand, have been shown to provide insights into the basis of the network's 's prediction (Bahdanau et al. 2015 Xu et al.)
2013 Yang et al. (2016). Incorporating attention in the process of symbol production/compeeeeee comprrhencoooonnnncncccccooooo will allow to inspect the meaning of each symbol in the messages.

---

Under review as a conference paper at ICLR 2023
Speaker
Listener
m
m
mestage-object matching score
ogt
Attention
Attention
Decoder
Encoder
mlg
m1
m
m2
mi
Figure
Illustration of the attention agents in the referential game.
In this paper, test attention agents with the referential game (Lewis) 1969 Lazaridou et al., 2017). which involves two agents: Speaker and Listener. The goal of the game to convey the type of object that Speaker sees Listener. We conduct extensive experiments with two types of agent architectures, L.TTM and Transformer, and two types of environments: the one-hot game and the Fashion-MNIS, game. We compare the attention agents against their non-attention counterparts to show that adding attention mechanisms with disentangled inputs to either/both Speaker or/and Listener helps develop more compositional language. We also analyze the attention weights and explore how they can help understand the learned language. We visualize the learned symbol- concept mapping and demonstrate how the emergent language can deviate from human language and, with experiments with pixel images, how the emergent language be affected by the visual similarity of referents. We also investigate the potential of the alignment between Speaker's and Listener's attention weights a proxy for the establishment of common understanding and show that the alignment correlates with task success.
2 EXPERIMENTAL FRAMEWORK
2.1 REFERENTIAL GAME
We study emergent language in the referential game (Lewis) 1969 Lazaridou 2017). The game focuses on the most basic feature of language, referring to things. The version of the referential game we adopt in this paper is structured follows:
1. Speaker is presented with target object Otgt O and generates a message m that consists of sequence of discrete symbols.
2. Listener receives the message m and a candidate set C = {ol, 02, C including the target object Orgt and distractor objects sampled randomly without replacement from O.
3. Listener chooses one object from the candidate set and if it is the target object, the game is considered successful.
The objects can be represented as a set of attributes. The focus of this game whether agents can represent the objects in compositional message that is supposedly based on the attributes.
2.2 AGENT ARCHITECTURES
Our goal in this paper is to test the effect of the attention mechanism on emergent language. The attention mechanism in its general form takes a query vector and key-value vectors fy1.... YL). The key-value vectors are optionally transformed into key vectors and value vectors or used as is. Then, attention scores (s1,..., 81) are calculated the similarity between the query vector and the key vectors to the attention weights via the softmax function. Finally, the attention weights are used to produce weighted sum of the value vectors. A key feature attention that it allows the agents to selectively attend to a part of disentangled vector inputs. Our intuition is that modeling direct associations between the symbol representations as query and disentangled input representations as key-value will bias agents toward packing each symbol with the information of meaningful subpart of the inputs rather than with opaque and non-compositional information. To test these hypotheses, design non-attention and attention agents for both Speaker and I. Listener (Figure TT.
2

---

Under review as a conference paper at ICLR 2023
2.2.1 OBJECT ENCODER
The objects are presented to the agents in the form of set of real-valued vectors, e.g., attribute-wise one-hot vectors patch-wise pretrained CNN feature vectors, as {o+ ..... 0²)
All the Speaker and Listener agents have their individual object encoders. The input vectors are independently fmnear-rransfrmeed into the size of the agent's hidden size and then successively go through the gelu activation (Hendrycks & Gimpel| 2016). The baseline non-attention agents av- erage the transformed vectors into single vector for the subsequent computations, whereas the attention agents leave the transformed vectors intact and attend to the set of vectors (o.,...6"))
2.2.2 SPEAKER AGENTS
Speaker has an ruto-segressive message decoder that takes the encoded vector(s) of the target object as input and generates a multi-symbol message m = (m1,..., my). There is a number of possible choices for the decoder architecture with attention. To provide extensive empirical evidence on the effect of attention mechanism, experiment with two common decoder architectures: the LSTM decoder from Luong et al. (2015) and the Transformer decoder from Vaswani al. (2017).
At each time step I, the decoders embed previously generated symbol into a vector m,-1 and produce an output hidden vector through three steps: (1) recurrent computation; (2) attention; (3) sottpprcessing. As the decoders basically follow the original architecture, only briefly describe each step in the LSTM and Transformer decoder with emphasis on how attention is incorporated. The more detailed descriptions can be found in Appendix C. The recurrent computation updates the input vector with the contextual information of previous inputs. The LSTM decoder uses a LSTM cell (Hochreiter Schmidhuber 1997) and the Transformer decoder uses the self-attention mechanism Vaaswani et al. (017)).
Then, with the contextualized input vector as the query x, and the object vectors the key-value vectors {ō ...., 6' the decoders perform attention. The LSTM decoder uses the bilinear attention, where the attention score s; is computed as si x, W,,', where W, learnable matrix. The attention vector is calculated as the weighted sum of the original key-value vectors. The Transformer attention fist transforrass the input vector as q W xt, k' W v W,,', where W,, W R, W. are learnable matrices. Then the attention score is calculated using the scaled dot attention: st (q. /kk//V, where d is the dimension of the query and key vectors. Finally, the attention vector is calculated as the weighted sum of the value vectors v*. The original Transformer also has the multi-head attention mechanism, but in the main experiments, we set the number of the attention heads to one for interpretability and ease of analysis. The effect of multi-head attention is investigated in Appendix E
Finally, in the post-proee poot-processing step, the original query vector and attended vector X, are integrated to produce the hidden vector to predict the next symbol.
Non-attention (NoAT) Speaker is a baseline agent that encodes the target object into a single vector uigt- The source-target attention in the decoder always attends to that single vector and does not change where to focus during message generation.
Attention (AT) Speaker, in contrast, encodes the target object into set of vectors (ōl ... og and the source-target attention dynamically changes its focus at each time step.
Our agent design is motivated by a fair comparison of con-attention and attention agents. They have the same architecture and number of parameters and only differ in whether they can dynamically change their focus when generating each symbol.
2.2.3 LISTNER AGENTS
Listener tries predict the target object from of candidate objects C = (01, 02, given the speaker message by computing message-object matching scores {8i, ..., ttcs} and choosing the object with the maximum score. Listener first encodes the objects using the object encoder and also encodes each symbol in the message into vectors {m" ...., 7 using message encoder, for which the LSTM based agent uses the bidirectional LSTM and the Transformer based agent uses the Transformer encoder.
3

---

Under review as a conference paper at ICLR 2023
Non-attention (NoAT) Listener encodes each candidate object into single vector ó,. The agent also averages the encoded symbol vectors into single vector m + m'. The message- object matching score is computed by taking the dot product of the object and message vector 8 ó; m. In our experiments, the LSTM model uses the bidirectional LSTM encoder and the Transformer model uses the Transformer encoder.
and use the encoded symbol vectors as it is. With each encoded symbol vector m' as query, the model produces an attention vector rn with the object attribute vectors as key-value using the dot-produce attention. Intuitively, the attention vector r is supposed to represent the attributes of the object o, relevant to the symbol m'. Then, the symbol-object matching score is computed by taking the dot product between the attention vector and the message vector: of m m Finally, the symbol-object matching scores are averaged to produce the message-object matching score: st = E. st.
Attention (AT) Listener encodes each object into set of attribute vectors
2.3 OPTIMIZATION
The parameters of Speaker As and Listener Oz are both optimized toward the task success.
Speaker is trained with the REINFORCE algorithm (Williams) 1992). The message decoder pro- duces the probability distribution of which symbol to generate n (-1t) at each time step t. At train- ing time, message symbols are randomly sampled according to the predicted probabilities and the loss function for the Speaker message policy is L (0s) E, r log((Te (mal()) where me denotes the t-th symbol in the message. The reward r is set to 1 for the task success and otherwise.
As an auxiliary loss function, - -E. H (TT (-16), where H is the entropy of a probability distribution, to encourage exploration. We also add KL loss (xs = E, DKL (TT (-It)[[=T, (-It), where the policy Tāg is obtained by taking an exponential moving average of the weights of 0g over training, to stabilize the train- ing (Chaabouni al., 2022). In summary, the final speaker loss is L(0s) Lx (0s) (nn((s)) + (.ss, where a and 3 are hyperparameters.
employ
entropy regularization loss (H
Listener is trained with a multi-class classification loss. The message-object matching scores converted through the softmax operation to pe, (o,[C), the probability of choosing the object o as the target from the candidate set C. Then Listener is trained to maximize the probability of predicting the target object by minimizing the loss function [(0r,) = log (Orge (C). The details of hyperparameter settings and training procedures can be found Appendix A.3
2.4 EVALUATION METRICS
We quantitatively evaluate emergent languages from how well the language can be used to solve the task and how well the language exhibits compositionality.
Training accuracy (TrainAec) measures the task performance with objects seen during training. This indicates how the agent architectures are simply effective to solve the referential game.
Generalization accuracy (GenAcc) measures the task performance with objects unseen during training. We split the distinct object types in the game into train and evaluation sets and the gen- eralization accuracy computed with the evaluation set. As each object can be represented as a combination of attribute values, what we expect for the agents is to learn to combine symbols denot- ing each attribute value in a systematic way so that the language can express unseen combinations of known attribute values.
Topographic similarity (TopSim), also known as Representational Similarity Analysis (Kriegesko- rte et al.) 2008), is one of the most commonly used metrics to assess the compositionality of emergent language (Brighton& & Kirby] 2006 Lazaridou et al. 2018: Ren et al.) 2020 Chaabouni et al. 2020). Intuitively, TopSim checks if similar objects have similar messages assigned. To compute TopSim, we enumerate all the object-message pairs (o' m )), ..... (ol01, m l with trained Speaker and define a distance function for objects (o o²) and messages (mmm', m²). Then we compute Spearman's 's correlation between pairwise distances in the object and message space. For the dis- tance function for objects do (o', o²), we use the cosine distance of the binary attribute value vectors and for the distance function of messages d m²) the edit distance of message symbols.
4

---

Under review as a conference paper at ICLR 2023
3
EXPERIMENTAL SETUP
3.1 ONE-HOT GAME
Our first experimental setting is the referential game with objects consisting of one-hot tors (Lazaridou et al.) 2017] Kottur et al. 2017 Chaabouni et al. 2020; Guo et al. (2022) The objects are represented as combination two attributes (e.g., shape and color), which a popular setting in the literature (Li & Bowling 2019 Ren et al. 2020). Each attribute has possible values, which results 64 distinct objects. To evaluate generalization
unseen objects, we randomly split the 64 objects into training and evaluation sets at ratio of 48/16. The number of candidates is set to 16, which the maximum number of objects available the evaluation set.
Input Representation.
The objects are disentangled in terms of attributes and represented as set of one-hot vectors where each dimension indicates a different value a different attribute (i.e., two 16-dim one-hot vectors).
Agent Configurations. The vocabulary size of the agents is set to and the message length of Speaker is 2. In this setting, perfectly compositional language that scores the maximum score for all the compositionality metries would assign different symbols to each attribute value, and symbols at each position consistently refer to single attribute.
3.2 SASHION-MIISS GAME
Attention has been shown to be able to associate a symbol and a relevant region of an image to solve the task (Xu et al. 2015 Yang al. 20T6). To develop human-like emergent languages, one important question is whether the attention agents are able to develop language such that we can understand the meaning of each symbol by inspecting the attended region. For this purpose, we design multi-item image referential game using the Fashion-MNIST datase (Xiao et al. 2017).
In this game, each object defined as combination of two classes from the Fashion- dataset (e.g., T-shirt and Sneaker). The dataset contains 10 classes and thus the game has 10 C2 = 45 different types of objects. The 45 types are randomly divided into training and evaluation sets at a ratio of 30/15, and the number of candidates is set to 15.
Input Representation. Each object is presented the agents as feature vectors extracted from a pixel image. The image is created by placing two item images, each of which is rescaled to the size of 48 x 48, on a 224 x 224 black canvas (Figure The places of the items are randomly sampled so that the items never overlap.
The specific item images and their positions are randomly sampled every time the agents process the objects both during training and evaluation time to avoid degenerated solutions that exploit spurious features of an image (Lazaridou et al. 2018 Bouchacourt & Baroni 2018). Also, the Speaker and Listener presented with different images the target object to facilitate learning robust communication protocol (Rodríguez Luna et al. (2020).
Each image is encoded into x 7 768-dim feature vectors with pretrained ConvNe Liu al. 2022). For non-attention models, the feature vectors are averaged across spatial axes into a single 768-dim feature vector.
Agent Configurations. The vocabulary size of the agents is set to 20 and the message length is 2. A perfectly compositional language would refer to each item in the image with different symbols with a consistent one-to-one mapping.
4 RESULTS
4.1 ATTENTION AGENTS FIND MORE COMPOSITIONAL SOLUTIONS
We evaluate non-attention agents and attention agents where either/both Speaker and Listener have dynamic attention. We perform hyperparameter tuning as described in Appendix A and plot the
https//githb..mm/aaaannccco com//alandorsaarcdiistiiiooooo The pretrained model is registered convrext tiny in the torchvi sion library
5

---

Under review as a conference paper at ICLR 2023
LSTM
Transfomer
LSTM
Transformer
LSTM
Transformer
1.0
0.8
0.98
0.9
0.96
0.6
0.8
0.94
NoAT NoAT
AT NoAT
NoAT
AT
NoAT NoAT
NoAT AT
AT
NoAT NoAT
NoAT AT
AT
NoAT
AT
NoAT
AT
(a) One-hot:
TrainAce
(b) One-hot:
GenAce
(c) One-hot:
TopSim
LSTM
Transformer
LSTM
Transformer
LSTM
Transforerr
0.9
0.5
0.4
0.7
NoAT NoAT
NoAT AT
AT AT
NoAT NoAT
NoAT AT
AT
NoAT NoAT
NoAT
NoAT
NoAT
AT
NoAT
AT
AT
(d) Fashion:
TrainAce
(c) Fashion:
GenAce
(f) Fashion:
TopSim
Figure 2: The results of the referential game. The color of the boxes indicates the base architecture of the agents (LSTM or Transformer) and the x-axis labels indicates whether Speaker and Listener use attention.
results of top-10 agents in terms of GenAce for each setting on Figure 2 difference between the LSTM and Transformer agents as they differ in multiple dimension in their architectures. Here, we focus on the difference between the non-attention and attention agents within each architecture.
We do not discuss the
We observe a general trend that the attention agents perform better than the non-attention baseline (NoAT-NoAT)), which is indicated by the better average scores in task generalization (GenAcc) and compositionality metrics (TopSim) with different degrees of significance. This observation provides evidence in support of the hypothesis that the attention mechanism creates pressure for learning more compositional emergent languages.
One possible interpretation is that the attention mechanism adds flexibility the model and it sim- ply leads to better learning of the task. We observe the effect in the TrainAcc of the Fashion-MNIST game, where the attention models consistently outperform their non-attention basclines. However, we can still see the contribution of the attention mechanism besides the flexibility. We can see some NoAT-NoAT agents and AT-AT agents exhibit comparable TrainAcc around 75% in the Fashion- MNIST game, which means they are successful at optimization to similar degree. However, observe all AT-AT agents significantly outperform any of the NoAT-NoAT agents, which indi- cates the degree of optimization alone cannot explain the better GenAcc and TopSim scores of the attention model. Therefore, the results supports our initial hypothesis that the attention mechanism facilitates developing compositional languages.
In some settings, we observe that adding the attention mechanism to only one of Speaker and Lis- tener does not lead better TopSim scores compared the NoAT-NoAT agents to in the AT-NoAT LSTM agents in the one-hot game, and the AT-NoAT and AoAT-AT LSTM agents in the Fashion- MNIST game. However, when both Speaker and Listener agents have the attention mechanism, they all exhibit more generalizable and compositional languages, which indicates the effect of attention in Speaker and Listener is multiplicative.
4.2 ATTENTION AGENTS LEARN TO ASSOCIATE INPUT ATTRIBUTES AND SYMBOLS
Having confirmed that the attention agents give rise to compositional languages, proceed to examine if they use attention in an expected way, i.e., producing/undersaandde each symbol by associating them with single input concept. We focus analyzing the Transformer agents below.

---

Under review as a conference paper at ICLR 2023
0.0
0.0
1.0
Attention weights
Attention weights
Attention weights
Attention weights
AT-NoAT:
(b) NoAT-AT:1.
(c) AT-AT:
(d)
Figure 3: The distribution of attention weights from Transformer agents in the one-hot game.
(a) Speaker
(b) Listener
(c) Speaker
(d) Listener
(c) Speaker
(f) Listener
Figure 4: Images with the attention maps produced by AT-AT agent pair.
Inspecting the distributions of the attention weights assigned to each input attribute vector in the one-hot game, we observed that most attention weights are around either or 1, indicating that the attention agents generally learn focus attention on single value (Figure 3}. We also inspect the attention weights in the Fashion-MNIST game and confirm that attention agents generally learn to focus on single object when generating a symbol in Figure ad), although there are some failure cases as shown in (i).re(fee)-()).
Given the observations above, we can associate each symbol in each message with the concepts defined in the game via attention weights. We visualize the association between each symbol in the vocabulary and each concept from pair of AT-AT agents in Figure 5 inspect the mapping patterns developed by the agents. A symbol is considered to be associated with concept when the attribute value has the largest attention weight in one-hot game or the center of gravity of attention weights is within the bounding box of the item in the Fashion-MNIST game. Overall, observe that the mappings learned by Speaker and Listener are mostly aligned except for the symbol in the one-hot hot game and the symbol in the Fashion-MNIS game. We identify three types of symbol-to- to-bccpt mapping patterns.
Monosemy. A single symbol always refers single concept, e.g., and d in the one-hot game; a, e, and in the Fashion-MNIST game. This is a desired mapping pattern that allows an unambiguous interpretation of symbols. However, we also observe "synonyms" where a single concept is referred to by multiple symbols, e.g., 1-3 by d and in the one-hot game.
Polysemy. A single symbol refers multiple concepts, e.g., j-n in the one-hot game; c, and d in the Mushion-MNIST" game. These symbols are somewhat ambiguous, but it is likely that the agents develop a way resolve the ambiguity, especially in the one-hot game where agents achieve more than 90% average accuracy scores with unseen objects. Moreover, these polysemous symbols always refer to two values of the different attributes. Given these observations, the agents are likely to have learned a consistent word order in terms of the object attributes and disambiguated the polysemous symbols by their position.
The polysemous symbols in the Dishion-MNIST game seem to be affected by the visual similarity of the fashion items, e.g., b refers tops (Pullover, Coat, and Shirt) and refers to shoes (Sandal, Sneaker, and Boot). This is in contrast the one-hot game, where the object attribute represen- tations are initialized randomly, which probably led to an arbitrary assignment of polysemy. The polysemous patterns in the Pashion-MNIS. game demonstrate that the semantics of emergent lan- guage can be heavily influenced by the property of the input objects.
Gibberish. We observe few cases where the attention weights do not consistently focus any particular regions in the Fashion-MNIST game, e.g., the symbol i. These gibberish symbols could have conveyed something informative but uninterpretable to humans, but confirmed that the

---

Under review as a conference paper at ICLR 2023
0-0
0-0
T-shirt
0-1
0-1
Fouser
Trouser
0-3
0-3
Pullover
Pullover
0-5
0-5
0.6
0-6
Coat
Coat
1:0
1:0
Sandal
Sandal
14
Shirt
Sneaker
Sneaker
Bag
Bag
Boot
Boat
NULL
NULL
abcdefghijj
abcdefghil Symbol
Symbol
Symbol
Symbol
(a) Onc-hot: Speaker
(b) One-hot: Listener
(c) Fashion-MNIST: Speaker
(d) Fahion-MNIST: Listener
Figure 5: The mappings between symbols and concepts derived from the attention weights of Trans- former AT-AT agent pairs. The darkness correslates with the frequency of the association.
Success
Fallure
Success
Fallure
Attention Discrepancy
Discrepancy
(a) Onehot
(b) Fashion-MNN MNIST
Figure 6: The distribution of attention discrepancy scores from the trials of the AT-AT agents. The frequency is normalized within successful and failed trials respectively.
communication success rate with the gibberish symbols is much lower (19.4%) than the overall success rate (45.3%), indicating that the gibberish symbols are the results of optimization failure.
4.3 THE COORDINATION OF SPEAKER AND LISTENER ATTENTIONS PREDICTS THE TASK PERFORMANCE
An important prerequisite of successful communication is the participants engaging in joint attention and establishing a shared understanding of each word (Garrod & Pickering 2004). Here we show that the degree of the alignment between Speaker's and Listener's 's attention weights can be regarded as proxy of their mutual understanding and predictive of communication success.
We define metric called attention discrepancy, which measures the difference between the attention weights of Speaker and Listener given the same inputs. The metric calculated by computing the Divergence (Lin 1991) between the Speaker's is and Listener's 's attention maps.
Figure shows the frequency of attention discrepancy scores within successful and failed commu- nications We can see that successful communications can be characterized by lower attention dis- crepancy compared failed communications, indicating that the alignment the attention weights does correlate with communication success. This suggests that the attention agents largely develop intuitive communication protocols that require a shared understanding of symbols.
5
RELATED WORK
5.1
EMERGENT LANGUAGE
One of the desired properties of emergent language is compositionality because more compositional languages supposed to be intuitive, interpretable, generalizable, and easier to learn (Li & Bowl-
We observe small peaks around 0.5 in the one-hot game for both successful and failed communications, and a closer look, this corresponds to cases where the Speaker and Listener attend to different attribute the symbols the message.

---

Under review as a conference paper at ICLR 2023
Compositional languages in this context usually have symbols each of which corresponds to one of the primitive concepts in the environment and combine them in a straightforward manner form a message. Existing studies have investigated various environmental pressures to improve the compositionality of emergent language: learning across generations (Li & Bowling 2019 Ren et al. 2020), learning with population (Rita al. 2022). applying noise to communication channel (Kuciński et al. (021).
ing 2019 Ren al. 2020).
These recent studies have been mainly conducted with simple RNN-based Speaker and Listener agents, where the inputs (e.g., attribute values or raw images) are encoded into a single input vec- tor for subsequence computations. Other agent architectures have been employed for other specific motivations. [Chaabouni et al. (2019) and Ryo et al. (2022) used the LSTM sequence-to- -o-sueeee-- (with attention) architecture to study how agents transduce a sequential input generated from a cer- tain grammar to their own emergent language. Gupta et al. (2021) developed patch-based Speaker architecture that produces messages by focusing important image patches.
One of our goals in this study is to investigate how the inductive biases of the Speaker and Listener architectures affect the compositionality of the emergent language. (220)) compared Speaker agents which process inputs in the form of graph, sequence, and bag-of- words and show that the graph architecture exhibits more compositional emergent languages. Our study offers additional empirical evidence in this direction with the attention mechanism.
similar spirit, (Slowik et al.
5.2 ATTENTION MECHANISM
The attention mechanism has been shown very effective in supervised learning (Xu et al., 2015 Vaswani al. 2017) and has become indispensable modeling piece in modern neural networks. Conceptually, the attention mechanism models pairwise associations between a query vector and a subset of key-value vectors. This enables the model focus on subpart of compositional represen- tation and has been shown to enforce composition solutions in various settings such as visual reason- ing (Hudson & Manning 2018), symbolic reasoning Korrel al. (2019), image generation (Hudson & Zitnick 2021). Our study demonstrates that by modeling association between symbol and an object attribute with attention, the agents can find more compositional languages in the referential game without any additional supervision.
Another potential benefit of attention interpretabilityy Attention has been shown to provide plausi- ble alignment patterns between inputs and outputs, for example, source and target words in machine translation (Bahdanau et al., 2015) and image regions and words in image captioning (Xu et al.) 2015 Yang et al. 2016). On the other hand, the alignment patterns may need careful interpretation. The patterns are not always consistent with human intuitions (Alkhouli et al.) 2018 Liu et al. 2020) and there is ongoing debate about to what extent the attention weights be used explanations for model prediction (Jain & Wallace 2019 Wiegreffe & Pinter 2019; Bibal et al. 2022). In this
paper, agents show relatively straighttorward attention patterns that allow unambiguous inter- pretations but the interpretability of attention weights with more complex models and environments needs further investigation in the future.
DISCUSSION AND CONCLUSION
When we face complex object, we likely dynamically change our focus its subpart de- scribe the whole (Rensink 2000). Hearing word helps us quickly recognize its referent (Boutonnet Lupyam 2015) and affects where focus in an image (Estes et al.) 2008). Motivated by these psychological observations, we implemented agents with the dynamic interaction between symbols and input representations in the form of the attention mechanism. We showed that the attention agents develop more compositional languages than their non-attention counterparts. This implies that the human ability to focus might have contributed to developing compositional language.
Overall, our results suggest that future work should explore more architectural variations that orig- inate from human cognitive processing to gain insights into the effect of the cognitive property on shaping a language. Given the observation that the alignment of Speaker's 's and Listener's attention somewhat indicative of successful communication in $4.3) for example, future work can explore how incorporating joint attention (Kwisthout et al. 2008 into training helps learn language.

---

Under review as a conference paper at ICLR 2023
REPRODUCIBILITY STATEMENT
The experiments described in this paper are ensured to be reproducible in the following ways:
We describe the task setup and models in $2 together with more specific descriptions of each task in $ The additional details and hyperparameters to reproduce the results are given in appendix A
The artifacts used in the experiments (the Fashion-MNIST" dataset and pretrained ConvNet encoder in $3. are all publicly available.
The source code will be released upon acceptance.
REFERENCES
Tamer Alkhouli, Gabriel Bretschner, and Hermann Ney. On the alignment problem in multi-head Aitention-based neural machine translation. In Proceedings of the Third Conference on Machine Translation: Research Papers, 2018.
Jimmy Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization. ArXiv, abs/ htdss/16770045500 2016.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations, 2015.
Adrien Bibal, Rémi Cardon, David Alfter, Rodrigo Wilkens, Xiaoou Wang, Thomas François, and Patrick Watrin. Is attention explanation? introduction the debate. In Proceedings of the 60th Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022.
Diane Bouchacourt and Marco Baroni. How agents things: On visual representations in emer- gent language game. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018
Bastien Boutonnet and Gary Lupyan. Words jump-start vision: label advantage in object recogni- tion. Journal of vision, 15 12:11, 2015.
Henry Brighton and Simon Kirby. Understanding linguistic evolution by visualizing the emergence of topographic mappings. Artificial Life, 12:229-244 2422,,2422, 2006.
Rahma Chaabouni, Eugene Kharitonov, Alessandro Lazaric, Emmanuel Dupoux, and Marco Baroni. Word-order biases in deep-agent emergent communication. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019.
Rahma Chaabouni, Eugene Kharitonov, Diane Bouchacourt, Emmanuel Dupoux, and Marco Baroni. Compositionality and generalization in emergent languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020.
Rahma Chaabouni, Florian Strub, Florent Altché, Eugene Tarassov, Corentin Tallec, Elnaz Davoodi, Kory Wallace Mathewson, Olivier Tieleman, Angeliki Lazaridou, and Bilal Piot. Emergent com- munication at scale. In International Conference on Learning Representations, 2022.
Zachary Estes, Michelle Verges, and Lawrence W. Barsalou. Head up, foot down. Psychological Science, 19:93 - 97, 2008.
Simon Garrod and Martin J. Pickering. Why is conversation so easy? Trends in Cognitive Sciences, 8(1):8 11, 2004. ISSN 1364-6613, doi: 101::0.1016/..icss. 2000....0.....
Shangmin Guo, Yi Ren, Kory Wallace Mathewson, Simon Kirby, Stefano V. Albrecht, and Kenny Smith. Expressivity of emergent languages is a trade-off between contextual complexity and unpredictability. In International Conference on Learning Representations, 2022.
Kamal Gupta, Gowthami Somepalli, Anubhav Gupta, Vinoj Jayasundara, Matthias Zwicker, and
Abhinay Shrivastava. Patchgame: Learning to signal mid-level patches in referential games. In NewrIPS, 2021.
10

---

Under review as a conference paper at ICLR 2023
Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). ArXiv, abs/ abs/1600.08415, 2016.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8): 1735-1780, 1997.
Drew A. Hudson and Christopher D. Manning. Compositional attention networks for machine rea- soning. In International Conference on Learning Representations, 2018.
Drew A. Hudson and C. Lawrence Zitnick. Compositional transformers for scene generation. In NeurIPS, 2021.
Sarthak Jain and Byron C. Wallace. Attention is not Explanation. In Proceedings the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers). 2019.
Kris Korrel, Dieuwke Hupkes, Verna Dankers, and Elia Bruni. Transcoding compositionally: Us- ing attention to find more generalizable solutions. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, 2019.
Satwik Kottur, José Moura, Stefan Lee, and Dhruv Batra. Natural language does not emerge 'nat- urally' in multi-agent dialog. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017.
Nikolaus Kriegeskorte, Marieke Mur, and Peter Bandettini. Representational similarity analysis connecting the branches of systems neuroscience. Frontiers in systems neuroscience, 2, 2008.
E ukasz Kucinski, Tomasz Korbak, PawelKot odziej, and Piotr Mit os. Catalytic role of noise and necessity of inductive biases the emergence of compositional communication. Advances in Neural Information Processing Systems, volume 34, 2021.
Johan Kwisthout, Paul Vogt, W.F.G Pim Haselager, and Ton Dijkstra. Joint attention and language evolution. Connection Science, 20: 155 171, 2008.
Angeliki Lazaridou and Marco Baroni. Emergent multi-agent communication in the deep learning era. ArXiv, abs/2006.024 02419, 2020.
Angeliki Lazaridou, Alexander Peysakhovich, and Marco Baroni. Multi-agent cooperation and the emergence of (natural) language. In International Conference on Learning Representations, 2017.
Angeliki Lazaridou, Karl Moritz Hermann, Karl Tuyls, and Stephen Clark. Emergence of linguistic communication from referential games with symbolic and pixel input. In International Confer- ence on Learning Representations, 2018.
David Kellogg Lewis. Convention: A Philosophical Study.
Cambridge, MA, USA: Wiley-
Blackwell, 1969.
Fushan Li and Michael Bowling. Ease-of-teaching and language structure from emergent commu-
nication. In Advances in Neural Information Processing Systems, volume 32, 2019.
Jianhua Lin. Divergence measures based on the shannon entropy. IEEE Transactions on Information Theory, 37:145-151, 1991.
Grace W. Lindsay. Attention psychology, neuroscience, and machine learning. Frontiers in
Computational Neuroscience, 14, 2020.
Fenglin Liu, Xuancheng Ren, Xian Wu, Shen Ge. Wei Fan, Yuexian Zou, and Xu Sun. Prophet attention: Predicting attention with future attention. NeurIPS, 2020.
Zhuang Liu, Hanzi Mao, Chao-Yure Wu, Christoph Feichtenhofer. Trevor Darrell, and Saining Xie. A convnet for the 2020s. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2022.
Ryan Lowe, Jakob N. Foerster, Y-Lan Boureau, Joelle Pineau, and Yann Dauphin, On the pitfalls of measuring emergent communication. In AAMAS, 2019.

---

Under review as a conference paper at ICLR 2023
Thang Luong, Hieu Pham, and Christopher D. Manning. Effective approaches to -based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015.
Igor Mordatch and Pieter Abbeel. Emergence of grounded compositional language in multi-agent populations, In The Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
Martin A. Nowak and D C Krakauer. The evolution of language. Proceedings of the National Academy of Sciences the United States of America, 96 14:8028 1999.
Yi Ren, Shangmin Guo, Matthieu Labeau, Shay B. Cohen, and Simon Kirby. Compositional lan- guages emerge in neural iterated learning model. In International Conference Learning Representations, 2020.
Ronald A. Rensink. The dynamic representation of scenes, Visual Cognition, 7:17 - 42, 2000.
Mathieu Rita, Florian Strub, lean-Bastien Grill, Olivier Pietquin, and Emmanuel Dupoux. On the role of population heterogeneity emergent communication. International Conference on Learning Representations, 2022.
Diana Rodríguez Luna, Edoardo Maria Ponti, Dieuwke Hupkes, and Elia Bruni. Internal and exter- nal pressures on language emergence: least effort, object constancy and frequency. In Findings of the Association for Computational Linguistics: EMNLP 2020, 2020.
Ueda Ryo, Ishi Taiga, Washio Koki, and Miyao Yusuke. Categorial grammar induction as a com- positionality measure for emergent languages in signaling games. In Proceedings of Emergent Communication Workshop at ICLR 2022, 2022.
Agnieszka Slowik, Abhinav Kumar Gupta, William L. Hamilton, Mateja Jamnik, S. Holden, and Christopher Joseph Pal. Structural inductive biases in emergent communication. In Proceedings of The 43rd Annual Meeting of the Cognitive Science Society, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Infor- mation Processing Systems, volume 30, 2017.
Sarah Wiegreffe and Yuval Pinter. Attention is not not explanation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019.
Ronald J. Williams. Simple statistical gradient algorithms for connectionist reinoorcmmet learning. Machine Learning. (0--442299-566 1992.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist. a novel image dataset for benchmark- ing machine learning algorithms. ArXiv, abs/ 2017.
Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard S. Zemel, and Yoshua Bengio. Show, attend and tell: Neural image caption generation with visual attention. In Proceedings of the International Conference on Machine Learning, volume 37, 2015.
Zichao Yang, Xiaodong He, Jianfeng Gao, Li Deng, and Alex Smola. Stacked attention networks for image question answering. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
12

---

Under review as a conference paper at ICLR 2023
APPENDIX
A ADDITIONAL DETAILS ON EXPERIMENTAL SETUP
A.1 MINI-BATCH CREATION
Each mini-batch in our referential game is specified via two parameters: the batch size and can- didate size |C|. We first sample b objects from the set of the objects available in the game O. Each object in mini-batch serves as the target object in round of the game, and we sample |C| distractors for each target object. Importantly, sample the objects so that every object round of the game has a different object type.
A.2 DATA PROCESSING IN THE FASHION-MNIST GAME
The images used in the Fashion- MNIST game (3.2) are created by sampling item images from the Fashion-MNIST dataset (Xiao et al. 2017). We first take 5,000 images for each item class in the dataset, and then create image objects for the referential game by sampling from the image pool. Instead of generating the image and input features on the fly during training, we precompute 3,000 images and their features for each object class (i.e., each combination of two items), and they are randomly sampled during training and evaluation.
A.3 HYPERPARAMETERS
The training setups and hyperparameters are shown Table The hyperparameter tuning is con- ducted for the weight of entropy loss a, and the scores in the paper computed from the agents with the best 10 generalization accuracy score for each agent.
One-hot (4, 4)
Fashion-MNIST
Speaker message embedding dim
64
256
Speaker LSTM decoder hideen dim
64
256
Speaker Transformer decoder dim
64
256
Speaker Transformer decoder feed-forward dim
128
512
Listener message embedding dim
64
256
Listener Bi-LSTTM encoder hidden dim
32
128
Listener Transformer encoder dim
64
256
Listener Transformer encoder feed-forward dim
128
512
Training Batch size
48
480
Max Training steps
15K
50K
Evaluation Rounds
1500
15000
Entropy loss weight o
[0.1, 0.01, 0.001]
[0.1, 0.01, 0.001]
KI. loss weight /
0.1
0.1
Learning rate
le-4
1e-4
Table Hyper-parameters of the experiments.
B ADDITIONAL RESULTS
For the configuration of the one-hot game in the main experiments, the number of attributes is 2 and the number of the possible values is 8. We denote this game setting as the one-hot game (8, 2). Here, show results from the one-hot games with different configurations with more attributes (4, 4) and more values (16. 2). The capacity of the communication channel is set to [V| = 16, T and [ 32, T = 2 respectively. Also, accommodate the increased task complexity,
double the model size (i.e., every hidden vector dimension) and set the training batch size to 192 and the max training steps to 30K.
The results are summarized in Figure 7 We generally observe the same trend as discussed in $4.1 AT-AT agents outperform their corresponding AT-AT baselines in GenAcc and TopSim. The only
13

---

Under review as a conference paper at ICLR 2023
LSTM
Transfomer
LSTM
Transformer
LSTM
Transformer
1.0
0.8
0.98
0.9
0.96
0.6
0.94
NoAT
AT NoAT
NoAT
AT
NoAT NoAT
NoAT AT
NoAT NoAT
NoAT AT
NoAT
AT
NoAT
AT
NoAT
AT
(a) One-hot (8, 2): TrainAcc
(b) One-hot (8, 2): GenAcc
(c) One-hot (8, 2): TopSim
LSTM
Transformer
Transformer
LSTM
Transformer
1.000
1.000
0.975
0.975
0.950
0.6
0.950
0.925
0.4
NoAT NoAT
NoAT AT
AT
NoAT NoAT
AT NoAT
NoAT AT
AT
NoAT NoAT
NoAT AT
NoAT
AT
AT
NoAT
AT
(d) One-hot (4, 4): TrainAce
(e) One-hot (4, 4): GenAce
(f) One-hot (4, 4): TopSim
LSTM
Transformer
LSTM
Transformer
LSTM
Transformer
1.00
0.99
0.98
0.95
0.6
0.97
0.90
0.96
0.4
NoAT NoAT
AT NoÂT
NoAT
AT AT
NoAT NoAT
AT
NoAT AT
AT
NoAT NoAT
NoAT AT
NoAT
AT
NoAT
AT
(g) One-hot (16, 2): TrainAcc
(h) One-hot (16, 2): GenAcc
(i) One-hot (16, 2): TopSim
Figure 7: The results of the one-hot game with different configurations. The color of the boxes indicates the base architecture of the agents (LSTM or Transformer) and the x-axis labels indicates whether Speaker and Listener attention.
exception is GenAcc of the AT-AT Transformer agents in the one-hot (4, 4) game. The AT-AT agents show significantly lower GenAce average score than NoAT-NoAT agents (Figure 7ch as well as the TrainAce score (Figure 7dp. We think that this is caused by the instability of jointly coordinating attention between Speaker and Listener during training. This can be particularly problematic with long message length. Yet, the average TopSim score is still higher than the NoAT-NoAV agents (Figure indicating the inductive bias towards more compositional languages is still effective in this case.
14

---

Under review as a conference paper at ICLR 2023
C
THE DETAILS OF THE SPEAKER ARCHITECTURES
In this section, we provide detailed descriptions of the Speaker agents used in the paper.
1+1
1+1
Symbol Projection
Symbol Projection
tanh
Sub-layer
Linear
Foed-Forward
Concat
Sub-layer
igt
0 ggr
Attention
Attention
Sub-layer
LSTM
Self-Atte Aiff-Attention
Embedding
Embedding
Layer
Layer
m
m,
m
m
(a) LSTM
(b) Transformer
Figure 8: The architecture the LSTM and Transformer decoders used in this paper.
C.I SPEAKER
Speaker has an auto-regressive message decoder that takes the encoded vector(s) of the target object (6',..,,'*) 04 as input and generates a multi-symbol message m (m1,... "ii,,.,,yy))) My).
At each time step t, the decoders embed previously generated symbol into vector m,-1 and produce an output hidden vector h, which fed into the output projection layer to predict the next symbol m,. The LSTM and Transformer decoders differ in how they transform the symbol vector m into the output hidden vector h,. We decompose the internal computation into three steps (1) recurrent computation; (2) attention; (3) post-processing, and how each step is implemented.
C.1.1 LSTM
The LSTM decoder based on the global attention decoder introduced in Luong al. (2015).
Recurrent computation. The LSTM decoder updates the input symbol vector m;-1 with the con- textual information using the LSTM cell (Hochreiter Schmidhuber, 1997). We initialize the mem- ory cell vector with the average of the object vectors 1 Σこ o' followed by a linear transformation and the hidden vector with zero. We also apply layer normalization (Ba et al. 2016) the hidden vector at each time step.
With the output of the LSTM cell xt, the attention vector is computed over the object vectors {ō), AA using the bilimear attention. The attention score for each object attribute of is computed sf x, Wo where W, is learnable matrix. Then the scores used to produce the attention weights via the softmax function of exp(st)/ Σ3 exp(s!). Finally, the object vectors used the value vectors produce the output attention vector x, Σ af0².
Attention.
15

---

Under review as a conference paper at ICLR 2023
Post-procet Post-processing. Finally, the attention vector X, is concatenated with and the vector is pro- jected into the dimension of the object vector followed by the tanh activation function: h, tanh( (W, [o( where W. is a learnable matrix.
C.1.2 TRANSFORMER
The Transformer decoder is based on the original architecture introduced in Vaswani et al. (2017) We first explain some building blocks repeatedly used in the computation.
Multi-head QKV Attention. The attention used in Transformer takes the query vector and key- value vectors (y1,..., YL) as input and is based on the QKV attention. It first project the vec- tors into query, key, value vector spaces respectively by linear transformation: q W Wg Wxy, v = W.y, where Wg. Wk. W, are learnable matrices. Then the attention score is cal- culated using the scaled dot attention: s (q, k') //V, where d is the dimension of the query and key vectors. The scores are transformed into the attention weights via the softmax function aj = exp(st)/ Ej exp(s)). Finally, the attention vector is calculated as the weighted sum of the value vectors: & Ei=1,3 a.v'
Transformer further combines the QKV attention with the multi-head mechanism. In essence, it divides the query, key and value vectors into sub-spaces with the same dimension, and perform the attention computation above within each sub- space, and then concatenate each output. We denote the attention computation
MultiHeadQKV(xx (y....,, yr.)) below.
Sub-layer connection. The Transformer architecture combats the gradient vamishing problem using sub-layer connection after each module. Let the input or hidden vector the current time step be and module to update the vector as Module, and the sub-layer connection can be described as: SubLaycr(x, Modulc(-)) = LN(Module(x) + x). where LN denotes layer normalization.
Recurrent computation. The Transformer decoder updates the input symbol vector m;-1 with the contextual information using the self-attention mechanism. The vector m attends to the previous imputs with the multi-head attention mechanism and produce the updated vector: x/ MultiHeadQKVmm {m), m,-1). Then the sub-layer connection is applied to produce x; LN(x) + m,-1).
Attention. The updated vector x₃ is further updated by attending to the object vectors (6' 64]: x MultiHeadKKV (6',..., (o.,,..,AA)) *1)) followed by the sub-layer connection: &; LN(*; + xt).
Post-processing. Finally, the attended vector & undergoes the feed-forward layer: h = W WeReLU(WW (&.,, where Wi and Wa learnable matrices. The dimension of hy should be the same as the input &, to apply the sub-layer connection: h LN(h' + xt).
16

---

Under review as a conference paper at ICLR 2023
D ANOTHER POSSIBLE BASELINE
Transformer
Transformer
Transformer
1.00
1.0
0.9
0.96
0.5
0.8
Avg.
Concat.
AT
Avg.
Concat.
Avg.
Concat.
AT
AT
(a) One-hot:
TrainAce
(b) Onc-hot: GenAcc
(c) Onc-hot: TopSim
Figure 9: The comparison of different types baseline object encoders.
In the main experiments, used the object encoder that simply averages the attribute vectors for the non-attention baseline. which is equivalent to the model attending to the attribute vectors uniformly (i.e.. with the same attention weights). Another possible baseline that aggregates multiple attribute vectors into one fixed-sized vector can incorporate the concatenation of the attribute vectors. Such method creates object vector where the attribute information is disentangled across the dimension and thus may lead to better compositionality in the emergent language.
Here, we replace the object encoder of the NoAT-NoAT agent in the one-hot game with the concatenation-based one. It firstly projects the input attribute vectors (o",..., into the dimen- sion of d/A A, where d is the dimension of the final object vector, with learnable matrix parameter. Then, the final object vector is derived by concatenating the projected vectors.
Figure 9 shows the result from the average-based and concatenation-basee object encoder. We observe that the concatenation- based encoder outperforms the peerage-based encoder in TopSim, indicating that having the attribute information disentangled across the vector dimension facilitates learning compositional languages. Still. the AT-AT agents perform the best, which shows that having dynamic attention each symbol generation is more effective.
17

---

Under review as a conference paper at ICLR 2023
E THE EFFECT OF MULTI-HEAD ATTENTION
In the main experiments, we used a single-head attention for the Transformer decoder to facilitate simpler analysis. In this section, check whether using the multi-head attention affects the perfor- mance of the agents. We modify the number of heads in the source-targel attention of the attentional Transformer Speaker and experiment with the non-attention Listener in the one-hot game. We could expect that the multi-head attention may improve the performance by adding more flexibility in learning or to the contrary hurt the performance lose the inductive bias of one-to-one mapping between symbol and an object attribute.
Transformer
Transformer
Transformer
1.00
0.95
0.90
0.6
0.95
0.85
0.90
2H
2H
4H
1H
2H
4H
8H
(a) One-hot: TrainAce
(b) One-hot: GenAcc
(c) One-hot: TopSim
Figure 10: The comparison of different numbers of heads in the source-target attention of the Trans- former Speaker. Listener is the non-attention Transformer model.
From Figure 10 changing the number heads basically does not affect the performance except for the two-head attention, where GenAce is significantly lower than the singe-head agent (p < 0.01) but the TopSim is significantly higher (p < 0.01). This possibly stems from some optimization instability. Overall, we do not observe significant impact of the multi-head attention in this case.
18


Reviews:
[{"review_id": "w51CVKRp9dJ", "reviewer": "Reviewer_kyNy", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper is concerned with emergent communication: How pairs of agents (neural networks in this case), develop joint symbols for communicating information, and is in particular concerned with the compositionality of those symbols. \n\nThe hypothesis of this paper is that the mechanism of attention, similar to the one popularised in transformers, lead to more compositionality in the emergent language. \n\nThe experimental setup in this paper is based on the \"referential game\", where one agent(\"speaker\") sees an object and need to convey this to the second (\"listener\"). The listener must then pick the correct object from a set of distractors. The speaker is implemented as a pre-trained CNN, the listener as a Transformer decoder. Experiments are run in a simple one-hot game and on inputs based on pairs of garments taken from the fashion MNIST dataset. \n\nThe experimental results are: \n- the use of attention lead to better performance in the game\n- the use of attention lead to more compositionality in the emergent language", "summary_of_the_review": "In summary, this is a well written paper, tackling an interesting and important question, namely compositionality in emergent language. \n\nThe paper tries to establish the effect/importance of transformer-like attention in compositionality, but it is unclear to me that the results really answer this question, beyond showing the expected result that attention improves performance. \n\nI have some concerns that the task may be too simple to effectively study the emergence of compositionality, as it may indeed not be needed at all. \n\nThe paper would also gain from a more in-depth discussion of the different mechanisms and approaches to attention and rationales why they would affect the emergence of compositionality. ", "strength_and_weaknesses": "### Strengths\n- The paper is generally clear and pleasant to read. \n- The question of language emergence and compositionality of codes in particular is an interesting and relevant one. \n- The experimental setup is well described and clear. \n\n### Weaknesses\n- It remain unclear to me whether the chosen experimental design is well suited to the research question. \n\t- First, the FashionMNIST scenario is only based on identifying pairs of items, there are no properties attached to the items, so it is not clear what type of compositionality we should expect.\n\t- Second, the message length is 2, therefore it is unclear how compositionality really comes into play. The argument is made on the pre-message dimensions of the attention model, but is that really the language in this experiment? This could be made clearer. \n- The attention mechanism used in transformers, if very successful in neural networks, are a very specific form of attention, there are other. This is not conveyed in the related work section. I would have liked some more discussion on the specific properties of this type of attention and why it is particularly suited for the emergence of compositionality. \n- The discussion of the results could be clearer: The results seem to show that attention increases compositionality (according to the chosen metrics), but also that attention generally improves performance at the task across the board. Is it the case that increased attention and therefore the benefit of attention is due to increased compositionality, or does compositionality increases due to better performance? I am not sure this question is answered by the experimental results. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "sXdUXE0_yL-", "reviewer": "Reviewer_gZp6", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper explores how attention may impact language emergence in two instances of the Lewis Game setting.\nThe authors observe that transformers seem to introduce an inductive bias toward compositionality.\nThey then look at the alignment between concepts and attention weights.\n", "summary_of_the_review": "While I value the initial research direction and some effort in analyzing the results (e.g., Fig5), the paper does not fully explore how transformers may impact the emergent language. I would recommend iterating over diverse tasks and architectures to discover potential correlations. Note that this does not require extensive computation if the network remains below a few million parameters on a simple one-hot or visual task. \nFurthermore, I am missing reproducibility points.\nTherefore, I cannot recommend paper acceptance in its current state. Yet, I recommend the authors to continue their research work.", "strength_and_weaknesses": "First of all, the authors point out a relevant fact: the language emergent models mostly remain stuck to RNN, while transformers were a game-changer in the Language modeling community. Therefore, updating the EmeCom modeling with transformers is worth exploring.\nHowever, I think that the paper only scratches the surface of the problem:\n - The transformer is 1-attention layers, which are merely basic memory networks. In this case, I would also add an RNN with attention.\n - there is no actual discussions about the difference transformer impact on language (depth, encoder/decoder, decoder-only). This would be by far the most exciting part for me\n - the task is extremely toyish, while transformers shine when complexity increase. For instance, a message length of 2 prevents advanced compositionality,\n - there is no clear disentanglement between the vision-transformer and language transformer in the paper, while compositionality may come from different input\n\nAs a result, my main takeaway from the paper is that the transformer (single-attention layers) outperforms RNN-based system. Yet, even this point must be taken with caution due to the lack of extensive architecture search. \n\nFor instance, a simple idea to increase paper generability: Given different X tasks of increasing difficulty (few attributes with many values or vice-versa) or with different settings (small/long message length), I would explore different combinations of neural architecture (RNN, RNN+attention, different transformers) where network capacity is fixed. Then, if an attention-based network consistently outperform RNN based model (acc/topsim), then it is an interesting outcome.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "eoBwc-sqIcQ", "reviewer": "Reviewer_4TD5", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper analyze the effect of attention on emergent communication. The paper hypothesizes that the attention mechanism will bias agents towards more compositional encodings. Experimental results on a small-scale symbolic game and image reference game derived from Fashion-MNIST demonstrate that attentive speaker and/or listeners result in improvements in task success on unseen objects and various proxy measures of compositionality such as topographic similarity and positional disentanglement. Further analysis builds on the intepretability of attention, and shows that attention can be used to analyze attribute and message alignment, as well as predict coordination between speakers and listeners.", "summary_of_the_review": "I recommend a borderline reject, since I am not confident that the contributions are novel.", "strength_and_weaknesses": "*Strengths*\n\nThe experimental results in this paper support the hypothesis that attention helps in the particular architecture considered. The improvements in all metrics over the ablated non-attention model are significant, despite large error bars. \n\n*Weaknesses*\n\nMy primary concern is that this paper has two main contributions: 1) weaker models result in worse performance and 2) attention is interpretable. These contributions are relatively limited, as a correlation between model flexibility and performance is common and the interpretability of attention has been analyzed previously [1].\n\nI am hesitant to ask for more experiments, but will suggest one. To increase the scope of the first contribution, one could test if more powerful models actually do result in task metric improvements. For example, one could run a multi-headed attention speaker and/or listener. It's possible that a single head attention model might perform better, since a compositional encoding may want to focus on one aspect of the referent at a time. This would contradict the \"weaker models result in worse performance\", and be an interesting additional contribution. \n\n*Questions*\n\n* The error bars in Table 1 are much larger than the ones reported in [2], which are run with a larger message space. I assume this is due to variance in the REINFORCE gradient estimator, which is one of the reasons for introducing the KL regularization. Is there an alternative explanation for this, and/or a way to reduce the size of the error bars? Perhaps other forms of variance reduction, such as multi-sample estimators and/or control variates.\n\n[1] Wiegreffe, Sarah and Yuval Pinter. “Attention is not not Explanation.” EMNLP (2019).\n[2] Chaabouni, Rahma, et al. \"Emergent communication at scale.\" International Conference on Learning Representations. 2021.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "88Z5QM2TqNg", "reviewer": "Reviewer_4G68", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper studies languages that emerge when two agents (Speaker and Listener) communicate to solve a referential game. The authors propose to endow the agents with attention mechanisms to create a pressure for learning more compositional emergent languages. The experimental study shows that indeed changing the architectures in this way results into increases in the compositionality metrics (topographic similarity, pos-dis, and bos-dis). The authors also provide interesting insights in the behaviour of the agents by analysing the emerging attention patterns.", "summary_of_the_review": "My main comments are:\n* It is not clear that the baselines are not overly simplistic;\n*  I think it is not surprising that a hand-crafted architecture, tailored for this specific game setup, can be created to promote compositionality in the emergent language. And it seems to me that this particular architecture is hand-crafted as it has a single-head attention. What makes this particular architecture interesting? \n* Ideally, to support the claim of the paper, one might want to disentangle the task success rate from how compositional language is;\n", "strength_and_weaknesses": "Strengths:\n* Generally the paper is well-written and has a good literature review. \n* The results (Table 1) show a significant improvement over the baseline and the interpretability section (section 4.3) is interesting, even if the difference between the failure and success settings is not significant.\n\nWeaknesses:\n* The main novelty of this paper is agents’ architecture (section 2.2). However, this attention mechanism is not sufficiently explained. \nThe authors state that the query is the symbol of the message and the keys/values are the inputs’ representation. This is still a vague description of the mechanism. \n* The baseline architecture simply averages input embeddings. Is this the most fair baseline we can come up with? For instance, can we have an architecture where the set of vectors are projected to a smaller dimension (/2) and concatenated?\n* Relatedly, the goal of the work is to show that the attention-enabled agents come up with more compositional languages. For now a possible interpretation of Table1 is that such agents are simply more effective at solving the task (have higher acc) and, consequently, they attain better compositionality. Is there a way to disentangle success rate vs compositionality, ie showing that for a fixed success rate the languages would be more compositional? Maybe the baseline architecture is simply too weak due to the averaging?\n* Stepping back, I think it is not surprising that a hand-crafted architecture, tailored for this specific game setup, can be created to promote compositionality in the emergent languages. What makes this particular architecture interesting? One specific way this architecture can be interesting is if it is very general and as powerful as standard Transformer yet provides additional benefit of compositionality. However, I believe the current experiments do not allow this conclusion, primarily because they only use the single-head attention. Would the compositionality bias persist with multiple heads?\n* In Figure 3, we only have examples where Speaker and Listener have access to the same example. It would be interesting to look at the attention map when the examples are different but from the same label (is the position important?).\n* For the Fashion-MNIST game: What is the size of the training dataset (it is only specified that there are 45 labels)? \n\n\n* Typo: S2.4, \"Positioanl\"\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject