Effect of Pressure for Compositionality on Language Emergence

Abstract: Humans can use natural languages compositionally, where complicated concepts are expressed using expressions grounded in simpler concepts. Hence, it has been argued that compositionality increases the ability of generalization. This behavior is acquired during natural language learning. Natural languages contain a large number of compositional phrases that function as examples of how to construct compositional expressions for human learners.  However, in language emergence, neural agents do not have access to such compositional language expressions. It can be circumvent by optimizing a suitably devised metric of compositionality, which does not require supervising examples. In this paper, we present a learning environment where agents are pressured to make their emerging languages compositional by incorporating a metric of topological similarity into the loss function. We observe that when this pressure is carefully adjusted,  agents can achieve higher generalizations. The optimal level of this pressure is highly dependent on the agent architecture, input, and structure of the message space. However, we find no simple correlation between high compositionality and generalization. The advantage offered by compositional pressure is situational. We observe instances where moderately compositional languages are showing generalizing behavior to the extent of some highly compositional ones.  

Reviews and author responses:

 First, we highly thank the reviewer for their valuable replies. We will explain our stance and address the reviewers' concerns below.

In the current draft, we only measure compositionality for the cases where we do not employ any type of compositionality loss on the discrete link, which we denote as the baseline scenario.  For this case, we study how generalization and compositionality relate, and find that there is indeed a strong but situational correlation.

In the case, where we use compositionality loss, we demonstrate the addition of  compositionality loss, if used in controlled amounts, can improve generalization and transmission speed than the baseline cases. We do not intend to highlight the correlation between test accuracy and topological similarity, for this case, since both are now being optimized simultaneously. 

Our loss is not based on a mechanism that is seen in human language evolution process, but we do not think that adding a loss does violate the existing practices of language emergence research. For example, there are several works which employ different types of metrics, which measure quality of communications in a human perspective, on the emerging communications . 

Studies have used various losses on emerging messages like; entropy regularization, loss on message length, loss on symbol usage, and selecting messages most understandable to the sender itself [1,2,3]. And humans, like how they try to use the shortest message, use compositional expressions not atomic messages. Hence, we think that adding a compositionality loss, which does not require any additional examples, is not against the existing research. As we understand, the main expectation of language emergence is to develop languages in a grounded manner without explicit supervising examples. In our environment, this property is not violated or removed, and agents still develop communications out of necessity. 

We are motivated to induce compositionality for two reasons; First, we observe natural languages are compositional and humans use natural language to express complex concepts, which hints about possible advantages of compositionality. Secondly, there have been studies, using methods motivated by language evolution, that show the generalizing advantage of compositional languages. In this research, we replace the generational transmission mechanisms with a loss which can be directly optimized, which does not need additional training stages. 

1. Katrina Evtimova, Andrew Drozdov, Douwe Kiela, and Kyunghyun Cho. Emergent communication in a multi-modal, multi-step referential game. In 6th International Conference on Learning Representations, 2018.
2. Edward Choi, Angeliki Lazaridou, and Nando De Freitas. Compositional obverter communication learning from raw visual input. In 6th International Conference on Learning Representations, 2018.
3. Angeliki Lazaridou, Karl Moritz Hermann, Karl Tuyls, and Stephen Clark. Emergence of linguistic communication from referential games with symbolic and pixel input. In 6th International Conference on Learning Representations, 2018.


 > topsim is from brighton and kirby 2006

This is actually true as we know, and it has been used other studies including [1] ,

[1] Yi Ren, Shangmin Guo, Matthieu Labeau, Shay B Cohen, and Simon Kirby. Compositional languages emerge in a neural iterated learning model. In International Conference on Learning Representations, 2020



 topsim is from brighton and kirby 2006

 Matrices A and B only depends on the size of the batch. The example above describe them for a batch size of 3. They are just permutation matrices that arrange the input batch in a vector, such that it is possible to obtain the pairwise distance, between each pair within a batch. It is easily constructed at the test time considering permutations. For the above example, we have three inputs $x_1, x_2, x_3 $, in this case to calculate the topographical similarity, we need to get the distances between $(x_1,x_2 )$, $(x_1,x_3 )$, and $(x_2,x_3 )$. The matrices just create the vectors $<x_1, x_1, x_2 >$ and $<x_2, x_3, x_3 >$, so when they are subtracted we can get the subtractions necessary to calculate the distances. 

Assume $x_1$ : "red square" ,  $x_2$ : "blue square" ,  $x_3$ : "red circle" . Then the corresponding distances would be <"red_square", "blue_square>, <"red_square", "red_circle">, <"blue_square", "red_circle">.  If we use word level edit distance, just for the sake of this example, the results will be <"red square", "blue_square> = 1, <"red square", "red circle"> = 1, <"blue square", "red circle"> = 2. Next we obtain agents messages for the inputs, and calculate the distances between every pair of messages, similarly.

 First, authors thank the reviewer for their valuable comments. We have submitted a new draft addressing reviews and with new experiments. We will address your concerns in the points below.

>I don't understand the purpose of the paper. It seems to fit into the emergence of language field but I cannot work out what the authors are saying. Do they propose such a mechanism was present in the evolution of human language? Or are they finding evidence that rejects this? Are they suggesting it be used in evolving communications in multi-agent environments? To be clear, I cannot see what research question is being asked, and I cannot see what the authors conclude about this question from the evidence they obtain.

1. Our main research question is; How to induce compositionality in emergent communications, through a straightforward and simple mechanism? and Whether such externally induced compositonality can increase desirable characteristics like generalization and transmission speed?  

* There is evidence showing that natural languages have a degree of compositionality. Compositional expressions are used by humans to deliver complex ideas using simpler words, hinting it is possible to gain advantages to generalization through compositioinality.

* Existing literature already suggest compositional lanuguages are easier to acquire by new learners.

* Literature contains various approaches for inducing compositionality to boost up emergent communications, including iterative generational transmission and using  supervised pre-training etc, which can be complex.

* If supervised pre-training is used it would require example dialogues, which otherwise entirely not needed for language emergence.

* Iterative generational transmission,inspired by evolutionary linguistics, requires one or more agents to be reinitialized over and over which can be time and resource.

* In this paper, we propose an approach which does not require examples or iterative learning to induce compositionality.

2. We incorporate a novel loss term into the loss function, which is straightforward to implement and does not require additional training cycles, to induce compositionality in controlled manner. We incorporate a well studied metric of compositionality into the training and steer agents to directly optimize the latter metric through their joint loss function. This makes our method straightforward and scalable to various other language emergence setups.

* We show that externally influencing agents via our method, yield improved generalizations and higher speeds of language transmission. 

* We analyse how our method performs with respect to different inputs and message spaces. 

>The relationship to past work is good. There is a sensible quantity of material cited throughout and the material referenced shows how the work fits into the existing material in the area. The bibliography contains a large proportion of papers from arXiv. This is not a peer reviewed source - we have no way of knowing whether claims in these papers meet normal professional criteria. I would guess most of this work actually has been published somewhere - please cite the published version.

1. Authors thank the reviewer for providing a point to improve the references. As suggested by the reviewer, we have updated the references to include articles published in peer reviewed venues. Actually, the arxiv versions were pre prints of the same articles published.



 >The description of the method appears to contain all the necessary details. It is sometimes a little hard to follow. It would be useful to have a non-technical overview in which the basic idea is described without getting bogged down in the details of the networks or using algebra. I didn't follow section 3 very well - I think more detail is needed on Chaabouni et al's measure of compositionality, as it is key to the approach and the paper cannot be fully understood without it.

1. We have included intuitive explanations in to the daft as requested by the reviewer.  And in the below point we will further explain our concepts in a non technical manner.

* Value attribute environment : The dataset used in our experiments appears in many other works including [1,2,3]. In a value attribute environment, a given input consists of a set of finite attributes, where each attribute can take a value. For example, an input with two attributes can be a “Red Square”, which has the attributes of color and shape. For this instance, the attributes color and shape have taken the values “Red” and “Square” respectively. Now if there are values {“Red”, “Blue”} and {“Square'', “Circle”}. This corresponds to a value attribute environment of X(2,2), with two attributes and two values per each. As done by other studies, we do not use direct pixel inputs in the study and employ symbolic inputs [1,2,3], where each value is represented as a one hot encoded vector. Hence, the concatenation of such vectors form an abstract object.

* Compositionality : Agents map environmental entities or concepts with symbols while reaching a consensus for the mapping. If agents follow compositionality, agents would connect individual symbols to atomic concepts in the environment, and combine multiple symbols to form composite messages for expressing complex  concepts, facilitating generalization. For example, Red Square is a compositional expression, containing attributes Color and Shape, constructed by using atomic concepts Red and Square, which corresponds to the values taken by the two attributes. Hence, whenever an agent observes a novel input, it can break down the input into a set of attributes and corresponding values. An expression describing the input can be constructed by suitably concatenating the symbols representing each value of the attributes.

* Generalization :- In this context, generalization means how agents could communicate about unseen combinations of attributes and values. It can be measured by test accuracy similar to other cases in machine learning.

* Reconstruction game : In a reconstruction game, there are two neural agents namely, speaker and listener. The speaker observes a given input and transmits a message consisting of multiple symbols towards the listener. Listener observes the message and reconstructs an approximation of the original input observed by the speaker. Both agents are jointly rewarded according to the degree to which the reconstruction approximates the original input.

[1] Yi Ren, Shangmin Guo, Matthieu Labeau, Shay B Cohen, and Simon Kirby. Compositional languages emerge in a neural iterated learning model. In International Conference on Learning Representations, 2020

[2] Cinjon Resnick, Abhinav Gupta, Jakob Foerster, Andrew M Dai, and Kyunghyun Cho. Capac-ity, bandwidth, and compositionality in emergent language learning. In Proceedings of the 19th International Conference on Autonomous Agents and Multi Agent Systems, pp. 1125–1133, 2020

[3] Abhinav Gupta, Cinjon Resnick, Jakob Foerster, Andrew Dai, and Kyunghyun Cho. Compositionality and capacity in emergent languages. InProceedings of the 5th Workshop on Representation Learning for NLP, pp. 34–38, 2020.


 >I think more detail is needed on Chaabouni et al's measure of compositionality....

1. Chaabouni et al. proposes three measures for compositionality, but we only use the topological similarity for our experiments. Because it has been used in many other studies and topological similarity can be implemented in a differentiable format using Pearson correlation. 

* Topological similarity: We obtain it as the Pearson correlation between distances of the input pairs and distances of the corresponding message pairs.

* Let's assume three inputs $x1, x2, x3 $ and corresponding messages $m1, m2, m3$.  
* First, all pairs of inputs and messages are obtained:- $[(x1,x2), (m1,m2)], [(x1,x3), (m1,m3)], [(x2,x3), (m2,m3)]$. 
* Next, cosine distance between each pair can be calculated as: $[1-cos(x1,x2), 1-cos(m1,m2)], [1-cos(x1,x3), 1-cos(m1,m3)], [1-cos(x2,x3), 1-cos(m2,m3)].$ 
* This yield the distances between each pair, $[d(x1,x2), d(m1,m2)], [d(x1,x3), d(m1,m3)], [d(x2,x3), d(m2,m3)]$. 
* The topological similarity is defined as the pearson correlation between the two distance sets : $<d(x1,x2),d(x1,x3),d(x2,x3)> $ and $<d(m1,m2),d(m1,m3),d(m2,m3)>$ . 
* The expression $[1-cos(Ah0, Bh0)]$ and [$1-cos(Am, Bm)] $ directly provide the above two vectors.  

Intuitively, Spearman correlation can also be used but its ranking operation is not directly differentiable. 




 

 The revision has addressed very little of my concern.  The single example helps somewhat but is not followed through in the description of the entire process.   The paper still cannot be read and understood standing alone as a self-contained piece of work, and still makes very little connection to reality in terms of scope of applicability and new insights obtained.  Therefore I would keep my current recommendation.


 It is not clear from the above explanation where A and B come from and why they are defined this way.   Also, this process of pairing needs to be explained in the context of the "red square" example.


 First authors highly thank the reviewer for their valuable comments. We will address the reviewers' concern below.

>Including other emergent communication tasks or effects of other types of pressures on emergent languages.

1. Most of the existing research on language emergence uses only one task, from reconstruction, classification or discrimination. So we think it would not be unfair to carry out our study using the reconstruction game, which is more robust [1,2,3, 4,5,6]. 

2. The type of improvements provided by other pressures are not as same as the compositional pressure we employed. For example, the pressure of entropy regularization and a loss encouraging shortest message are not functionally identical to each other. The first increases the exploration and the latter make messages more concise, which is a desired property in language usage.  Hence, as we think, their behavior is not directly comparable to each other. 

3. In the case of compositional pressure, it provides improvements to both  the language acquisition speed by new learners, and to the generalization. We believe that functional pressures concerning different aspects of communication can be carefully combined together to gain more improved performance. This is seen in general machine learning, for example, dropout and weight decay, which are frequently used together.

>Although, to my knowledge, directly optimizing the topological similarity metric has not been studied before, the metric itself is well studied. I am not fully convinced that directly optimizing this metric tells us very much (as is apparent in the experimental section). The paper also bears a lot of similarity to parts of [1].

1. First, it is true that the improvements can vary across different environment configurations. But there are configurations which give good improvements to the generalization alone  (Appendix B):
* Input space X(50,2) Recurrent Listener :- M(10,4), M(10,6)
* Input space X(50,2) Linear Listener:- M(10,4), M(10,6)
* Input space X(10,4) Recurrent Listener:- M(10,6)

2. In configurations with lower improvements to generalization, either the message space is too tight or it is excessively large. First case hinders agents from converging to a good protocol, which is not a problem confined to our research as many studies in the area report this issue. In the second case, the high degree of freedom provided by the message space, permits agents developing unstructured yet generalizing protocols. In this case, we show that even if the improvements to the generalization are very small, languages emerged under high compositional pressure are much easier to capture by new learners (Section 4.2.3)

[1] Yi Ren, Shangmin Guo, Matthieu Labeau, Shay B Cohen, and Simon Kirby.  Compositional languages  emerge  in  a  neural  iterated  learning  model.   In International  Conference  on  Learning Representations, 2020

[2] Cinjon Resnick,  Abhinav Gupta,  Jakob Foerster,  Andrew M Dai,  and Kyunghyun Cho.   Capac-ity, bandwidth, and compositionality in emergent language learning.  In Proceedings of the 19th International Conference on Autonomous Agents and Multi Agent Systems, pp. 1125–1133, 2020.

[3] Abhinav Gupta, Cinjon Resnick, Jakob Foerster, Andrew Dai, and Kyunghyun Cho.  Compositionality and capacity in emergent languages. InProceedings of the 5th Workshop on Representation Learning for NLP, pp. 34–38, 2020.

[4] Angeliki Lazaridou, Alexander Peysakhovich, and Marco Baroni.  Multi-agent cooperation and the emergence of (natural) language.  In 5th International Conference on Learning Representations,2017. 

[5] Emilio Jorge, Mikael K ̊ageb ̈ack, Fredrik D Johansson, and Emil Gustavsson. Learning to play guess who?  and inventing a grounded language as a consequence.arXiv preprint arXiv:1611.03218,2016

[6] Serhii Havrylov and Ivan Titov.  Emergence of language with multi-agent games: Learning to communicate with sequences of symbols.  In Advances in neural information processing systems, pp.2149–2159, 2017



 >the paper also bears a lot of similarity to parts of [1].

1. (Chaabouni et al) discuss compositionality and generalization, and they claim there is no correlation between generalization and compositionality, given sufficiently large input spaces generalization will automatically happen, and compositionality provides advantages to the language transmission. They use the value attribute environment and the reconstruction setting. 

2. The value attribute environment and reconstruction game, which we use in our experiments, is used in other language emergence studies [1,2,3].  As we understand, it is one of the three widely used game configurations (reconstruction, discrimination, classification).

3. In the first experiments we do as a baseline, we do not incorporate our compositionality based loss term. With that we found that there is indeed strong but situational correlation between compositionality and generalization. We further elaborate on the factors that determine the strength of this relationship.  The following, as far as we know, are not discussed in (Chaabouni et al).
* Recurrent networks display much correlation between generalization and compositinality, than linear networks. We think this is due to recurrent networks processing inputs sequentially along the message, starting from a fixed position. Hence, it is better to have information neatly distributed along the whole message length. 
*  When the available training data is low (smaller data spaces), the latter correlation becomes stronger.
* When agents can train better, the strength of correlation becomes higher. 
* In addition, we observe evidence that further strengths claims of (Chaabouni et al) like:- Positive effect of large input spaces

2. Secondly as our main contribution, we introduce the topological similarity based loss which can externally pressure agents towards compositionality. To the best of our knowledge, (Chaabouni et al) does not employ such external pressure in their study. The advantages offered by our loss are twofold:
* First, it increases the generalization of emergent languages by various amounts depending on the environment parameters (Appendix B)..
* Secondly, when the input space is large, where both our method and the baseline can converge to good generalizing representations, we show that our method can achieve significant improvements in transmission speeds compared to the baseline (Section 4.2.3) .

3. Furthermore we discuss on the behavior of the external pressure with respect to other environment parameters.
* Instances, with recurrent listeners and small amount of training data, gets the most improvement in generalization.
* Higher compositional pressures are favored when vocabulary size is small and messages are longer.

[1] Yi Ren, Shangmin Guo, Matthieu Labeau, Shay B Cohen, and Simon Kirby. Compositional languages emerge in a neural iterated learning model. In International Conference on Learning Representations, 2020

[2] Cinjon Resnick, Abhinav Gupta, Jakob Foerster, Andrew M Dai, and Kyunghyun Cho. Capac-ity, bandwidth, and compositionality in emergent language learning. In Proceedings of the 19th International Conference on Autonomous Agents and Multi Agent Systems, pp. 1125–1133, 2020.

[3] Abhinav Gupta, Cinjon Resnick, Jakob Foerster, Andrew Dai, and Kyunghyun Cho. Compositionality and capacity in emergent languages. InProceedings of the 5th Workshop on Representation Learning for NLP, pp. 34–38, 2020.





 >The experimental setup is not clear. What are some examples from this dataset? What are some examples of attributes and values? How is it related to other tasks in the literature? Comparisons to the reconstruction task in [2] are made but this does not seem like the same task. It looks similar/identical to the reconstruction task used in [1] but no comparison to that task is made in the paper.

1. The dataset used in our experiments appears in many other works including [1,2,3]. In a value attribute environment, a given input consists of a set of finite attributes, where each attribute can take a value. For example, an input with two attributes can be a “Red Square”, which has the attributes of color and shape. For this instance, the attributes color and shape have taken the values “Red” and “Square” respectively. Now if there are values {“Red”, “Blue”} and {“Square'', “Circle”}. This corresponds to a value attribute environment of X(2,2), with two attributes and two values per each. As done by other studies, we do not use direct pixel inputs in the study and employ symbolic inputs [1,2,3], where each value is represented as a one hot encoded vector. Hence, the concatenation of such vectors form an abstract object. 

2. As suggested we have included an intuitive overview for the reconstruction game and have improved the presentation of the overall paper. In a reconstruction game, there are two neural agents namely, speaker and listener. The speaker observes a given input and transmits a message consisting of multiple symbols towards the listener. Listener observes the message and reconstructs an approximation of the original input observed by the speaker. Both agents are jointly rewarded according to the degree to which the reconstruction approximates the original input. 

3. Value attribute environment can be used in all three types of tasks. [1] Use it in a discrimination setting. In any type of game, objects of value attribute environment can be used as inputs.

>I don't this that these plot adds much towards the paper. It only seems to work as a sanity check to see if the implementation is accurate. Optimizing a separate loss would naturally improve that loss. It would be more interesting if a different compositionality metric than what was used during training is used instead. 

We have redo the whole set of plots in the current draft of the paper. 

>These tables are missing descriptions and does not depict the metrics being measured. The tables are also missing confidence intervals which makes drawing any conclusion from these tables misleading

We have rearranged the tables and have added confidence intervals to the all plots (Appendix B). Moreover we have corrected the typos which have been pointed by the reviewer in the current draft.

[1] Yi Ren, Shangmin Guo, Matthieu Labeau, Shay B Cohen, and Simon Kirby. Compositional languages emerge in a neural iterated learning model. In International Conference on Learning Representations, 2020

[2] Cinjon Resnick, Abhinav Gupta, Jakob Foerster, Andrew M Dai, and Kyunghyun Cho. Capac-ity, bandwidth, and compositionality in emergent language learning. In Proceedings of the 19th International Conference on Autonomous Agents and Multi Agent Systems, pp. 1125–1133, 2020.

[3] Abhinav Gupta, Cinjon Resnick, Jakob Foerster, Andrew Dai, and Kyunghyun Cho. Compositionality and capacity in emergent languages. InProceedings of the 5th Workshop on Representation Learning for NLP, pp. 34–38, 2020.



 >The paper claims that carefully tuning the topological similarity metric improves the speed of convergence and generalization. However, this claim is not fully supported due the usage of only 1 learning rate. In particular, 4/6 settings in Figure 1, adding no additional loss, performs among the best. In figure 1, a), d), there is evidence that different values performs better but unless additional learning rates and more seeds are used, this claim is not fully supported.

1. We have included additional experiments on the experiments, including a configuration with recurrent listener and 10 value 4 attribute environment, experiments with two new learning rates, and further experiments proving the transmission speed advantage of languages developed under high compositional pressures. In these experiments, we show that even when the improvement to generalization is small, it can emerge languages which are easier to acquire by new learners.

* Test accuracy for $X(10,4)$; 10 values and 4 attribute input space, with recurrent listener.

|$M(W,L)$ | $0$ | $1e-06$ | $0.1$ | $0.5$ | $1$ |
|------|----|---------|--------|-----|-----|
|$(10,4)$  | $0.711 \pm 0.046$ | $0.710 \pm 0.033$ | $0.694 \pm 0.036$ | $0.733 \pm 0.161$ | $0.549 \pm 0.018$ |
|$(10,6)$  | $0.846 0.145$ | $0.829 \pm 0.090$ | $0.890 \pm 0.178$ | $0.811 \pm 0.126$  |  $0.740 \pm 0.173$ |
|$(100,2)$ | $0.671 \pm 0.030$ | $0.662 \pm 0.048$ | $0.624 \pm 0.053$ | $0.559 \pm 0.020$ | $0.526 \pm 0.063$ |
|$(100,4)$ | $1.0 \pm 0.0$ | $1.0 \pm 0.0$ | $1.0 \pm 0.0$ | $0.912 \pm 0.081$ | $0.737 \pm 0.099$|

2. We have carried out experiments using two additional learning rates, which are already incorporated in to the current draft.  We use $X(50,2)$ input space with the recurrent listener, and when $C_t>0$, use the pressures that gave best results under $lr=0.01$. So we use $C_t = 1e-6,$ for $M(100,2), M(100,4)$ and $C_t =0.5$ for $M(10,4), M(10,6)$.

|$M(W,L)$| $lr:0.005, C_t =0$ | $lr:0.005, C_t > 0$ | $lr:0.05, C_t =0$ | $lr:0.05, C_t >0$ |
|----------------|---------------|-----|---------------|-----------|
| $M(10, 4)$  | $0.726 \pm 0.053$ | $0.750 \pm 0.073$  | $0.125 \pm 0.228$ | $0.211 \pm 0.126$ |
| $M(10, 6)$  | $0.665 \pm 0.051$  | $0.870 \pm 0.005$ | $0.328 \pm 0.122$ | $0.228 \pm 0.108$ |
| $M(100, 2)$  | $0.779 \pm 0.023$  | $0.833 \pm 0.046$  | $0.820 \pm 0.046$ | $0.858 \pm 0.079$ |
| $M(100, 4)$  | $0.973 \pm 0.011$ | $0.975 \pm 0.012$   | $0.025 \pm 0.031$ | $0.025 \pm 0.000$ |

The results below, indicate the pressure values that gave the best test accuracy for $lr=0.01$

|$M(W,L)$  | $0$   |   $1e-06$  |  $0.1$    |    $0.5$    |     $1$ |
|--------|------------|--------|-------|---------|---|
|$(10,4)$ |  $0.543 \pm 0.418$ | $0.472 \pm 0.331$ | $0.643 \pm 0.451$ | $0.742 \pm 0.024$ | $0.519 \pm 0.202$ 
|$(10,6)$  | $0.558 \pm 0.345$ | $0.468 \pm 0.042$ | $0.477 \pm 0.043$ | $0.615 \pm 0.541$ | $0.504 \pm 0.381$ 
|$(100,2)$ |  $0.916 \pm 0.124$ | $0.934 \pm 0.092$ | $0.815 \pm 0.099$ | $0.692 \pm 0.097$ | $0.618 \pm 0.258$ 
|$(100,4)$ |  $0.938 \pm 0.106$ | $0.968 \pm 0.027$ | $0.953 \pm 0.027$ | $0.930 \pm 0.051$ | $0.905 \pm 0.126$ 

The optimum values selected for $lr=0.01$  still remains desirable, across the new learning rates, giving increased test accuracies in most of the cases.

 I feel that the paper needs to go into a bit more depth, encounter a few more challenges, and solve them. Currently the paper contains:

- an additional data point showing that compositionality helps generalization performance
- an additional data point showing that compositionality helps language transmission
- adding a compositional metric to the loss improves performance on that compositional loss
- adding a compositional metric to the loss improves generalization
- adding a compositional metric to the loss improves language transmission
- adding too much compositional metric loss causes task performance to go down

I feel that there is nothing in the above felt new/surprising to me. I think one thing the paper stated which was interesting to me was how to make topsim differentiable: using Spearman's Rank is not differentiable, but using Pearson coefficient is. Also, the idea of adding a compositional metric to the loss was surprising, and flies in the face of convention.

I'm unsure what the paper would need to do to make me feel that it should be accepted. I think that it would need to dive deeper into their chosen direction of adding compositional losses, e.g. how do different compositional loss metrics compare? But I'm not sure whether that would be enough. I feel the authors need to be clear about what research questions they are trying to answer, and why those are high impact questions.

 This paper is a first in my role as reviewer, in that generally, I feel that the papers I have reviewed before where I've rejected them have simply not gone into enough depth/rigor; and adding depth/rigor would, in my opinion, likely move the paper from a reject to a possible accept.

However, in this case, whilst it would be easy for me to say 'well, youve shown that the optimal level of external pressure varies with environmental parameters, but perhaps it could be interesting to dig into this in more depth?', I'm not sure that will fix a more fundamental issue, which is I'm not clear about the motivation of the paper. I'm not quite sure how to phrase this, and perhaps I should just stop replying (which I probalby will do soonish, maybe after this reply, unsure). Stating that the motivation of the paper seems unclear seems a litte 'he said/she said' to me, so I'd ideally like to formulate that in a more objective/rigorous way, but I'm not quite sure how to do that.

Since there are, as far as I know, many maths-like papers, that go off into a great many esoteric directions, and who's to say that a direction that seems esoteric and niche now won't turn out to be useful at some point in the future?

Indeed, such an accusation seems easy to level against emergent communication as a whole, since it doesn't clearly benefit any currently deployed models, such as chatbots, question and answering etc, which are almost always grounded.

Personally, I like emergent communicaiton as a playground to investigate how language evolved. I dislike the nature of supervised learning on groudned languages where it's not clear if the model is actually being 'clever' in any way, and not just sort of regurgitating by rote learning what it's been taught.

but ok, that's just my own personal preferences. Let's say that it's ok to go off in an esoteric direction. In that case, well then, my reply would be what I allude to earlier: you've shown that the optimal level of external pressure varies with environmental parameters, but perhaps it could be interesting to dig into this in more depth?

Still, I do feel that you might face an up-hill battle in motivating this paper. As the other reviewers allude to, well we have:

-  "I don't understand what the purpose of performing the experiments is, or what conclusions we should draw based on the results. The research question really needs to be made explicit."
- "idid not try to connect back to any natural learning scenarios."

I worry that even if you do go into more depth, and come up with interesting challenges along the way, and interesting solutions to those challenges, that this won't solve the fundamental potential issue that it's not clear how or why other people might use the research?


 We thank the reviewer for their replies.

We provide evidences that applying external compositional functional pressure is advantageous to both generalization and transmission speed. We provide a straightforward technique to externally induce compositionality, which in return give the above mentioned advantages. We show the relationship between the level of external pressure and environment factors, discussing how the optimal level of external functional pressure varies with the environment parameters. 

We believe capturing the functional and interactive aspects as two of the main goals of emergent communications, where agents emerge the necessary concepts by themselves out of necessity.  

 >  we are interested in generalization and transmission speed
> The end results concern generalization and transmission speed, which we measure and compare with the baseline. 

There are existing works which show a relationship between compositionality, generalization and transmission speed. What specific, concrete new information do you feel you are providing?

> There are multiple functional losses, which we hope, will increase the performance of emergence languages.

What do you see as the goal(s) of emergent communication research?


 We thank the reviewer for their quick responses.

>So that leaves the use of the compositional metric loss to encourage compositionality. But I kind of feel that training on the test loss is a little like training on the test set, in the case of emergent communication?  

We are not intending to do this in our paper.  There are two experiment sets in the draft.

1. Experiments that measure compositionality and generalization, but do not employ any external pressure. In these experiments, we do  not see any problem.

2. Experiments employing compositional pressure: in this case, we are interested in generalization and transmission speed. Hence we do not see any problem in here too. In this draft, we show  experimentally, that our method can improve the transmission speed and the test accuracy. Of course our method increases the compositionality of the emerged languages, but it is not our end result. The end results concern generalization and transmission speed, which we measure and compare with the baseline. We propose our approach as a method that increase these two metrics in conjunction. 

>And as far as your claiming that you show that compositionality makes a language easier to learn, this was observed in many works in the last couple of years, inspired by Kirby's ILM, but also including the same Chaabouni et al paper, at the end of the same introduction:

Actually, we do not  intend to pose as the first to observe this relationship, although our experiments verify this claim. What we do is that we propose a method based on compositional pressure to make languages more transmission friendly as we show in our transmission speed experiments. 

>are you suggesting that we start adding such a loss to our experiments? I don't see myself doing that...

There are multiple functional losses, which we hope, will increase the performance of emergence languages. Instead of using supervising examples, we can create such losses, that mimic desired characteristics and steer language emerging agents to optimize them in a simultaneous manner. We do not think this as a violation of the grounding aspects, since agents are free to invent their own concepts. We hope, by using such functional pressures, the emerging languages could improve considerably.


 We highly thank the reviewer for their continuous interest.

There are existing research that emerge their language within a single generation of neural agents, as we do in this draft. In that type of research, the languages they emerge, as we think, are not evolving through multiple generations.  Next, we see that existing natural languages are displaying compositional features which help them generalizing, but we do not investigate how these existing languages have become compositional. It may due to evolutionary processes, which we are not refuting. 

Although, we do not employ or study the mechanisms that made the existing natural languages compositional,  what we observe is that, since existing languages are compositional, human learners learn to use language in a compositional manner and it helps generalizing. This acquired compositionality of a single human learner, as we think, is motivated by learning through compositional examples already existing. But in the language emergence it is best to avoid using supervising examples to make sure that agents are free to ground their words with meanings on their own. 

Hence, we use our compositional loss, that does not require explicit linguistic (or communicative) examples, into the loss function. We do not change the original reconstruction loss, therefore we do not think it affects the primary aspects of language emergence; grounding and capturing functional aspects. t is still a non-example based loss and so the compositional loss. Hence, agents  still have much freedom to emerge their own concepts, and  we are only asking them to make their representations compositional.  And we show through our experiments, that such steering can increase the generalizing power and transmissibility. 

We believe, language emergence research have been motivated like explained above; observing the prevailing characteristics of natural languages. For example, most frequently seen communicating link, between agents in language emergence, is in discrete format. We think, this is because it resembles the discrete nature of existing human languages. We do not refuse  evolutionary mechanisms behind the discrete nature of human languages.  But also not refuse the possibility of directly observing the existing natural languages and  obtaining inspiration from its characteristics.  If incorporating them do not restrict the agents from inventing or emerging their own concepts.

>score higher on compositional metrics, by adding compositional metrics into the loss

We are actually not doing this. Of course we measure compositionlaity in a set of experiments, where we do not employ any external pressure, hence it is not circular.

For the second part of experiments, we employ the pressure, which increase the compositionality.  But most importantly we do not present it as the end effect we desire. Our desired end effects are generalization and transmission speed. In the current draft, we show that, the increased compositionality in our communications, improve generalization and transmission speed compared with the baseline. Hence, the advantage offered by incorporating the compositionality loss is, the increase of generalization and transmission speed, not the mere increase of compositionality alone.  

 > Basically, we study the effect of compositionality across two parallel dimensions; generalization and transmissibility. Our intention is not to show that there is a universal relationship between generalization and compositionality, but to show a situationally strong relationship.
> Chaabouni et al 2020, claims that there is no correlation between generalization and compositionality. 

From what I remember of previous work, compositionality is a sufficient, but not a necessary, condition for generalization. That is:
- when compositionality is low, the language might or might not show good generalization. coin-flip
- when compositionality is high, the language shows good generalization

Have a look at the end of section 1 of the Chaabouni et al 2020:

"compositionality, however, if not necessary, turns out to be a sufficient condition for generalization."

And as far as your claiming that you show that compositionality makes a language easier to learn, this was observed in many works in the last couple of years, inspired by Kirby's ILM, but also including the same Chaabouni et al paper, at the end of the same introduction:

"Finally, more compositional languages are easier to learn for new agents, including agents that are architecturally different from the ones that evolved the language. This suggests that, while composition might not be a “natural” outcome of the need to generalize, it is a highly desirable one, as compositional languages will more easily be adopted by a large community of different agents."

So, the links between generalization and compositionality that you propose as novel are existing information I feel.

So that leaves the use of the compositional metric loss  to encourage compositionality. But I kind of feel that training on the test loss is a little like training on the test set, in the case of emergent communication?

What are you hoping will be the outcome of the paper?
- are you suggesting that we start adding such a loss to our experiments? I don't see myself doing that...
- do you feel that you are presenting information that is new or surprising in some way? The points about generalization vs compositionality have already been covered in at least chaabouni et al 2020 I feel
- something else?

 | M(W,L)   | Test Accuracy     |
|----------|-------------------|
| M(10,4)  | 0.407 $\pm$ 0.073 |
| M(10,6)  | 0.507 $\pm$ 0.074 |
| M(100,2) | 0.941 $\pm$ 0.124 |
| M(100,4) | 0.983 $\pm$ 0.011 |

Weight decay performs better than applying dropout, where, on M(100,4) and M(100,2),  test accuracy is better than what obtained from compositional pressures. However, for the first two cases performance is still below the baseline.
Moreover, we believe that these regularization techniques,  dropout and weight decay, operate on a different level than our compositional loss. Weight decay and dropout concerns about the weights of the neural network and nodes on the neural network respectively, where our loss is only focused on the representations emerged on the discrete link. Hence, we think that it is better to combine them and use them on top of each other for better test accuracies and transmission speeds simultaneously. 

>What is the intention of doing this?

The main intention of our research is to demonstrate that inducing compositionality is  beneficial in language emergence and introduce straightforward implementation to achieve this. Basically, we study the effect of compositionality across two parallel dimensions; generalization and transmissibility. Our intention is not to show that there is a universal relationship between generalization and compositionality, but to show a situationally strong relationship. 

>However, given the clear knowledge of prior work expressed in the introduction, I'm unclear why the observation that there is no simple correlation between generalization and compositionality is novel? There are multiple works (eg Chaabouni et al 2020, which you cited), which show this already?

Chaabouni et al 2020, claims that there is no correlation between generalization and compositionality.  But in this draft, we show that there is indeed a strong but situational correlation. The setting they use is similar to a value attribute environment of X(100,2) in a M(100,3) message space. In this setting the message space is fairly larger than the input space, which itself contains an adequate amount of training examples. In such  settings agents can converge to representations that are not systematic or compsitonal but generalizing, which we point out in our draft. In our case, we employ multiple configurations of input and message spaces. In our mixed settings, we find strong evidence of a situational correlation, where its strength is determined by environment parameters like agent architecture and input space. The reconstruction game and the value attribute inputs are widely used in existing research other than Chaabouni et al 2020. Moreover, we propose methods to induce compositionality in emerging communications, which is not proposed in Chaabouni et al 2020.

>I thought the paragraphs were going in the direction of using non-symbolic input, so I was a bit disappointed to find their arguing in favor of symbolic inputs.

In Fact there is plenty of research that is confined to symbolic inputs, so authors do not think using symbolic input is not an issue.

>Ok, this bit explains taht the pairwise is for topological similarity. However, I feel the ordering of the explanation wrt the text in 2.2 makes the text in 2.2 challenging to understand. I feel it might be good to rethink the ordering of these texts a little?

We have rearranged the overall presentation of the current draft as suggested. 

>Why do you test with both recurrent and linear architecture for hte listener? Why not just choose one, to keep things simple?

The overall behaviour significantly changed according to the architecture of the listener. The optimal levels of compositional pressure is different for a recurrent listener than a linear listener. Moreover, even without external pressure, the relationship between compositionality and test accuracy is dependent on the type of the listener used. Hence, in addition to a recurrent listener we carried our experiments on linear listeners too.

> find figure 1 hard to read: I think we care about seeing the effect of $C_t$? Therefore, strong preference for the x axis to be $C_t$ and for the runs, rather than showing training curves, why not use early stopping somehow (or just train always for 100 epochs, as you state in the text), and report the value either at early stopping, or at 100 epochs? Then you can draw a single curve of test accuracy vs C_t. And you can put all six curves, for each different message space, on the same set of axes. I believe this would be much more intuitive for me to understand/read.

We thank the reviewer for their comments for improving the presentation of the paper. We have redo the graph and included all experiment results with their confidence interval in Appendix B.

We have included experiments with other regularizers; weight decay and dropout to the new draft. In addition, we have corrected the typos and have improved the presentation of the paper.

 > we observe natural languages are compositional and humans use natural language to express complex concepts, which hints about possible advantages of compositionality. Secondly, there have been studies, using methods motivated by language evolution, that show the generalizing advantage of compositional languages.

Right. I'm much more interested in understanding how evolution and such gave rise to compositionality. It appears that in our artificial experiments, it emerges only very weakly, for reasons unclear. I'm personally interested in why it doesn't emerge naturally in our experiments, and what needs to change so that it does. However, directly postulating a pre-existing loss function that encourages compositionality seems problematic to me. One would have to then explain where this loss function came from.

In emergent communication literature in general, one supposes some kind of collaborative task (occasionally non-collaborative, or more ambiguous, but usually at least somewhat collaborative), where agents gain a reward for completing the task.

This all seems biologically plausible to me. We can imagine a pair of creatures hunting together, and needing to collaborate in order to hunt effectively. Those who collaborate more effectively kill the prey with higher probability, and are thus both more likely to survive having food), and to reproduce (staying alive, and perhaps being attractive to potential partners).

However, the loss you are adding has no obvious biological plausibility. By what means are you motivating it?

Ok, so you say, paraphrasing "but entropy regularization". Well, yes, that's a challenging point to refute I agree. However, one can see such a thing as being part of the learning/optimizing process (I'm thinking this through... I might end up agreeing with you, unclear a this point...). And one can see the learning/optimizing process as having evolved over time. It's a very general 'thing' that could have evolved in order to learn more basic things, like walking, flying etc.

So, you could argue "but pressure for compositionality" could have evolved too. Yes, but by what kind of mechanism? Not from learning to fly, nor by learning to walk. One would have to posit an advantage to something else. Communicating perhaps? In that case, I think I'd be interested in seeing an experiment that somehow simulates an environment that causes this loss function for pressure on compositionality to emerge naturally.

So, you could argue, "well babysteps. first we should show that a loss function that encourages compositionality shows a benefit. Later on we can make a new paper that shows that pressure to communicate is sufficient for such a loss function to evolve, naturally." And that would be a fair point too.

So, let me open your paper again a sec....

Well, you aren't arguing this case. In fact, you are arguing that your technique is better than evolution. You are positioning your technique as a competitive technique, better than evolution. So, you are not making the argument that I just wrote in your favor. So, I feel you need to write your own argument then. There are no 'rules' as such in emergent communication domain, no-one has rigorously defined what are the 'goals', which are fairly nebulous I feel (there is perhaps an opportunity for someone to write a work synthesizing the various goals; perhaps I can do that sometime...). Still, I feel there needs to be more motivation than just 'score higher on compositional metrics, by adding compositional metrics into the loss'?


 First, the authors highly thank the reviewer, for their invaluable comments to improve the paper, and their quick response to our previous comments. We will address reviewers' concerns below.

>adding topological similarity as a loss term seems to me to fly in the face of the goals of emergent communication, which is to allow the agents to freely generate their own language, and then to evaluate the resulting language? This is not to say that the approach espoused is not useful, but I feel the motivation provided seems unclear to me?

In the current draft, we only measure compositionality for the cases where we do not employ any type of compositionality loss on the discrete link, which we denote as the baseline scenario. For this case, we study how generalization and compositionality relate, and find that there is indeed a strong but situational correlation.

In the case, where we use compositionality loss, we demonstrate the addition of compositionality loss, if used in controlled amounts, can improve generalization and transmission speed than the baseline cases. We do not intend to highlight the correlation between test accuracy and topological similarity, for this case, since both are now being optimized simultaneously.
Our loss is not based on a mechanism that is seen in the human language evolution process, but we do not think that adding a loss does violate the existing practices of language emergence research. For example, there are several works which employ different types of metrics, which measure quality of communications in a human perspective, on the emerging communications .

Studies have used various losses on emerging messages like; entropy regularization, loss on message length, loss on symbol usage, and selecting messages most understandable to the sender itself [1,2,3]. And humans, like how they try to use the shortest message, use compositional expressions not atomic messages. Hence, we think that adding a compositionality loss, which does not require any additional examples, is not against the existing research. As we understand, the main expectation of language emergence is to develop languages in a grounded manner without explicit supervising examples. In our environment, this property is not violated or removed, and agents still develop communications out of necessity.

We are motivated to induce compositionality for two reasons; First, we observe natural languages are compositional and humans use natural language to express complex concepts, which hints about possible advantages of compositionality. Secondly, there have been studies, using methods motivated by language evolution, that show the generalizing advantage of compositional languages. In this research, we replace the generational transmission mechanisms with a loss which can be directly optimized, which does not need additional training stages.

1. Katrina Evtimova, Andrew Drozdov, Douwe Kiela, and Kyunghyun Cho. Emergent communication in a multi-modal, multi-step referential game. In 6th International Conference on Learning Representations, 2018.


2. Edward Choi, Angeliki Lazaridou, and Nando De Freitas. Compositional obverter communication learning from raw visual input. In 6th International Conference on Learning Representations, 2018.


3. Angeliki Lazaridou, Karl Moritz Hermann, Karl Tuyls, and Stephen Clark. Emergence of linguistic communication from referential games with symbolic and pixel input. In 6th International Conference on Learning Representations, 2018.

>experiments do not appear to me to eliminate the possibility that the role of the compositional pressure is purely as a regularizer. e.g. can the results be reproduced by using other regularizers such as dropout, L1 weight reg, or L2 weight reg?
We have included additional experiments as requested by the reviewer for the cases of dropout and weight decay.

| M(W,L)   | Dropout = 0.5     | Dropout = 0.7  
|----------|-------------------|-------------------|
| M(10,4)  | 0.478 $\pm$ 0.040 | 0.499 $\pm$ 0.002 | 
| M(10,6)  | 0.455 $\pm$ 0.108 | 0.498 $\pm$ 0.001 | 
| M(100,2) | 0.811 $\pm$ 0.365 | 0.512 $\pm$ 0.012 |
| M(100,4) | 0.879 $\pm$ 0.013 | 0.501 $\pm$ 0.009 |

With the experiments with dropout we did not see any improvements over the baseline case for any configuration. In fact the displayed results are less than the baseline scenario. Increasing the dropout has increased the performance for the first two message spaces but for the last two cases, results have been degraded. For these configurations,  even with a very high compositional pressure ($C_t = 1$), test accuracy is still higher than the results gained with dropout = 0.7.




 First authors highly thank the reviewer for their valuable comments.We will address the reviewers concerns in below.
>On the other hand the paper cannot be read and understood standing alone as a self-contained piece of work. The entire discussion is heavily dependent on assumed knowledge of the topic, the broader area, and the usual practices of such simulation experiments. It does not try to explain what is the purpose of training, what is supposed to happen in testing, and what composition and generalization mean in this context. Therefore the results, even if it is strong enough to be useful, appeal only to those closely following the topic.

We have included broader explanations to the current draft including intuitive explanations. Below we will further explain the concepts of compositionality and generalization. 

1. Compositionality :- Agents map environmental entities or concepts with symbols while reaching a consensus for the mapping. If agents follow compositionality, agents would connect individual symbols to atomic concepts in the environment, and combine multiple symbols to form composite messages for expressing complex novel concepts, facilitating generalization. For example, \emph{Red Square} is a compositional expression, containing attributes \emph{Color} and \emph{Shape}, constructed by using atomic concepts \emph{Red} and \emph{Square}, which corresponds to the values taken by the two attributes. Hence, whenever an agent observes a novel input, it can break down the input into a set of attributes and corresponding values. An expression describing the input can be constructed by suitably concatenating the symbols representing each value of the attributes.
2. Generalization :- In this context, generalization means how agents could communicate about unseen combinations of attributes and values. It can be measured by test accuracy similar to other cases in machine learning.

The main purpose of training is to steer agents to make their communications compositional. We do this by incorporating a loss term into the original reconstruction loss. Loss is based on the topological similarity, which is a well studied metric of compositionality,  between input space and message space. Our loss term does not require any additional stages to induce compositionality in communications as other methods, which are based on evolutionary linguistics. 

In testing we expect, and in experiments we show, that when agents trained with non-zero external compositional loss, they can converge to higher test accuracies. Moreover, if agents are trained with large compositional losses they emerge languages that are significantly easier to acquire by new learners. 

>The pairing between the object space and the message space is said to be done with some matrices but it is not clear how such matrices are constructed..

We have added additional explanations on the matrices into the current draft. Let's assume three inputs $x1, x2, x3 $ and corresponding messages $m1, m2, m3$.  First, all pairs of inputs and messages are obtained:- $[(x1,x2), (m1,m2)], [(x1,x3), (m1,m3)], [(x2,x3), (m2,m3)]$. Next, cosine distance between each pair can be calculated as: $[1-cos(x1,x2), 1-cos(m1,m2)], [1-cos(x1,x3), 1-cos(m1,m3)], [1-cos(x2,x3), 1-cos(m2,m3)].$ Then the distances of each pair is as follows, $[d(x1,x2), d(m1,m2)], [d(x1,x3), d(m1,m3)], [d(x2,x3), d(m2,m3)]$. The topological similarity is defined as the pearson correlation between the two distance vectors : $<d(x1,x2),d(x1,x3),d(x2,x3)> $ and $<d(m1,m2),d(m1,m3),d(m2,m3)>$ . The expression $[1-cos(Ah0, Bh0)]$ and [$1-cos(Am, Bm)] $ directly provide the above two vectors.  

The matrices A and B in the paper permutate the inputs and their messages to construct every possible pairwise formation which is later converted to cosine distances. Topological similarity measures how the distances between points in the input space correlate with the distances of the corresponding points in the message space. 

Lets assume three inputs $X = <x_1, x_2, x_3>^T$, and their corresponding messages $M = <m_1, m_2, m_3>^T$, in a situation where the batch size is equal to $3$. Then we define the matrices $\mathbf{A}$ and $\mathbf{B}$ as follows.

\begin{equation*}
\mathbf{A} = 
\begin{pmatrix}
1 & 0 & 0 \\\\
1 & 0 & 0 \\\\
0 & 1 & 0
\end{pmatrix}
\end{equation*}

\begin{equation*}
\mathbf{B} = 
\begin{pmatrix}
0 & 1 & 0 \\\\
0 & 0 & 1 \\\\
0 & 0 & 1
\end{pmatrix}
\end{equation*}

Which yields pairing vectors as follows,

\begin{equation*}
\mathbf{A}X = X_A = <x_1, x_1, x_2>^T 
\end{equation*}
\begin{equation*}
\mathbf{B}X = X_B = <x_2, x_3, x_3>^T 
\end{equation*}
\begin{equation*}
\mathbf{A}M = M_A = <m_1, m_1, m_2>^T 
\end{equation*}
\begin{equation*}
\mathbf{B} = M_B = <m_2, m_3, m_3>^T
\end{equation*}

Using vectors $X_A, X_B$, cosine distance between each input pair,

For inputs,
\begin{equation*}
d_x = 1 - cos(X_A - X_B)
\end{equation*}

Distance for outputs can be calculated similarity


The paper introduces a novel loss term, which encourages high topological similarity, in the context of emergent communication. The paper hypothesizes that increasing the topological similarity of utterances from the two agents will increase the generalization ability of the agents to unseen inputs.

The paper presents evidence that mmoderate amounts of compositional pressure do improve the generalization ability, but large amounts of compositional pressure negatively affect the generalization ability.

The paper goes on to show evidence that the correlation between topological similarity of utterances and the generalization ability is complex and not always clear-cut. ## Good points

- investigates the use of topological similarity as a loss term, in the context of emergent communication, which loss term might be novel
- presents additional data points showing that the link between topological similarity and generalization is tenuous

## Bad points

- adding topological similarity as a loss term seems to me to fly in the face of the goals of emergent communication, which is to allow the agents to freely generate their own language, and then to evaluate the resulting language? This is not to say that the approach espoused is not useful, but I feel the motivation provided seems unclear to me?
- no code provided
- experiments do not appear to me to eliminate the possibility that the role of the compositional pressure is purely as a regularizer. e.g. can the results be reproduced by using other regularizers such as dropout, L1 weight reg, or L2 weight reg?

## Notes

I wrote the following notes whilst reading through the paper

### Abstract

interesting idea to incorporate metric of compositionality into learning
- doesnt this go against the idea of letting languages emerge naturally, and then evaluating them?
- what is the intention of doing this?
- as far as generatlization vs compositoinality, there is existing work that shows this, eg baroni, kharitonov, andreas

### 1. Introduction

obverter reference seems incorrect to me: I believe you are thinking of Choi et al 2018, not Lazararidou et al 2016?

The introduction seems to me well-written, and fairly complete, on the whole.

However, given the clear knowledge of prior work expressed in the introduction, I'm unclear why the observation that there is no simple correlation between generalization and compositionality is novel? There are multiple works (eg Chaabouni et al 2020, which you cited), which show this already?

Contributions 2 and 3 appear to be highly related to contribution 4.

Using a compositional metric to guide learning seems to me to be novel, but also seems to me to be somewhat against the spirit of emergent communication? i.e. the goal of emergent communication appears to me to be to let languages emerge, and then see what kinds of languages emerge; rather than to add our opinion of ideal languages into the loss function? Nevertheless, it does appear to me to be novel to add the compositional metric into the loss function. Measuring the same metric as our output seems somewhat circular, but measuring generalization ability seems a plausibly reasonable thing to do, to me.

### 2. Language emergence game

I thought the paragraphs were going in the direction of using non-symbolic input, so I was a bit disappointed to find their arguing in favor of symbolic inputs.

#### 2.1 setup

Nice that you are using the recommended notation for sets for ICLR :)

#### 2.2 agents

I don't understand what you are pairing. You suddnely talk about it, out of nowhere. From my understanding, for each example, we take a single object, x, which is sampled from N_A attributes, each with N_V possible values. We feed this through our network, which predicts what was the original x. There is no negative sampling involved. I'm not sure I understand what the pairs relate to. Please could you explain the pairing motivation more clearly?

## 3. Pressure for compositionality

### 3.1 Metrics for compositionality

Ok, this bit explains taht the pairwise is for topological similarity. However, I feel the ordering of the explanation wrt the text in 2.2 makes the text in 2.2 challenging to understand. I feel it might be good to rethink the ordering of these texts a little?

Interesting observation that Pearson correlation is differentiable.

## 4. Experiments

Figure 1: I feel that 1e-6 should be in between 0.0 and 0.1 in the legend?

Why are the random seeds 1,3,5,7,11,13? why not just choose 1,2,3,4,5,6?

I'm not sure what it means for an exampoe to be 'up to five times the size of the input space'?

I believe 'thrive' might be intended to be 'strive'?

Figure 1: you are dnoeted message space size as eg (10,4), but I believe per your notation, defined in 2.1, this hsould be written M(10, 4)? This would alo reduce my confusion, since I initially assumed taht (10,4) was input space size, not message space size.

$C_t$ is not defined in the text. It's implicitly defined in algorithm 1. It's obviously the coefficient in front of the topological similarity regularization term, however I think it would be good to state this explicitly in the text?

I find figure 1 hard to read: I think we care about seeing the effect of $C_t$? Therefore, strong preference for the x axis to be $C_t$. and for the runs, rather than showing training curves, why not use early stopping somehow (or just train always for 100 epochs, as you state in the text), and report the value either at early stopping, or at 100 epochs? Then you can draw a single curve of test accuracy vs C_t. And you can put all six curves, for each different message space, on the same set of axes. I believe this would be much more intuitive for me to understand/read.

### 4.1 Generalization

Why do you test with both recurrent and linear architecture for hte listener? Why not just choose one, to keep things simple?

"It is evident from the plots that there is no clear winner for Ct, which is suitable for all the scenarios." => not obvious at all to me tbh. Not without looking really carefully. Please consider redoing the plots ,as alluded to above, so that it is in fact really obvious :)

The graphs are all for linear listener. In reality, in emergent communication literature, almost all listeners I've ever seen are some kind of RNN. I think it might be more meaningful then to just use an RNN listener, and move linear listener results into the appendix?

### 4.2 Compositionality

You compare your compositional loss term to other regularization techniques. How do you know that the term is not functioning simply as a regularization term? For example, what happens if you flip the compositionality term, so that it encourages low compositionality. This would also act as a regularizer, by increasing entropy. Does this also increase test accuracy too.

"Excessive compositional pressure" implies that the compositional pressure is bad, but if I understand correctly, in this context the high compositional pressure is actually good, for test accuracy? So, perhaps use "High compositional pressure" instead of "Excessive compositional pressure"?

Don't your results at M(50,2) align with a hypothesis that the compositionality loss could be acting uniquely as a regularizer, rather than by improving something concrete that correlates with generalization?

"However there is more than enough evidence to accept advantage of the compositional pressure on generalization" => I feel this is overly strong. There is some small amount of evidence, but nuanced by at least 1. high compositional pressure reduces test accuracy, and 2. the hypothesis that it is acting purely as any other regularizer. e.g. what happens if you use other regularizers such as L1 weight norm, L2 weight norm, or dropout? Can you reproduce similar graphs as for using compositional pressure? The paper investigates the use of topological similarity as a loss function to improve the generalization ability of agents, in the context of emergent communication. I feel that this approach is sort of the opposite direction of other emergent communication research, which allows the agents to freely emerge their own language, and then evaluates the resulting language. Thus, I feel it is important to motivate clearly what the goal of the additional loss function is.

The paper does give an additional data point showing that the link between compositionality, as measured by topological similarity, and generalization is tenuous. However, there already exist multiple works that present such evidence, as alluded to in the Related Work section of this work.

The paper shows that the compositional pressure does improve generalization ability, but only in moderation, and that the amount to be used is an empirical question, which depends on many factors, in a complex way. Thus, is it different from any other regularizer? Could a similar effect be observed by using other regularizers such as dropout, or weight decay?

Overall, I feel that I'd like to see the paper delve in a little deeper somehow. For example, comparing to other regularizers, comparing with applying negative compositional pressure. How do agents with naturally high compositionality compare with those which had high compositionality under conditions of high compositional pressure? How do other metrics of compositionality change? Can we apply compositional pressure using other metrics, such as Resnick's residual entropy?


 >One other issue with the paper is that there are no examples at all throughout. The entire discussion about setup, experiments, and results is abstract. Even if the objects, attributes, messages, etc. are all synthetic and symbolic, there should still be a way to show what a message looks like. Without that, the reader has no way to get an intuitive sense of what messages are being exchanged, what language emerges, what it means by generalizing, and how far off the results are. To better convey the ideas, an example should be give as early as in Section 2.1, and followed through in the discussions.

Messages are concatenated one hot encoded vector, each vector has a size equal to the size of the vocabulary. The number of vectors per message is the message length and it is externally controlled. We included a more intuitive explanation into Section 2 in the current draft. 
The existing value attribute environment is frequently used in the language emergence literature and most of the research does not explicitly delve into representing or analyzing the exchanged messages. 

>The fact that the entire work uses only synthetic objects with no connection to the real-world is also a weakness. Given that the effects depend on the structure of the input data, environmental parameters, etc., there should at least be some discussion on what type of real-world scenarios would best match the conditions that the method requires to work well.

According to our understanding, many published studies in this field use abstract or synthetic objects [1,2,3,4], hence we do not think that employing such types of objects in our research goes against the norm. Nevertheless, we can give an example for a real world scenario of a reconstruction game. In a reconstruction game, there are two neural agents namely, speaker and listener. The speaker observes a given input and transmits a message consisting of multiple symbols towards the listener. Listener observes the message, and reconstructs an approximation of the original input, observed by the speaker. Both agents are jointly rewarded according to the degree to which the reconstruction approximates the original input. It can be thought of as a game, where we have two human participants. First participant sees a picture and describes that image to the other one, after hearing what the first participant says, the second participant redraws that image. Both participants obtain a score proportional to what extent the two images match with each other.

1. Cinjon Resnick, Abhinav Gupta, Jakob Foerster, Andrew M Dai, and Kyunghyun Cho. Capacity, bandwidth, and compositionality in emergent language learning. In Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, pp. 1125–1133, 2020.
2. Yi Ren, Shangmin Guo, Matthieu Labeau, Shay B Cohen, and Simon Kirby. Compositional languages emerge in a neural iterated learning model. In International Conference on Learning Representations, 2020.
3. Abhinav Gupta, Cinjon Resnick, Jakob Foerster, Andrew Dai, and Kyunghyun Cho. Compositionality and capacity in emergent languages. In Proceedings of the 5th Workshop on Representation Learning for NLP, pp. 34–38, 2020.
4. Rahma Chaabouni, Eugene Kharitonov, Diane Bouchacourt, Emmanuel Dupoux, and Marco Baroni. Compositionality and generalization in emergent languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4427–4442, 2020.


 None of these changes address the fundamental issue that most reviewers highlighted, including myself, that it is not clear how to motivate adding in a loss using the metric we are trying to measure, in the context of emergent communication. This might make sense in other tasks, where are goal is to achieve the best metric possible, using any available means. But in emergent communication, the goals are a little up in the air, but are in some sense, perhaps, an empirical investigation into trying to recreate human evolution of language from scratch. As wKT2 asks, "Do they propose such a mechanism was present in the evolution of human language?".

I'm not saying that adding this loss in cannot be motivated. But I feel that the current abstract does not provide such motivation.

 Authors thank all reviewers for their valuable comments. We have upload a new draft that have addressed comments by the reviewers. It basically contains the same stance, but the presentation has been improved. New experiments with additional explanations are included in the current draft.

1. The abstract has been updated better explaining the purpose and the intention of the paper.
2. We include sections explaining the intuition behind value attribute environment, our motivation what type of experiments conducted and the expected results in to the introduction.
3. We have included an intuitive explanation to the reconstruction game and reasons for selecting reconstruction game over other settings in to the Section 2.
4. Examples of previous studies using value attribute environment and how to compare such setups with the environment used in this paper (Section 2.)
5. Reasons for using topological similarity as a metric and how the pairing matrices operate, in constructing the input and message pairs (Section 3, Appendix D). 
6. Additional experiments for $X(10,4) input space with recurrent listener.
7. Evidence of situational but strong correlation between generalization and compositionality, and the factors that affect the latter relationship (Section 4.1).
8. Redoing of graphs as requested by the reviewers (Section 4.2)
9. Experimental results with confidence intervals (Appendix B)
9. Experiments comparing performance with other commonly used regularizing techniques (Section 4.2, Appendix C).
10. Experiments conducted with other learning rates and comparison of results (Section 4.2.1)
11. An additional set of experiments comparing the language transmission speed, between instances with different external compositional pressure.
12. Updated conclusion. 

The authors create a language game in which one neural agent describes an item to another.  They study what happens to the language that emerges if an explicit parameter promoting compositionality is altered, considering a range of different environmental factors. The results show an impact of this parameter on the speed at which communication converges and the degree of generalisation of the language I don't understand the purpose of the paper. It seems to fit into the emergence of language field but I cannot work out what the authors are saying. Do they propose such a mechanism was present in the evolution of human language? Or are they finding evidence that rejects this? Are they suggesting it be used in evolving communications in multi-agent environments? To be clear, I cannot see what research question is being asked, and I cannot see what the authors conclude about this question from the evidence they obtain.

The abstract and introduction introduces the general field and makes clear what the experiments involve. As noted above, this fails to articulate what issue the experiments are contributing to.

The relationship to past work is good. There is a sensible quantity of material cited throughout and the material referenced shows how the work fits into the existing material in the area. The bibliography contains a large proportion of papers from arXiv. This is not a peer reviewed source - we have no way of knowing whether claims in these papers meet normal professional criteria. I would guess most of this work actually has been published somewhere - please cite the published version.

The description of the method appears to contain all the necessary details. It is sometimes a little hard to follow. It would be useful to have a non-technical overview in which the basic idea is described without getting bogged down in the details of the networks or using algebra. I didn't follow section 3 very well - I think more detail is needed on Chaabouni et al's measure of compositionality, as it is key to the approach and the paper cannot be fully understood without it.

The experiments seem fine: there is nothing weird in the parameter choices; the factors studied make sense; graphs are clear. There is no comparison to other approaches but this is fine in this work, as it makes no real sense to compare to other methods.

The conclusions do not conclude anything. They are a summary of the experiments and results. They don't tell us what the results mean in terms of any research question. I understand what experiments the authors have done and what the results are. I don't understand what the purpose of performing the experiments is, or what conclusions we should draw based on the results. The research question really needs to be made explicit.

This paper proposes to use a well-studied compositionality metric as an additional auxiliary loss function in a reconstruction task used to study language emergence. In certain cases, this loss function allows for improved generalization. The paper additionally finds that there is no simple correlation in increased weighting of this loss and increased generalization. **Strengths:**

- The paper uses the topological similarity metric [1, 3, 4] commonly used in the emergent communication literature to evaluate compositionality of emergent languages. The key contribution in this paper being the fact that it is used as an additional loss function to explicity tune the pressure for compositionality. To my knowledge, directly optimizing the topological similarity metric has not been studied before which is interesting.
- The paper also includes some evidence that explicitly optimizing this compositionality metric can improve generalization. This insight could be very useful in other NLP settings.
- For the most part, the paper is easy to follow.

**Weaknesses:**

I have included actions that can be taken to strengthen this paper as it stands. However, even with these improvements, it is unlikely that I would recommend the paper to be accepted without drastic changes like including other emergent communication tasks or effects of other types of pressures on emergent languages. 

- **Novelty and Motivation:**
    - Although, to my knowledge, directly optimizing the topological similarity metric has not been studied before, the metric itself is well studied. I am not fully convinced that directly optimizing this metric tells us very much (as is apparent in the experimental section). The paper also bears a lot of similarity to parts of [1].
- **Language Emergence Game and Setup:**
    - The experimental setup is not clear. What are some examples from this dataset? What are some examples of attributes and values? How is it related to other tasks in the literature? Comparisons to the reconstruction task in [2] is made but this does not seem like the same task. It looks similar/identical to the reconstruction task used in [1] but no comparison to that task is made in the paper.
        - **Action:** Improve clarity of the experimental task being used in this paper. Include comparisons to [1].
- **Experiments:**
    - **Figure 1:** The paper claims that carefully tuning the topological similarity metric improves the speed of convergence and generalization. However, this claim is not fully supported due the usage of only 1 learning rate. In particular, 4/6 settings in Figure 1, adding no additional loss performs among the best. In figure 1, a), d), there is evidence that different $C_t$ values performs better but unless additional learning rates and more seeds are used, this claim is not fully supported.
        - **Action:** Run experiments with more learning rates and more seeds.
    - **Figure 2:** I don't this that these plot adds much towards the paper. It only seems to work as a sanity check to see if the implementation is accurate. Optimizing a separate loss would naturally improve that loss. It would be more interesting if a different compositionality metric than what was used during training is used instead.
        - **Action:** Use other compositionality metrics like **positional disentanglement [1].**
    - **Table 1, Table 2:** These tables are missing descriptions and does not depict the metrics being measured. The tables are also missing confidence intervals which makes drawing any conclusion from these tables misleading.
        - **Action:** Add descriptions to the tables and include confidence intervals.
- **Overall:**
    - Paper has numerous typos and punctuation mistakes that make the paper a little difficult to follow in some places.
    - For example, in section 2.1:
        - discreet → discrete.
        - Similar to inputs, each symbols ... → .... symbol ...
    - In section 4.3:
        - ... lower $C_t$ coefficient than $10^6$ → ... $10^{-6}$ ...
    - **Action:** Fix typos and errors

[1] Chaabouni, R., Kharitonov, E., Bouchacourt, D., Dupoux, E., & Baroni, M. (2020). Compositionality and generalization in emergent languages. *arXiv preprint arXiv:2004.09124*.

[2] Gupta, A., Resnick, C., Foerster, J., Dai, A., & Cho, K. (2020, July). Compositionality and Capacity in Emergent Languages. In *Proceedings of the 5th Workshop on Representation Learning for NLP* (pp. 34-38).

[3] Ren, Y., Guo, S., Labeau, M., Cohen, S. B., & Kirby, S. (2020). Compositional languages emerge in a neural iterated learning model. *arXiv preprint arXiv:2002.01365*.

[4] Lazaridou, A., Hermann, K. M., Tuyls, K., & Clark, S. (2018). Emergence of linguistic communication from referential games with symbolic and pixel input. *arXiv preprint arXiv:1804.03984*. This paper proposes to use a well-studied compositionality metric as an additional auxiliary loss function in a reconstruction task used to study language emergence. Due to the lack of rigor in the experimental sections, the claims made in the paper are not well supported. The paper's novelty is also highly incremental to existing works, particularly compared to [1]. As such, I would recommend that the paper is rejected.

The paper is an incremental study that tries to go beyond some
existing baseline in the generalization ability of language learned
through message exchanges between two neural agents (sender and listener).

The proposed method is to add to the optimization objective of the
agents the effect of compositional pressure using a metric called
topological similarity, under a set-up of the game where the primary
objective is reconstruction.  Senders are said to sample from some
message space a certain representation about an object.  The listener
tries to decode the message and reconstruct the object.

Using some synthetic experiments, it is shown that though in some cases
the method has an advantage on generalization, the advantage is not
consistent but depends substantially on the agent architecture,
structure of input data, and parameters of the game environment. The paper is motivated by an ambitious objective of capturing some
observed advantages of leveraging composition in natural language learning.
It also cites many references in the area for prior art.

On the other hand the paper cannot be read and understood standing
alone as a self-contained piece of work.  The entire discussion is
heavily dependent on assumed
knowledge of the topic, the broader area, and the usual practices of
such simulation experiments.  It does not try to explain what is the
purpose of training, what is supposed to happen in testing, and what
composition and generalization mean in this context.  Therefore the results, even if
it is strong enough to be useful, appeal only to those closely
following the topic. 

The pairing between the object space and the message space is said to
be done with some matrices but it is not clear how such matrices are constructed.

One other issue with the paper is that there are no examples at all throughout.
The entire discussion about setup, experiments, results is abstract.
Even if the objects, attributes, messages, etc. are all synthetic and
symbolic, there should still be a way to show what a message looks
like.  Without that, the reader has no way to get an intuitive sense
of what messages are being exchanged, what language emerges, what it
means by generalizing, and how far off the results are.  To better
convey the ideas, an example should be give as early as in Section
2.1, and followed through in the discussions.

The fact that the entire work uses only synthetic objects with no
connection to the real-world is also a weakness.  Given that the
effects depend on the structure of the input data, environmental
parameters, etc., there should at least be some discussion on what
type of real-world scenarios would best match the conditions that the
method requires to work well.   
 The work is a small incremental experiment that probes for the advantage of a perturbed objective in language learning, with some weak results.  It took inspirations from natural language learning but idid not try to connect back to any natural learning scenarios.


Meta-review: This work presents a proposal for increasing the compositionality of emergent languages that uses a measure of topographic similarity as an auxiliary loss function on the communication game. The authors find that in certain cases this loss indeed results in increased generalization but overall the authors do not find a strong relation between high weights for weighting loss and and generalization.
All reviewers agree that the of compositionality is very important and the idea of explicitly optimizing for compositionality (through the topographic similarity metric) is also novel. At the same time a number of concerns are raised by reviewers: 

a) B3Jo and WEY2 raise the point that more evidence is required to establish the robustness of the current findings, e.g., by controlling whether topsim is merely inducing a regularization behaviour and providing more confidence on the current presented results (e.g., Figure 2 currently provides a somewhat perplexed pictured as the additional loss doesn't seem to improve across the board).

b) The relation between compositionality and generalization is not a new one and it is not clear what exactly the current paper is adding on this discussion and, as zB6g and  B3Jo point, this makes it seem rather incremental.

c) the paper is currently somewhat hard to follow with numerous results reported in a somewhat raw format, little to no examples and important details being presented only in Appendix (e.g., the loss function is only given in p12 and the actual format of L_{C} is never provided)

As such, I cannot recommend acceptance at this time but, given the importance of the topic, I sincerely hope the authors will work on incorporating reviewers' feedback for a later resubmission.

Decision: Reject