Question: Why is predictability of the future given the past is important for these communicating agents, considering they are memoryless across episodes?
Evidence: Context Communication between humans always takes place w.r.t. some context. For humans, that context includes: 1) features of the environment that can be directly perceived by the dialogue participants and used for interpreting an utterance (e.g. "cup" is interpreted as an object in the immediate physical context); and 2) information that cannot be directly perceived but can affect the interpretation of utterances (e.g. using and interpreting "cup" assumes some shared knowledge of what counts as a "cup"; once a particular cup has been established as salient in particular dialogue, further utterances of "the cup" or "it" will be interpreted as referring to that specific cup).
Inspired by this conclusion from pragmatics, in the DL-based language games, we define context as the supervision information involved in the calculation of losses, i.e. the context of a specific target x t , denoted as Cpx t q, is the space of samples involved in the calculation of loss. As for in referential games, the cross entropy loss is calculated based on the distance between the message embedding h L pm Bi q and candidate embedding f L cenc px i q where x i P C, thus Cpx t q " C Ď X in referential games.
We assume that the goal of communication is to distinguish a target object from other possibilities in the context (as defined above). It therefore follows that the similarity of distractors in the context to the target influences the communicative precision required, and that greater precision is required to distinguish the target from a more similar distractor. For example, it is relatively easy in natural language to distinguish e.g. a cat from a table (a relatively general label like "cat" or "animal" would suffice), but harder to make fine-grained distinctions between very similar objects e.g. a Scottish Fold Cat and an American Shorthair Cat (a specialist vocabulary or a lengthy description is necessary). Following the assumption (described verbally above) that a context which contains more similar objects makes the game harder because there are fewer unique features that suffice to distinguish the target from the distractors, we first define a neighbour set in k-th degree of a given target x t as N k px t q " tx j : dpx t , x j q ď ku where d is a distance function that can properly capture the similarity between inputs, e.g Hamming distance in our setting. The complexity of Cpx t q is then defined as the expectation of the probability that Cpx t q contains an object from N k px t q, i.e. E xt rg pCpx t q, N k px t qqs ,where g pCpx t q, N k px t qq " " 1, if Dx i P Cpx t q s.t.
In our referential games, since the sampling procedure is independent Bernoulli without replacement, the value of the above expectation is then
¯|C| which is a monotonically increasing function w.r.t |C| and a fixed k. That said, larger contexts are more complex since they are more likely to include items which are very similar to the target.
Unpredictability of context Our notion of unpredictability comes from experimental work with human participants, e.g.. Suppose the aim is to distinguish a striped t-shirt from a number of distractors, and there are two sequences of context: 1) three runs of games where distractors are all cups; 2) three runs of games where distractors are a cup, a plain t-shirt, and a pencil. In the first sequence of games, participants would become certain that "t-shirt" is enough for distinguishing the target, whereas in the second sequence participants would learn that a more overspecified utterance (e.g. "striped t-shirt") is necessary to guarantee comprehension after a failure on the trial involving the plain t-shirt distractor. That is, the context in the first sequence is more predictable than the second. show that human participants are sensitive to this kind of unpredictability, and adapt their communicative strategies accordingly.
In DL-based games, we refer to the context at the e-th trial, i.e. the e-th training epoch, of a target x t as C e px t q. Following the above example, the unpredictability of context is then defined as the proportion of C e`1 px t q that are not from C e px t q, i.e.
In our referential games, since the sampling procedure is independent Bernoulli without replacement, the proportion of objects not from C e px t q in C e`1 px t q is then simply
" 1 ´|C| |X | (i.e. smaller contexts are more unpredictable, since contexts on successive trials are more likely to differ in their composition).
More details about the above discussion are provided in Appendix B.
Unpredictability of context Our notion of unpredictability comes from experimental work with human participants, e.g. Winters et al. (2018). Suppose the aim is to distinguish a striped t-shirt from a number of distractors, and there are two sequences of context: 1) three runs of games where distractors are all cups; 2) three runs of games where distractors are a cup, a plain t-shirt, and a pencil. In the first sequence of games, participants would become certain that “t-shirt” is enough for distinguishing the target, whereas in the second sequence participants would learn that a more overspecified utterance (e.g. "striped t-shirt") is necessary to guarantee comprehension after a failure on the trial involving the plain t-shirt distractor. That is, the context in the first sequence is more predictable than the second. Winters et al. (2018) show that human participants are sensitive to this kind of unpredictability, and adapt their communicative strategies accordingly.
Answer: Predictability of the future given the past is important for these agents, even though they do not store information in an explicit memory module for use across training epochs (i.e. "memoryless across episodes"), because: 1) they store knowledge in their gradients and parameters, 2) because the data is sampled without replacement, 3) the data sampling and training dynamics will affect the learned representations.
We gave in our common reply how our definition of unpredictability follows Winters et al. (2018), which leads us to a meaningful measure of unpredictability of context in our work in emergent communication.
We appreciate that you are aligned with our intuition that “unpredictable environments force the speaker to be maximally informative in its utterances”, and we believe that our proposed metric does provide an informative perspective on this relationship.
We agree with your intuition on the effect of predictability, i.e. there’s an implicit effect brought by the data presentation order in the training dynamics. In fact, introducing unpredictability (i.e. variance in the input and input context space) has also been commonly used in the computer vision community. For example, the preprocessing for images used in deep reinforcement learning tasks can include transformations such as random shifts to make the inputs vary across epochs (e.g Kostrikov et al. 2021). The unpredictability/variance of inputs are not explicitly measured by the online learning methods, but the effect of them still can improve the learned representation, which is shown by the improvement of performance (see Figure 1 from Kostrikov et al. 2021).
To fully understand the implicit effect caused by the data presentation order covers much wider topics than emergent communication. That said,we look forward to exploring these questions in our future work. In this work, we align with previous related work in cognitive science and proposed our definition of unpredictability in the context of emergent communication, as detailed in Section 3.1.
Regarding the “online learnability using prequential coding” (Blier and Ollivier 2018), one challenge of this measure in our context is that it would conflate the learnability of emergent languages with the definition of unpredictability. Learnability is a connected research challenge in emergent communication, for example as investigated by Li and Bowling (2019).
Since we focus on the effect from context on the expressivity of emergent languages, we opted to separate our definition of unpredictability from learnability which is again influenced by both communicating agents and emergent languages. In this work, we focus on the influence of the unpredictability of the contexts from epoch to epoch on expressivity of emergent languages. Similar to learnability, expressivity is also a specific property of emergent languages, which is learned by agents through participation in language games. On the other hand, context is a part of the environment (i.e. part of the definition of the language games), thus its properties and characteristics should be controlled by experimenters as best as possible to understand the influence of agents learning in these environments.
Therefore, we chose a measure of unpredictability and complexity that can separate the effects from the outputs of the language games including the emergent languages and the trained communicating agents.
Reference: [1] Li, F., & Bowling, M. (2019). Ease-of-teaching and language structure from emergent communication. NeurIPS. [2] Kostrikov, I., Yarats, D., & Fergus, R. (2020). Image augmentation is all you need: Regularizing deep reinforcement learning from pixels. arXiv preprint arXiv:2004.13649.