Speaking Your Language: Spatial Relationships in Interpretable Emergent Communication

Authors: olaf  lipinski
adam j sobey
federico  cerutti
timothy j norman
Venue: NeurIPS_cc_2024_Conference
Published: 2024-04-23 10:37:20

Abstract: Effective communication requires the ability to refer to specific parts of an observation in relation to others. While emergent communication literature shows success in developing various language properties, no research has shown the emergence of such positional references. This paper demonstrates how agents can communicate about spatial relationships within their observations. The results indicate that agents can develop a language capable of expressing the relationships between parts of their observation, achieving over 90% accuracy when trained in a referential game which requires such communication. Using a collocation measure, we demonstrate how the agents create such references. This analysis suggests that agents use a mixture of non-compositional and compositional messages to convey spatial relationships. We also show that the emergent language is interpretable by humans. The translation accuracy is tested by communicating with the receiver agent, where the receiver achieves over 78% accuracy using parts of this lexicon, confirming that the interpretation of the emergent language was successful.

Main paper: 
 spatial referencing in emergent communication: 
 Emergent communication allows agents to develop bespoke languages for their environment. While there are many successful examples of efficient (Rita et al., 2020) and compositional (Chaabouni et al., 2020) languages, they often lack fundamental aspects seen in human language, such as syntax (Lazaridou and Baroni, 2020) or recursion (Baroni, 2020) . It is argued that these aspects of communication are important to improve the efficiency and generalisability of emergent languages (Baroni, 2020; Boldt and Mortensen, 2024; Rita et al., 2024) . However, the current architectures, environments, and reward schemes are yet to exhibit such fundamental properties.One such aspect is the development of deixis (Rita et al., 2024) , which has been described as a way of pointing through language. Examples of temporal deixis include words such as "yesterday" or "before," and spatial deixis include words such as "here" or "next to" (Lyons, 1977) . In emergent communication, Lipinski et al. (2023) investigate how agents may refer to repeating observations, which could also be viewed from the linguistic perspective as investigating temporal deixis. However, while there are advocates to investigate how emergent languages can develop key concepts from human language (Rita et al., 2024) , no work has demonstrated the emergence of relative references to specific locations within an observation, or spatial deixis.Spatial references would be valuable in establishing shared context between agents, increasing communication efficiency by reducing the need for detailed descriptions, and adaptability, by removing the need for unique references per object. For example, instead of describing a new, previously unseen object, such as "a blue vase with intricate motifs on the table," one could simply use spatial relationships and say "the object left of the plate." Spatial referencing streamlines communication by leveraging the shared environment as a reference point. In dynamic environments where objects might change positions, spatial references enable agents to easily track and refer to objects without having to update their descriptions. This enhances communication efficiency and improves interaction and collaboration between agents. These elements may also help the evolved language become human interpretable, allowing the development of trustworthy emergent communication (Lazaridou and Baroni, 2020; Mu and Goodman, 2021) . This paper therefore explores how agents can develop communication with spatial references. While Rita et al. (2024) posit that the emergence of these references might require complex settings, we show that even agents trained in a modified version of the simple referential game (Lazaridou et al., 2018; Lewis, 1969) can develop spatial references. 2 This resulting language is segmented and analysed using a collocation measure, Normalised Pointwise Mutual Information (NPMI) adapted from computational linguistics. NPMI allows us to measure the strength of associations between message parts and their context, making it a valuable tool for gaining insights into the underlying structure of the emergent language. Using NPMI, we show how the agents compose such spatial references, providing the first hint of a syntactic structure, and showing that the emergent language can be interpreted by humans. 
development of a spatial referential game: 
 Current emergent communication environments have not produced languages incorporating spatial references. To address this, we present a referential game (Lazaridou et al., 2018) environment where an effective language requires communication about spatial relationships.In the referential game, there are two agents, a sender and a receiver. The sender observes a vector and transmits its compressed representation through a discrete channel to the receiver. The receiver observes a set of vectors and the sender's message. One of these vectors is the same as the one the sender has observed. The receiver's goal is to correctly identify the vector the sender has described, among other vectors referred to as distractors. The simplicity of the referential games enables the reduction of extraneous factors which could impact the emergence of spatial references, such as transfer learning of the vision network or exploring action spaces in more complex environments.In this work, the sender's input is an observation in the form of a vector o = [o 1 , o 2 , o 3 , o 4 , o 5 ], where ∀o ∈ {-1, 0, 1 . . . 59}. The vector o is always composed of 5 integers. The observation includes a -1 in only one position, e.g., o 3 = -1 for o = [x, x, -1, x, x], to indicate the target integer for the receiver to identify. o represents a window into a longer sequence s, which is randomly generated using the integers {0 . . . 59} without repetitions. This sequence is visible to the receiver, but not to the sender. As the target's position in the sequence is unknown to the sender, it has to rely on the relative positional information present in its observation, necessitating the use of spatial referencing.Due to the window into the sequence being of length 5, it is necessary to shift the window when it approaches either extent of the sequence. The window is then shifted to the other side, maintaining the size of 5. For example, given a short sequence s = [7, 5, 2, 12, 10, 4, 3, 15, 16, 13, 14, 6, 9, 8, 11 , 1], if the selected target is 1, since there are no integers to the right of 1 the vector o would be o = [6, 9, 8, 11, -1] where it is shifted to the left as it approaches this rightmost extent of the sequence.Due to the necessity of maintaining the window size, some observations provide additional positional information to the sender agent. Given the same example sequence s, we can categorise all observations into 5 types. The begin and begin+1, where the target integer is either at, or one after, the beginning of the sequence, i.e., o = [-1, 5, 2, 12, 10] or o = [7, -1, 2, 12, 10] . The end and end-1, where the target integer is either at, or one before, the end of the sequence, i.e., o = [6, 9, 8, 11, -1] or o = [6, 9, 8, -1, 1]. The most common case is the middle observation, where the target integer is anywhere in the sequence, excluding the first, second, second to last, and last positions, e.g., o = [12, 10, -1, 3, 15] . Given a window of length 5, only 4 specific target integer positions per sequence can result in the other observations (begin, begin+1, end-1, and end). All other target integer positions within the sequence fall into the middle category, as they do not occupy the first, second, second to last, or last positions. Consequently, the majority of the target integer positions result in a middle type observation. The sender's output is a message defined as a vector m = [m 1 , m 2 , m 3 ], where m ∈ {1 . . . 26}. 26 is chosen to allow for a high degree of expressivity, with the agents being able to use over 17k different messages, while also matching the size of the Latin alphabet. Since such a vocabulary size is enough to convey any information in natural languages like English, we consider that this should also apply to the agents. The vector m is always composed of 3 integers. The receiver's input is an observation consisting of three vectors: the sender's message m, the sequence s, and the set of distractor integers together with the target integer td. The distractor integers are randomly generated, without repetitions, given the same range of integers as the original sequence s, i.e., {0 . . . 59}, excluding the target object itself. Given an environment with 3 distractors, td could be [d 1 , t, d 2 , d 3 ], where t is the target object and d 1 , d 2 , d 3 are distractor objects. The position of the target object in td is randomised.For example, given the sequence s = [7, 5, 2, 12, 10, 4, 3, 15, 16, 13, 14, 6, 9, 8, 11, 1] , and the sender's observation o = [4, 3, -1, 16, 13], the vector td could be td = [7, 15, 11, 9] , with 15 being the target that the receiver needs to identify. The sender could produce a message m = [3, 1, 1], which would mean that the target integer is one after the integer 3. This message would then be passed to the receiver, together with s and td. The receiver would then have to correctly understand the message m (i.e., that the target is one after 3) and find the integer 3 together with the following integer in the sequence s. Having identified the target 15 given the message m and the sequence s, it would output the correct position of this target in the td vector, i.e., 2, since td 2 = 15.To provide a generalisation of our results, we formalise what we refer to as spatio-temporal references. Let O represent an abstract observation that an agent perceives from its environment, O ∈ R m , where m represents the dimensions of the observation. For a 3D observation, m could be m = j × k × d. Such an m could represent a j × k matrix of d = 3 values, which, for example, could be an RGB picture, with j × k pixels and one value for each of the RGB colours (d = 3). The m dimensions can represent the spatial, temporal, or other positions.Let O p and O t be the coordinates of some elements in O, represented by an m-tuple of natural numbers (x 1 , x 2 ...x m ) and (y 1 , y 2 ...y m ), respectively. O p represents the reference point and O t represents a target point.Then, the relative distance function d(O p , O t ) returns an m-tuple of integers (z 1 , z 2 ...z m ), such that z i = x i -y i . This relative distance function allows for unambiguous identification of the target object O t , given that the position of O p is known.We define the spatio-temporally referent expression as a mapping of the value of d(O p , O t ), the reference point O p , and their context O, to a specific linguistic or symbolic phrase that describes the relationship between O p and O t . This mapping can be represented as:(O, d(O p , O t ), O p ) → Phrase(O, d(O p , O t ), O p )where the resulting expression Phrase(O, d(O p , O t ), O p ) is a description of the reference point O p and its relative distances to the target point O t , given the context O.The version of spatial referencing in our environment is a specific case of the general spatial reference formalisation, where the observation O is represented as a one-dimensional tensor, and the target point O t is always indicated by the value -1 within the tensor. The sender's task is to describe the relative position of the target O t within this sequence, using a message that effectively communicates the spatial relationship between a chosen O p and the target O t .Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: A discussion of the limitations is provided in Section 7.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
agent architecture: 
 The agent architecture follows that of the most commonly used EGG agents (Kharitonov et al., 2019) . This architecture is used to maintain consistency with the common approaches in emergent communication research (Chaabouni et al., 2019 (Chaabouni et al., , 2020;; Kharitonov et al., 2019; Lipinski et al., Target and Distractors [5, 10, 7, 3] Full sequence [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] GRU 2 (Combine) (b) The receiver architecture.Figure 1 : The sender and receiver architectures. Adapted from (Lipinski et al., 2023) . Ueda and Washio, 2021) , increasing the generalization of the results presented in this work. All environmental observations, i.e., o, s, and td, are passed in as scalars, as one-hot encoding of the observation vectors leads to agents memorising the dataset.The sender agent, shown in Figure 1a , receives a single input, the vector o, which is passed through the first GRU of the sender. The resulting hidden state is used as the initial hidden state for the message generation GRU (Cho et al., 2014) . The message generation GRU is used to produce the message, character by character, using the Gumbel-Softmax reparametrization trick (Jang et al., 2017; Kharitonov et al., 2019; Mordatch and Abbeel, 2018) . The sequence of character probabilities generated from the sender is used to output the message m. m is input to the receiver agent, shown in Figure 1b , together with the full sequence s and the target and distractors td. The message is processed by the first receiver GRU, which produces a hidden state used as the initial hidden state for the GRU processing the sequence s. This is the only change from the standard EGG architecture (Kharitonov et al., 2019) . This additional GRU allows the receiver agent to process the additional input sequence s, using the information contained within the message m. The goal of this GRU is to use the information provided by the sender to correctly identify which integer from the sequence s is the target integer. The final hidden state from the additional GRU is multiplied with an embedding of the targets and distractors, to output the receiver's prediction. This prediction is in the form of the index of the target within td.Following the commonly used approach (Kharitonov et al., 2019) , agent optimisation is performed using the Gumbel-Softmax reparametrization (Jang et al., 2017; Mordatch and Abbeel, 2018) , allowing for direct gradient flow through the discrete channel. The agents' loss is computed by applying the cross entropy loss, using the receiver target prediction and the true target label. The resulting gradients are passed to the Adam optimiser and backpropagated through the network.Detailed training hyperparameters are provided in Appendix A.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: The code for the experiments is provided on GitHub (Footnote 2), together with the instructions on reproducing the paper's results.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
message interpretability and analysis using npmi: 
 To analyse spatial references in emergent language, a way to identify their presence is essential. In discrete emergent languages, interpretation is typically done by either using dataset labels in natural language (Dessì et al., 2021) , or by qualitative analysis of specific messages (Havrylov and Titov, 2017) . However, both of these techniques require message-meaning pairs, and so neither would be able to identify the presence of spatial references, as the labels for spatial relationships that the agents refer to would not necessarily be available. One approach that could overcome this problem is emergent language segmentation using Harris' Articulation Scheme, recently employed by Ueda et al. (2023) . Ueda et al. (2023) compute the conditional entropy of each character in the emergent language, segmenting the messages where the conditional entropy increases. However, even after language segmentation, there is no easy way to interpret the segments, as no method has been proposed to map them to specific meanings.We present an approach to both segment the emergent language and map the segments to their meanings. We use a collocation measure called Normalised Pointwise Mutual Information (NPMI) (Bouma, 2009) , often used in computational linguistics (Lim and Lauw, 2024; Thielmann et al., 2024; Yamaki et al., 2023) . It is used to determine which messages are used for which observations and to analyse how the messages are composed, including whether they are trivially compositional (Korbak et al., 2020; Perkins, 2021; Steinert-Threlkeld, 2020) . By applying a collocation measure to different parts of each message as well as the whole message, we can address the problems of both segmentation and interpretation of the message segments. This approach allows any part of the message to carry a different meaning. For example, if an emergent message contains segments that frequently appear in contexts involving specific integers, NPMI can help identify these segments and their meanings based on their statistical association with those integers.NPMI is a normalised version of the Pointwise Mutual Information (PMI) (Church and Hanks, 1989) , which is a measure of association between two events. PMI is widely used in computational linguistics, to measure the association between words (Han et al., 2013; Paperno and Baroni, 2016) . Normalising the PMI measure results in its codomain being defined between -1 and 1, with -1 indicating a purely negative association (i.e., events never occurring together), 0 indicating no association (i.e., events being independent), and 1 indicating a purely positive association (i.e., events always occurring together). Normalised PMI is used for convenience when defining a threshold at which we consider a message or n-gram to carry a specific meaning, as the threshold can be between 0 and 1, instead of unbounded numbers in the case of PMI. 3To determine which parts of each message are used for a given meaning, two algorithms are proposed.1. PMI nc The algorithm to measure non-compositional monolithic messages, most often used for target positional information (e.g., begin+1 (Section 2)); and 2. PMI c the algorithm to measure trivially compositional messages and their n-grams, used to refer to different integers in different positions.A visual representation of the different types of messages that the algorithms can identify is provided in Figure 2 . The PMI nc algorithm can identify any non-compositional messages, while the PMI c algorithm identifies both position variant and invariant compositional messages. The positional variance of the emergent language means that the position of an n-gram in the message also carries a part of its meaning. In this work, n-grams refer to a contiguous sequence of n integers from the sender's message. Consequently, in one message there are3 unigrams (m 1 , m 2 , m 3 ), two bigrams ([m 1 , m 2 ], [m 2 , m 3 ]), and one trigram (i.e., the whole message [m 1 , m 2 , m 3 ]).Figure 2 shows that in the position invariant case, the bigram [5, 6] always carries the meaning of 4. While in the position variant case, the bigram [5, 6] in position 1 of the message means 4, but [5, 6] in position 2 of the message means 8. This can also be interpreted as the position of the bigram containing additional information, meaning a single "word" could be represented as a tuple of the bigram and its position in the message, as both contribute to its underlying information.Non-compositional messages are monolithic, i.e., the whole message carries the entire meaning. For example, message [5, 6, 8 ] means the target is in the first position, while [5, 6, 6] means the target is one to the right of 9, even though the two messages share the bigram [5, 6] .The PMI nc algorithm The PMI nc algorithm calculates the NPMI per message by first building a dictionary of all counts of each message being sent, together with an observation that may provide positional information (e.g., begin+1) or refer to an integer in a given position (e.g., 1 left of the target).The counts of that message and the counts of the observation, including the integer position, are also collected. For example, consider the observation o = [4, -1, 15, 16, 13] . For the corresponding message m, the counts for each integer in each position relative to the target would increase by 1 (i.e., lef t1[4]+ = 1, right1[15]+ = 1 etc.). The count for the message signifying begin+1 would also be increased. Given these counts, the algorithm then estimates the probabilities of all respective events (messages, positional observations, and integers in given positions) and calculates the NPMI measure.Figure 2 : Examples of the different types of message compositionality that are possible to identify using the PMI algorithms.The PMI c algorithm The PMI c algorithm first creates a dictionary of all possible n-grams, given the message space (m) and maximum message length (3). The list of all possible n-grams is pruned to contain only the n-grams present in the agents' language, avoiding unnecessary computation in the later parts of the algorithm. Given the pruned list of n-grams, the algorithm checks the context in which the n-grams have been used. The occurrence of each n-gram is counted, together with the n-gram position in the messages and the context in which it has been sent, or the integers in the observation. The n-gram position in the message is considered to account for the possible position variance of the compositional messages.Consider the previous example, with o = [4, -1, 15, 16, 13] and a message m = [11, 13, 5] . For all n-grams ([11], [13], [5] , [11, 13] , etc.) of the message, all integers are counted, irrespective of their positions (i.e., counts[4]+ = 1, counts[15]+ = 1, etc.).Given these counts, the PMI c algorithm estimates the NPMI measure for all n-grams and all integers in the observations. These probabilities are estimated from the dataset using the count of their respective occurrences divided by the number of all observations/messages.Once the NPMI measure is obtained for the n-gram-integer pairs, the algorithm calculates the NPMI measure for n-grams and referent positions or the positions of the integer in the observation the message refers to. For example, given an observation o = [4, -1, 15, 16, 13], if the message contains an n-gram which has been identified as referring to the integer 15, the rest of the message (i.e., the unigram or bigram, depending on the length of the integer n-gram) is counted as a possible reference to that position, in this case, to position right1, or 1 to the right of the target. This procedure follows for all messages, building a count for each time an n-gram was used together with a possible n-gram for an integer. These counts are used to calculate the NPMI measure for n-gram and position pairs.The PMI c algorithm also accounts for the possible position invariance of the n-grams, i.e., where in the message the n-gram appears. This is achieved by calculating the respective probabilities regardless of the position of the n-gram in the message, by summing the individual counts for each n-gram position.Pseudocode We provide a condensed pseudocode for both algorithms in Algorithm 1. In the case of the PMI nc , the n-grams in the pseudocode would be whole messages, i.e., trigrams. This base pseudocode would then be duplicated, interpreting the context as either an observation that may provide positional information (e.g., begin+1) or an integer.For the PMI c algorithm, only the unigrams and bigrams would be evaluated. The base pseudocode would also be duplicated, once for the integer in a given position, and second for the referent position.Each would be used as the context in which to evaluate the NPMI for each n-gram. A detailed commented pseudocode for both the PMI nc and PMI c algorithms is available in Algorithm 2 and Algorithm 3 in Appendix D, respectively.Both algorithms use two hyperparameters: a confidence threshold t c and top_n t n . The confidence threshold refers to the value of the NPMI measure at which a message or n-gram can be considered to refer to the given part of the observation unambiguously. To account for polysemy (where one symbol can have multiple meanings), the agents can use a single n-gram to refer to multiple integers.Algorithm 1: PMI Algorithm Base Gather ngram_counts, context_counts, joint_counts, n_grams; for each n-gram g in position p and context c doP (g, p) = ngram_counts[g] • 1 total n-grams ; P (c) = context_counts[c] • 1 total contexts ; P (g, p; c) = joint_counts[(g, c)] • 1 total n-grams ; NPMI(g, p; c) = log 2 P (g, p, c) P (g)P (c) • 1 -log 2 P (g, p, c) ; end return NPMI;This is given by the second hyperparameter, top_n, which sets the degree of the polysemy, or the number of integers to be considered for a given n-gram. 
spatial referencing experiments: 
 The agent pairs are trained over 16 different seeds to verify the results' significance. All agent pairs achieve above 98% accuracy on the referential task, showing that the agents develop a way to communicate about spatial relationships in their observations. The analysis provided in this section is based on the messages collected from the test dataset after the training has finished.The two hyperparameters, t c and t n (Section 4), governing the NPMI measure have been determined through a grid search to maximise the understanding of the emergent language, by maximising the translation accuracy. The results in this section are obtained using the best-performing values for each of the hyperparameters. We provide the values for the grid search in Appendix A.Using the PMI nc algorithm, we detect the emergence of messages tailored to convey the positional information contained in the observations. As mentioned in Section 2, sender observations which require shifting convey additional information about the position of the target within the sequence. In over 90% of agent pairs, these observations are assigned unique messages, used only for each kind of observation, i.e., begin, begin+1, end-1 and, end.In 20% of runs which develop these specialised messages, the same repeating character is used to convey the message. The characters used for these observations are reserved only for these kinds of observations. For example, in one of the runs the agents use character 11 to signify the beginning of the sequence, with the character 11 being used only in two contexts: as the messages [11, 11, 11] to signify begin, or as a message [0, 11, 11] to signify begin+1. In other cases, characters are fully reserved for specific messages. e.g., 22 is used only for end, in the message [22, 22, 22] .The emergence of non-compositional references used for other observations is also detected using the P M I nc algorithm. Such messages refer to a specific integer in a specific position of the sender observation, e.g., o 5 = 10. While we allow for polysemy of the message in our analysis using t_n = [1, 2, 3, 5, 10, 15], we observe the highest translation accuracy with t_n = 1, indicating that the non-compositional messages do not have any additional meanings.Using the P M I c algorithm, we also detect the emergence of compositional spatial references for 25% of agent pairs. Such messages are composed of two parts, a positional reference and an integer reference. The positional reference specifies where a given integer can be found in the observation, in relation to the masked target integer -1. The integer reference specifies which integer the positional reference is referring to. For example, one pair of agents has assigned the unigram 7 to mean that the target integer is 2 to the right of the given integer, and the bigram [0, 2] to mean the integer 18. Together, a message can be composed [7, 0, 2], which means that the target integer for the receiver to identify is 2 to the right of the integer 18, i.e., o = [18, X, -1, X, X]. This allows the sender to identify the target integer exactly, given the sequence s.In Table 1 , we summarise the emergence of each type of message across all runs, together with the percentage of the vocabulary that they represent. The entries in the table are composed of average percentages, across all t n and t c choices. In the parentheses, we show the maximum and minimum values across all t n and t c choices. The average % of emergence represents the absolute % of runs which developed that message type or message feature. For all messages, the average % of messages which are of a given type or exhibit a given feature is only counted for in runs where these features emerged.To ensure the validity of our message analysis, we present two hypotheses which, if supported by the results, would indicate that the mappings generated by the NPMI measure are correct.Hypothesis 1 (H1) If the correlations exist and do not require non-trivial compositionality (Perkins, 2021) , and are not highly context-dependent (Nikolaus, 2023) , then the evaluation accuracy should be significantly higher than chance, or above 20%, when using the identified mappings. Hypothesis 2 (H2) If the positional components of compositional messages are correctly identified and carry the intended meaning, then their inclusion should result in an increase in accuracy.Given the messages identified by the NPMI method, we test H1 and H2 by using a dictionary of all messages successfully identified, given a value of both NPMI hyperparameters t n and t c . A dataset is generated to contain only targets which can be described with the messages present in the dictionary.For the non-compositional messages, the dataset is generated by selecting a message from the dictionary at random, and creating an observation that can be described with that message. Given a non-compositional message that corresponds to the target being on the right of the integer 15, an observation o = [1, 15, -1, 5, 36] would be created. Analogously, for non-compositional positional messages such as begin an observation o = [-1, 15, 8, 5, 36] would be created.For the compositional messages, we create the observations by randomly selecting a positional component and an integer component from the dictionary. For example, given the unigram 7 meaning that X is 2 to the left of the target, we could select the bigram [8, 14] corresponding to the integer 30. The observation created could then be o = [30, 8, -1, 36, 5] . The dataset creation process for the compositional messages also checks if the observations can be described given the two n-grams in their required positions within the message.To test H2, a dataset is created using only the integers that can be described by the dictionaries, randomly selecting integer components from the dictionary, and creating the respective observations. This process also accounts for the required positions of the message components so that a message describing the observation can always be created. For example, if the unigram 9 described the integer 11, and the bigram [5, 1] described the integer 6, a corresponding observation could be o = [11, 6, -1, 8, 9] . The positions of the integers in the observations are chosen at random. By generating both compositional datasets using a stochastic process, we do not assume a specific syntax. Rather, the syntax can only be identified by looking at messages understood by the receiver. These datasets, together with their respective dictionaries, are then used to query the receiver agent, testing if the messages are identified correctly. We run this test for all of our trained agents, with the dictionaries that were identified for each agent pair. We provide the details in Table 2 .Using just the non-compositional positional messages, we observe a significant increase in the performance of the agents, compared to random chance accuracy of 20%. This proves H1, showing that at least some messages do not require complex functions to be composed, or contextual information to be interpreted. As the accuracy for these messages reaches over 90% on average, we argue that the NPMI method has captured almost all the information transmitted using these messages.As mentioned in H2, we examine the impact of the positional components and whether they carry the information the NPMI method has identified. We, therefore, separate the compositional analysis into two parts: Compositional-NP, where the positional components are replaced with 0, and Compositional-P, which includes the identified positional components. In the Compositional-NP case, the agents achieve a close to random accuracy, whereas, in the Compositional-P case, agents achieve above random accuracy, with some agent pairs reaching over 75% accuracy. This proves our H2 correct, showing that the NPMI method has successfully identified the positional information contained in the messages, together with the integer information. 
discussion: 
 Having successfully verified both H1 and H2, we confirmed the validity of the language analysis.We also verify the generalisation ability of the agents, by evaluating varying training and evaluation sequence lengths, vocabulary sizes, and hidden size in Appendix C.To provide human interpretability of the emergent language, we use the NPMI method to create a dictionary providing an understanding of both the positional and compositional messages. We present an excerpt from an example dictionary in Table 3 . With human interpretability, we can gain a deeper understanding of the principles underlying the agents' communication protocol.We posit that the emergence of compositional spatial references points to a first emergence of a simple syntactic structure in an emergent language. Both of the n-grams in our example from Section 5.2, also shown in Table 3 , are assigned specific positions in the message by the agents. The unigram 7 must always be in the first position of the message, while the bigram [0, 2] must always be in the second position. The emergence of this structure shows that even though referential games have been considered obsolete in recent research (Chaabouni et al., 2022; Rita et al., 2024) , a careful design of the environment may yet elicit more of the fundamental properties of natural language.We hypothesise that the emergence of non-compositional spatial references tailored to specific observations, such as begin+1, is due to observation sparsity. Compositionality would bring no benefit since the observations which they describe are usually rare, representing 1-2% of the dataset and are monolithic, i.e., begin, begin+1, end-1, and end. We therefore argue that the emergence of noncompositional references in these cases is advantageous, since these messages are easily compressible. Since these messages are monolithic, they could be compressed to a single token/character in simple encoding schemes. In contrast, compositional messages require at least two tokens/characters, one for each integer/positional component. With a linguistic parsimony pressure (Chaabouni et al., 2019; Rita et al., 2020) applied, these messages could be more efficient at transmitting the information contained within these observations than compositional ones. Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: The training details are provided in Appendix A, with the optimisation method outlined in Section 3. Our code also includes detailed information about the training and test parameters.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
limitations: 
 The accuracy for the Non-Compositional Integer, and Compositional-P messages averages about 33%. While still above random, showing that some meaning is captured in non-compositional messages, it points to there being more to be understood about these messages. We hypothesise this may be due to the higher degree of message pragmatism, or context dependence (Nikolaus, 2023) . Our method of message generation, using randomly selected parts, may not be able to capture the complexity of the messages. For example, the context in which they are used might be crucial for some n-grams, requiring the use of a specific n-gram instead of another when referring to certain integers, or when specific integers are present in the observation. Just like in English, certain verbs are only used with certain nouns, such as "pilot a plane" vs "pilot a car". While the word "pilot" in the broad sense refers to operating a vehicle, it is not used with cars specifically. This may also be the case for the emergent language. For compositional messages, an additional issue may be that some messages are non-trivially compositional, using functions apart from simple concatenation to convey compositional meaning (Perkins, 2021) , making them impossible to analyse with the NPMI measure. However, these issues may be addressed by scaling the emergent communication experiments as the languages become more general with the increased complexity of their environment (Chaabouni et al., 2022) .Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: We include the value of the 1-sigma standard deviation for reported accuracies in Table 2 .Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
conclusion: 
 Recent work in the field of emergent communication has advocated for better alignment of emergent languages with natural language (Boldt and Mortensen, 2024; Rita et al., 2024) , such as through the investigation of deixis (Rita et al., 2024) . Aligned to this approach, we provide a first reported emergent language containing spatial references (Lyons, 1977) , together with a method to interpret the agents' messages in natural language. We show that agents can learn to communicate about spatial relationships with over 90% accuracy. We identify both compositional and non-compositional spatial referencing, showing that the agents use a mixture of both. We hypothesise why the agents choose non-compositional representations of observation types which are sparse in the dataset, arguing that this behaviour can be used to increase communicative efficiency. We show that, using the NPMI language analysis method, we can create a human interpretable dictionary, of the agents' own language. We confirm that our method of language interpretation is accurate, achieving over 94% accuracy for certain dictionaries. 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [NA] Justification: We do not believe that our methods for emergent language interpretability or the ability to use spatial references would have a path to significant negative societal impacts at this stage. We briefly discuss the positive impact of using spatial references and of more interpretable emergent languages in Section 1.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: We do not expect increased emergent language efficiency or the ability to use spatial references to have a risk of misuse. We would argue more transparency into the emergent languages makes the systems less susceptible to misuse.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: The codebase used is credited with a citation in Section 3. The URL to the original code is also provided on GitHub. Both the original codebase, and our code for training and dataset creation, are released under the MIT Licence.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] Justification: Our code is well commented, and the documentation is available with the code on GitHub (Footnote 2) under the MIT Licence. Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)? Answer: [NA] Justification: Human participants were not involved. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human SubjectsQuestion: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained? Answer: [NA] Justification: Human participants were not involved. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: The computational resources needed to reproduce this work are shown in Table 4 , with the hyperparameters in Table 5 and Table 6 . The Table 4 shows resources required for all training and evaluation. The processors used were a mixture of Intel Xeon Silver 4216s and AMD EPYC 7502s. The GPUs used were a mixture of NVIDIA Quadro RTX 8000s, NVIDIA Tesla V100s, and NVIDIA A100s. These nodes used in our experiments were hosted on the IRIDIS cluster. The development process consumed more compute, which we estimate would have added 10 CPU and GPU hours, to account for experimentation. To train and evaluate the agents, we use datasets consisting of 200,000 samples for training, 200,000 for validation, and 20,000 for testing. Each dataset is generated independently, with sequences created randomly. Given the sequence length of 60 and the fact that no integers are repeated, the number of possible permutations is 60! ≈ 8 × 10 81 , which vastly exceeds the number of samples we generate. We further ensure that there is no overlap between datasets by empirically checking the overlap rates across 1,000 randomly generated datasets, confirming an overlap rate of 0%.To generalise the results presented in this paper, we also run additional tests, varying the vocabulary size, training sequence length, evaluation sequence length, and the hidden size of the agents, as outlined in Appendix A. We observe no performance decline with either increasing or decreasing the vocabulary size or the training sequence length, given that the agents have enough capacity within their network to still learn the longer sequence lengths. We observe a decline in task accuracy at sequence lengths of 100, when the agents have a hidden size of 64. However, increasing the hidden size to 128 brings the training and validation accuracy back to over 90%.When agents are evaluated on sequence lengths that are different from the ones they were trained on, we observe a small performance decline for small differences in sequence lengths. We present the average accuracies for the base case (Sequence shortened by 0), as well as the average difference in accuracy as compared to the baseline for different sequence lengths in Table 7 . We observe a significant difference if the agents are evaluated on sequences that are over 50% shorter than the ones they were trained on. We hypothesise that this is due to the agents missing certain integers that they used more often than others, therefore reducing their accuracy. However, even in the worst case, the accuracy remains above 70%. For our pseudocode we will be using the Python assignments convention, i.e., = and ← are equivalent, and x+=1 is equivalent to x ← x + 1. The algorithms presented are for top_n = 1. To improve the computational efficiency. the probability of the integer appearing is statically defined as 1 60 for top_n = 1, or in Equation (1) for top_n > 1. In the case of top_n > 1 we use the probability for the integer as per Equation (1), to account for the polysemy, i.e., the probability for any of top_n integers occurring in the observation. The lower part of the binomial is 4, as there are 4 integers that can be sampled from the 60 possible integers, instead of 5, as we exclude the target integer. (1) Additionally, in the PMI c algorithm, we specify a probability to equal to 0.98 in Line 74 and Line 77. This is a simplification of the calculation for clarity of the pseudocode. This probability is instead obtained using the count of a given type of observation, divided by the number of total observations. This calculation is performed for each type of observation, i.e., begin, begin+1, end, end-1 and middle. The probability of the middle observation is very close to 1, being on average 0.98, while the other probabilities are on average 0.005. Since the middle observation is most common, we included its value in the pseudocode. Justification: We claim that we demonstrate how agents can communicate about spatial relationships, and how such a language can be interpreted. These claims are supported by our results in Section 5.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.Our code is available on GitHub at https://github.com/olipinski/TPGOur implementation of NPMI is not numerically stable due to probability approximation, sometimes exceeding the [-1,1] co-domain. We provide more details in the code.tn for the referent position n-grams is set to 0.3 


Decision: The paper proposes a variant of the referential game in which agents must learn to perform a simplified form of spatial deixis, by referring to a target item in terms of its relative position. The authors perform an information-theoretic analysis suggesting that the agents develop a language that is successfully incorporating deixis and is partially compositional.

The reviewers appreciated the interest and novelty of the research question. There were several concerns about generalization, for example in terms of length of the sequences, and a more general worry that the paper is only considering a toy setup for deixis, which might not be sufficient to build upon in future research.

There was a lively and constructive discussion, in which the authors addressed the length generalization issue, and they worked towards a formalization of deixis that establishes a clearer link between the paper setup and the general notion of deixis in linguistics.

The authors have addressed the main concern of the reviewers, and I think the revised paper will be a nice addition to the literature on emergent communication.

Reviews: review: paper_summary: paper_summary: This paper shows that emergent communication can learn spatial references. They first create a modified referential game which requires the agents to communicate by messages that indicate the relative position of a number. The proposed agent architecture shows that the GRU-based agents can achieve good performance. The analysis uses NPMI to identify the meaning of the ngrams in the message (i.e., the correlation between the ngram and the referred positions). This paper further shows that the mapping generated by NPMI is correct by generating additional datasets based on the identified dictionaries to show that both non-compositional and compositional messages carry the intended meanings.
strength_weakness: strength_weakness: - The paper is not very easy to follow especially the definition and design of different types of sequences, examples of the messages, and how the hypotheses are tested. The presentation can still be improved.
- It is unclear how much the test set overlaps with the training set when measuring the accuracy. There is no control of generalization tests such as varying full sequence length or observation of certain patterns of sequences. So, it is hard to understand if the learned messages are effective or memorization of part of sequences in training. For example, does the ngram that means “leftmost” can effectively communicate in a longer sequence?
- The design of the game put high communication pressure on the agents. The agents need to develop messages conveying relative positions in order to succeed. How does the success relate to the communication protocol, for example, when the message length is longer, is it still necessary to develop messages that convey relative positions? It is unclear about the role of channel bandwidth, effective communication, and developed messages.
- The test in Compositional-NP is to generate the dataset by removing the positional component of the message. This is an extreme case of H2. In reality, the message is most likely to be corrupted rather than removed. To reject the null hypothesis, it will be more convincing to have a corrupted message version.
questions: questions: - In the experiment, the observation is always fixed length which makes the communication easier. What happens when the observation is longer? I can imagine if the longer sequence contains the same number at different positions, it will introduce some ambiguity in the messages.
limitations: limitations: - The experiment is based on one-dimensional sequences, the types of spatial references are limited in this case. It will be helpful if the author can discuss how it can extend to more complex types of spatial references.
paper_summary: paper_summary: The authors design a referential game environment intended to motivate the emergence of spatial references, cast in the form of a task where the target integer must be selected from an integer sequence.  The character vocabulary for the message is smaller in size than the set of integers in the list, and this necessitates an alternative to directly specifying the target integers.  Using a traditional GRU-based speaker/listener architecture, the model achieves high task accuracy.  Using existing information theoretic measures the authors are able to roughly decode the semantics of the messages and show some degree of compositionality in the messages.
strength_weakness: strength_weakness: - Overall I find the biggest weakness to be in the scope of the paper and the degree to which the design of the environment caters to the type of messages the authors want to elicit here.  It comes across as a toy problem, and through the lens of the field as a whole, I think it raises the question of whether there is sufficient novelty in making such small and targetted tweaks to the referential game formula.

This might be best highlighted by revisiting the motivating example, such as "a blue vase with intricate motifs on the table".  Why refer to this as "the vase over there" or "that vase near you"?  There are pressures in the referential game to draw out these spatial references, but these feel artificial and devoid of broader understanding about linguistic pressures when we compare them to the pragmatic concerns that would motivate a spatial reference (and what type of spatial reference) in the motivating example.

I also think claims are over-stated.  The authors claim this is the first paper in EC to have syntax and make comparisons to SVO ordering.  This comes across as a flimsy attempt to connect to human language and to signal a degree of progress in the complexity of ECs, but a lengthier discussion is warranted.  Syntax can't be treated as something that does or does not exist, but rather, discussion of what formal language class the emergent language falls into would be relevant, and nothing here would necessitate a CFG or the degree of syntax that is meaningful when it comes to discussing natural language.

Despite the simplicity I do see value in this work but I would have liked to have seen a less contrived environment with a more difficult learning problem / substantial scope in the necessary semantics to feel comparable to the degree of contributions typical of a paper at this venue.  I think it would be far more appropriate at a more targetted venue where it can also recieve appropriate attention and discussion.
questions: questions: - There are some fairly trivial solutions to this problem.  It seems compositional integer style gets at this -- are there cases where compositional integer would fail?  I'm not seeing the need to learn a spatial solution to this problem when it seems that a two-character code could cover all possible target integers.  Does this vary as the length of sequence or size of alphabet are increased/decreased?

- Similarly, is there any reason to motivate the choice of vocabulary size with respect to latin alphabet?  The chunks of the messages are more akin to words than characters.  To me it just read as an attempt to have a connection to human language, but that relationship was not meaningful.

- It is mentioned that the hyperparameters for MI are optimized for translation accuracy.  I can make some guesses as to what might be done here, but it wasn't clear to me exactly what is being compared here.
limitations: limitations: Some limitations have been discussed, though limitations of the experimental design (such as 3 character messages, "nouns" being integers and therefore being able to reference without features, etc.) are not.  The authors mention some doubts in previous work regarding the value of referential game -- I think this is precisely one of the ways that these doubts emerge, i.e., "we try very hard to setup a task where success must be achieved in this way, and the model finds it".  These sorts of limitations may not be addressable in the scope of the paper, but they are also not mentioned.  There are no negative societal impacts from such work.
paper_summary: paper_summary: This paper proposes a new communication game in the emergent communication framework to analyze the emergence of _deictic reference, i.e. expressions akin to demonstratives like "this" and "that".  These are important expressions in natural language and especially in this emergence literature, since their meaning is context-dependent and "functional", i.e. cannot be reduced to objective properties of the object of reference. The paper also introduces an application of normalized pointwise information to the analysis of the emergent communication protocol in order to identify holistic messages (where a message refers to an entire meaning) and compositional ones (where certain n-grams and/or positions refer to specific "components" of the meaning).  Both of these are welcome contributions and will be of interest to many people working on emergent communication.  The core idea in their game is to use integers within longer sequences as the object of reference, provide a _partial observation_ of the true context to the sender (so that absolute positional information cannot be used) and to _mask out_ the target object in a sequence (so that the integer itself cannot be used); what remains as possible information to convey are things like "two to the right of 13".
strength_weakness: strength_weakness: * Some experimental details could be more carefully reported and some analyses could be more systematic/quantitative (see questions below).
* The artificial messages used to validate their NPMI metric does not yield results as strong as one would like (as discussed in the Limitations section); this makes it not entirely clear that the metric does what its intended to do.
questions: questions: * Why did you choose a fixed-length of 3 for the messages (as opposed to either a single token, or variable-length)? 
* Line 92 and 114: should "target integers" and "targets" both be singular?  There's one target integer, correct?  Or is the plural here just over a batch of examples?  If the latter, the wording is a bit confusing since the worked case in the paper is just one example ("batch size 1" so to speak). 
* Can $PMI_c$ and $PMI_{nc}$ be seen as one metric, with the latter a special case of the former (i.e. for the full tri-grams)?  The discussion just before Section 5 seems to suggest so, so I would encourage more elaboration on whether these are really two separate degrees or not.  For instance: does high $nc$ entail low $c$, and vice versa?
* "The analysis provided in this section is based on the messages collected from the test dataset after the training has finished".  What was the train/test split here?  Appendix A provides model / optimizer hyper-parameters, but what are the game/environment/data choices?
* Can the observations in Section 5.1 and 5.2 be made more quantitative?  I would appreciate a more detailed analysis of the types of composition observed, their frequency, and other factors like that.
* H2 and Table 1: while the Comp-P case is above chance, if the NPMI method correctly identified "genuinely compositional" messages, we would expect nearly perfect accuracy in this case, right?
* Very minor typographic point: I think that the "n" in "$n$-gram" and similarly in the main text should be in math mode.
limitations: limitations: Yes
paper_summary: paper_summary: This work investigates the presence of spatial deixis (e.g., spatial references in language dependent on the context of the utterance) in a signalling game within the paradigm of emergent communication.
It begins by introducing a variant of the signalling game which requires the sender to communicate the relative position of some element in a sequence of integers visible to the receiver.
Analysis of the semantics of the emergent communication is performed primarily through looking at normalized point-wise mutual information.
These analyses show that the emergent communication regularly uses spatially-referent messages (or sub-message units), validating the presence of spatial deixis in the environment.
strength_weakness: strength_weakness: ### Quality
- Nothing of note.
### Clarity
- (minor) One or two more of the points in Section 5 need to be illustrated (likely with just a table); although the text is mostly clear, some tables which aggregate what is said there would make referencing the paper much easier.
- (minor) It is a little bit confusing that your hypotheses are what you expect to be false; it might be clearer to state hypotheses in positive (even if the statistical test you are using is rejecting a null hypothesis of a random baseline).
### Reusability
- Nothing of note.
### Generalizability
- (major) It is not clearly the case that the environment addresses deixis in a way that applies to emergent communication environments more generally (see first question for more).
- (major) There is not much discussion on how the deixis investigated in this paper is applicable to emergent communication more generally.
### Directedness
- Nothing of note.
questions: questions: - The biggest question regarding the paper for me is how do we make the jump for the simple example of deixis presented in the empirical investigation of the paper to a more robust form of deixis.  It is not wrong (and likely correct, in fact) to have started out with a toy problem, but I believe there either needs to be empirical work and/or some light theoretical work on what exactly is meant by "deixis" in this paper, how the environment investigated satisfies that definition, and how this is relevant for further environments.  I might raise my score if the authors could respond with a slightly more formal characterization of deixis and how it maps both to the current environment and more sophisticated, "natural" environments (e.g., embodied multi-agent environment).
- In the environment, are the integers actually represented as integers in the neural network or are they encoded as one-hot vectors?  If they are OHVs, it is not clear that it is the case; if they are not OHVs, it seems like an odd design choice to feed a scalar into an NN when it is representing something categorical.
- How are the various inputs to the receiver agent actually fed into the network.  Are they just concatenated "temporally" and given as a sequence to the GRU?


### Comments

- The fact that this paper is inducing a segmentation of emergent communication is (minor) contribution in and of itself, so I think it deserves a mention in the introduction.
- The "td" variable needs to be introduced before the example; I am assuming it is the list of distractors plus the correct answer, but it should be stated explicitly when defining the vectors earlier.
- Table 2: Don't reuse X; use other variables.
- Include the actual URL for the (Anonymous) GitHub so that it is obvious it is a link.
- I don't understand the point being made at Line 337.
- I do not think it is appropriate to specifically mention "SVO" as an interpretation of the language since there is no clear way to distinguish between nouns, verbs, subjects, or objects; I think it is fine to say that there is syntactic structure, but I a skeptical of there being evidence to make any claim further than that here.
limitations: limitations: N/A.

Comments: title: Thanks for the reply
Reply
comment: Regarding the simple environment, I agree and disagree.  Of course there is prudence in starting from a simple, well-controlled environment, as presented in the paper.  But of course the risk is that it will be difficult to scale the approach up or reconcile it with existing work.  Frankly this is an all too common problem in the EC literature, which is rampant with one-off papers which each show some semblance of the target structures in emergent languages, and which are never incorporated or built upon ever again, and cited only in passing as yet another linguistic phenomena which now has a toy world which can give rise to some simple form of it.

The question should probably be, is it too much to ask for both?  Is it too much to expect that an EC paper at least show some consideration for what comes next, or to attempt to show both a contrived toy scenario and a more sophisticated one which explores the limitations of the approach in the face of a more realistic and more difficult learning problem?

To this end, I greatly appreciate the prodding by Reviewer vZ26, and the author response, to formalize a more general notion of deixis, such that we could at least have some understanding of what scaling this work up might entail.

With that in mind, with the inclusion of this formalisation I am willing to increase my score to 5, on the accept side of the decision boundary, as I think the work as a whole might represent an adequately thorough investigation of this specific issue.  This is a difficult decision -- I am still a bit concerned about the potential lack of impact from the work, especially when submitted to a competitive and general interest venue, and where the additions to the draft (which seek to formalize the broader phenomenon of deixis) are still a work in progress.
Thank you for providing this.  It is pretty close to what I had in mind in writing my prior comment, and I think it will facilitate applying the results of this paper more broadly.
I thank the authors for the further explanation. It is good to see it generalize to a longer sequence. However, this is not the only form of generalization; also, non-overlapping training and test sets do not mean the messages generalize to different combinations of deixis. For example, seeing "4 to the left" and "3 to the right" and generalizing to "3 to the left". These generalization tests will need to reflect what a formal deixis definition this paper uses.

Regarding the design to avoid repetition, I understand the difficulty in identifying the referent position, but this restriction can limit the learned messages to simple ones. So, it will be helpful to discuss this limitation in the design too. 

Overall, I appreciate the authors' response and it addressed some of my questions. I will increase my score to 5 and suggest a further discussion on the generalization that reflects the definition of deixis.
Thank you for the detailed and helpful reply!  I am already a fan of this paper, and these responses clarify any last remaining questions that I had.
Thank you for your continued engagement with our paper! We improve the formalisation of deixis below, making it more environment-agnostic. 

To do so, we expand the observation to be an $m$-dimensional $R^n$ tensor, which can represent any possible properties of the observation.

## Formalisation of deixis

Let $O$ represent an abstract observation that an agent perceives from its environment, $O \in R^{n \times n \times n ... n = n^m}$. The $m$ dimensions can represent the spatial, temporal, or other positions. For example $R^{n^3}$ could represent a 3-D observatrion.

Let $O_p$ be the reference point in $O$, which coordinates are represented by an $n$-tuple of real numbers $(x_1,x_2 ... x_m)$ and $O_t$ be the target point in $O$, with its coordinates represented by an $n$-tuple of real numbers $(y_1, y_2 ... y_m)$.

Then, the relative distance function $d(O_p,O_t)$ returns an $n$-tuple of real numbers $(z_1,z_2 … z_m)$, such that $z_i = x_i - y_i$. This relative distance function allows for unambiguous identification of the target object $O_t$, given that the position of $O_p$ is known.

A deictic expression can be defined as a mapping of the value of $d(O_p,O_t)$, the reference point $O_p$, and their context $O$, to a specific linguistic or symbolic expression that describes the relationship between $O_p$ and $O_t$. This mapping can be represented as:

$(O,d(O_p,O_t),O_p) \rightarrow \text{Expression}(O,d(O_p,O_t),O_p)$

where the resulting expression $\text{Expression}(O,d(O_p,O_t),O_p)$ is a description of the reference point $O_p$ and its relative distances to the target point $O_t$, given the context $O$.

### Examples

Consider two objects, $O_p$ and $O_t$, located at Cartesian coordinates $(x_1, x_2)$ and $(y_1, y_2)$, respectively. The relative distance function $d(O_p, O_t) = (z_1, z_2)$ captures the differences in both dimensions.

In this context, suppose the expression "2 to the left and 3 up" is used. This would correspond to $z_1 = -2$ (indicating 2 to the left) and $z_2 = 3$ (indicating 3 up) relative to the reference point $O_p$. However, without knowing the exact position of $O_p$, this information alone is insufficient to identify $O_t$.

To resolve this, the reference point $O_p$ must either be explicitly included in the deictic expression or implicitly understood by all interlocutors. For example, if it is agreed that the reference point is at $(5, 4)$, where $(5,4)$ could be an abstraction of any object, the expression "2 to the left and 3 up" clearly locates $O_t$ at $(3, 7)$. Without such an agreement, a full deictic expression is necessary, such as "2 to the left and 3 up from $(5, 4)$".

Depending on the context $O$ and the need for clarity between agents, some information provided by the relative distance function $d$ may be omitted. For example, if the target object $O_t$ is the only item of a given type in the environment to the left of the reference point $O_p$, simply stating "to the left" might be sufficient without specifying the exact distances or the $y$ direction. Similarly, if the agents are conversing about objects on a flat surface, the height difference may be irrelevant and ignored, simplifying the expression further.

The examples above could also be rewritten to showcase the ability to specify any other deictic basis, by simply changing the underlying meaning of a given dimension. The expression "5 minutes after 10:00" specifies a temporal deixis by explicitly locating an event $O_t$ relative to a reference time $O_p$. In a 3D spatial context, an expression like "4 to the left, 2 forward, and 1 down" captures the exact relative position of an object $O\_t$ in Euclidean space. 

This formalization will be included in the revised version of our paper.
Regarding the response to the "generalization" point of the review starting with "Let O be the observation available to an agent", I think this is a step in the right direction but still needs more development (although it is a little hard to judge the proposed formalization out of context).
I think the main shortcoming of the sketch of formalization proposed is that it is still not quite clear what "deixis" is in a sort of environment-agnostic way, which is what I was getting at in my review (although potentially unclearly).

For example, $O_p$ is a point in the observation serving as the origin, but what does "point in the observation" mean in some environment-independent way?
Maybe one could start off by specifying that the environment has some structure which supports deictic expressions in the first place (e.g., space, time, succession, 1st-2nd-3rd person distinction).
Additionally, there needs to be some way that the context in which the potentially deictic expression is grounded; that is, we can't just have some arbitrary $O_p$, there needs to be some way in which $O_p$ is derivable from the context of the utterance.
With these in mind, we could maybe define deixis in an EC environment along the lines of the following:
- The observations of the environment support some notion of position (or something more general, if you can think of it).
- Using this notion of position, we can take some original point, apply a transformation (i.e., moving a distance in space), and derive the target point.
- The original point is derived from the context of the utterance.

I am not saying this the best or only way to do, but hopefully this better illustrates what I meant by "generalization".  I think the account offered in the rebuttal is on the right track, but maybe a couple extra steps could be taken to make it environment agnostic.
I am moving recommendation from a 5 to a 6 because think the proposed changes are close to making this paper more widely applicable to other EC environments.
Nevertheless, I probably will move my rating to a 7 because I am not able to see a fully-developed formalization in context.

---

As an aside, regarding OHVs vs scalars, it seems odd that OHVs would perform worse since it essentially pre-processes the scalar values into something the neural networks can work with---maybe this has something to do with over-parameterization when switching from scalars to OHVs, but I don't think this all that important (aside from making it clear that scalars are being used since one might naturally assume integers are represented as OHVs), as it does not invalidate the experiments.

Everything else not mentioned here I found was addressed suitably by the rebuttal.