V ARIABLE C OMPOSITIONALITY R ELIABLY E MERGES IN N EURAL N ETWORKS
Anonymous authors
Paper under double-blind review
A BSTRACT
Human languages are pervasively compositional: because the meaning of an expression is composed from the meaning of its parts, we can productively leverage our prior experience in order to communicate about novel meanings.
Work looking at the languages that emerge when neural networks solve communicative tasks has shown that networks regularly develop languages that allow them to communicate and generalize to novel examples; surprisingly, that work has struggled to show that compositional systems reliably develop, leading to claims that a language’s degree of compositionality has little bearing on how well it can generalise.
We argue that the languages that emerge between networks are in fact straightforwardly compositional, just with variation.
We introduce a variation-based framework for interpreting the mappings produced by neural networks in emergent communication games and find that they reliably exhibit straight-forward compositional structure, with a degree of natural language-like variation that obscures their compositionality under measures used in previous work.
We show that early in training measures of variation correlate with generalization performance, but that this effect goes away later in training as the languages become regular enough to generalize robustly.
In an effort to decrease the variability of the emergent languages we show how reducing a model’s capacity results in greater regularity, in line with claims about factors shaping the emergence of regularity in human language.
I NTRODUCTION
Compositionality is a defining feature of natural language; the meaning of a phrase is composed from the meaning of its parts and the way they are combined (Cann, 1993).
This underpins the powerful generalization abilities of the average speaker: compositionality allows us to readily interpret novel sentences and express novel concepts.
Robust generalization like this is a core goal of machine-learning: foundational to how we evaluate our models is seeing how well they generalize to examples that were withheld during training (Bishop, 2006).
Deep neural networks show remarkable aptitude for generalization in-distribution (e.g. Dong & Lapata, 2016; Vaswani et al.), but a growing body of work questions whether or not these networks are generalizing compositionally (e.g. Kim & Linzen, 2020; Lake & Baroni, 2018), highlighting contexts where models consistently fail to generalize (e.g. in cases of distributional shift; Keysers et al.
These questions follow from earlier work that challenges if these architectures are even capable of behaving compositionally (Fodor & Pylyshyn, 1988).
Recent work has looked at whether compositional communicative systems emerge in neural networks when placed in conditions analogous to those that gave rise to human language (e.g. Choi et al., 2018;
Kottur et al., 2017).
In these simulations, multiple separate networks need to learn to communicate with one another about concepts, environmental information, instructions or goals via discrete signals - like sequences of letters - but are given no prior information about how to do so.
A common setup is a ‘reconstruction game’ modelled after a Lewisian signalling game (Lewis, 1970), where a sender network describes a meaning using a signal, and a receiver network needs to reconstruct that meaning given the signal alone.
The resulting set of mappings from meanings to signals can be thought of as a language.
Previous work has shown that in this setup models reliably develop a language that succeeds not only in describing the examples seen during training but also successfully generalizes to a held-out test set, allowing accurate communication about novel meanings.
Despite this evidence of the capacity to generalize, which is a product of compositionality in natural languages, existing analyses of those emergent languages provide little evidence of reliable compositional structure (see Lazaridou &
Baroni, 2020, for a review), leading some to suggest that compositionality is not required in order to generalise (Chaabouni et al., 2020; Kharitonov & Baroni, 2020).
However, this interpretation leaves us with a major puzzle: if the languages that develop in these emergent models are non-compositional, how do they allow successful communication about thousands of unseen examples (e.g. Lazaridou et al., 2018;
Havrylov & Titov, 2017)?
If the meaning of a form is completely arbitrary rather than being in some way composed from its parts there should be no reliable way to use such a mapping to generalize to novel examples (Brighton, 2002).
Here we provide an answer to this question, showing that emergent languages are characterised by variation , which masks their compositionality from many of the measures used in the existing literature.
Existing measures take regularity as the defining feature of a compositional system, assuming that in order to be compositional separate semantic atoms need to be represented separately in the signal (Chaabouni et al., 2020), or that symbols in the signal must have the same meaning regardless of the context they occur in (Kottur et al., 2017; Resnick et al., 2020).
Alternately they expect that each part of meaning will be encoded in only one way, or that the resulting languages will have a strict canonical word order (Brighton & Kirby (2006) used in Lazaridou et al.
However, natural languages exhibit rich patterns of variation (Weinreich et al., 1968; Goldberg, 2006; Dankers et al., 2021), frequently violating these four properties: forms often encode multiple elements of meaning (e.g. fusional inflection of person and number or gender and case), language is rife with homonymy (where the meaning of a form depends on context) and synonymy (where there are many ways of encoding a meaning in form), and many natural languages exhibit relatively free word order.
This offers us a different explanation of previous results: compositional systems may emerge, but exhibit variation.
An emergent language’s compositionality is related to, but distinct from, its regularity.
Accordingly we look at compositionality and regularity separately, taking a language’s generalization performance as an indication of whether or not it is compositional, and introducemeasures of natural language-like variation including those mentioned above.
We present results from a standard emergent communication model, and show that every run results in a highly-generalizing, and therefore compositional language.
We also show that the languages which emerge tend to exhibit a high degree of variability, which explains why previous metrics of compositionality would classify them as non-compositional.
Natural languages show us that systems can have considerable variation while retaining their compositionality.
Accordingly when we compare each emergent language’s regularity to its generalization performance it shows that degree of regularity correlates strongly with generalization early in training, but as the emergent language becomes regular enough to generalize reliably this correlation goes away.
In a final set of experiments we look at how to increase the regularity of an emergent language.
Limitations on human’s memory and cognitive capacity are thought to be a driving force in the emergence of compositional structure and regularity in natural language (Kirby, 2001; Hudson Kam & Chang, 2009; Smith & Wonnacott, 2010).
Learners with less memory are believed to regularize their input because they are more constrained in their ability to store low-frequency forms (Newport, 1990;
Ferdinand et al., 2019).
We reduce the capacity of our models by reducing the size of the hidden layers, and show that lower capacity models develop more regular languages, as predicted by accounts linking learner capacity and regularity in natural language and in line with previous work in this area (Resnick et al., 2020).
Q UANTIFYING V ARIATION
A number of measures have been introduced for assessing the degree of ‘compositionality’ in a mapping from meaning to form.
Many of these assess the degree of regularity in that mapping as a proxy for compositionality, and as a result treat natural-language-like variation as evidence of non-compositionality.
A A
A A B B
B B C C
C C D D
D D E E
E E F
F
F
F
O b j ec t V a l u e s
E l s a
K i r s t y A
nn i e
S i m on J e nny 13 4 P
o s iti on i n S i gn a l
A A
A A B B
B B C C
C C D D
D D E E
E E F
F
F F
H o m ony m y ( E i n po s iti on 4 ) 13 4 P
o s iti on i n S i gn a l
S ynony m y
( E l s a i n po s iti on 4 )
Regularity in language has been studied in broad array of contexts (see Ferdinand et al., 2019, for discussion).
In line with this work, we follow Smith & Wonnacott (2010) viewing it as the predictability of a form given a meaning, where a maximally regular language expresses the same (part of) meaning with the same (part of) form every time.
In our context - mapping meanings to discrete signals - regularity is maximized by a language consisting of one-to-one mappings: each position in the signal encodes one role in the meaning (e.g. subject, verb, object), each character in that position refers to only one semantic atom in that role (Subject: Ollie, Ernest, Isla, ... ) and is the only character ever used to refer to that atom.
While such a system is intuitively compositionalthe meaning of the signal would be composed from the atoms its characters map to and the position they’re in (in line with Cann (1993)) it is by no means the only kind of compositional system.
We briefly discuss some of the most relevant existing measures of regularity then introduce four kinds of variation - drawn from kinds of variation attested in natural language - and introduce ways of quantifying each of them individually.
In what follows we refer to four properties of a meaning-signal mapping: meanings are comprised of roles - e.g. Subject, Verb, and Object - with semantic atoms which can occur in each role (e.g. Subject: Ollie, Isla ...
Verb: loves, hates, ... ).
Prior work in this area sometimes refers to these as attribute-value pairs (see Lazaridou & Baroni, 2020, for a review including some mention of attribute-value pairs, p. 11).
Signals are strings comprised of positions (indices), and the character that occurs in each.
We can frame linguistic concepts of variation in terms of how semantics (roles and atoms) map to signals (positions, characters).
2. E XISTING M EASURES
Topographic Similarity (Topsim) (Brighton & Kirby, 2006) has been used as a measure of compositionality in a wide array of contexts from early computational models of language emergence (Smith et al., 2003), to human experiments (Kirby et al., 2008), and more recently in neural network based studies (Lazaridou et al., 2018).
It assumes that in a compositional system where a whole signal is composed from reusable parts, similar meanings will map to similar signals.
This can be assessed by measuring the correlation between pairwise meaning-distances and edit distances between their associated signals: a perfectly regular compositional language without variation achieves a correlation score close to 1, while a non-compositional (random) mapping between meanings and signals achieves a correlation close to 0.
Often the emergent languages that develop in neural networks have topsim somewhere between these values ( g. in our results reported in Table indicate that the mapping is not
Posdis & Residual Entropy Posdis (Chaabouni et al., 2020) and Residual entropy (Resnick et al., 2020) provide entropy-based measures of regularity.
Posdis captures the extent to which each position of the signal univocally refers to a role in the meaning (e.g. subject, object, verb) and looks for each signal position to refer to only one role.
Posdis is maximised when separate roles are encoded separately in the signal.
Similarly, residual entropy assesses the degree to which a sub-string of the signal encodes a single atom in a role (e.g. Ollie in the Subject) and is minimized when a sub-string refers to only one atom in a role.
This requires there to be minimal homonymy and entanglement in a subset of the signal (across 1 or more positions), with each unique sub-string in those positions referring to only one atom in a role.
For posdis, like topsim, a maximally regular language with one-to-one mappings between meaning and form will score near 1, while for residual entropy it will score near 0.
We show empirically that a maximally regular language maximizes topsim and posdis - see table 1while minimizing residual entropy (for brevity residual entropy results are deferred to the appendix).
While such a language is compositional, not all compositional languages will score highly; these measures provide a way of evaluating the degree to which a language is maximally regular without allowing for the kinds of variability seen in (compositional) natural languages.
2. K INDS OF V ARIATION
Synonymy & Homonymy: Synonymy is minimised when each atom in a meaning maps to a single character in a position.
Homonymy is minimised when each character in a position maps to a single atom (Hurford, 2003).
A perfectly regular compositional language minimises these, but natural language is rife with both synonymy and homonymy (e.g. ‘loves’, ‘adores’, ‘fancies’ all map to approximately the same concept; the homonymous ‘bank’ maps to a financial institution, the act of turning a plane, and the land at the side of a river).
One-to-many mappings (synonymy) aren’t a problem for compositionality, as each different synonym can still be composed with the rest of a signal.
Similarly many-to-one mappings (homonymy) can be used compositionally, with meaning disambiguated by context.
Word Order Freedom: Word order freedom is minimized when each role in the meaning is always encoded in the same position(s) in the signal, resulting in a single canonical word order.
Looking at a language like Basque we see that a compositional language can support a number of different grammatical word orders (Laka Mugarza, 1996), with at least two equivalently valid translations of ‘Ollie saw Ernest:’ Ollie Ernest ikusi zuen, Ollie ikusi zuen Ernest.
Even in English which has relatively strict word order we see processes like topicalization that result in alternate orders that are equally acceptable Let’s go down to the lake for some fun; For some fun, let’s go down to the lake , or even more commonly dative alternations (Chomsky, 1957) like Ollie gave Orson a book;
Ollie gave a book to Orson .
While many languages have some constraints on word-order, even when there is maximal word order freedom the resulting language can still be clearly compositional, with characters encoding the meaning and their order conveying little information.
Entanglement: Entanglement is minimised when each role is encoded in different positions in the signal.
While a dis-entangled language is likely compositional, consider the English past tense form of ‘go.’
‘Went’ is irregular, encoding action and tense together, in contrast to the hypothetical regular form ‘goed’ where action and tense are encoded in separate parts of form (Anderson, 1992, p. 55).
Despite this we can go on to use the entangled form ‘went’ compositionally in a sentence:
Ollie went down to the shore (for discussion O’Donnell, 2015, p. 105).
While maximal entanglement where every role is encoded in every position would be non-compositional, the existence of even a high degree of entanglement does not preclude compositionality, given the entangled forms can be straight-forwardly recomposed with others.
2. V ARIATION -B ASED M EASURES
We define measures of the four kinds of variation outlined above: Synonymy, Entanglement, WordOrder Freedom, and Homonymy.
Each of these measures is bounded between 0 and 1, whereindicates a perfectly regular language with no variation, and 1 represents a maximally variable language.
We generate a maximally regular compositional language, and maximally irregular noncompositional language (where each meaning maps to a unique randomly-generated signal).
The regular language scores near 0 across our measures of variation, while the random mapping scores near 1, as shown in table 1.
All four measures start with the same underlying process, generating a tensor that describes the mapping between meanings and signals probabilistically, in terms of a probability distribution over characters in each signal position given a semantic atom in a role, encoding, for example, how likely character ‘A’ is in signal position 1 given that ‘Ollie’ is in the subject role of the signal’s meaning.
We can quantify this as a straightforward conditional probability using maximum likelihood estimation, shown in equation 1.
We estimate this repeatedly for every atom ( ∀ atom r,i ∈
A r ) in every role ( ∀ r ∈ R ), looking at every character ( ∀ char p,j ∈ C ) in every position of every signal ( ∀ p ∈ P ).
[Equation]
The resulting tensor describes how often each letter occurs in a position, given a certain atom (like
Ollie) in a role in the meaning 2 .
This tensor has dimensions semantic roles × semantic atoms × max signal length × characters 3 , where the last axis is a probability distribution over all possible characters in a given position - here denoted by P ( char p | atom r,i ) .
Synonymy is how many different characters appear in each position in the signal for each atom in each role.
For example when atom i = Ollie and r = Subject how many characters have non-zero probability in each signal position?
A perfectly regular language where ‘Ollie’ is always encoded by ‘A’ in position 1 would have a probability of 1.0 on ‘A’ in position 1.
A maximally variable language would have a uniform distribution over all characters.
We can take the entropy over characters in a position H ( P ( char p | atom r,i )) as a measure of synonymy in that position.
Equation 2a gives us a vector of | P | values corresponding to entropy over characters in each position ∀ p ∈ P (illustrated in figure 1).
[Equation]
(2a)
We take the position with the lowest entropy as a lower-bound estimate of synonymy for that atom r,i .
In order to bound the value we divide it by the entropy of a same-sized uniform distribution
H u (cid:16) P ( char p | atom r,i ) (cid:17) .
The synonymy of an entire language ( L ) is obtained by averaging across all atoms in a role, then across all roles.
A language with no synonymy where each atom is encoded by a single character in a position achieving close to 0, and maximal synonymy where any character can refer to each atom achieving close to 1 (shown empirically in table 1).
[Equation]
Word Order Freedom We also want to know if each role is consistently encoded in a particular part of the signal - reflecting a regular word order.
A language with free word order is equally likely to encode any Role ∈ R in any position, while a maximally regular language always encodes atoms from the same role in the same position(s).
If a given atom r,i is not encoded in a position we expect its distribution over characters to be roughly uniform.
So we can take the entropy for each position S ( atom r,i ) obtained in equation 2a, and average across all atoms in that role ∀ i ∈
A r .
If all the atoms in a role are encoded in the same position the distribution resulting from the mean will be non-uniform, with some positions having lower mean entropy than others (the appendix includes an illustration of this in figure 4 and some discussion of why we opt not to directly estimate P ( position | char j , atom r,i ) ).
[Equation]
To get a lower-bound estimate of the language-level word-order freedom we take the minimum from the mean distribution F ( role r ) and divide it by the entropy of a uniform distribution over characters to bound between 0 and 1, then average across all roles:
[Equation]
Entanglement
In addition to looking at whether a role is consistently encoded in a position, we can see if two roles are consistently encoded in the same (or different) positions.
We compare the means F ( role ) from equation 3a for each possible pair of roles r i , r j ∈ R C 2 by taking the magnitude of their difference, if two roles are encoded in the same position the result will be close to zero.
If the roles are maximally disentangled then the result will be close to the max (cid:16) F ( role i ) , F ( role j ) (cid:17) for that position.
To get a lower bound estimate of two roles’ entanglement we take the maximum of the difference and divide by the pre-difference max.
When the resulting value approaches 0 all roles are mapped to different parts of the signal, as it approaches 1 all roles are encoded in the same positions (illustrated in appendix figure 5).
[Equation]
Homonymy Finally we assess homonomy to see how uniquely a character in a position is used to refer to an atom.
This measure is similar to the synonymy measure applied to a different axis of the probability tensor P as depicted in figure 1.
For each character in each position we re-normalize P to create a probability distribution over atoms in a role H ( char p,j , r )
(The result is equivalent to
[Equation]
To get a lower-bound estimate of language level homonymy we take the position with the lowest entropy over atoms and divide by the entropy of a uniform distribution to bound between 0 and 1, then average across all characters and roles.
When the resulting value is close to 1 every atom maps to the same character(s).
Approaching
each character uniquely refers to a single atom.
[Equation]
M ETHODS
Models We implement a reconstruction game with a sender network and receiver network.
The overall architecture used is similar to Resnick et al.
Guo et al.
(2021) and identical to
Chaabouni et al.
(2020) when they use a GRU as the recurrent unit with a hidden size of 500.
The sender network is comprised of an embedding layer, linear layer, and a GRU (Cho et al., 2014) - the receiver architecture is the inverse.
A linear layer is used as the input is of fixed length, so can be presented at once as a one-hot encoding - while a GRU spells out the variable-length signal a character at a time.
The maximum signal length used here is 6, with 26 characters available to the model in each position.
The sender is optimized using REINFORCE (Williams, 1992) due to the discrete channel, while the receiver is optimised using ADAM (Kingma & Ba, 2014).
Models are implemented using pytorch (Paszke et al., 2019), and make use of portions of code from the EGG repository (Kharitonov et al., 2019).
Full hyperparameters for the experiments presented here can be found in the appendix.
Data The sender is shown examples drawn from a meaning space of two place predicates (e.g.
Ollie loves Osgood )
generated using a context free grammar, with three roles: subject, verb, and object and 25 atoms per role, resulting in a total of 15625 examples.
This is equivalent to the attribute, value setup used in previous work and closely follows Resnick et al.
Chaabouni et al.
(2020) to allow direct comparison of results.
Data is divided into 4 splits for training: 60%, validation 10%, i.i.d.
testing 10%, and o.o.d.
testing 20%.
O.O.D. Evaluation Previous emergent communication work typically evaluates generalization on an in-distribution held out test-set.
In order to better align our findings with the broader literature on compostional generalization in neural networks (e.g. Lake & Baroni, 2018;
Kim & Linzen, 2020)
we implement a version of the maximum compound divergence (MCD) algorithm from Keysers et al.
(2020), and report results for both in-distribution generalization, and out-of-distribution generalization to an MCD split.
Additionally we use an O.O.D. split because models often converge to ceiling i.i.d.
performance, which potentially makes it difficult to look for correlations between generalization performance and attributes of the language, like regularity.
For our small model i.i.d.
performance 95% confidence intervals are ± 0 .
99% while o.o.d.
performance is ± 14 .
14% , allowing a broader range of values with which to look for correlations (we include the same analyses on i.i.d.
performance in the appendix).
The effect of capacity on variation To test if model capacity affects the variability of emergent languages in neural networks, and to confirm that our measures of variation work as intended, we vary the capacity of our models.
We look to see if models with less capacity arrive at more regular languages than their larger counterparts as predicted by work in natural language (e.g. Hudson Kam & Chang, 2009).
We vary model capacity by varying the size of the hidden layers used by the model reporting and comparing results of three different model sizes small , medium , and large with hidden layer sizes 250, 500, and 800 respectively.
[Figure]
R ESULTS AND DISCUSSION
Our results for all model sizes are summarised in table 1.
As stated in the section 1, a language must be compositional in order to generalize, in line with previous work in this area (e.g. Kottur et al., 2017) and in linguistics (Brighton & Kirby, 2006).
All versions of our model get near ceiling i.i.d generalization and robust o.o.d.
generalization indicating a compositional system.
Compositionality and regularity are related, but distinct ; while a system needs to be more regular than a completely random mapping in order to generalize compositionally it does not need to be perfectly regular.
Natural languages show us that a system can support a high-degree of variation while remaining compositional.
In line with this in all conditions of our model the language that emerges is substantially more regular than a random mapping, but more variable than a perfectly regular language consisting of one-to-one mappings.
The relationship between regularity and generalization We use linear mixed effect models to measure the relationship between our four measures of variation and o.o.d.
performance, fitting multiple models on sliding windows covering the time course of training (implementation details in appendix).
The resulting regression coefficient (b value) in a particular time window indicates how strong a predictor our measures are of generalization performance in that time window.
As shown in figure 3 early on a language’s regularity is a strong predictor of how well it generalizes, but later in training this effect goes away.
This is consistent with the idea that some regularity is needed for generalization, but maximal regularity is not required.
Later in training, as a language emerges that is regular enough to succeed at the task (achieving ceiling i.i.d.
generalization performance), the relationship between regularity and generalization trends toward non-significance.
Further support for languages becoming regular enough can be found in table 1 where we see that in every condition the model decreases the variation in its language between early training and the best generalizing epoch - while remaining well below the theoretical max.
An important limitation of these results is that every language that results is still highly-variable (with the lowest mean variation score of any run being 0.43), possibly because the task here is quite simple in comparison to compositional generalization datasets in other domains (e.g. Kim & Linzen, 2020) - see Chaabouni et al.
(2020) for some discussion of the relationship between input-space complexity and regularity.
It is possible that as languages approach maximal regularity, regularity may again be a strong predictor of generalization performance - but given our models do not approach perfect regularity this remains an open question.
How can a variable language still be compositional?
Figure 2.2 helps us to understand what these highly variable but robustly generalizing languages look like.
It visualizes the word order for the run of our small model with the highest variation.
It shows that while the language is still much more variable than a perfectly regular one (this language has freedom = 0 .
57 , a compositional language with fixed word order has freedom = 0 ), it nonetheless exhibits a high degree of word order regularity, with verbs most likely encoded at the start of the signal, subjects in the middle, and objects at the end, but with each individual atom sometimes being encoded slightly differently.
It’s worth noting this plot is from the run of the small with the highest word order freedom - meaning all other runs of that model exhibit even stricter word order.
Given compositionality requires the meaning of a whole to be a function of its parts the pattern seen here where each role is encoded in part of the signal appears to meet that threshold.
The effect of capacity We see a consistent effect of capacity on regularity, with an increase in the number of trainable parameters resulting in an increase in variability across all measures with large arriving at significantly more variable languages than small or medium ( p < 0 . 05 ).
Spearman correlations show model size does not correlate significantly with o.o.d.
accuracy ( p = 0 . 24 ) but correlates with synonymy ( r = 0 .
67 ), word order freedom ( r = 0 . 69 ), entanglement ( r = 0 . 68 ), and homonymy ( r = 0 . 68 ) indicating larger models develop more variable languages (all of which are significant p < 0 . 00001 ).
This result is in line with work that points to constraints on human cognition as a key driver of regularization in natural language, suggesting that similar factors shape the regularity of emergent communication in neural networks.
Previous work studying the effect of network capacity on emergent languages (Resnick et al., 2020) found that while all model sizes could generalize well, larger models did so using a ‘non-compositional code’ indicated by a higher degree of variability according to their residual entropy measure (which has similarities to our measures of homonymy and entanglement).
This is consistent with our results, although we believe this indicates that larger models develop a language characterised by higher variability rather than non-compositionality (residual entropy scores for our model can be found in the appendix).
Interpreting previous measures and results in terms of regularity Existing measures (topsim and posdis) correlate negatively with model size ( r = − 0 .
63 , r = − 0 . 71 ) strongly suggesting that rather than tracking compositionality these measures implicitly track the degree of regularity in a language, especially given that the magnitude of their correlation coefficient is similar to that of our measures that explicitly assess variation.
This helps us to interpret results suggesting compositionality doesn’t correlate with generalization - if these measures assess regularity instead we know a wide array of languages can be regular enough to generalize well - without needing to maximize regularity to do so.
Once a language becomes regular enough for a task higher regularity doesn’t necessarily mean that language will generalize more robustly than its more variable counterparts.
C ONCLUSION
Neural networks reliably arrive at compositional languages when natural language-like variation is taken into account.
Previously these languages’ compositionality has been assessed on the basis of their regularity, but natural languages show us a system can be rich with variation while retaining the generalizability that makes compositionality so desirable.
In contrast to previous work which shows no relationship between regularity and generalization we have shown that degree of regularity matters early in training but as the language becomes regular enough for the task this effect goes away.
Similar to natural language the capacity of learners is a key driver of the degree of regularity that emerges.
By accounting for variation we can see striking similarities between the structure of the languages that emerge and structures in natural language.
A. E NTANGLEMENT G RAPHIC
H H H
H m ea n m ea n m ea n m ea n
O b j ec t V a l u e s E l s a
K i r s t y A
nn i e
S i m on J e nny 1 24 P
o s iti on i n S i gn a l
H H H
H m ea n m ea n m e a n m ea n
V e r b V a l u e s l ov e s h a t e s a do r e s a nnoy s k ill s 1 24 P o s iti on i n S i gn a l
M ea n E n t r op i e s v e r b − ob j ec t | d i ff e r e n ce |
A. R OLLING M IXED E FFECTS M ODEL I MPLEMENTATION
We implement a rolling mixed effects model using the python statsmodels package.
We fit a separate model for each of our 6 independent variables: synonymy, entanglement, freedom, homonymy, topsim, posids.
The dependent variable across all of them is o.o.d.
generalization performance.
For each model we include two random effects, random intercepts based on the seed used to initialize that run of the model, and random slopes for the epochs of training.
This allows the model to account for variation between different models, given that some initializations outperform others.
The model is fit to a window of 100 epochs of training data at a time, at each step it fits a regression to 100 epochs of data for 20 initializations of the model.
It then moves forward one epoch at a time (e.g. the first fit of the model is on epochs 0-100, the second on 1-101, the third 2-102, etc.).
We plot the resulting regression coefficient (b value) obtained from each mixed effects model fit to each IV, at each window of data.
For reference the corresponding command to run this model with the LMER package in R (a standard method for fitting these kinds of models) is: lmer(ood acc ∼ IV + (1 + Epoch | Seed)) where IV is one of the variation measures.
A. O.O.D. A CCURACY V S . V ARIATION S LICES
In addition to the regression analysis presented in the results section we show relational plots for two different epochs in training: one from mid way through and one from late in training near convergence.
In line with the regression analysis a more linear relationship between o.o.d.
performance and variation is visible earlier in training before the language becomes regular enough for the task.
Entanglement in particular shows a steep negative relationship in the 100 epoch plot but is totally scattered by epoch 500.
Were we to only assess the relationship between generalization and variation at the end of training we would could easily conclude in line with previous work that they were not meaningfully related.
It’s worth noting that the pattern here may not appear as salient as it appears in the rolling mixed effects model presented earlier, there are two major reasons for this: first the rolling model considers 100 epochs at a time, rather than a single slice with only 20 data points, providing it with 100 times the data visualized here by which to assess the relationship between variation and generalization.
Secondly the rolling model has a random intercept based on the seed used in each run of the model.
This is important because in line with other work on o.o.d.
generalization we see a substantial effect of initialization on generalization performance, by including it as a random effect the rolling model can look at each seed separately to see if each seed’s generalization performance over the run is related to its language’s variation.
So while in the visualizations below we may see some seeds which appear like outliers, the rolling model accounts for this, fitting a separate intercept for each run.
A. U SE OF E QUATION 1 FOR ALL 4 MEASURES OF VARIATION
We use equation 1 (copied below in simplified notation) to calculate the conditional probabilities used in all 4 measures of variation:
[Equation]
This gives us a distribution over characters for each position, for each atom, in each role.
This distribution is intuitively useful for estimating synonymy which can be seen as the entropy over characters in a position.
A.5. F REEDOM AND E NTANGLEMENT
However it’s natural to wonder why we use this distribution again when calculating measures of word order freedom and entanglement.
Both of these measures refer to how likely it is that a given role is encoded in a position in the signal.
Freedom looks at how consistently the atoms in a role are encoded in a position, while entanglement looks at how consistently any two roles are encoded in the same position.
Intuitively we might want to calculate a distribution over positions given roles instead, like:
[Equation]
then marginalize over characters and atoms so that we could directly estimate the probability of a position given a role P ( position | role ) .
The problem with this is that every signal has a character in every position, and every meaning has more than one role (i.e. subject, verb, and object, rather than just having a subject) meaning that the distribution over positions is always uniform.
If we were to only marginalize over atoms, to get a distribution over characters and positions
P ( position, char | role ) this is also nearly uniform, because different atoms are encoded using different characters.
So marginalizing over atoms combines distinct distributions for each atom into a near-uniform one.
Similarly if we only marginalize over characters to get a distribution over positions and atoms P ( position, atoms | role ) because every signal has a character in every position the resulting distribution is also uniform.
Fundamentally the only relevant probability distribution which is consistently non-uniform is the one described in equation 1: P ( char | position, atom, role ) .
Even though every signal has a character in every position every character is not equally likely given a specific position, atom, role combination.
As a result this is the distribution we use to calculate measures of word order freedom and entanglement.
In order to do so we first observe that in signal positions where an atom, role combination is not encoded P ( char | position, atom, role ) is uniform as the distribution is not conditioned by the selected atom and role.
Accordingly we take lower conditional entropy
H ( P ( char | position, atom, role ))
(defined in equation 2a) as an indication that an atom, role combination is more likely to be encoded in a position.
By taking a mean of this conditional entropy across all atoms in a given role (described in equation 3a) we can see if it is consistently low in the same position(s) of the signal for all atoms in that role - indicating adherence to a single word order.
Equation 3b then aggregates this across all roles.
Entanglement looks to see if there is consistent encoding of multiple roles in a single position.
Seeing as we take low conditional entropy as an indication that an atom, role combination is encoded in a given position we compare the mean from equation 3a with means from other roles to see if they are consistently low in the same parts of the signal.
A.5. H OMONYMY
Given that homonymy assesses the probability that a letter in a position encodes each atom in a role, it is possible to look at this by estimating the distribution P ( atom | char, position, role ) directly.
The same distribution can be calculated by instead taking the P ( char | position, atom, role ) distribution and re-normalizing it along the atom axis as described in equation 5a.
We find empirically that this is equivalent to computing H ( P ( atom | char, position, role ))
(see results in table 2) with the only differences between the two resulting from small rounding errors.
In table 2 we report results for both approaches to computing homonomy across model sizes to show their equivalency.
Note these results are the means of 6 seeds so differ slightly from figures in the core results.
When introducing the measures in section 2.3 of these two approaches we opt for the re-normalization of P ( char | position, atom, role ) rather than computing a new probability distribution because we believe this makes the formulation of the homonymy measure more intuitively related to the others, and makes the visualizations in figure 1 a direct reflection of how the measures are computed while producing equivalent results.
[Table]
A. R ESIDUAL E NTROPY
In addition to topsim and posdis reported in the main results we also report results from one other measure from previous work, residual entropy (Resnick et al., 2020).
The results here follow the same pattern as the other measures of variation with larger models arriving and more irregular languages.
Additionally all conditions increase the regularity of the emergent language over the course of training.
Also shown is the correlation analysis between Residual entropy and O.O.D. performance, showing like other measures of variation residual entropy is a strong predictor early in training but that this effect goes away later on.
[Table]
A. I.I.D. C ORRELATION R ESULTS
We also include the correlation results between the measures of variation and in-distribution generalization.
The results follow a similar pattern with degree of regularity being a strong predictor of generalization performance early in training but this effect goes away as the emergent language becomes regular enough to generalize well.
Interestingly in-distribution and out-of-distribution correlations align almost exactly.
This is reassuring in that it shows degree of regularity is important for generalization in general whether it is in or out of distribution.
A. S IGNIFICANCE T ESTING FOR V ARIATION D IFFERENCES
[Table]
A. H YPERPARAMETERS
• Recurrent Unit: GRU • Hidden Size: 250, 500,• Entropy Regularization Coefficient: (sender 0.5, receiver 0.0) • Batch Size:• Learning Rate:
1e- • Signal Length:• Character Inventory:• Training Epochs:•
Embedding Size:• Roles:• Atoms:• Optimizer:
(Sender: Reinforce, Receiver: ADAM)