Original: A RI EL: VOLUME CODING FOR SENTENCE GENERATION COMPARISONS
Revised: A RI EL: VOLUME CODING FOR SENTENCE GENERATION COMPARISONS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Mapping sequences of discrete data to a point in a continuous space makes it difﬁcult to retrieve those sequences via random sampling.
Revised: Mapping sequences of discrete data to a point in a continuous space makes it difﬁcult to retrieve those sequences via random sampling.

Original: Mapping the input to a volume would make it easier to retrieve at test time, and that is the strategy followed by the family of approaches based on Variational Autoencoder.
Revised: Mapping the input to a volume would make it easier to retrieve at test time, and that is the strategy followed by the family of approaches based on Variational Autoencoder.

Original: However the fact that they are at the same time optimizing for prediction and for smoothness of representation, forces them to trade-off between the two.
Revised: However the fact that they are at the same time optimizing for prediction and for smoothness of representation, forces them to trade-off between the two.

Original: We benchmark the performance of some of the standard methods in deep learning to generate sentences by uniformly sampling a continuous space.
Revised: We benchmark the performance of some of the standard methods in deep learning to generate sentences by uniformly sampling a continuous space.

Original: We do it by proposing AriEL, that constructs volumes in a continuous space, without the need of encouraging the creation of volumes through the loss function.
Revised: We do it by proposing AriEL, that constructs volumes in a continuous space, without the need of encouraging the creation of volumes through the loss function.

Original: We ﬁrst benchmark on a toy grammar, that allows to automatically evaluate the language learned and generated by the models.
Revised: We ﬁrst benchmark on a toy grammar, that allows to automatically evaluate the language learned and generated by the models.

Original: Then, we benchmark on a real dataset of human dialogues.
Revised: Then, we benchmark on a real dataset of human dialogues.

Original: Our results indicate that the random access to the stored information can be signiﬁcantly improved, since our method AriEL is able to generate a wider variety of correct language by randomly sampling the latent space.
Revised: Our results indicate that the random access to the stored information can be signiﬁcantly improved, since our method AriEL is able to generate a wider variety of correct language by randomly sampling the latent space.

Original: VAE follows in performance for the toy dataset while, AE and Transformer follow for the real dataset.
Revised: VAE follows in performance for the toy dataset while, AE and Transformer follow for the real dataset.

Original: This partially supports the hypothesis that encoding information into volumes instead of into points, leads to improved retrieval of learned information with random sampling.
Revised: This partially supports the hypothesis that encoding information into volumes instead of into points, leads to improved retrieval of learned information with random sampling.

Original: We hope this analysis can clarify directions to lead to better generators.
Revised: We hope this analysis can clarify directions to lead to better generators.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: It is standard for neural networks to map an input to a point in a d -dimensional real space (15; 43; 22).
Revised: It is standard for neural networks to map an input to a point in a d -dimensional real space (Hochreiter and Schmidhuber, 1997; Vaswani et al., 2017; LeCun et al., 1989).

Original: However, that makes it difﬁcult to ﬁnd a speciﬁc point when the real space is being sampled randomly.
Revised: However, that makes it difﬁcult to ﬁnd a speciﬁc point when the real space is being sampled randomly.

Original: That can limit the applicability of pre-trained models to their initial scope.
Revised: That can limit the applicability of pre-trained models to their initial scope.

Original: Some approaches do map an input into volumes in the latent space.
Revised: Some approaches do map an input into volumes in the latent space.

Original: The family of approaches that stems out of the idea of Variational Autoencoders (20; 3; 35; 4) are trained to encourage such type of representations.
Revised: The family of approaches that stems out of the idea of Variational Autoencoders (Kingma and Welling, 2014; Bowman et al., 2016; Rezende and Mohamed, 2015; Chen et al., 2018) are trained to encourage such type of representations.

Original: By encoding an input into a probability distribution that is sampled before decoding, several neighbouring points in R d can end up representing the same input.
Revised: By encoding an input into a probability distribution that is sampled before decoding, several neighbouring points in R d can end up representing the same input.

Original: However, it often implies having two summands in the loss, a log-prior term and a log-likelihood term (20; 3), that ﬁght for two different causes.
Revised: However, it often implies having two summands in the loss, a log-prior term and a log-likelihood term (Kingma and Welling, 2014; Bowman et al., 2016), that ﬁght for two different causes.

Original: In fact, if we want a smooth and volumetric representation, encouraged by the log-prior, it might come at the cost of having worse reconstruction or classiﬁcation, encouraged by the log-likelihood.
Revised: In fact, if we want a smooth and volumetric representation, encouraged by the log-prior, it might come at the cost of having worse reconstruction or classiﬁcation, encouraged by the log-likelihood.

Original: Therefore, each diminishes the strength and inﬂuence of the other.
Revised: Therefore, each diminishes the strength and inﬂuence of the other.

Original: By giving partially up on the smoothness of the representation, we propose instead a method to explicitly construct volumes, without a loss that is implicitly encouraging such behavior.
Revised: By giving partially up on the smoothness of the representation, we propose instead a method to explicitly construct volumes, without a loss that is implicitly encouraging such behavior.

Original: We propose AriEL, a method to map sentences to volumes in R d for efﬁcient retrieval with either random sampling, or a network that operates in its continuous space.
Revised: We propose AriEL, a method to map sentences to volumes in R d for efﬁcient retrieval with either random sampling, or a network that operates in its continuous space.

Original: It draws inspiration from arithmetic coding (AC)  and k-d trees (KdT), and we name it after them
Revised: It draws inspiration from arithmetic coding (AC) (Elias and Abramson, 1963) and k-d trees (KdT) (Bentley, 1975), and we name it after them

Original: Arithmetic coding and k-d trEes for Language (AriEL).
Revised: Arithmetic coding and k-d trEes for Language (AriEL).

Original: For simplicity we choose to focus on language, even though the technique is applicable for the coding of any variable length sequence of discrete symbols.
Revised: For simplicity we choose to focus on language, even though the technique is applicable for the coding of any variable length sequence of discrete symbols.

Revised: We focus on dialogue with the goal to provide a tool to optimize interactive agents, and the interaction of AriEL with longer text will be developped in future work.

Original: In particular, it can be used as an objective benchmark to compare other methods and to understand how their latent space is used.
Revised: In particular, it can be used as an objective benchmark to compare other methods and to understand how their latent space is used.

Original: AriEL attempts to ﬁll completely the latent space with the sentences present in the training dataset, using notions from information theory.
Revised: AriEL attempts to ﬁll completely the latent space with the sentences present in the training dataset, using notions from information theory.

Original: AriEL uses a language model to split the latent space in volumes guided by the probability assigned to the next symbol in a sentence.
Revised: AriEL uses a language model to split the latent space in volumes guided by the probability assigned to the next symbol in a sentence.

Original: For this reason, it can simplify as well the reuse of pretrained language models for new tasks and in larger architectures.
Revised: For this reason, it can simplify as well the reuse of pretrained language models for new tasks and in larger architectures.

Original: In fact, it can provide a training agent with a simpler interface with a language model, e.g. a GPT-2 (34), where the agent could choose the optimal dimensionality of the interwface.
Revised: In fact, it can provide a training agent with a simpler interface with a language model, e.g. a GPT-2 (Radford et al., 2019), where the agent could choose the optimal dimensionality of the interwface.

Original: We prove how such a volume representation eases the retrieval of stored learned patterns and how to use it to set references for other models.
Revised: We prove how such a volume representation eases the retrieval of stored learned patterns and how to use it to set references for other models.

Original: Our contributions are therefore:
Revised: Our contributions are therefore:

Original: • AriEL, a volume coding technique based on arithmetic coding(9) and k-d trees(2) (Section 3.1), to improve the retrieval of learned patterns with random sampling; • the use of a context-free grammar and a random bias in the dataset (Section 3.3), that allows us to automatically quantify the quality of the generated language; • the notion that explicit volume coding (Section 2 and 5) can be a useful technique in tasks that involve the generation of sequences of discrete symbols, such as sentences.
Revised: • AriEL, a volume coding technique based on arithmetic coding and k-d trees (Section 3.1), to improve the retrieval of learned patterns with random sampling; • the use of a context-free grammar and a random bias in the dataset (Section 3.3), that allows us to automatically quantify the quality of the generated language; • the notion that explicit volume coding (Section 2 and 5) can be a useful technique in tasks that involve the generation of sequences of discrete symbols, such as sentences.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Volume codes: We deﬁne a volume code as a pair of functions, an encoder and a decoder functions, where the encoder maps an input x into a set that contains compact and connected sets of R d (29), and the decoder maps every point within that set back to x .
Revised: Volume codes: We deﬁne a volume code as a pair of functions, an encoder and a decoder functions, where the encoder maps an input x into a set that contains compact and connected sets of R d (Munkres, 2018), and the decoder maps every point within that set back to x .

Original: It is a form of distributed representations (14) since the latter only assumes that the input x will be represented as a point in R d .
Revised: It is a form of distributed representations (Hinton et al., 1984) since the latter only assumes that the input x will be represented as a point in R d .

Original: We deﬁne point codes as the distributed representations that are not volume codes.
Revised: We deﬁne point codes as the distributed representations that are not volume codes.

Original: Volume codes differ from coarse coding (14) since in this case the code is represented by a list of zeros and ones that identiﬁes in which overlapping sets x falls into.
Revised: Volume codes differ from coarse coding (Hinton et al., 1984) since in this case the code is represented by a list of zeros and ones that identiﬁes in which overlapping sets x falls into.

Original: We call implicit volume codes , when the volume code is encouraged through a term in the loss function (1).
Revised: We call implicit volume codes , when the volume code is encouraged through a term in the loss function bengio2013representation.

Original: Both generative and discriminative models (30; 20; 16) can learn volume codes this way.
Revised: Both generative and discriminative models (Ng and Jordan, 2002; Kingma and Welling, 2014; Jebara, 2012) can learn volume codes this way.

Original: We call explicit volume code , when the volumes are constructed instead through the operations that deﬁne the architecture, and are created independently from any loss and optimizer choice.
Revised: We call explicit volume code , when the volumes are constructed instead through the operations that deﬁne the architecture, and are created independently from any loss and optimizer choice.

Original: Sentence generation through random sampling: Generative Adversarial Networks (GAN) (12) map random samples to a learned generation through a 2-players game training procedure.
Revised: Sentence generation through random sampling: Generative Adversarial Networks (GAN) (Goodfellow et al., 2014) map random samples to a learned generation through a 2-players game training procedure.

Original: They have had trouble for text generation, due to the non differentiability of the argmax at the end of the generator, and given that partially generated sequences are non trivial to score (51).
Revised: They have had trouble for text generation, due to the non differentiability of the argmax at the end of the generator, and given that partially generated sequences are non trivial to score (Yu et al., 2017).

Original: Several advances have signiﬁcantly improved their performance for text generation, such as using the generator as a reinforcement learning agent trained through Policy Gradient (51), avoiding a binary classiﬁcation in favor of a cross-entropy for the discriminator that evaluates each word generated (48), or with the Gumbel-Softmax distribution (21).
Revised: Several advances have signiﬁcantly improved their performance for text generation, such as using the generator as a reinforcement learning agent trained through Policy Gradient (Yu et al., 2017), avoiding a binary classiﬁcation in favor of a cross-entropy for the discriminator that evaluates each word generated (Xu et al., 2018), or with the Gumbel-Softmax distribution (Kusner and Hern´andez-Lobato, 2016).

Original: Random sampling the latent space is used as well by Variational Autoencoders (VAE) (20), to smooth the representation of the learned patterns.
Revised: Random sampling the latent space is used as well by Variational Autoencoders (VAE) (Kingma and Welling, 2014), to smooth the representation of the learned patterns.

Original: Training VAE for text has been shown to be possible with KL annealing and word dropout (3), and made easier with convolutional decoders (37; 49).
Revised: Training VAE for text has been shown to be possible with KL annealing and word dropout (Bowman et al., 2016), and made easier with convolutional decoders (Severyn et al., 2017; Yang et al., 2017).

Original: Several works explore how VAE and GAN can be combined (26; 40; 27).
Revised: Several works explore how VAE and GAN can be combined (Makhzani et al., 2015; Tolstikhin et al., 2017; Mescheder et al., 2017).

Original: AriEL can be used as a generator or a discriminator in a GAN, or as an encoder or a decoder in an autoencoder.
Revised: AriEL can be used as a generator or a discriminator in a GAN, or as an encoder or a decoder in an autoencoder.

Original: However it differs from them in the explicit procedure to construct volumes in the latent space that correspond to different inputs.
Revised: However it differs from them in the explicit procedure to construct volumes in the latent space that correspond to different inputs.

Original: The intention is to ﬁll the entire latent space with the learned patterns, to ease the retrieval by uniform random sampling.
Revised: The intention is to ﬁll the entire latent space with the learned patterns, to ease the retrieval by uniform random sampling.

Original: Arithmetic coding and neural networks: AC is one of the most efﬁcient lossless data compression techniques (46; 9).
Revised: Arithmetic coding and neural networks: AC is one of the most efﬁcient lossless data compression techniques (Witten et al., 1987; Elias and Abramson, 1963).

Original: AC assigns a sequence to a segment in [0,1] whose length is proportional to its frequency in the dataset.
Revised: AC assigns a sequence to a segment in [0,1] whose length is proportional to its frequency in the dataset.

Original: AC is used for neural network compression (44) but typically, neural networks are used in AC as the model of the data distribution, to perform prediction based compression (32; 41; 17; 24; 39).
Revised: AC is used for neural network compression (Wiedemann et al., 2019) but typically, neural networks are used in AC as the model of the data distribution, to perform prediction based compression (Pasero and Montuori, 2003; Triantafyllidis and Strintzis, 2002; Jiang et al., 1993; Ma et al., 2019; Tatwawadi, 2018).

Original: We turn AC into a compression algorithm in d real numbers, to combine its properties with the properties of high-dimensional spaces, which is the domain of neural networks.
Revised: We turn AC into a compression algorithm in d real numbers, to combine its properties with the properties of high-dimensional spaces, which is the domain of neural networks.

Original: K-d trees and neural networks: KdT (2) is a data structure for storage that can handle different types of queries efﬁciently.
Revised: K-d trees and neural networks: KdT (Bentley, 1975) is a data structure for storage that can handle different types of queries efﬁciently.

Original: It is typically used as a fast approximation to k-nearest neighbours in low dimensions (10).
Revised: It is typically used as a fast approximation to k-nearest neighbours in low dimensions (Friedman et al., 1977).

Original: It gives a binary label to the data with respect to its median.
Revised: It gives a binary label to the data with respect to its median.

Original: It moves through the k dimensions of the data and repeats the process.
Revised: It moves through the k dimensions of the data and repeats the process.

Original: Neural networks are typically used in conjuction with KdT to reduce the dimensionality of the search space, for KdT to be able to perform queries efﬁciently (47; 50; 42).
Revised: Neural networks are typically used in conjuction with KdT to reduce the dimensionality of the search space, for KdT to be able to perform queries efﬁciently (Woodbridge et al., 2018;

Revised: Yin et al., 2017; Vasudevan et al., 2009).

Original: KdT has been used as well in combination with Delaunay triangulation for function learning, as an alternative to NN with Backpropagation (13).
Revised: KdT has been used as well in combination with Delaunay triangulation for function learning, as an alternative to NN with Backpropagation (Gross, 1995).

Original: A KdT inspired algorithm is used in (25) to guide the creation of neurons to grow a neural network.
Revised: A KdT inspired algorithm is used in (Maillard and Solaiman, 1994) to guide the creation of neurons to grow a neural network.

Original: We use KdT to make sure that when we make AC turn multidimensional, it makes use of all the space available.
Revised: We use KdT to make sure that when we make AC turn multidimensional, it makes use of all the space available.

Original:  M ETHODOLOGY 3. A 
Revised:  M ETHODOLOGY 3. A RI EL: VOLUME CODING OF LANGUAGE IN CONTINUOUS SPACES

Original: A C S -> A | B | AA | AB | AC | BC | ABC | BCC A A

Original: A B C

Original: A B

Original: B

Original: B C

Original: B C C

Original: A    1 B B C A A A A B A C
Revised: [Figure]

Original: Arithmetic Coding AriEL B A d 1 = 1 mod 2 d 1 =  1 B
Revised: [Equation]

Original: B C B C C A A A A B A C A B C d 2 = 2 mod 2 d 2 =d 3 = 3 mod 2 d 3 =
Revised: CFG preserved pro vides boundaries bols found

Original: [Figure]

Original: AriEL maps the sentence ( s 1 , · · · , s n ) to a d -dimensional volume of P (( s 1 , · · · , s n ))
Revised: AriEL maps the sentence ( s 1 , · · · , s n ) to a d -dimensional volume of P (( s 1 , · · · , s n ))

Original: =
Revised: =

Original: Π ni =1 P
Revised: Π ni =1 P

Original: s i | ( s j ) j<i ) .
Revised: s i | ( s j ) j<i ) .

Original: The sentence is encoded as the center of that volume for simplicity and any point within it is decoded to the same sentence.
Revised: The sentence is encoded as the center of that volume for simplicity, and any point within it is decoded to the same sentence.

Original: Decoding iteratively computes the bounds of the volumes for all possible next symbols and checks inside which bounds the vector is, to ﬁnd the next symbol at each step.
Revised: Decoding iteratively computes the bounds of the volumes for all possible next symbols and checks inside which bounds the vector is, to ﬁnd the next symbol at each step.

Original: The algorithm is described in detail in the Supplementary Material.
Revised: The algorithm is described in detail in the Supplementary Material.

Original: To adapt KdT to more splits than binary, we split the chosen dimension giving a space from 0 to 1 to each possible next symbol, proportional to its probability.
Revised: To adapt KdT to more splits than binary, we split the chosen dimension giving a space from 0 to 1 to each possible next symbol, proportional to its probability.

Original: The ﬁrst symbol in the sentence will be assigned a segment of length P ( s 1 ) in the ﬁrst axis chosen d 1 , and next symbols will be assigned a segment proportional to their probability conditional to the symbols previously seen e.g.
Revised: The ﬁrst symbol in the sentence will be assigned a segment of length P ( s 1 ) in the ﬁrst axis chosen d 1 , and next symbols will be assigned a segment proportional to their probability conditional to the symbols previously seen e.g.

Original: P ( s 3 | ( s 2 , s 1 )) on the axis d 3 , where s 1 , s 2 and s 3 are the ﬁrst three symbols in the sentence.
Revised: P ( s 3 | ( s 2 , s 1 )) on the axis d 3 , where s 1 , s 2 and s 3 are the ﬁrst three symbols in the sentence.

Original: Then  turn to the following axis and continue the process of splitting and turning (ﬁgure 1).
Revised: Then we turn to the following axis and continue the process of splitting and turning (ﬁgure 1).

Original: We select the next axis in R d to split to be d i = i mod d , where i ∈ { 1 , 2 , . . . , n } and n is the length of the sequence.
Revised: We select the next axis in R d to split to be d i = i mod d , where i ∈ { 1 , 2 , . . . , n } and n is the length of the sequence.

Original: If n is larger than the dimension d , then the segment in d i is split again.
Revised: If n is larger than the dimension d , then the segment in d i is split again.

Original: We applied a neural network to approximate the true statistics of the data P ( s i | ( s j ) j<i ) , the Language Model (LM) of AriEL, P LM ( s i | ( s j ) j<i ) .
Revised: We applied a neural network to approximate the true statistics of the data P ( s i | ( s j ) j<i ) , the Language Model (LM) of AriEL, P LM ( s i | ( s j ) j<i ) .

Original: This will approximate the frequency information that makes
Revised: This will approximate the frequency information that makes

Original: AC entropically efﬁcient since after a successful training, P LM ( · ) will converge to P ( · ) .
Revised: AC entropically efﬁcient since after a successful training, P LM ( · ) will converge to P ( · ) .

Original: AriEL conserves then the arithmetic coding property of assigning larger volume to frequent sentences.
Revised: AriEL conserves then the arithmetic coding property of assigning larger volume to frequent sentences.

Original: AriEL only uses a bounded region of R d , the interval [0 , 1] d , so encoder and decoder map each input to a compact set and from a compact set.
Revised: AriEL only uses a bounded region of R d , the interval [0 , 1] d , so encoder and decoder map each input to a compact set and from a compact set.

Original: Moreover, they assign sequence x to a hyper-rectangle (18) and back.
Revised: Moreover, they assign sequence x to a hyperrectangle (Johnson, 2018) and back.

Original: Since hyper-rectangles cannot be divided into two disjoint non-empty closed sets, they are connected (29).
Revised: Since hyper-rectangles cannot be divided into two disjoint nonempty closed sets, they are connected (Munkres, 2018).

Original: Therefore AriEL is a volume code .
Revised: Therefore AriEL is a volume code .

Original: AriEL is an explicit volume code since its LM is trained only on a next word prediction log-likelihood loss, without a regularization term that encourages smoothness, and the volumes are constructed by arranging the softmax outputs into a d dimensional grid, operation performed with any choice of loss or optimizer.
Revised: AriEL is an explicit volume code since its LM is trained only on a next word prediction log-likelihood loss, without a regularization term that encourages smoothness, and the volumes are constructed by arranging the softmax outputs into a d dimensional grid, operation performed with any choice of loss or optimizer.

Original: AriEL with a RNN-based language model has a computational complexity of O ( nD 2 ) for both encoding and decoding, as it can be seen in algorithms 1 and 2, where n is the length of the sequence and D is the dimensionality of the RNN hidden state.
Revised: AriEL with a RNN-based language model has a computational complexity of O ( nD 2 ) for both encoding and decoding, as it can be seen in algorithms 1 and 2, where n is the length of the sequence and D is the dimensionality of the RNN hidden state.

Original: We use capital D to refer to the length of the longest hidden layer among the recurrent layers in the encoder or in the decoder, while d refers to the length of the latent space.
Revised: We use capital D to refer to the length of the longest hidden layer among the recurrent layers in the encoder or in the decoder, while d refers to the length of the latent space.

Original: AriEL has a time complexity of O ( n ) for both encoding and decoding, which is on par with conventional recurrent networks for seq2seq learning.
Revised: AriEL has a minimum number of sequential operations of O ( n ) for both encoding and decoding, which is on par with conventional recurrent networks for seq2seq learning.

Original: 3. N EURAL N ETWORKS : MODELS AND EXPERIMENTAL CONDITIONS
Revised: 3. N EURAL N ETWORKS : MODELS AND EXPERIMENTAL CONDITIONS

Original: We compare AriEL to some of the classical approaches to map variable length discrete spaces to ﬁxed length continuous spaces.
Revised: We compare AriEL to some of the classical approaches to map variable length discrete spaces to ﬁxed length continuous spaces.

Original: These are the sequence to sequence recurrent autoencoders (AE) (38), their variational version (VAE) (3) and Tranformer (43).
Revised: These are the sequence to sequence recurrent autoencoders (AE) (Sutskever et al., 2014), their variational version (VAE) (Bowman et al., 2016) and Tranformer (Vaswani et al., 2017).

Original: We trained them for next word prediction of word s i , when all the previous words are given as input, and they are trained over the biased train set, deﬁned in section 3.3.
Revised: We trained them for next word prediction of word s i , when all the previous words are given as input, and they are trained over the biased train set, deﬁned in section 3.3.

Original: All of them can be split into an encoder that maps the sentences at the input into R d , and a decoder that maps back from R d into a sentence.
Revised: All of them can be split into an encoder that maps the sentences at the input into R d , and a decoder that maps back from R d into a sentence.

Original: More training details can be found in the appendix.
Revised: More training details can be found in the appendix.

Original: In this work, AriEL’s language model neural network P LM consists of a word embedding of size 64, followed by a 140 units LSTM, a feedforward layer and a softmax over next possible symbols.
Revised: In this work, AriEL’s language model neural network P LM consists of a word embedding of size 64, followed by a 140-unit LSTM, a feedforward layer and a softmax over next possible symbols.

Original: At test time the argmax is not applied directly to the softmax, but the probabilities deﬁned by the latter are used as a deterministic russian roulette with the latent space point as the deterministic pointer that chooses the position in the roulette.
Revised: At test time the argmax is not applied directly to the softmax, but the probabilities deﬁned by the latter are used as a deterministic russian roulette with the latent space point as the deterministic pointer that chooses the position in the roulette.

Original: However the language model is trained for next time step prediction through cross-entropy.
Revised: However the language model is trained for next time step prediction through cross-entropy.

Original: For both AE and VAE, we stack two GRU layers (5) withunits at both, the encoder and the decoder, to increase their representational capabilities (31).
Revised: For both AE and VAE, we stack two GRU layers (Cho et al., 2014) with 128 units at both, the encoder and the decoder, to increase their representational capabilities (Pascanu et al., 2014).

Original: Other recurrent networks gave similar results (15; 23).
Revised: Other recurrent networks gave similar results (Hochreiter and Schmidhuber, 1997; Li et al., 2018).

Original: The last encoder layer has either d = 16 units or d = 512 for all methods.
Revised: The last encoder layer has either d = 16 units or d = 512 for all methods.

Original: The decoder outputs a softmax over the entire vocabulary.
Revised: The decoder outputs a softmax over the entire vocabulary.

Original: Tranformer (43) is the state-of-the-art in many S2S problems (43; 6; 33).
Revised: Tranformer (Vaswani et al., 2017) is the state-of-the-art in many S2S problems (Vaswani et al., 2017;

Revised: Dai et al., 2019; Radford et al., 2018).

Original: Since it is a ﬁxed-length representation at the word level but it is variable-length at the sentence level, we padded all sentences to the maximum length in the dataset to be able to compare its latent space capacity to the other models.
Revised: Since it is a ﬁxed-length representation at the word level but it is variable-length at the sentence level, we padded all sentences to the maximum length in the dataset to be able to compare its latent space capacity to the other models.

Original: We take as its latent dimension the connection between the encoder and decoder, with d model size, that will take a value of 16 or 512.
Revised: We take as its latent dimension the connection between the encoder and decoder, with d model size, that will take a value of 16 or 512.

Original: We choose most parameters as in the original work (43): the number of attention heads as n head = 8 , the key and value dimension as d key = 64 and d value = 64 , a dropout regularization of 0.1, and we only change the stack of identical decoders and encoders to n layers = 2 , and the dimension of the inner feed-forward network to d inner layer =to have a number of parameters similar to the other methods.
Revised: We choose most parameters as in the original work (Vaswani et al., 2017): the number of attention heads as n head = 8 , the key and value dimension as d key = 64 and d value = 64 , a dropout regularization of 0.1, and we only change the stack of identical decoders and encoders to n layers = 2 , and the dimension of the inner feed-forward network to d inner layer = 256 to have a number of parameters similar to the other methods.

Original: In the GuessWhat?!
Revised: In the GuessWhat?!

Original: dataset we tested n layers = 20 to have an amount of parameters comparable for d = 16 to the other methods, but performed worse than n layers = 2 , so we report the latter.
Revised: dataset we tested n layers = 20 to have an amount of parameters comparable for d = 16 to the other methods, but performed worse than n layers = 2 , so we report the latter.

Original: 3. D ATASETS : TOY AND HUMAN SENTENCES
Revised: 3. D ATASETS : TOY AND HUMAN SENTENCES

Original: We perform our analysis on two datasets.
Revised: We perform our analysis on two datasets.

Original: A toy dataset of sentences generated from a context-free grammar (CFG) and a realistic dataset of sentences written by humans playing a cooperative game.
Revised: A toy dataset of sentences generated from a context-free grammar (CFG) and a realistic dataset of sentences written by humans playing a cooperative game.

Original: The toy dataset: we generate questions about objects with a CFG (appendix 1).
Revised: The toy dataset: we generate questions about objects with a CFG (appendix 1).

Original: To stress the learning methods and understand their limits we choose a CFG with a large vocabulary and numerous grammar rules, rather than more classic alternatives (e.g. REBER).
Revised: To stress the learning methods and understand their limits we choose a CFG with a large vocabulary and numerous grammar rules, rather than more classic alternatives (e.g. REBER).

Original: The intention is as well to focus on dialogue agents and that’s the reason why all sentences are framed as questions about objects.
Revised: The intention is as well to focus on dialogue agents and that’s the reason why all sentences are framed as questions about objects.

Original: We distinguish between unbiased sentences, those that have been simply sampled from the CFG, and biased sentences, those that after being sampled from the CFG have been selected according to an additional structural constraint.
Revised: We distinguish between unbiased sentences, those that have been simply sampled from the CFG, and biased sentences, those that after being sampled from the CFG have been selected according to an additional structural constraint.

Original: To do so we generate an adjacency matrix of words that can occur together in the same sentence, and we use that as the ﬁlter to bias the sentences.
Revised: To do so we generate an adjacency matrix of words that can occur together in the same sentence, and we use that as the ﬁlter to bias the sentences.

Revised: Once a sentence is produced from the CFG, if all its words can be together in a sentence judged by the adjacency matrix, the sentence is considered as biased, and unbiased otherwise.

Original: For simplicity the adjacency matrix has been generated randomly.
Revised: For simplicity the adjacency matrix is a random matrix of zeros and ones, making sure that some symbols such as the , it or ? , can be found in both types of sentences.

Original: The intention is to emulate the setting were a CFG is constrained by realistic scenes, where not all the grammatically correct sentences are semantically correct: e.g.”Is it the wooden shower in the kitchen ?”
Revised: The intention is to emulate the setting were a CFG is constrained by realistic scenes, where not all the grammatically correct sentences are semantically correct: e.g.

Original:  could be grammatical, but semantically incorrect given that it is unusual in a realistic scene.
Revised: ”Is it the wooden shower in the kitchen ?” could be grammatical, but semantically incorrect given that it is unusual in a realistic scene.

Original: We use it to detect how each learning method is able to extract the grammar and extract the roles of each word, despite a bias that makes it harder.
Revised: We use it to detect how each learning method is able to extract the grammar and extract the roles of each word, despite a bias that makes it harder.

Original: The vocabulary consists of 840 words.
Revised: The vocabulary consists of 840 words.

Original: The maximal and mean length of the sentences is of 19 and 9.9 symbols.
Revised: The maximal and mean length of the sentences is of 19 and 9.9 symbols.

Original: We split the biased dataset into 1M train, 10k test and 512 validation sentences, where no sentence is shared between sets.
Revised: We split the biased dataset into 1M train, 10k test and 512 validation sentences, where no sentence is shared between sets.

Original: We created another set of 10k unbiased test sentences with the same CFG, where we only gather sentences that do not follow the adjacency matrix, to make sure that the overlap of this test set is zero with previous ones.
Revised: We created another set of 10k unbiased test sentences with the same CFG, where we only gather sentences that do not follow the adjacency matrix, to make sure that the overlap of this test set is zero with previous ones.

Original: We train on the biased sentences and we test if they grasped the grammar behind, with the unbiased.
Revised: We train on the biased sentences and we test if they grasped the grammar behind, with the unbiased.

Original: The real dataset: we choose the GuessWhat?!
Revised: The real dataset: we choose the GuessWhat?!

Original: dataset (7), a dataset of sentences asked by humans to humans to solve a cooperative game.
Revised: dataset (De Vries et al., 2017), a dataset of sentences asked by humans to humans to solve a cooperative game.

Original: It has a vocabulary of 10,469 words, an order of magnitude larger than the toy CFG.
Revised: It has a vocabulary of 10,469 words, an order of magnitude larger than the toy CFG.

Original: The maximal and mean length of the sentences are of 57 and 5.9 symbols.
Revised: The maximal and mean length of the sentences are of 57 and 5.9 symbols.

Original: 3. E VALUATION M ETRICS 3.4. Q UALITATIVE EVALUATIONS
Revised: 3. E VALUATION M ETRICS 3.4. Q UALITATIVE EVALUATIONS

Original: The two qualitative studies are: (1) we list a few samples of reconstruction via next word prediction of unbiased sentences, to understand the generalization capabilities of the different models (table 1), (2) we list a few samples of generated sentences when the latent space is sampled randomly, to understand the generation capabilities (table 2).
Revised: The two qualitative studies are: (1) we list a few samples of reconstruction via next word prediction of unbiased sentences, to understand the generalization capabilities of the different models (table 1), (2) we list a few samples of generated sentences when the latent space is sampled randomly, to understand the generation capabilities (table 2).

Original: 3.4. Q UANTITATIVE EVALUATIONS ON THE TOY GRAMMAR , CFG
Revised: 3.4. Q UANTITATIVE EVALUATIONS ON THE TOY GRAMMAR , CFG

Original: We propose measures for 3 properties of an autoencoder: the quality of generation, prediction and generalization.
Revised: We propose measures for 3 properties of an autoencoder: the quality of generation, prediction and generalization.

Original: We perform our studies for networks with a latent dimension of 16 units, to understand their compression limits, and for 512 units, which is often taken as the default size (20; 43).
Revised: We perform our studies for networks with a latent dimension of 16 units, to understand their compression limits, and for 512 units, which is often taken as the default size (Kingma and Welling, 2014; Vaswani et al., 2017).

Original: Generation/Decoding Quality is evaluated with sentences produced by the decoder when the latent space of each model is sampled randomly.
Revised: Generation/Decoding Quality is evaluated with sentences produced by the decoder when the latent space of each model is sampled randomly.

Original: The sampling is done uniformly in the continuous latent space, within the maximal hyper-cube deﬁned by the encoded test sentences.
Revised: The sampling is done uniformly in the continuous latent space, within the maximal hyper-cube deﬁned by the encoded test sentences.

Original: We sample 10k sentences and apply four measures: i) grammar coverage (GC) as the number of grammar rules (e.g. single adjective, multiple adjectives) that could be parsed in the sampled sentences, over four, the maximal number of adjectives plus one for sentences without adjectives; ii) vocabulary coverage (VC) as the ratio between the number of words in the sampled sentences, over 840, the size of the complete vocabulary; iii) uniqueness (U) as a ratio of unique sampled sentences; and iv) validity (V) as a ratio of valid sampled sentences, sentences that were unique and grammatically correct. 
Revised: We sample 10k sentences and apply four measures: i) grammar coverage (GC) as the number of grammar rules (e.g. single adjective, multiple adjectives) that could be parsed in the sampled sentences, over four, the maximal number of adjectives plus one for sentences without adjectives; ii) vocabulary coverage (VC) as the ratio between the number of words in the sampled sentences, over 840, the size of the complete vocabulary; iii) uniqueness (U) as a ratio of unique sampled sentences; and iv) validity (V) as a ratio of valid sampled sentences, sentences that were unique and grammatically correct. We keep our deﬁnition of grammar rule simple, for computational efﬁciency, and for clarity, given that the grammar tree is deﬁned with an ambiguous number of placeholders and terminal symbols.

Original: Prediction Quality is evaluated by encoding and decoding the 10k biased test sentences as follows:
Revised: Prediction Quality is evaluated by encoding and decoding the 10k biased test sentences as follows:

Original: i) prediction accuracy biased (PAB) as a ratio of correctly reconstructed sentences (i.e. all words must match); ii) grammar accuracy (GA) as a ratio of grammatically correct reconstructions (i.e. can be parsed by the CFG, even if the reconstruction is not accurate).
Revised: i) prediction accuracy biased (PAB) as a ratio of correctly reconstructed sentences (i.e all words must match); ii) grammar accuracy (GA) as a ratio of grammatically correct reconstructions (i.e. can be parsed by the CFG, even if the reconstruction is not accurate).

Original: and

Original:  iii) bias accuracy (BA) as the ratio of inaccurate reconstructions that are still grammatical and keep the bias of the training set.
Revised: and iii) bias accuracy (BA) as the ratio of inaccurate reconstructions that are still grammatical and keep the bias of the training set.

Original: Generalization Quality is evaluated using the 10k unbiased test sentences while the embeddings were trained on the biased training set.
Revised: Generalization Quality is evaluated using the 10k unbiased test sentences while the embeddings were trained on the biased training set.

Original: The prediction accuracy unbiased (PAU) is computed in the same way as PAB , as the ratio of correctly reconstructed ubiased sentences.
Revised: The prediction accuracy unbiased (PAU) is computed in the same way as PAB , as the ratio of correctly reconstructed ubiased sentences.

Original: It allows us to measure how well the latent space generalizes to grammatically correct sentences outside the bias.
Revised: It allows us to measure how well the latent space generalizes to grammatically correct sentences outside the bias.

Original: 3.4. Q UANTITATIVE EVALUATIONS ON THE REAL DATASET , G UESS W HAT ?!
Revised: 3.4. Q UANTITATIVE EVALUATIONS ON THE REAL DATASET , G UESS W HAT ?!

Original: In a real dataset we don’t have a notion of what is grammatically correct, since humans can use spontaneously ungrammatical constructions.
Revised: In a real dataset we don’t have a notion of what is grammatically correct, since humans can use spontaneously ungrammatical constructions.

Original: We quantiﬁed the quality of the language learned with two measures: uniqueness is the percentage of the sentences generated with random sampling that was unique over the 10K generations and validity was the percentage of the unique sentences that could be found in the training data, indicating how easy it was to retrieve the learned information.
Revised: We quantiﬁed the quality of the language learned with two measures: uniqueness is the percentage of the sentences generated with random sampling that was unique over the 10K generations and validity was the percentage of the unique sentences that could be found in the training data, indicating how easy it was to retrieve the learned information.

Original: 3.4. Q UANTITATIVE EVALUATIONS : RANDOM INTERPOLATIONS WITHIN A RI EL
Revised: 3.4. Q UANTITATIVE EVALUATIONS : RANDOM INTERPOLATIONS WITHIN A RI EL

Original: In ﬁgure 2 we show what we call the interpolation diversity given the dimension of the latent space.
Revised: In ﬁgure 2 we show what we call the interpolation diversity given the dimension of the latent space.

Original: It measures how many of the sentences generated through a straight line between two random points in R d were unique and grammatically correct for AriEL for different values of d .
Revised: It measures how many of the sentences generated through a straight line between two random points in R d were unique and grammatically correct for AriEL for different values of d .

Original: The Language
Revised: The Language

Original: Model tested is the one trained on the toy grammar.
Revised: Model tested is the one trained on the toy grammar.

Original:  R ESULTS 4. Q UALITATIVE E VALUATIONS
Revised:  R ESULTS 4. Q UALITATIVE E VALUATIONS

Original: We present the qualitative studies performed for d = 16 .
Revised: We present the qualitative studies performed for d = 16 .

Original: Table 1 shows the output of the generalization study.
Revised: Table 1 shows the output of the generalization study.

Original: To avoid cherry picking, we display the ﬁrst 4 reconstructed sentences.
Revised: To avoid cherry picking, we display the ﬁrst 4 reconstructed sentences.

Original: AE and
Revised: AE and

Original: VAE fail to generalize to the unbiased language, however both manage to keep the structure at the output of the input sentence.
Revised: VAE fail to generalize to the unbiased language, however both manage to keep the structure at the output of the input sentence.

Original: Their behavior improved signiﬁcantly when the latent space dimension is increased to d = 512 , with the corresponding increase of parameters.
Revised: Their behavior improved signiﬁcantly when the latent space dimension is increased to d = 512 , with the corresponding increase of parameters.

Original: In theory, AriEL is able to reconstruct any sequence by design, by keeping a volume for each of them.
Revised: In theory, AriEL is able to reconstruct any sequence by design, by keeping a volume for each of them.

Original: However in practice, it failed only slightly less often than the Transformer.
Revised: However in practice, it failed only slightly less often than the Transformer.

Original: Both produce reconstructions of the unbiased input at a similar rate, as it can be see in table 1 and in the metric PAU in table 3 and ﬁgure S2.
Revised: Both produce reconstructions of the unbiased input at a similar rate, as it can be see in table 1 and in the metric PAU in table 3 and ﬁgure S2.

Original: This means that to a reasonable degree, the areas that represent unseen data during training, are available and relatively easy to track for AriEL and Transformer.
Revised: This means that to a reasonable degree, the areas that represent unseen data during training, are available and relatively easy to track for AriEL and Transformer.

Original: Instead, all the latent space seems to be taken almost exclusively by the content of the training set for AE and VAE, since sentences that are not seen during training (the unbiased sentences) cannot be reconstructed at all.
Revised: Instead, all the latent space seems to be taken almost exclusively by the content of the training set for AE and VAE, since sentences that are not seen during training (the unbiased sentences) cannot be reconstructed at all.

Original: The generation study is shown in Table 2 (ﬁrst 4 samples for each model).
Revised: The generation study is shown in Table 2 (ﬁrst 4 samples for each model).

Original: AriEL excels at this task, and almost all generations are unique and grammatically correct ( valid ).
Revised: AriEL excels at this task, and almost all generations are unique and grammatically correct ( valid ).

Original: AE and VAE perform remarkably well given the small latent space.
Revised: AE and VAE perform remarkably well given the small latent space.

Original: As it is shown in the quantitative study, VAE almost triples AE performance in terms of generation of valid sentences when d = 16 (table 3).
Revised: As it is shown in the quantitative study, VAE almost triples AE performance in terms of generation of valid sentences when d = 16 (table 3).

Original: Transformer performs poorly at generating grammatical sentences when the latent space is sampled randomly.
Revised: Transformer performs poorly at generating grammatical sentences when the latent space is sampled randomly.

Original: The quantitative analysis reveals however that with the increase of the latent space, Transformer,
Revised: The quantitative analysis reveals however that with the increase of the latent space, Transformer,

Original: AE and VAE achieve all improved validity, remaining at one third the reference set by AriEL.[Table] [Table]
Revised: AE and VAE achieve all improved validity, remaining at one third the reference set by AriEL.

Original: when sampled uniformly in the latent space.

Original: Red deﬁnes grammatically incorrect generations according to the CFG the models are trained on.

Original: AriEL produces an extremely varied set of grammatically correct sentences, most of which keep the bias of the training set.

Original: Transformer reveals itself to be hard to control via random sampling of the latent space, since it almost never produces correct sentences with this method.

Original: AE and VAE manage to produce several different sentences, the latter producing more non grammatical, but as well more varied grammatical ones.

Original: [Table]

Original: [Figure]

Original: i n t e r p o l a t i o n d

Original: i v e r s i t y

Original: [Table]

Original: Questioner data.

Original: For the real dataset the pattern is repeated: AriEL shows that a larger value of valid sentences is possible .

Original: Transformer 16 gave better results than when n layers was increased from 2 to 20 to increase its learnable parameters from 588K to 2,666K.

Original: 4. Q UANTITATIVE E VALUATIONS
Revised: 4. Q UANTITATIVE E VALUATIONS

Original: The results of the quantitative study are shown in table 3 and in ﬁgure S2.
Revised: The results of the quantitative study are shown in table 3 and in ﬁgure S2.

Original: AriEL outperforms or closely matches every other method for all the 8 measures, outperforming the rest by a large margin for validity, i.e. unique and grammatical sentences generated, the most important of the metrics.
Revised: AriEL outperforms or closely matches every other method for all the 8 measures, outperforming the rest by a large margin for validity, i.e. unique and grammatical sentences generated, the most important of the metrics.

Original: Transformer performs remarkably well at not overﬁtting and it is able to reconstruct biased and unbiased sentences better than the other non-AriEL methods, even under-parameterized ( d = 16 ).
Revised: Transformer performs remarkably well at not overﬁtting and it is able to reconstruct biased and unbiased sentences better than the other non-AriEL methods, even under-parameterized ( d = 16 ).

Original: It manages to cover all grammar rules in generation but it performs very poorly at generating a diverse set of valid sentences by random sampling.
Revised: It manages to cover all grammar rules in generation but it performs very poorly at generating a diverse set of valid sentences by random sampling.

Original: It only needed one iteration through the data to achieve almost perfect validation accuracy, without losing performance when we trained for the remaining 9 epochs.
Revised: It only needed one iteration through the data to achieve almost perfect validation accuracy, without losing performance when we trained for the remaining 9 epochs.

Original: VAE 16 despite the poor generalization to the biased and the unbiased test set, results in the best non-AriEL generator, measured by validity.
Revised: VAE 16 despite the poor generalization to the biased and the unbiased test set, results in the best non-AriEL generator, measured by validity.

Original: The conﬂict between log-prior and log-likelihood, encouraged VAE to look for sentences outside the bias, since it was able to produce more grammatically correct sentences, albeit unbiased, than AE.
Revised: The conﬂict between log-prior and log-likelihood, encouraged VAE to look for sentences outside the bias, since it was able to produce more grammatically correct sentences, albeit unbiased, than AE.

Original: Increasing the learned parameters
Revised: Increasing the learned parameters

Revised: [Table]

Revised: [Table]

Revised: when sampled uniformly in the latent

Revised: Red deﬁnes grammatically incorrect generations according to the CFG the models are trained on.

Original:  ( d = 512 ), had no effect on Transformer, that was already excellent in several of the metrics, apart from a signiﬁcant improvement in validity.
Revised: AriEL produces an extremely varied set of grammatically correct sentences, most of which keep the bias of the training set. Transformer reveals itself to be hard to control via random sampling of the latent space, since it almost never produces correct sentences with this method. AE and VAE manage to produce several different sentences, the latter producing more non grammatical, but as well more varied grammatical ones. ( d = 512 ), had no effect on Transformer, that was already excellent in several of the metrics, apart from a signiﬁcant improvement in validity.

Original: However, a larger latent space and the increase in number of parameters that followed, prevented AE and VAE from overﬁtting (better PAU and PAB).
Revised: However, a larger latent space and the increase in number of parameters that followed, prevented AE and VAE from overﬁtting (better PAU and PAB).

Original: When trained on human sentences, on the GuessWhat?!
Revised: When trained on human sentences, on the GuessWhat?!

Original: dataset, AriEL sets again a large validity to be reached.
Revised: dataset, AriEL sets again a large validity to be reached.

Original: Every approach seems to generate more unique sentences than AriEL, but the fraction of them that is a good generation is very small.
Revised: Every approach seems to generate more unique sentences than AriEL, but the fraction of them that is a good generation is very small.

Original: Less than 6% of the unique sentences generated by
Revised: Less than 6% of the unique sentences generated by

Original: AE, VAE and Transformer are in the training set, while AriEL achieves 22 .
Revised: AE, VAE and Transformer are in the training set, while AriEL achieves 22 .

Original: 47% and more.
Revised: 47% and more.

Original: In the interpolation diversity study (ﬁgure 2) we see that for low d , we have to pass through many sentences in between two random points in the latent space, while as we augment the dimensionality, we distribute the sentences in different directions.
Revised: In the interpolation diversity study (ﬁgure 2) we see that for low d , we have to pass through many sentences in between two random points in the latent space, while as we augment the dimensionality, we distribute the sentences in different directions.

Original: Therefore we ﬁnd less sentences when we move on the straight line between two random points.
Revised: Therefore we ﬁnd less sentences when we move on the straight line between two random points.

Original: The speciﬁc curve, lower threshold and speed of decay, will vary for different vocabulary sizes and given the complexity of the language learned. 
Revised: The speciﬁc curve, lower threshold and speed of decay, will vary for different vocabulary sizes and given the complexity of the language learned. [Table] [Figure]

Revised: i n t e r p o l a t i o n d

Revised: i v e r s i t y

Revised: [Table]

Original:   D ISCUSSION
Revised: Questioner data. For the real dataset the pattern is repeated: AriEL shows that a larger value of valid sentences is possible . Transformer 16 gave better results than when n layers was increased from 2 to 20 to increase its learnable parameters from 588K to 2,666K.  D ISCUSSION

Original: AriEL latent space d is a free parameter.
Revised: AriEL latent space d is a free parameter.

Original: It is worth to stress that the size d of the latent space of AriEL can be deﬁned at any time, for a ﬁxed Language Model.
Revised: It is worth to stress that the size d of the latent space of AriEL can be deﬁned at any time, for a ﬁxed Language Model.

Original: It could therefore be controlled with a learnable parameter, with the activity of another neuron, or as another function of the input.
Revised: It could therefore be controlled with a learnable parameter, with the activity of another neuron, or as another function of the input.

Original: In fact, as we increase d , the volumes will have more neighbouring volumes that represent different sentences, as conﬁrmed by the interpolation study, ﬁgure 2.
Revised: In fact, as we increase d , the volumes will have more neighbouring volumes that represent different sentences, as conﬁrmed by the interpolation study, ﬁgure 2.

Original: It could have implications as well during training e.g. to have a gradient that can rely more on its angle than on its magnitude.
Revised: It could have implications as well during training e.g. to have a gradient that can rely more on its angle than on its magnitude.

Original: What to choose for a learning agent with a language module?
Revised: What to choose for a learning agent with a language module?

Original: Our study suggests that a learning agent that needs a language model to interact with other agents, would beneﬁt from AriEL to generate a diverse language.
Revised: Our study suggests that a learning agent that needs a language model to interact with other agents, would beneﬁt from AriEL to generate a diverse language.

Original: It outperformed the rest both in the toy dataset and on the real data.
Revised: It outperformed the rest both in the toy dataset and on the real data.

Original: Partial evidence for volume codes.
Revised: Partial evidence for volume codes.

Original: The experiments performed suggest that the volume aspect of
Revised: The experiments performed suggest that the volume aspect of

Original: AriEL is to be held responsible of its success.
Revised: AriEL is to be held responsible of its success.

Original: We have provided evidence on how volume coding can be beneﬁcial for retrieval of stored information that is composed of discrete symbols, and variable length, by random sampling, in contrast with simply distributed representations.
Revised: We have provided evidence on how volume coding can be beneﬁcial for retrieval of stored information that is composed of discrete symbols, and variable length, by random sampling, in contrast with simply distributed representations.

Original: It is in fact AriEL to generate more valid sentences, an explicit volume coding method.
Revised: It is in fact AriEL to generate more valid sentences, an explicit volume coding method.

Original: VAE is the second on the toy dataset, an implicit volume coding method, but it performed poorly on the real dataset.
Revised: VAE is the second on the toy dataset, an implicit volume coding method, but it performed poorly on the real dataset.

Revised: The low prediction accuracy of AE/VAE, has to be read in conjuction with the grammar accuracy: it basically means that those methods are overﬁtting the training data, and even if they often manage to produce grammatically correct sentences when a test sentence is given at the input, the volumes/points that would represent new test sentences, seem to have disappeared, all the latent space is dedicated to only the training set.

Revised: Transformers are hard to sample from the latent space.

Revised: The Transformer has been used in this work in an uncommon way: by sampling its latent space instead of its input space.

Revised: Its low validity score reﬂects that.

Original: C ONCLUSION AND F UTURE W ORK
Revised: Our aim was to better understand the latent organization of language, so, we do not want to suggest this is the most effective way to use Transformer. Transformer is excellent when sampled in the input space, but it’s difﬁcult to sample from the latent space. This is so because Transformer represents each word by a d dimensional vector while the other approaches represent whole sentences in d dimensional vectors, Transformer needs an extremely high dimensional vector to represent a sentence, n · d where n is the number of words in a sentence.

Revised: This makes it extremely hard to ﬁnd sentences using uniform random sampling in the latent space.

Revised:  C ONCLUSION AND F UTURE W ORK

Original: We proposed AriEL, a volume mapping of language into a continuous hypercube to be used as a reference system.
Revised: We proposed AriEL, a volume mapping of language into a continuous hypercube to be used as a reference system.

Original: It provides a latent organization of language that excels at several metrics related to the use of language, and especially at generating many unique and grammatically correct sentences sampling uniformly the latent space.
Revised: It provides a latent organization of language that excels at several metrics related to the use of language, and especially at generating many unique and grammatically correct sentences sampling uniformly the latent space.

Original: AriEL fuses arithmetic coding and k-d trees to construct volumes that preserve the statistics of a dataset.
Revised: AriEL fuses arithmetic coding and k-d trees to construct volumes that preserve the statistics of a dataset.

Original: In this way we construct a latent representation that assigns a data sample to a volume, instead of a point.
Revised: In this way we construct a latent representation that assigns a data sample to a volume, instead of a point.

Original: When compared to standard techniques it highlights room for improvement in their capacity for generation, prediction and generalization.
Revised: When compared to standard techniques it highlights room for improvement in their capacity for generation, prediction and generalization.

Original: Recurrent-based continuous sentence embeddings largely overﬁt the training data and only cover a small subset of the possible language space, particularly when the size of the latent space is small.
Revised: Recurrent-based continuous sentence embeddings largely overﬁt the training data and only cover a small subset of the possible language space, particularly when the size of the latent space is small.

Original: They also fail to learn the underlying CFG and generalize to unbiased sentences from that CFG.
Revised: They also fail to learn the underlying CFG and generalize to unbiased sentences from that CFG.

Original: However they manage to generate quite a few diverse valid sentences.
Revised: However they manage to generate quite a few diverse valid sentences.

Original: Transformer managed to avoid overﬁtting even after being overtrained, proving its robustness.
Revised: Transformer managed to avoid overﬁtting even after being overtrained, proving its robustness.

Original: It performed a remarkable generalization to the unbiased data.
Revised: It performed a remarkable generalization to the unbiased data.

Original: However it proves hard to use as a generator from the continuous latent space using random sampling.
Revised: However it proves hard to use as a generator from the continuous latent space using random sampling.

Revised: On the one hand, this study helps to realize how much of the latent space lies unused by standard architectures.

Revised: On the other hand, AriEL can be seen as a technique to provide an effective interface between multi-modal RL agents that need a pretrained language model for language interaction.

Original: We stress that volume based codes can provide an advantage over point codes in generation tasks.
Revised: We stress that volume based codes can provide an advantage over point codes in generation tasks.

Original: AriEL allows us to sample/generate in theory the same probability distribution as the training set and in practice a more diverse set of sentences, as demonstrated on the toy and on the human dataset.
Revised: AriEL allows us to sample/generate in theory the same probability distribution as the training set and in practice a more diverse set of sentences, as demonstrated on the toy and on the human dataset.

Original: Our planned next step is to use AriEL as a module in a learning agent.
Revised: Our planned next step is to use AriEL as a module in a learning agent.

Original: This study has been performed for dialogue based language generation, which implies short sentences.
Revised: This study has been performed for dialogue based language generation, which implies short sentences.

Original: It would be useful for the
Revised: It would be useful for the

Original: NLP community to understand if this method generalizes to the compression of longer texts.
Revised: NLP community to understand if this method generalizes to the compression of longer texts.

Original: S UPPLEMENTARY M ATERIAL C ONTEXT - FREE GRAMMAR (CFG) USED IN THE EXPERIMENTS
Revised: S UPPLEMENTARY M ATERIAL C ONTEXT - FREE GRAMMAR (CFG) USED IN THE EXPERIMENTS

Original: The context free grammar used to generate the biased and unbiased sentences is composed by the following rules: 
Revised: The context free grammar used to generate the biased and unbiased sentences is composed by the following rules: s → q

Original: [Equation]
Revised: [Equation]

Revised: ’ adjective ’?’

Original: material [Equation] q 
Revised: q → qword adjective ’?

Original: [Equation]
Revised: [Equation]

Original: qword preposition np
Revised: qword ’made’ ’of

Original: [Equation]
Revised: [Equation]

Original:  q [Equation]
Revised: noun material ’?’ q → qword preposition np ’?’

Original:  qword np [Equation]
Revised: q → qword np ’?’

Original: [Table]
Revised: [Table]

Original:  S IZE OF THE LANGUAGE SPACE
Revised:  S IZE OF THE LANGUAGE SPACE

Original: From the CFG used in the experiment
Revised: From the CFG used in the experiment

Original: [Table]
Revised: [Table]

Original: it is possible to extract a total of 15,396 distinct grammar rules, some are shown below.
Revised: it is possible to extract a total of 15,396 distinct grammar rules, some are shown below.

Original: However, for simplicity
Revised: However, for simplicity

Original: [Table]
Revised: [Table]

Original: we deﬁned only 4, related to the number of adjectives in it.
Revised: we deﬁned only 4, related to the number of adjectives in it.

Original: In the case of the unbiased dataset
Revised: In the case of the unbiased dataset

Original: [Table]
Revised: [Table]

Original: those rules can produce a total of 9.81e+ unique sentences.
Revised: those rules can produce a total of 9.81e+ unique sentences.

Original: The total number of unique sentences for the biased dataset is expected to be an order of magnitude smaller
Revised: The total number of unique sentences for the biased dataset is expected to be an order of magnitude smaller

Original: [Table]
Revised: [Table]

Original:  E XAMPLE OF SENTENCES GENERATED FROM THE CFG 3. B IASED SAMPLE SENTENCES
Revised:  E XAMPLE OF SENTENCES GENERATED FROM THE CFG 3. B IASED SAMPLE SENTENCES

Original: • is it large , light yellow and light ?
Revised: • is it large , light yellow and light ?

Original: • is it white , deep pink and average-sized ?
Revised: • is it white , deep pink and average-sized ?

Original: • is it a light , huge and shallow laminate ?
Revised: • is it a light , huge and shallow laminate ?

Original: • is the object average-sized and light ?
Revised: • is the object average-sized and light ?

Original: • is the object fashionable , ghost white and pale turquoise ?
Revised: • is the object fashionable , ghost white and pale turquoise ?

Original: • is the thing huge , huge and khaki ?
Revised: • is the thing huge , huge and khaki ?

Original: • is the thing small , ignitable and very light ?
Revised: • is the thing small , ignitable and very light ?

Original: • is the object a notable very light orange carpet ?
Revised: • is the object a notable very light orange carpet ?

Original: • is the object this small wood made of facing stone ?
Revised: • is the object this small wood made of facing stone ?

Original: • is the object a textured and combinable ﬂoor cover made of laminate ?
Revised: • is the object a textured and combinable ﬂoor cover made of laminate ?

Original: 3. U NBIASED SAMPLE SENTENCES
Revised: 3. U NBIASED SAMPLE SENTENCES

Original: • is the object the huge tiny lovable guest room ?
Revised: • is the object the huge tiny lovable guest room ?

Original: • is the object the closed closed transparent textile ?
Revised: • is the object the closed closed transparent textile ?

Original: • is the thing a transparent , narrow and slightly heavy textile ?
Revised: • is the thing a transparent , narrow and slightly heavy textile ?

Original: • is it steerable , dark orange and light ?
Revised: • is it steerable , dark orange and light ?

Original: • is it gray , very heavy and textured ?
Revised: • is it gray , very heavy and textured ?

Original: • is it closed , heavy and moderately light ?
Revised: • is it closed , heavy and moderately light ?

Original: • is it transparent , transformable and moderately light ?
Revised: • is it transparent , transformable and moderately light ?

Original: • is the thing average-sized and dark red ?
Revised: • is the thing average-sized and dark red ?

Original: • is the thing large and deep garage ?
Revised: • is the thing large and deep garage ?

Original: • is it that slightly heavy stucco made of grass ?
Revised: • is it that slightly heavy stucco made of grass ?

Original:  V OCABULARY
Revised:  V OCABULARY

Original: [Table]

Original: Example

Original: [Table]

Original: Color

Original: [Table]
Revised: [Table]

Original:  U SE OF LATENT SPACE
Revised:  U SE OF LATENT SPACE

Original: In ﬁgure S1, each dot represents a sentence in the latent space.
Revised: In ﬁgure S1, each dot represents a sentence in the latent space.

Original: In the ﬁrst row the dot in the latent space is passed as input to the decoder, while in the second and third row the dot is the output of the encoder when the biased test sentence is fed at its input.
Revised: In the ﬁrst row the dot in the latent space is passed as input to the decoder, while in the second and third row the dot is the output of the encoder when the biased test sentence is fed at its input.

Original: Two random axis in R d are chosen for the generator, ﬁrst row, while two axis were chosen subjectively among the ﬁrst components of a PCA for the encoder, second and third row.
Revised: Two random axis in R d are chosen for the generator, ﬁrst row, while two axis were chosen subjectively among the ﬁrst components of a PCA for the encoder, second and third row.

Original: In every case, the values in the latent space where normalized between zero and one to ease the visualization.
Revised: In every case, the values in the latent space where normalized between zero and one to ease the visualization.

Original: Lines are used to ease the visualization of the clusters and shifts of data with their label, since the point clouds overlap and are hard to see.
Revised: Lines are used to ease the visualization of the clusters and shifts of data with their label, since the point clouds overlap and are hard to see.

Original: The curves are constructed as concave hulls of the dots based on their Delaunay triangulation, a method called alpha shapes (8).
Revised: The curves are constructed as concave hulls of the dots based on their Delaunay triangulation, a method called alpha shapes Edelsbrunner et al.

Original: We can see in ﬁgure S1 (ﬁrst row) how easy it is to ﬁnd grammatical sentences when randomly sampling the latent space for each model.
Revised: We can see in ﬁgure S1 (ﬁrst row) how easy it is to ﬁnd grammatical sentences when randomly sampling the latent space for each model.

Original: AriEL practically only generates grammatical sentences and AE and VAE perform reasonably well too, while Transformer fails.
Revised: AriEL practically only generates grammatical sentences and AE and VAE perform reasonably well too, while Transformer fails.

Original: AriEL failures are plot on top, to remark how few they are, while AE and VAE failures are plot at the bottom, otherwise they would hide the rest given how numerous they are.
Revised: AriEL failures are plot on top, to remark how few they are, while AE and VAE failures are plot at the bottom, otherwise they would hide the rest given how numerous they are.

Original: In the same ﬁgure (rows two and three) we can observe how different methods structure the input in the latent space, each with prototypical clusters and shifts.
Revised: In the same ﬁgure (rows two and three) we can observe how different methods structure the input in the latent space, each with prototypical clusters and shifts.

Original: The Transformer presents an interesting structure of clusters whose purpose remains unclear.
Revised: The Transformer presents an interesting structure of clusters whose purpose remains unclear.

Original: Interestingly, the encoding maps seem to be more organized than the decoding ones.
Revised: Interestingly, the encoding maps seem to be more organized than the decoding ones.

Original: All the models seem to cluster or shift data belonging to different classes at the encoding, that could be taken advantage of by a learning agent placed in the latent space.
Revised: All the models seem to cluster or shift data belonging to different classes at the encoding, that could be taken advantage of by a learning agent placed in the latent space.

Original: However it seems hard to use the Transformer as a generator module for an agent.
Revised: However it seems hard to use the Transformer as a generator module for an agent.

Original: The good performance of AriEL is a consequence of the fact that all the latent space is utilized, and in no directions large gaps can be observed.
Revised: The good performance of AriEL is a consequence of the fact that all the latent space is utilized, and in no directions large gaps can be observed.

Original: This can be seen in the two encoding rows, where the white spaces around the cloud of dots are consequence of the rotation performed by the PCA, otherwise all the space between 0 and 1 would be utilized by AriEL.
Revised: This can be seen in the two encoding rows, where the white spaces around the cloud of dots are consequence of the rotation performed by the PCA, otherwise all the space between 0 and 1 would be utilized by AriEL.

Original:  V ISUALIZATION OF PERFORMANCE ON TOY DATA
Revised:  V ISUALIZATION OF PERFORMANCE ON TOY DATA

Original: G e n e r a t i o n VC
Revised: G e n e r a t i o n VC

Original: V
Revised: V

Original: U
Revised: U

Original: BA
Revised: BA

Original: GA
Revised: GA

Original: PAB
Revised: PAB

Original: PAU
Revised: PAU

Original: VAE
Revised: VAE

Original: AriEL AETransformer
Revised: AriEL AETransformer

Original: G e n e r a l i z a t i o n
Revised: G e n e r a l i z a t i o n

Original: P r e d i c t i o n GCVC V UBAGARAGRAU
Revised: P r e d i c t i o n GCVC V UBAGARAGRAU

Original: GC
Revised: GC

Original: VC
Revised: VC

Original: V
Revised: V

Original: U
Revised: U

Original: BA
Revised: BA

Original: GA
Revised: GA

Original: PAB
Revised: PAB

Original: PAU G e n e r a l i z a t i o n
Revised: PAU G e n e r a l i z a t i o n

Original: P r e d i c t i o n
Revised: P r e d i c t i o n

Original: GC
Revised: GC

Original: GCVC V UBAGARAGRAU
Revised: GCVC V UBAGARAGRAU

Original: G e n e r a t i o nperforms well given performs exceptionally not in and , it manages to cover all grammar with proved to be an improved For
Revised: G e n e r a t i o n

Original:  A RI EL ALGORITHM
Revised:  A RI EL ALGORITHM

Original: Input: sentence: S =
Revised: Input: sentence: S =

Original: ( s j ) nj =
Revised: ( s j ) nj =

Original: Output: z represents S in [0 , 1] d 1: function A RI EL ENCODE ( S ) 2: d = latent space dimension 3:
Revised: Output: z represents S in [0 , 1] d 1: function A RI EL ENCODE ( S ) 2: d = latent space dimension 3:

Original: B low =
Revised: B low =

Original: zeros ( d ) 4:
Revised: zeros ( d ) 4:

Original: B up = ones ( d ) 5: n = length ( S ) 6: for i = 0 , · · · , n − 1 do (cid:46) choose dimension to split 7: d i
Revised: B up = ones ( d ) 5: n = length ( S ) 6: for i = 0 , · · · , n − 1 do (cid:46) choose dimension to split 7: d i

Original: = i mod d 8:
Revised: = i mod d 8:

Original: P next ( s ) =
Revised: P next ( s ) =

Original: P LM ( s | ( s j ) j<i ) 9: c low ( s ) =
Revised: P LM ( s | ( s j ) j<i ) 9: c low ( s ) =

Original: (cid:80) s
Revised: (cid:80) s

Original: >s (cid:48)
Revised: >s (cid:48)

Original: P next ( s (cid:48) )
Revised: P next ( s (cid:48) )

Original: 10: c up ( s ) =
Revised: 10: c up ( s ) =

Original: (cid:80) s
Revised: (cid:80) s

Original: >s (cid:48) − 1 P next ( s (cid:48) )
Revised: >s (cid:48) − 1 P next ( s (cid:48) )

Original: 11: range =
Revised: 11: range =

Original: B up ( d i ) − B low ( d i ) (cid:46) update volume bounds 12:
Revised: B up ( d i ) − B low ( d i ) (cid:46) update volume bounds 12:

Original: B up ( d i ) =
Revised: B up ( d i ) =

Original: B low ( d i ) + range · c up ( s i ) 13:
Revised: B low ( d i ) + range · c up ( s i ) 13:

Original: B low ( d i ) =
Revised: B low ( d i ) =

Original: B low ( d i ) + range · c low ( s i ) 14: end for (cid:46) represent the volume by its center 15:
Revised: B low ( d i ) + range · c low ( s i ) 14: end for (cid:46) represent the volume by its center 15:

Original: z =
Revised: z =

Original: ( B low + B up ) /

Revised: ( B low + B up ) /

Original: : return z 17: end function
Revised: : return z 17: end function

Original: Input: z represents S in [0 , 1] d
Revised: Input: z represents S in [0 , 1] d

Original: Output: sentence: S =
Revised: Output: sentence: S =

Original: ( s j ) nj = 1: function A RI EL DECODE ( z ) 2:
Revised: ( s j ) nj = 1: function A RI EL DECODE ( z ) 2:

Original: d = dimension ( z ) 3:
Revised: d = dimension ( z ) 3:

Original: B low =
Revised: B low =

Original: zeros ( d ) 4:
Revised: zeros ( d ) 4:

Original: B up = ones ( d ) 5:
Revised: B up = ones ( d ) 5:

Original: S = (cid:104) START (cid:105) 6: for i = 0 , · · · , n max − 1 do (cid:46) choose dimension to unsplit 7: d i
Revised: S = (cid:104) START (cid:105) 6: for i = 0 , · · · , n max − 1 do (cid:46) choose dimension to unsplit 7: d i

Original: = i mod d 8:
Revised: = i mod d 8:

Original: P next ( s ) =
Revised: P next ( s ) =

Original: P LM ( s | S ) 9: c low ( s ) =
Revised: P LM ( s | S ) 9: c low ( s ) =

Original: (cid:80) s
Revised: (cid:80) s

Original: >s (cid:48)
Revised: >s (cid:48)

Original: P next ( s (cid:48) )
Revised: P next ( s (cid:48) )

Original: 10: c up ( s ) =
Revised: 10: c up ( s ) =

Original: (cid:80) s
Revised: (cid:80) s

Original: >s (cid:48) − 1 P next ( s (cid:48) )
Revised: >s (cid:48) − 1 P next ( s (cid:48) )

Original: 11: range =
Revised: 11: range =

Original: B up ( d i ) − B low ( d i ) (cid:46) update volume bounds 12:
Revised: B up ( d i ) − B low ( d i ) (cid:46) update volume bounds 12:

Original: Bs up ( s )
Revised: Bs up ( s )

Original: =
Revised: =

Original: B up ( d i ) + range · c up ( s ) 13:
Revised: B up ( d i ) + range · c up ( s ) 13:

Original: Bs low ( s ) =
Revised: Bs low ( s ) =

Original: B low ( d i ) + range · c low ( s ) (cid:46) any point in the volume is assigned the symbol s i 14: s i = find s (cid:16) Bs low ( s )
Revised: B low ( d i ) + range · c low ( s ) (cid:46) any point in the volume is assigned the symbol s i 14: s i = find s (cid:16) Bs low ( s )

Original: <
Revised: <

Original: z ( d i ) <
Revised: z ( d i ) <

Original: Bs up ( s ) (cid:17) 15:
Revised: Bs up ( s ) (cid:17) 15:

Original: B up ( d i ) =
Revised: B up ( d i ) =

Original: Bs up ( s i ) 16:
Revised: Bs up ( s i ) 16:

Original: B low ( d i ) =
Revised: B low ( d i ) =

Original: Bs low
Revised: Bs low

Original: ( s i ) 17:
Revised: ( s i ) 17:

Original: S =
Revised: S =

Original: S.append ( s i ) 18: end for 19: return S 20: end function
Revised: S.append ( s i ) 18: end for 19: return S 20: end function

Original:  T RAINING DETAILS
Revised:  T RAINING DETAILS

Original: We go through the training data 10 times, in mini-batches of 256 sentences.
Revised: We go through the training data 10 times, in mini-batches of 256 sentences.

Original: We applied teacher forcing (45) during training.
Revised: We applied teacher forcing (Williams and Zipser, 1989) during training.

Original: We use the Adam (19) optimizer with a learning rate of 1e-3 and gradient clipping at 0.5 magnitude.
Revised: We use the Adam (Kingma and Ba, 2015) optimizer with a learning rate of 1e-3 and gradient clipping at 0.5 magnitude.

Original: Learning rate was reduced by a factor of 0.2 if the loss function didn’t decrease within 5 epochs, with a minimum learning rate of 1e-5.
Revised: Learning rate was reduced by a factor of 0.2 if the loss function didn’t decrease within 5 epochs, with a minimum learning rate of 1e-5.

Original: For all RNN-based embeddings, kernel weights used the Xavier uniform initialization (11), while recurrent weights used random orthogonal matrix initialization (36).
Revised: For all RNN-based embeddings, kernel weights used the Xavier uniform initialization (Glorot and Bengio, 2010), while recurrent weights used random orthogonal matrix initialization (Saxe et al., 2014).

Original: Biases are initialized to zero.
Revised: Biases are initialized to zero.

Original: Embeddings layers are initialized with a uniform distribution between [-1, 1].
Revised: Embeddings layers are initialized with a uniform distribution between [-1, 1].

Original: For Transformer the multihead attention matrices and the feedforward module matrices, used the Xavier uniform initialization (11), the beta of the layer normalization uses zeros, and its gamma uses ones for initialization.
Revised: For Transformer the multihead attention matrices and the feedforward module matrices, used the Xavier uniform initialization (Glorot and Bengio, 2010), the beta of the layer normalization uses zeros, and its gamma uses ones for initialization.

Original: AE and VAE are trained with a word dropout of 0.25 at the input, and VAE is trained with KL loss annealing that moves the weight of the KL loss from zero to one during the 7th epoch, similarly tothe original work (3).
Revised: AE and VAE are trained with a word dropout of 0.25 at the input, and VAE is trained with KL loss annealing that moves the weight of the KL loss from zero to one during the 7th epoch, similarly to original work (Bowman et al., 2016).