Original: D IFFERENTIABLE P ERTURB - AND -P ARSE : S EMI -S UPERVISED P ARSING WITH A S TRUCTURED V ARIATIONAL A UTOENCODER
Revised: D IFFERENTIABLE P ERTURB - AND -P ARSE : S EMI -S UPERVISED P ARSING WITH A S TRUCTURED V ARIATIONAL A UTOENCODER

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Human annotation for syntactic parsing is expensive, and large resources are available only for a fraction of languages.
Revised: Human annotation for syntactic parsing is expensive, and large resources are available only for a fraction of languages.

Original: A question we ask is whether one can leverage abundant unlabeled texts to improve syntactic parsers, beyond just using the texts to obtain more generalisable lexical features (i.e. beyond word embeddings).
Revised: A question we ask is whether one can leverage abundant unlabeled texts to improve syntactic parsers, beyond just using the texts to obtain more generalisable lexical features (i e. beyond word embeddings).

Original: To this end, we propose a novel latent-variable generative model for semi-supervised syntactic dependency parsing.
Revised: To this end, we propose a novel latent-variable generative model for semi-supervised syntactic dependency parsing.

Original: As exact inference is intractable, we introduce a differentiable relaxation to obtain approximate samples and compute gradients with respect to the parser parameters.
Revised: As exact inference is intractable, we introduce a differentiable relaxation to obtain approximate samples and compute gradients with respect to the parser parameters.

Original: Our method (Differentiable
Revised: Our method (Differentiable

Original: Perturb-and-Parse) relies on differentiable dynamic programming over stochastically perturbed arc weights.
Revised: Perturb-and-Parse) relies on differentiable dynamic programming over stochastically perturbed arc weights.

Original: We demonstrate effectiveness of our approach with experiments on English, French and Swedish.
Revised: We demonstrate effectiveness of our approach with experiments on English, French and Swedish.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: A dependency tree is a lightweight syntactic structure exposing (possibly labeled) bi-lexical relations between words (Tesni ` ere, 1959; Kaplan & Bresnan, 1982), see Figure 1.
Revised: A dependency tree is a lightweight syntactic structure exposing (possibly labeled) bi-lexical relations between words (Tesni`ere, 1959; Kaplan & Bresnan, 1982), see Figure 1.

Original: This representation has been widely studied by the NLP community leading to very efﬁcient state-of-the-art parsers (Kiperwasser & Goldberg, 2016; Dozat & Manning, 2017; Ma & Hovy, 2017), motivated by the fact that dependency trees are useful in downstream tasks such as semantic parsing (Reddy et al., 2016; Marcheggiani & Titov, 2017), machine translation (Ding & Palmer, 2005; Bastings et al., 2017), information extraction (Culotta & Sorensen, 2004;
Revised: This representation has been widely studied by the NLP community leading to very efﬁcient state-of-the-art parsers (Kiperwasser & Goldberg, 2016; Dozat & Manning, 2017; Ma & Hovy, 2017), motivated by the fact that dependency trees are useful in downstream tasks such as semantic parsing (Reddy et al., 2016; Marcheggiani & Titov, 2017), machine translation (Ding & Palmer, 2005; Bastings et al., 2017), information extraction (Culotta & Sorensen, 2004;

Original: Liu et al., 2015), question answering (Cui et al., 2005) and even as a ﬁltering method for constituency parsing (Kong et al., 2015), among others.
Revised: Liu et al., 2015), question answering (Cui et al., 2005) and even as a ﬁltering method for constituency parsing (Kong et al., 2015), among others.

Original: Unfortunately, syntactic annotation is a tedious and expensive task, requiring highly-skilled human annotators.
Revised: Unfortunately, syntactic annotation is a tedious and expensive task, requiring highly-skilled human annotators.

Original: Consequently, even though syntactic annotation is now available for many languages, the datasets are often small.
Revised: Consequently, even though syntactic annotation is now available for many languages, the datasets are often small.

Original: For example, 31 languages in the Universal Dependency Treebank 1 , the largest dependency annotation resource, have fewer than 5,000 sentences, including such major languages as Vietnamese and Telugu.
Revised: For example, 31 languages in the Universal Dependency Treebank,the largest dependency annotation resource, have fewer than 5,000 sentences, including such major languages as Vietnamese and Telugu.

Original: This makes the idea of using unlabeled texts as an additional source of supervision especially attractive.
Revised: This makes the idea of using unlabeled texts as an additional source of supervision especially attractive.

Original: In previous work, before the rise of deep learning, the semi-supervised parsing setting has been mainly tackled with two-step algorithms.
Revised: In previous work, before the rise of deep learning, the semi-supervised parsing setting has been mainly tackled with two-step algorithms.

Original: On the one hand, feature extraction methods ﬁrst learn an intermediate representation using an unlabeled dataset which is then used as input to train a supervised parser (Koo et al., 2008; Yu et al., 2008; Chen et al., 2009; Suzuki et al., 2011).
Revised: On the one hand, feature extraction methods ﬁrst learn an intermediate representation using an unlabeled dataset which is then used as input to train a supervised parser (Koo et al., 2008; Yu et al., 2008; Chen et al., 2009; Suzuki et al., 2011).

Original: On the other hand, the self-training and co-training methods start by learning a supervised parser that is then used to label extra data.
Revised: On the other hand, the self-training and co-training methods start by learning a supervised parser that is then used to label extra data.

Original: Then, the parser is retrained with this additional annotation (Sagae & Tsujii, 2007; Kawahara & Uchimoto, 2008;
Revised: Then, the parser is retrained with this additional annotation (Sagae & Tsujii, 2007; Kawahara & Uchimoto, 2008;

Original: McClosky et al., 2006).
Revised: McClosky et al., 2006).

Original: Nowadays, unsupervised feature extraction is achieved in neural parsers by the means of word embeddings (Mikolov et al., 2013; Peters et al., 2018).
Revised: Nowadays, unsupervised feature extraction is achieved in neural parsers by the means of word embeddings (Mikolov et al., 2013; Peters et al., 2018).

Original: The natural question to ask is whether one can exploit unlabeled data in neural parsers beyond only inducing generalizable word representations. 
Revised: The natural question to ask is whether one can exploit unlabeled data in neural parsers beyond only inducing generalizable word representations. [Figure] [Table]

Original: Our method can be regarded as semi-supervised Variational Auto-Encoder, VAE (Kingma et al., 2014).
Revised: Our method can be regarded as semi-supervised Variational Auto-Encoder (VAE, Kingma et al., 2014).

Original: Speciﬁcally, we introduce a probabilistic model (Section 3) parametrized with a neural net- [Figure] [Table] [Figure]
Revised: Speciﬁcally, we introduce a probabilistic model (Section 3) parametrized with a neural network (Section 4).

Original: [Table]

Original: work (Section 4).

Original: The model assumes that a sentence is generated conditioned on a latent dependency tree.
Revised: The model assumes that a sentence is generated conditioned on a latent dependency tree.

Original: Dependency parsing corresponds to approximating the posterior distribution over the latent trees within this model, achieved by the encoder component of VAE, see Figure 2a.
Revised: Dependency parsing corresponds to approximating the posterior distribution over the latent trees within this model, achieved by the encoder component of VAE, see Figure 2a.

Original: The parameters of the generative model and the parser (i.e. the encoder) are estimated by maximizing the likelihood of unlabeled sentences.
Revised: The parameters of the generative model and the parser (i.e. the encoder) are estimated by maximizing the likelihood of unlabeled sentences.

Original: In order to ensure that the latent representation is consistent with treebank annotation, we combine the above objective with maximizing the likelihood of gold parse trees in the labeled data.
Revised: In order to ensure that the latent representation is consistent with treebank annotation, we combine the above objective with maximizing the likelihood of gold parse trees in the labeled data.

Original: Training a VAE via backpropagation requires marginalization over the latent variables, which is intractable for dependency trees.
Revised: Training a VAE via backpropagation requires marginalization over the latent variables, which is intractable for dependency trees.

Original: In this case, previous work proposed approximate training methods, mainly differentiable Monte-Carlo estimation (Kingma & Welling, 2013; Rezende et al., 2014) and score function estimation, e.g. REINFORCE (Williams, 1992).
Revised: In this case, previous work proposed approximate training methods, mainly differentiable Monte-Carlo estimation (Kingma & Welling, 2013; Rezende et al., 2014) and score function estimation, e.g. REINFORCE (Williams, 1992).

Original: However, REINFORCE is known to suffer from high variance (Mnih & Gregor, 2014).
Revised: However, REINFORCE is known to suffer from high variance (Mnih & Gregor, 2014).

Original: Therefore, we propose an approximate differentiable Monte-Carlo approach that we call Differentiable Perturb-and-Parse (Section 5).
Revised: Therefore, we propose an approximate differentiable Monte-Carlo approach that we call Differentiable Perturb-and-Parse (Section 5).

Original: The key idea is that we can obtain a differentiable relaxation of an approximate sample by (1) perturbing weights of candidate dependencies and (2) performing structured argmax inference with differentiable dynamic programming, relying on the perturbed scores.
Revised: The key idea is that we can obtain a differentiable relaxation of an approximate sample by (1) perturbing weights of candidate dependencies and (2) performing structured argmax inference with differentiable dynamic programming, relying on the perturbed scores.

Original: In this way we bring together ideas of perturb-and-map inference (Papandreou & Yuille, 2011; Maddison et al., 2017) and continuous relaxation for dynamic programming (Mensch & Blondel, 2018).
Revised: In this way we bring together ideas of perturb-and-map inference (Papandreou & Yuille, 2011; Maddison et al., 2017) and continuous relaxation for dynamic programming (Mensch & Blondel, 2018).

Original: Our model differs from previous works on latent structured models which compute marginal probabilities of individual edges Kim et al.
Revised: Our model differs from previous works on latent structured models which compute marginal probabilities of individual edges Kim et al.

Original: ; Liu & Lapata (2018).
Revised: ; Liu & Lapata (2018).

Original: Instead, we sample a single tree from the distribution that is represented with a soft selection of arcs.
Revised: Instead, we sample a single tree from the distribution that is represented with a soft selection of arcs.

Original: Therefore, we preserve higher-order statistics, which can then inform the decoder.
Revised: Therefore, we preserve higher-order statistics, which can then inform the decoder.

Original: Computing marginals would correspond to making strong independence assumptions.
Revised: Computing marginals would correspond to making strong independence assumptions.

Original: We evaluate our semi-supervised parser on English, French and Swedish and show improvement over a comparable supervised baseline (Section 6).
Revised: We evaluate our semi-supervised parser on English, French and Swedish and show improvement over a comparable supervised baseline (Section 6).

Original: Our main contributions can be summarized as follows: • we introduce a variational autoencoder for semi-supervised dependency parsing; • we propose the Differentiable Perturb-and-Parse method for its estimation; • we demonstrate the effectiveness of the approach on three different languages.
Revised: Our main contributions can be summarized as follows: (1) we introduce a variational autoencoder for semi-supervised dependency parsing; (2) we propose the Differentiable Perturb-and-Parse method for its estimation; (3) we demonstrate the effectiveness of the approach on three different languages.

Original: In short, we introduce a novel generative model for learning latent syntactic structures.
Revised: In short, we introduce a novel generative model for learning latent syntactic structures.

Original:  D EPENDENCY PARSING
Revised:  D EPENDENCY PARSING

Original: A dependency is a bi-lexical relation between a head word (the source) and a modiﬁer word (the target), see Figure 1.
Revised: A dependency is a bi-lexical relation between a head word (the source) and a modiﬁer word (the target), see Figure 1.

Original: The set of dependencies of a sentence deﬁnes a tree-shaped structure.
Revised: The set of dependencies of a sentence deﬁnes a tree-shaped structure.

Original: In the parsing problem, we aim to compute the dependency tree of a given sentence.
Revised: In the parsing problem, we aim to compute the dependency tree of a given sentence.

Original: Formally, we deﬁne a sentence as a sequence of tokens (words) from vocabulary W .
Revised: Formally, we deﬁne a sentence as a sequence of tokens (words) from vocabulary W .

Original: We assume a one-to-one mapping between W and integers 1 . . .
Revised: We assume a one-to-one mapping between W and integers 1 . . .

Original: | W | .
Revised: | W | .

Original: Therefore, we write a sentence of length n as a vector of integers s of size n + 1 with 1 ≤ s i ≤ | W | and where s 0 is a special root symbol.
Revised: Therefore, we write a sentence of length n as a vector of integers s of size n + 1 with 1 ≤ s i ≤ | W | and where s 0 is a special root symbol.

Original: A dependency tree of sentence s is a matrix of booleans T ∈ { 0 , 1 } ( n +1) ×
Revised: A dependency tree of sentence s is a matrix of booleans T ∈ { 0 , 1 } ( n +1) ×

Original: ( n +1) with T h,m =
Revised: ( n +1) with T h,m =

Original: meaning that word
Revised: meaning that word

Original: s h is the head of word s m in the dependency tree.
Revised: s h is the head of word s m in the dependency tree

Original: [Figure]
Revised: [Figure]

Revised: z s

Revised: [Figure]

Revised: ( s

Revised: [Figure]

Revised: s

Original:  s m , v
Revised: [Figure] p ( z | s ) [Figure] s m , v

Original: W z
Revised: W z

Original: T
Revised: T

Original: LSTM + GCN s ( [Figure] Computation Graph
Revised: LSTM + GCN s (b) Computation Graph

Original: L STM
Revised: L STM

Original: L STM L STM
Revised: L STM L STM

Original: L STM ssssz
Revised: L STM ssssz

Original: M LP →
Revised: M LP →

Original: M LP ←
Revised: M LP ←

Original: M LP (cid:35) tanh( (cid:80) · ) s [Figure]
Revised: M LP (cid:35) tanh( (cid:80) · ) s(c) Decoder

Original: More speciﬁcally, a dependency tree T is the adjacency matrix of a directed graph with n +vertices v 0 . . .
Revised: More speciﬁcally, a dependency tree T is the adjacency matrix of a directed graph with n +vertices v 0 . . .

Original: v n .
Revised: v n .

Original: A matrix T is a valid dependency tree if and only if this graph is a v 0 -rooted spanning arborescence, 3 i.e. the graph is connected, each vertex has at most one incoming arc and the only vertex without incoming arc is v 0 .
Revised: A matrix T is a valid dependency tree if and only if this graph is a v 0 -rooted spanning arborescence, 3 i.e. the graph is connected, each vertex has at most one incoming arc and the only vertex without incoming arc is v 0 .

Original: A dependency tree is projective if and only if, for each arc v h
Revised: A dependency tree is projective if and only if, for each arc v h

Original: → v m , if h < m (resp.
Revised: → v m , if h < m (resp.

Original: m < h ) then there exists a path with arcs T from v h to each vertex v k such that h < k < m (resp.
Revised: m < h ) then there exists a path with arcs T from v h to each vertex v k such that h < k < m (resp.

Original: m < k < h ).
Revised: m < k < h ).

Original: From a linguistic point of view, projective dependency trees combine contiguous phrases (sequence of words) only.
Revised: From a linguistic point of view, projective dependency trees combine contiguous phrases (sequence of words) only.

Original: Intuitively, this means that we can draw the dependency tree above the sentence without crossing arcs.
Revised: Intuitively, this means that we can draw the dependency tree above the sentence without crossing arcs.

Original: Given a sentence s , an arc-factored dependency parser computes the dependency tree T which maximizes a weighting function f ( T ; W ) =
Revised: Given a sentence s , an arc-factored dependency parser computes the dependency tree T which maximizes a weighting function f ( T ; W ) =

Original: (cid:80) h,m T h,m W h,m , where W is a matrix of dependency (arc) weights.
Revised: (cid:80) h,m T h,m W h,m , where W is a matrix of dependency (arc) weights.

Original: This problem can be solved with a O ( n 2 ) time complexity (Tarjan, 1977; McDonald et al., 2005).
Revised: This problem can be solved with a O ( n 2 ) time complexity (Tarjan, 1977; McDonald et al., 2005).

Original: If we restrict T to be a projective dependency tree, then the optimal solution can be computed with a O ( n 3 ) time complexity using dynamic programming (Eisner, 1996).
Revised: If we restrict T to be a projective dependency tree, then the optimal solution can be computed with a O ( n 3 ) time complexity using dynamic programming (Eisner, 1996).

Original: Restricting the search space to projective trees is appealing for treebanks exhibiting this property (either exactly or approximately): they enforce a structural constraint that can be beneﬁcial for accuracy, especially in a low-resource scenario.
Revised: Restricting the search space to projective trees is appealing for treebanks exhibiting this property (either exactly or approximately): they enforce a structural constraint that can be beneﬁcial for accuracy, especially in a low-resource scenario.

Original: Moreover, using a more restricted search space of potential trees may be especially beneﬁcial in a semi-supervised scenario: with a more restricted space a model is less likely to diverge from a treebank grammar and capture non-syntactic phenomena.
Revised: Moreover, using a more restricted search space of potential trees may be especially beneﬁcial in a semi-supervised scenario: with a more restricted space a model is less likely to diverge from a treebank grammar and capture non-syntactic phenomena.

Original: Finally, Eisner’s algorithm (Eisner, 1996) can be described as a deduction system (Pereira & Warren, 1983), a framework that uniﬁes many parsing algorithms.
Revised: Finally, Eisner’s algorithm (Eisner, 1996) can be described as a deduction system (Pereira & Warren, 1983), a framework that uniﬁes many parsing algorithms.

Original: As such, our methodology could be applied to other grammar formalisms.
Revised: As such, our methodology could be applied to other grammar formalisms.

Original: For all these reasons, in this paper, we focus on projective dependency trees only.
Revised: For all these reasons, in this paper, we focus on projective dependency trees only.

Original:  G ENERATIVE MODEL
Revised:  G ENERATIVE MODEL

Original: We now turn to the learning problem, i.e. estimation of the matrix W .
Revised: We now turn to the learning problem, i.e. estimation of the matrix W .

Original: We assume that we have access to a set of i.i.d.
Revised: We assume that we have access to a set of i.i.d.

Original: labeled sentences L = {(cid:104) s , T (cid:105) , . . . } and a set of i.i.d.
Revised: labeled sentences L = {(cid:104) s , T (cid:105) , . . . } and a set of i.i.d.

Original: unlabeled sentences U = { s , . . . } .
Revised: unlabeled sentences U = { s , . . . } .

Original: In order to incorporate unlabeled data in the learning process, we introduce a gen- Tree linguistic structure arborescence erative model where the dependency tree is latent (Subsection 3.1).
Revised: In order to incorporate unlabeled data in the learning process, we introduce a generative model where the dependency tree is latent (Subsection 3.1).

Original: As such, we can maximize the  likelihood of observed sentences even if the ground-truth dependency tree is unknown.
Revised: As such, we can maximize the graph likelihood of observed sentences even if the ground-truth dependency tree is unknown.

Original: We learn the parameters of this model using a variational Bayes approximation (Subsection 3.2) augmented with a discriminative objective on labeled data (Subsection 3.3).
Revised: We learn the parameters of this model using a variational Bayes approximation (Subsection 3.2) augmented with a discriminative objective on labeled data (Subsection 3.3).

Original: 3. G ENERATIVE STORY
Revised: 3. G ENERATIVE STORY

Original: Under our probabilistic model, a sentence s is generated from a continuous sentence embedding z and with respect to a syntactic structure T .
Revised: Under our probabilistic model, a sentence s is generated from a continuous sentence embedding z and with respect to a syntactic structure T .

Original: We formally deﬁne the generative process of a sentence of length n as:
Revised: We formally deﬁne the generative process of a sentence of length n as:

Original: [Equation]
Revised: [Equation]

Original: This Bayesian network is shown in Figure 2a.
Revised: This Bayesian network is shown in Figure 2a.

Original: In order to simplify notation, we omit conditioning on n in the following.
Revised: In order to simplify notation, we omit conditioning on n in the following.

Original: T and z are latent variables and p ( s | T , z ) is the conditional likelihood of observations.
Revised: T and z are latent variables and p ( s | T , z ) is the conditional likelihood of observations.

Original: We assume that the priors p ( T ) and p ( z ) are the uniform distribution over projective trees and the multivariate standard normal distribution, respectively.
Revised: We assume that the priors p ( T ) and p ( z ) are the uniform distribution over projective trees and the multivariate standard normal distribution, respectively.

Original: The true distribution underlying the observed data is unknown, so we have to learn a model p θ ( s | T , z ) parametrized by θ that best ﬁts the given samples:
Revised: The true distribution underlying the observed data is unknown, so we have to learn a model p θ ( s | T , z ) parametrized by θ that best ﬁts the given samples:

Original: [Equation]
Revised: [Equation]

Original: Then, the posterior distribution of latent variables p θ ( T , z | s ) models the probability of underlying representations (including dependency trees) with respect to a sentence.
Revised: Then, the posterior distribution of latent variables p θ ( T , z | s ) models the probability of underlying representations (including dependency trees) with respect to a sentence.

Original: This conditional distribution can be written as:
Revised: This conditional distribution can be written as:

Original: [Equation]
Revised: [Equation]

Original: In the next subsection, we explain how these two quantities can be estimated from data.
Revised: In the next subsection, we explain how these two quantities can be estimated from data.

Original: 3. V ARIATIONAL A UTO -E NCODERS
Revised: 3. V ARIATIONAL A UTO -E NCODERS

Original: Computations in Equation 1 and Equation 2 require marginalization over the latent variables:
Revised: Computations in Equation 1 and Equation 2 require marginalization over the latent variables:

Original: [Equation]
Revised: [Equation]

Original: which is intractable in general.
Revised: which is intractable in general.

Original: We rely on the Variational Auto-Encoder (VAE) framework to tackle this challenge (Kingma & Welling, 2013; Rezende et al., 2014).
Revised: We rely on the Variational Auto-Encoder (VAE) framework to tackle this challenge (Kingma & Welling, 2013; Rezende et al., 2014).

Original: We introduce a variational distribution q φ ( T , z | s ) which is intended to be similar to p θ ( T , z | s ) .
Revised: We introduce a variational distribution q φ ( T , z | s ) which is intended to be similar to p θ ( T , z | s ) .

Original: More formally, we want
Revised: More formally, we want

Original: KL [ q φ ( T , z | s ) (cid:107) p θ ( T , z | s )] to be as small as possible, where KL is the Kulback-Leibler (KL) divergence.
Revised: KL [ q φ ( T , z | s ) (cid:107) p θ ( T , z | s )] to be as small as possible, where KL is the Kulback-Leibler (KL) divergence.

Original: Then, the following equality holds:
Revised: Then, the following equality holds:

Original: [Equation]
Revised: [Equation]

Original: where log p θ ( s ) is called the evidence.
Revised: where log p θ ( s ) is called the evidence.

Original: The KL divergence is always positive, therefore by removing the last term we have:
Revised: The KL divergence is always positive, therefore by removing the last term we have:

Original: [Equation]
Revised: [Equation]

Original: where the right-hand side is called the Evidence Lower Bound (ELBO).
Revised: where the right-hand side is called the Evidence Lower Bound (ELBO).

Original: By maximizing the ELBO term, the divergence KL [ q φ ( T , z | s ) (cid:107) p θ ( T , z | s )] is implicitly minimized.
Revised: By maximizing the ELBO term, the divergence KL [ q φ ( T , z | s ) (cid:107) p θ ( T , z | s )] is implicitly minimized.

Original: Therefore, we deﬁne a surrogate objective, replacing the objective in Equation 1:
Revised: Therefore, we deﬁne a surrogate objective, replacing the objective in Equation 1:

Original: [Equation]
Revised: [Equation]

Original: The ELBO in Equation 4 has two components.
Revised: The ELBO in Equation 4 has two components.

Original: First, the KL divergence with the prior, which usually has a closed form solution.
Revised: First, the KL divergence with the prior, which usually has a closed form solution.

Original: For the distribution over dependency trees, it can be computed with the semiring algorithm of Li & Eisner (2009).
Revised: For the distribution over dependency trees, it can be computed with the semiring algorithm of Li & Eisner (2009).

Original: Second, the non-trivial term E q φ ( T , z | s ) [log p θ ( s | T , z )] .
Revised: Second, the non-trivial term E q φ ( T , z | s ) [log p θ ( s | T , z )] .

Original: During training, Monte-Carlo method provides a tractable and unbiased estimation of the expectation.
Revised: During training, Monte-Carlo method provides a tractable and unbiased estimation of the expectation.

Original: Note that a single sample from q φ ( T , z | s ) can be understood as encoding the observation into the latent space, whereas sampling a sentence from the latent space can be understood as decoding.
Revised: Note that a single sample from q φ ( T , z | s ) can be understood as encoding the observation into the latent space, whereas regenerating a sentence from the latent space can be understood as decoding.

Original: However, training a VAE requires the sampling process to be differentiable.
Revised: However, training a VAE requires the sampling process to be differentiable.

Original: In the case of the sentence embedding, we follow the usual setting and deﬁne q φ ( z | s ) as a diagonal Gaussian: backpropagation through the the sampling process z ∼ q φ ( z | s ) can be achieved thanks to the reparametrization trick (Kingma & Welling, 2013; Rezende et al., 2014).
Revised: In the case of the sentence embedding, we follow the usual setting and deﬁne q φ ( z | s ) as a diagonal Gaussian: backpropagation through the the sampling process z ∼ q φ ( z | s ) can be achieved thanks to the reparametrization trick (Kingma & Welling, 2013; Rezende et al., 2014).

Original: Unfortunately, this approach cannot be applied to dependency tree sampling T ∼ q φ ( T | s ) .
Revised: Unfortunately, this approach cannot be applied to dependency tree sampling T ∼ q φ ( T | s ) .

Original: We tackle this issue in
Revised: We tackle this issue in

Original: Section 5.
Revised: Section 5.

Original: 3. S EMI - SUPERVISED LEARNING
Revised: 3. S EMI - SUPERVISED LEARNING

Original: VAEs are a convenient approach for semi-supervised learning (Kingma et al., 2014) and have been successfully applied in NLP (Koˇcisk´y et al., 2016; Xu et al., 2017; Zhou & Neubig, 2017; Yin et al., 2018).
Revised: VAEs are a convenient approach for semi-supervised learning (Kingma et al., 2014) and have been successfully applied in NLP (Koˇcisk´y et al., 2016; Xu et al., 2017; Zhou & Neubig, 2017; Yin et al., 2018).

Original: In this scenario, we are given the dependency structure of a subset of the observations, i.e. T is an observed variable.
Revised: In this scenario, we are given the dependency structure of a subset of the observations, i.e. T is an observed variable.

Original: Then, the supervised ELBO term is deﬁned as:
Revised: Then, the supervised ELBO term is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: [Equation] Note that our end goal is to estimate the posterior ditribution over dependency trees q φ ( T | s ) , i.e. the dependency parser, which does not appear in the supervised ELBO.
Revised: Note that our end goal is to estimate the posterior ditribution over dependency trees q φ ( T | s ) , i.e. the dependency parser, which does not appear in the supervised ELBO.

Original: We want to explicitly use the labeled data in order to learn the parameters of this parser.
Revised: We want to explicitly use the labeled data in order to learn the parameters of this parser.

Original: This can be achieved by adding a discriminative training term to the overall loss.
Revised: This can be achieved by adding a discriminative training term to the overall loss.

Original: The loss function for training a semi-supervised VAE is:
Revised: The loss function for training a semi-supervised VAE is:

Original: [Equation]
Revised: [Equation]

Original: where the ﬁrst term is the standard loss for supervised learning of log-linear models (Johnson et al., 1999; Lafferty et al., 2001).
Revised: where the ﬁrst term is the standard loss for supervised learning of log-linear models (Johnson et al., 1999; Lafferty et al., 2001).

Original:  N EURAL P ARAMETRIZATION
Revised:  N EURAL P ARAMETRIZATION

Original: In this section, we describe the neural parametrization of the encoder distribution q φ (Subsection 4.1) and the decoder distribution p θ (Subsection 4.2).
Revised: In this section, we describe the neural parametrization of the encoder distribution q φ (Subsection 4.1) and the decoder distribution p θ (Subsection 4.2).

Original: A visual representation is given in Figure 2b.
Revised: A visual representation is given in Figure 2b.

Original: 4. E NCODER
Revised: 4. E NCODER

Original: We factorize the encoder as q φ ( T , z | s ) =
Revised: We factorize the encoder as q φ ( T , z | s ) =

Original: q φ ( T | s ) q φ ( z | s ) .
Revised: q φ ( T | s ) q φ ( z | s ) .

Original: The categorical distribution over dependency trees is parametrized by a log-linear model (Lafferty et al., 2001) where the weight of an arc is given by the neural network of Kiperwasser & Goldberg (2016).The sentence embedding model is speciﬁed as a diagonal Gaussian parametrized by a LSTM, similarly to the seq2seq framework (Sutskever et al., 2014; Bowman et al., 2016).
Revised: The categorical distribution over dependency trees is parametrized by a log-linear model (Lafferty et al., 2001) where the weight of an arc is given by the neural network of Kiperwasser & Goldberg (2016).The sentence embedding model is speciﬁed as a diagonal Gaussian parametrized by a LSTM, similarly to the seq2seq framework (Sutskever et al., 2014; Bowman et al., 2016).

Original: That is:
Revised: That is:

Original: [Equation]
Revised: [Equation]

Original: where m and v are mean and variance vectors, respectively.
Revised: where m and v are mean and variance vectors, respectively.

Original: 4. D ECODER
Revised: 4. D ECODER

Original: We use an autoregressive decoder that combines an LSTM and a Graph Convolutional Network (GCN) (Kipf & Welling, 2016; Marcheggiani & Titov, 2017).
Revised: We use an autoregressive decoder that combines an LSTM and a Graph Convolutional Network (GCN, Kipf & Welling, 2016; Marcheggiani & Titov, 2017).

Original: The LSTM keeps the history of generated words, while the Graph Convolutional Network incorporate information about syntactic dependencies.
Revised: The LSTM keeps the history of generated words, while the GCN incorporate information about syntactic dependencies.

Original: The hidden state of the LSTM is initialized with latent variable z (the sentence embedding).
Revised: The hidden state of the LSTM is initialized with latent variable z (the sentence embedding).

Original: Then, at each step 1 ≤ i ≤ n , an embedding associated with word at position i − 1 is fed as input.
Revised: Then, at each step 1 ≤ i ≤ n , an embedding associated with word at position i − 1 is fed as input.

Original: A special start-of-sentence symbol embedding is used at the ﬁrst position.
Revised: A special start-of-sentence symbol embedding is used at the ﬁrst position.

Original: Let o i be the hidden state of the LSTM at position i .
Revised: Let o i be the hidden state of the LSTM at position i .

Original: The standard seq2seq architecture uses this vector to predict the word at position i .
Revised: The standard seq2seq architecture uses this vector to predict the word at position i .

Original: Instead, we transform it in order to take into account the syntactic structure described by the latent variable T .
Revised: Instead, we transform it in order to take into account the syntactic structure described by the latent variable T .

Original: Due to the autoregressive nature of the decoder, we can only take into account dependencies T h,m such that h < i and m < i .
Revised: Due to the autoregressive nature of the decoder, we can only take into account dependencies T h,m such that h < i and m < i .

Original: Before being fed to the GCN, the output of the LSTM is fed to distinct multi-layer perceptrons 6 that characterize syntactic relations: if s h is the head of s i , o h is transformed with M LP (cid:121) , if s m is a modiﬁer of s i , o m is transformed with M LP (cid:120) , and lastly o i is transformed with M LP (cid:35) .
Revised: Before being fed to the GCN, the output of the LSTM is fed to distinct multi-layer perceptrons 6 that characterize syntactic relations: if s h is the head of s i , o h is transformed with M LP (cid:121) , if s m is a modiﬁer of s i , o m is transformed with M LP (cid:120) , and lastly o i is transformed with M LP (cid:35) .

Original: Formally, the GCN is deﬁned as follows:
Revised: Formally, the GCN is deﬁned as follows:

Original: [Equation]
Revised: [Equation]

Original: The output vector g i is then used to estimate the probability of word s i .
Revised: The output vector g i is then used to estimate the probability of word s i .

Original: The neural architecture of the decoder is illustrated on Figure 2c.
Revised: The neural architecture of the decoder is illustrated on Figure 2c.

Original:  D IFFERENTIABLE P ERTURB - AND -P ARSE
Revised:  D IFFERENTIABLE P ERTURB - AND -P ARSE

Original: Encoder-decoder architectures are usually straightforward to optimize with the back-propagation algorithm (Linnainmaa, 1976; LeCun et al., 2012) using any autodiff library.
Revised: Encoder-decoder architectures are usually straightforward to optimize with the back-propagation algorithm (Linnainmaa, 1976; LeCun et al., 2012) using any autodiff library.

Original: Unfortunately, our
Revised: Unfortunately, our

Original: VAE contains stochastic nodes that can not be differentiated efﬁciently as marginalization is too expensive or intractable (see Figure 2b for the list of stochastic nodes in our computation graph).
Revised: VAE contains stochastic nodes that can not be differentiated efﬁciently as marginalization is too expensive or intractable (see Figure 2b for the list of stochastic nodes in our computation graph).

Original: Kingma & Welling (2013) and Rezende et al.
Revised: Kingma & Welling (2013) and Rezende et al.

Original: (2014) proposed to rely on a Monte-Carlo estimation of the gradient.
Revised: (2014) proposed to rely on a Monte-Carlo estimation of the gradient.

Original: This approximation is differentiable because the sampling process is moved out of the backpropagation path.
Revised: This approximation is differentiable because the sampling process is moved out of the backpropagation path.

Original: In this section, we introduce our Differentiable Perturb-and-Parse operator to cope with the distribution over dependency trees.
Revised: In this section, we introduce our Differentiable Perturb-and-Parse operator to cope with the distribution over dependency trees.

Original: Firstly, in Subsection 5.1, we propose an approximate sampling process by computing the best parse tree with respect to independently perturbed arc weights.
Revised: Firstly, in Subsection 5.1, we propose an approximate sampling process by computing the best parse tree with respect to independently perturbed arc weights.

Original: Secondly, we propose a differentiable surrogate of the parsing algorithm in Subsection 5.2.
Revised: Secondly, we propose a differentiable surrogate of the parsing algorithm in Subsection 5.2.

Original: 5. P ERTURB - AND -P ARSE
Revised: 5. P ERTURB - AND -P ARSE

Original: Sampling from a categorical distributions can be achieved through the Gumbel-Max trick (Gumbel, 1954; Maddison et al., 2014).
Revised: Sampling from a categorical distributions can be achieved through the Gumbel-Max trick (Gumbel, 1954; Maddison et al., 2014).

Original: Unfortunately, this reparametrization is difﬁcult to apply when the discrete variable can take an exponential number of values as in Markov Random Fields (MRF).
Revised: Unfortunately, this reparametrization is difﬁcult to apply when the discrete variable can take an exponential number of values as in Markov Random Fields (MRF).

Original: Papandreou & Yuille (2011) proposed an approximate sampling process: each component is perturbed independently.
Revised: Papandreou & Yuille (2011) proposed an approximate sampling process: each component is perturbed independently.

Original: Then, standard MAP inference algorithm computes the sample.
Revised: Then, standard MAP inference algorithm computes the sample.

Original: This technique is called perturb-and-map.
Revised: This technique is called perturb-and-map.

Original: Arc-factored dependency parsing can be expressed as a MRF where variable nodes represent arcs, singleton factors weight arcs and a fully connected factor forces the variable assignation to describe a valid dependency tree (Smith & Eisner, 2008).
Revised: Arc-factored dependency parsing can be expressed as a MRF where variable nodes represent arcs, singleton factors weight arcs and a fully connected factor forces the variable assignation to describe a valid dependency tree (Smith & Eisner, 2008).

Original: Therefore, we can apply the perturb-and-map method to dependency tree sampling:
Revised: Therefore, we can apply the perturb-and-map method to dependency tree sampling:

Original: [Equation]
Revised: [Equation]

Original: where G (0 , 1) is the Gumbel distribution, that is sampling matrix P is equivalent to setting P i,j = − log( −
Revised: where G (0 , 1) is the Gumbel distribution, that is sampling matrix P is equivalent to setting P i,j = − log( −

Original: log U i,j ))
Revised: log U i,j ))

Original: where U i,j ∼ Uniform (0 , 1) .
Revised: where U i,j ∼ Uniform (0 , 1) .

Original:  Alternatively, it is possible to sample from the set of projective dependency trees by running the inside-out algorithm (Eisner, 2016).
Revised: We Alternatively, it is possible to sample from the set of projective dependency trees by running the inside-out algorithm (Eisner, 2016).

Original: However, it is then not straightforward to formally derive a path derivative gradient estimation.
Revised: However, it is then not straightforward to formally derive a path derivative gradient estimation.

Original: Algorithm split point for constructing an element given its span.
Revised: Algorithm split point for constructing an element given its span.

Original: b is a one-hot vector such that b
Revised: b is a one-hot vector such that b

Original: i − k
Revised: i − k

Original: = 

Revised: =

Original: iff k is the best split position.
Revised: iff k is the best split position.

Original: 1: function DEDUCE - URIGHT ( i, j, W ) 2: s ← null-initialized vec.
Revised: 1: function DEDUCE - URIGHT ( i, j, W ) 2: s ← null-initialized vec.

Original: of size j − i 3:
Revised: of size j − i 3:

Original: for i ≤ k < j do 4: s i − k ← [ i k ] + [ k +j ] + W j,i 5: b ← ONE - HOT - ARGMAX ( s ) 6:
Revised: for i ≤ k < j do 4: s i − k ← [ i k ] + [ k +j ] + W j,i 5: b ← ONE - HOT - ARGMAX ( s ) 6:

Original: BACKPTR [ i j ] ← b 7:
Revised: BACKPTR [ i j ] ← b 7:

Original: WEIGHT [ i j ] ← b (cid:62) s
Revised: WEIGHT [ i j ] ← b (cid:62) s

Original: Algorithm j ] has contributed the optimal objective, this function sets T i,j to 1 .
Revised: Algorithm j ] has contributed the optimal objective, this function sets T i,j to 1 .

Original: Then, it propagates the contribution information to its antecedents.
Revised: Then, it propagates the contribution information to its antecedents.

Original: 1: function B ACKTRACK - URIGHT ( i, j, T ) 2:
Revised: 1: function B ACKTRACK - URIGHT ( i, j, T ) 2:

Original: T i,j ← CONTRIB [ i j ] 3: b ← BACKPTR [ i j ] 4: for i ≤ k < j do 5:
Revised: T i,j ← CONTRIB [ i j ] 3: b ← BACKPTR [ i j ] 4: for i ≤ k < j do 5:

Original: CONTRIB [ i k ] + ← b i − k T i,j 6:
Revised: CONTRIB [ i k ] + ← b i − k T i,j 6:

Original: CONTRIB [ k +j ] + ← b i − k T i,j
Revised: CONTRIB [ k +j ] + ← b i − k T i,j

Original: The (approximate) Monte-Carlo estimation of the expectation in Equation 3 is then deﬁned as:
Revised: The (approximate) Monte-Carlo estimation of the expectation in Equation 3 is then deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: W
Revised: W

Original: [Equation]
Revised: [Equation]

Original: P )
Revised: P ))

Original: [Equation] where (cid:39) denotes a Monte-Carlo estimation of the gradient, P ∼ G (0 , 1) is sampled in the last line and E ISNER is an algorithm that compute the projective dependency tree with maximum (perturbed) weight
Revised: where (cid:39) denotes a Monte-Carlo estimation of the gradient, P ∼ G (0 , 1) is sampled in the last line and E ISNER is an algorithm that compute the projective dependency tree with maximum (perturbed) weight

Original: (Eisner, 1996).
Revised: (Eisner, 1996).

Original: Therefore, the sampling process is outside of the backpropagation path.
Revised: Therefore, the sampling process is outside of the backpropagation path.

Original: Unfortunately, the E ISNER algorithm is built using O NE -H OT -A RGMAX operations that have illdeﬁned partial derivatives.
Revised: Unfortunately, the E ISNER algorithm is built using O NE -H OT -A RGMAX operations that have illdeﬁned partial derivatives.

Original: We propose a differentiable surrogate in the next section.
Revised: We propose a differentiable surrogate in the next section.

Original: 5. D IFFERENTIABLE P ARSING A LGORITHM
Revised: 5. D IFFERENTIABLE P ARSING A LGORITHM

Original: We now propose a continuous relaxation of the projective dependency parsing algorithm.
Revised: We now propose a continuous relaxation of the projective dependency parsing algorithm.

Original: We start with a brief outline of the algorithm using the parsing-as-deduction formalism, restricting this presentation to the minimum needed to describe our continuous relaxation.
Revised: We start with a brief outline of the algorithm using the parsing-as-deduction formalism, restricting this presentation to the minimum needed to describe our continuous relaxation.

Original: We refer the reader to Eisner (1996) for an in-depth presentation.
Revised: We refer the reader to Eisner (1996) for an in-depth presentation.

Original: The parsing-as-deduction formalism provides an uniﬁed presentation of many parsing algorithms (Pereira & Warren, 1983; Shieber et al., 1995).
Revised: The parsing-as-deduction formalism provides an uniﬁed presentation of many parsing algorithms (Pereira & Warren, 1983; Shieber et al., 1995).

Original: In this framework, a parsing algorithm is deﬁned as a deductive system, i.e. as a set of axioms, a goal item and a set of deduction rules.
Revised: In this framework, a parsing algorithm is deﬁned as a deductive system, i.e. as a set of axioms, a goal item and a set of deduction rules.

Original: Each deduced item represents a sub-analysis of the input.
Revised: Each deduced item represents a sub-analysis of the input.

Original: Regarding implementation, the common way is to rely on dynamic programming: items are deduced in a bottom-up fashion, from smaller sub-analyses to large ones.
Revised: Regarding implementation, the common way is to rely on dynamic programming: items are deduced in a bottom-up fashion, from smaller sub-analyses to large ones.

Original: To this end, intermediate results are stored in a global chart.
Revised: To this end, intermediate results are stored in a global chart.

Original: For projective dependency parsing, the algorithm builds a chart whose items are of the form [ i j ] , [ i j ] , [ i j ] and [ i j ] that represent sub-analyses from word i to word j .
Revised: For projective dependency parsing, the algorithm builds a chart whose items are of the form [ i j ] , [ i j ] , [ i j ] and [ i j ] that represent sub-analyses from word i to word j .

Original: An item [ i j ] (resp.
Revised: An item [ i j ] (resp.

Original: [ i j ] ) represents a sub-analysis where every word s k , i ≤ k ≤ j is a descendant of s i and where s j cannot have any other modiﬁer (resp.
Revised: [ i j ] ) represents a sub-analysis where every word s k , i ≤ k ≤ j is a descendant of s i and where s j cannot have any other modiﬁer (resp.

Original: can have).
Revised: can have).

Original: The two other types are deﬁned similarly for descendants of word s j .
Revised: The two other types are deﬁned similarly for descendants of word s j .

Original: In the ﬁrst stage of the algorithm, the maximum weight of items are computed (deduced) in a bottom-up fashion.
Revised: In the ﬁrst stage of the algorithm, the maximum weight of items are computed (deduced) in a bottom-up fashion.

Original: For example, the weight WEIGHT
Revised: For example, the weight WEIGHT

Original: [ i j ] is deﬁned as the maximum of WEIGHT
Revised: [ i j ] is deﬁned as the maximum of WEIGHT

Original: [ i k ] + WEIGHT
Revised: [ i k ] + WEIGHT

Original: [ k +j ] , ∀ k s.t. i ≤ k < j , plus W i,j because [ i j ] assumes a dependency with head s i and modiﬁer s j .
Revised: [ k +j ] , ∀ k s.t. i ≤ k < j , plus W i,j because [ i j ] assumes a dependency with head s i and modiﬁer s j .

Original: In the second stage, the algorithm retrieves arcs whose scores have contributed to the optimal objective.
Revised: In the second stage, the algorithm retrieves arcs whose scores have contributed to the optimal objective.

Original: Part of the pseudo-code for the ﬁrst and second stages are given in Algorithm 1 and Algorithm 2, respectively.
Revised: Part of the pseudo-code for the ﬁrst and second stages are given in Algorithm 1 and Algorithm 2, respectively.

Original: Note that, usually, the second stage is implemented with a linear time complexity but we cannot rely on this optimization for our continuous relaxation.
Revised: Note that, usually, the second stage is implemented with a linear time complexity but we cannot rely on this optimization for our continuous relaxation.

Original: This algorithm can be thought of as the construction of a computational graph where WEIGHT ,
Revised: This algorithm can be thought of as the construction of a computational graph where WEIGHT ,

Original: BACKPTR and CONTRIB are sets of nodes (variables).
Revised: BACKPTR and CONTRIB are sets of nodes (variables).

Original: This graph includes ONE - HOT - ARGMAX operations that are not differentiable (see line 5 in Algorithm 1).
Revised: This graph includes ONE - HOT - ARGMAX operations that are not differentiable (see line 5 in Algorithm 1).

Original: This operation takes as input a vector of weights v of size k and returns a one-hot vector o of the same size with oi
Revised: This operation takes as input a vector of weights v of size k and returns a one-hot vector o of the same size with o

Original: [-= 
-]
Revised: [-= 
-]i

Original:  if and only if v i is the element of maximum value:
Revised: 1 if and only if v i is the element of maximum value:

Original: [Equation]
Revised: [Equation]

Original: We follow a recent trend (Jang et al., 2017; Maddison et al., 2017; Goyal et al., 2017;2018; Mensch & Blondel, 2018) in differentiable approximation of the ONE - HOT - ARGMAX function and replace it with the PEAKED - SOFTMAX operator:
Revised: We follow a recent trend (Jang et al., 2017; Maddison et al., 2017; Goyal et al., 2017; 2018) in differentiable approximation of the ONE - HOT - ARGMAX function and replace it with the PEAKEDSOFTMAX operator:

Original: [Equation]
Revised: [Equation]

Original: where τ > 0 is a temperature hyperparameter controlling the smoothness of the relaxation: when τ → ∞ the relaxation becomes equivalent to ONE - HOT - ARGMAX .
Revised: where τ > 0 is a temperature hyperparameter controlling the smoothness of the relaxation: when τ → ∞ the relaxation becomes equivalent to ONE - HOT - ARGMAX .

Original: With this update, the parsing algorithm is fully differentiable.
Revised: With this update, the parsing algorithm is fully differentiable.

Original: 12 Note, however, that outputs are not valid dependency trees anymore.
Revised: 12 Note, however, that outputs are not valid dependency trees anymore.

Original: Indeed, then an output matrix T contains continuous values that represent soft selection of arcs.
Revised: Indeed, then an output matrix T contains continuous values that represent soft selection of arcs.

Original: Mensch & Blondel (2018) introduced a  similar approach for tagging with the Viterbi algorithm.
Revised: Mensch & Blondel (2018) introduced a alternative but similar approach for tagging with the Viterbi algorithm.

Original: We report pseudo-codes for the forward and backward passes of our continuous relaxation of E ISNER ’s algorithm in Appendix F.
Revised: We report pseudo-codes for the forward and backward passes of our continuous relaxation of E ISNER ’s algorithm in Appendix F.

Original: 5. D ISCUSSION
Revised: 5. D ISCUSSION

Original: The fact that T is a soft selection of arcs, and not a combinatorial structure, does not impact the decoder.
Revised: The fact that T is a soft selection of arcs, and not a combinatorial structure, does not impact the decoder.

Original: Indeed, a GCN can be run over weighted graphs, the message passed between nodes is simply multiplied by the continuous weights.
Revised: Indeed, a GCN can be run over weighted graphs, the message passed between nodes is simply multiplied by the continuous weights.

Original: This is one of motivations for using GCNs rather than a Recursive LSTMs Tai et al. (2015) in the decoder.
Revised: This is one of motivations for using GCNs rather than a Recursive LSTMs (Tai et al., 2015) in the decoder.

Original: On the one hand, running a GCN with a matrix that represents a soft selection of arcs (i.e. with real values) has the same computational cost than using a standard adjacency matrix (i.e. with binary elements) if we use matrix multiplication on GPU.
Revised: On the one hand, running a GCN with a matrix that represents a soft selection of arcs (i.e. with real values) has the same computational cost than using a standard adjacency matrix (i.e. with binary elements) if we use matrix multiplication on GPU.

Original: On the other hand, a recursive network over a soft selection of arcs requires to build a
Revised: On the other hand, a recursive network over a soft selection of arcs requires to build a

Original: O ( n 2 ) set of RNN-cells that follow the dynamic programming chart where the possible inputs of a cell are multiplied by their corresponding weight in T, which is expensive and not GPU-friendly.
Revised: O ( n 2 ) set of RNN-cells that follow the dynamic programming chart where the possible inputs of a cell are multiplied by their corresponding weight in T, which is expensive and not GPU-friendly.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We ran a series of experiments on 3 different languages to test our method for semi-supervised dependency parsing: English, French and Swedish.
Revised: We ran a series of experiments on 3 different languages to test our method for semi-supervised dependency parsing: English, French and Swedish.

Original: Details about corpora can be found in Appendix C.
Revised: Details about corpora can be found in Appendix C.

Original: The size of each dataset is reported in Table 1.
Revised: The size of each dataset is reported in Table 1.

Original: Note that the setting is especially challenging for
Revised: Note that the setting is especially challenging for

Original: Swedish: the amount of unlabeled data we use here barely exceeds that of labeled data.
Revised: Swedish: the amount of unlabeled data we use here barely exceeds that of labeled data.

Original: The hyperparameters of our network are described in Appendix D.
Revised: The hyperparameters of our network are described in Appendix D.

Original: In order to ensure that we do not bias our model for the beneﬁt of the semi-supervised scenario, we use the same parameters as Kiperwasser & Goldberg (2016) for the parser.
Revised: In order to ensure that we do not bias our model for the beneﬁt of the semi-supervised scenario, we use the same parameters as Kiperwasser & Goldberg (2016) for the parser.

Original: Also, we did not perform any language-speciﬁc parameter selections.
Revised: Also, we did not perform any language-speciﬁc parameter selections.

Original: This makes us hope that our method can be applied to other languages with little extra effort.
Revised: This makes us hope that our method can be applied to other languages with little extra effort.

Original: We stress that no part-of-speech tags are used as input in any part of our network.
Revised: We stress that no part-of-speech tags are used as input in any part of our network.

Original: For English, the supervised parser took 1.5 hours to train on a NVIDIA Titan X GPU while the semi-supervised parser without sentence embedding, which sees 2 times more instances per epoch, took 3.5 hours to train.
Revised: For English, the supervised parser took 1.5 hours to train on a NVIDIA Titan X GPU while the semi-supervised parser without sentence embedding, which sees 2 times more instances per epoch, took 3.5 hours to train.

Original: Previous work has shown that learned latent structures tend to differ from linguistic syntactic structures (Kim et al., 2017; Williams et al., 2018).
Revised: Previous work has shown that learned latent structures tend to differ from linguistic syntactic structures (Kim et al., 2017; Williams et al., 2018).

Original: Therefore, we encourage the VAE to rely on latent structures close to the targeted ones by bootstrapping the training procedure with labeled data only.
Revised: Therefore, we encourage the VAE to rely on latent structures close to the targeted ones by bootstrapping the training procedure with labeled data only.

Original: We follow a common practice for VAEs: we experimented with scaling down the KL-divergence of priors (Bowman et al., 2016; Miao et al., 2017; Yin et al., 2018).
Revised: We follow a common practice for VAEs: we experimented with scaling down the KL-divergence of priors (Bowman et al., 2016; Miao et al., 2017; Yin et al., 2018).

Original: We use weights 0.01 the KLdivergence with the prior for distributions over sentence embeddings.
Revised: We use weights 0.01 the KLdivergence with the prior for distributions over sentence embeddings.

Original: For dependencies trees, we report all experiments with the weight of 0, as removing the term or heavily downweighting it was yielding the best results.
Revised: For dependency trees, we report all experiments with the weight of 0, as removing the term or heavily downweighting it was yielding the best results.

Original: As the encoder is bootstrapped with the supervised loss, it is implicitly regularized toward linguistic trees, and the KL term would negate this effect.
Revised: As the encoder is bootstrapped with the supervised loss, it is implicitly regularized toward linguistic trees, and the KL term would negate this effect.

Original: Intuitively, the KL term favors models which are uncertain on unlabeled examples, which may also be problematic, given that we would expect a strong parser to have sharp posteriors.
Revised: Intuitively, the KL term favors models which are uncertain on unlabeled examples, which may also be problematic, given that we would expect a strong parser to have sharp posteriors

Original: [Table]
Revised: [Table]

Original: 6. P ARSING RESULTS
Revised: 6. P ARSING RESULTS

Original: For each dataset, we train under the supervised and the semi-supervised scenario.
Revised: For each dataset [Table] we train under the supervised and the semi-supervised scenario.

Original: Moreover, in the semi-supervised setting, we experiment with and without latent sentence embedding z .
Revised: Moreover, in the semi-supervised setting, we experiment with and without latent sentence embedding z .

Original: We compare only to the model of Kiperwasser & Goldberg (2016).
Revised: We compare only to the model of Kiperwasser & Goldberg (2016).

Original: Recently, even more accurate models have been proposed (e.g, Dozat & Manning (2017)).
Revised: Recently, even more accurate models have been proposed (e.g., Dozat & Manning, 2017).

Original: In principle, the ideas introduced in recent work are mostly orthogonal to our proposal as we can modify our VAE model accordingly.
Revised: In principle, the ideas introduced in recent work are mostly orthogonal to our proposal as we can modify our VAE model accordingly.

Original: For example, we experimented with using bi-afﬁne attention of Dozat & Manning (2017), though it has not turned out beneﬁcial in our low-resource setting.
Revised: For example, we experimented with using bi-afﬁne attention of Dozat & Manning (2017), though it has not turned out beneﬁcial in our low-resource setting.

Original: Comparing to multiple previous parsers would have also required tuning each of them on our dataset, which is infeasible.
Revised: Comparing to multiple previous parsers would have also required tuning each of them on our dataset, which is infeasible.

Original: Therefore, we only report results with a comparable baseline, i.e. trained with a structured hinge loss (Kiperwasser & Goldberg, 2016; Taskar et al., 2005).We did not perform further tuning in order to ensure that our analysis is not skewed toward one setting.
Revised: Therefore, we only report results with a comparable baseline, i.e. trained with a structured hinge loss (Kiperwasser & Goldberg, 2016; Taskar et al., 2005). We did not perform further tuning in order to ensure that our analysis is not skewed toward one setting.

Original: Parsing results are summarized in Table 2a.
Revised: Parsing results are summarized in Table 2a.

Original: We observe a score increase in all three languages.
Revised: We observe a score increase in all three languages.

Original: Moreover, we observe that VAE performs slightly better without latent sentence embedding.
Revised: Moreover, we observe that VAE performs slightly better without latent sentence embedding.

Original: We assume this is due to the fact that dependencies are more useful when no information leaks in the decoder through z .
Revised: We assume this is due to the fact that dependencies are more useful when no information leaks in the decoder through z .

Original: Interestingly, we observe an improvement, albeit smaller, even on Swedish, where we used a very limited amount of unlabeled data.
Revised: Interestingly, we observe an improvement, albeit smaller, even on Swedish, where we used a very limited amount of unlabeled data.

Original: We note that training with structured hinge loss gives stronger results than our supervised baseline.
Revised: We note that training with structured hinge loss gives stronger results than our supervised baseline.

Original: In order to maintain the probabilistic interpretation of our model, we did not include a similar term in our model.
Revised: In order to maintain the probabilistic interpretation of our model, we did not include a similar term in our model.

Original: We conducted qualitative analyses for English.
Revised: We conducted qualitative analyses for English.

Original: 14 We report scores with respect to dependency lengths in Table 2b.
Revised: 14 We report scores with respect to dependency lengths in Table 2b.

Original: We observe that the semi-supervised parser tends to correct two kind of errors.
Revised: We observe that the semi-supervised parser tends to correct two kind of errors.

Original: Firstly, it makes fewer mistakes on root attachments, i.e. the recall is similar between the two parsers but the precision of the semi-supervised one is higher.
Revised: Firstly, it makes fewer mistakes on root attachments, i.e. the recall is similar between the two parsers but the precision of the semi-supervised one is higher.

Original: We hypothesis that root attachment errors come at a high price in the decoder because there is only a small fraction of the vocabulary that is observed with this syntactic function.
Revised: We hypothesis that root attachment errors come at a high price in the decoder because there is only a small fraction of the vocabulary that is observed with this syntactic function.

Original: Secondly, the semi-supervised parser recovers more long distance relations, i.e. the recall for dependencies with a distance superior or equal to 7 is higher.
Revised: Secondly, the semi-supervised parser recovers more long distance relations, i.e. the recall for dependencies with a distance superior or equal to 7 is higher.

Original: Intuitively, we assume these dependencies are more useful in the decoder: for short distance dependencies, the LSTM efﬁciently captures the context of the word to predict, whereas this infor- mation could be vanishing for long distances, meaning the GCN has more impact on the prediction.
Revised: Intuitively, we assume these dependencies are more useful in the decoder: for short distance dependencies, the LSTM efﬁciently captures the context of the word to predict, whereas this infor- mation could be vanishing for long distances, meaning the GCN has more impact on the prediction.

Original: We also checked how the scores differ across dependency labels.
Revised: We also checked how the scores differ across dependency labels.

Original: We report main differences in
Revised: We report main differences in

Original: Tables 2c.
Revised: Tables 2c.

Original: The largest improvements are obtained for multi-word expressions: this is particularly interesting because they are known to be challenging in NLP.
Revised: The largest improvements are obtained for multi-word expressions: this is particularly interesting because they are known to be challenging in NLP.

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: Dependency parsing in the low-ressource scenario has been of interest in the NLP community due to the expensive nature of annotation.
Revised: Dependency parsing in the low-ressource scenario has been of interest in the NLP community due to the expensive nature of annotation.

Original: On the one hand, transfer approaches learn a delexicalized parser for a resource-rich language which is then used to parse a low-resource one (Agi´c et al., 2016;
Revised: On the one hand, transfer approaches learn a delexicalized parser for a resource-rich language which is then used to parse a low-resource one (Agi´c et al., 2016;

Original: McDonald et al., 2011).
Revised: McDonald et al., 2011).

Original: On the other hand, the grammar induction approach learns a dependency parser in an unsupervised manner.
Revised: On the other hand, the grammar induction approach learns a dependency parser in an unsupervised manner.

Original: Klein & Manning (2004) introduced the ﬁrst generative model that outperforms the right-branching heuristic in English.
Revised: Klein & Manning (2004) introduced the ﬁrst generative model that outperforms the right-branching heuristic in English.

Original: Close to our work, Cai et al.
Revised: Close to our work, Cai et al.

Original: (2017) use an auto-encoder setting where the decoder tries to rebuild the source sentence.
Revised: (2017) use an auto-encoder setting where the decoder tries to rebuild the source sentence.

Original: However, their decoder is unstructured (e.g. it is not auto-regressive).
Revised: However, their decoder is unstructured (e.g. it is not auto-regressive).

Original: Variational Auto-Encoders (Kingma & Welling, 2013; Rezende et al., 2014) have been investigated in the semi-supervised settings (Kingma et al., 2014) for NLP.
Revised: Variational Auto-Encoders (Kingma & Welling, 2013; Rezende et al., 2014) have been investigated in the semi-supervised settings (Kingma et al., 2014) for NLP.

Original: Ko ˇ cisk ´ y et al.
Revised: Ko ˇ cisk ´ y et al.

Original: (2016) learn a semantic parser where the latent variable is a discrete sequence of symbols.
Revised: (2016) learn a semantic parser where the latent variable is a discrete sequence of symbols.

Original: Zhou & Neubig (2017) successfully applied the variational method to semi-supervised morphological re-inﬂection where discrete latent variables represent linguistic features (e.g. tense, part-of-speech tag).
Revised: Zhou & Neubig (2017) successfully applied the variational method to semi-supervised morphological re-inﬂection where discrete latent variables represent linguistic features (e.g. tense, part-of-speech tag).

Original: Yin et al.
Revised: Yin et al.

Original: (2018) proposed a semi-supervised semantic parser.
Revised: (2018) proposed a semi-supervised semantic parser.

Original: Similarly to our model, they rely on a structured latent variable.
Revised: Similarly to our model, they rely on a structured latent variable.

Original: However, all of these systems use either categorical random variables or the REINFORCE score estimator.
Revised: However, all of these systems use either categorical random variables or the REINFORCE score estimator.

Original: To the best of our knowledge, no previous work used continuous relaxation of a dynamic programming latent variable in the VAE setting.
Revised: To the best of our knowledge, no previous work used continuous relaxation of a dynamic programming latent variable in the VAE setting.

Original: The main challenge is backpropagation through discrete random variables.
Revised: The main challenge is backpropagation through discrete random variables.

Original: Maddison et al.
Revised: Maddison et al.

Original: (2017) and Jang et al.
Revised: (2017) and Jang et al.

Original: (2017) ﬁrst introduced the Gumbel-Softmax operator for the categorical distribution.
Revised: (2017) ﬁrst introduced the Gumbel-Softmax operator for the categorical distribution.

Original: There are two issues regarding more complex discrete distributions.
Revised: There are two issues regarding more complex discrete distributions.

Original: Firstly, one have to build a reparametrization of the the sampling process.
Revised: Firstly, one have to build a reparametrization of the the sampling process.

Original: Papandreou & Yuille (2011) showed that low-order perturbations provide samples of good qualities for graphical models.
Revised: Papandreou & Yuille (2011) showed that low-order perturbations provide samples of good qualities for graphical models.

Original: While independent factor perturbation in a graphical model may not capture the underlying structure of the latter, Gane et al.

Original: (2014) proposed to learn perturbations that implicitly model the structure.

Original: Either way, the sampling process is reduced to a structured arg max with a perturbed objective function.

Original: Secondly, one have to build a good differentiable surrogate to the structured arg max operator.
Revised: Secondly, one have to build a good differentiable surrogate to the structured arg max operator.

Original: Early work replaced the structured arg max with structured attention (Kim et al., 2017).
Revised: Early work replaced the structured arg max with structured attention (Kim et al., 2017).

Original: However, computing the marginals over the parse forest is sensitive to numerical stability outside speciﬁc cases like non-projective dependency parsing (Liu & Lapata, 2018; Tran & Bisk, 2018).
Revised: However, computing the marginals over the parse forest is sensitive to numerical stability outside speciﬁc cases like non-projective dependency parsing (Liu & Lapata, 2018; Tran & Bisk, 2018).

Original: Moreover, we are interested in continuous surrogates that are more peaked toward the maximum.

Original: Mensch & Blondel (2018) studied this setting for dynamic program smoothing.
Revised: Mensch & Blondel (2018) proposed a stable algorithm based on dynamic program smoothing.

Original: They experiment with the linear-chain Viterbi and the

Original: DTW algorithm. Our approach is highly related but we describe a practical implementation with a different type of dynamic programs, i.e. using the parsing-as-deduction formalism.
Revised: Our approach is highly related but we describe a continuous relaxation using the parsing-as-deduction formalism.

Original: Peng et al.
Revised: Peng et al.

Original: (2018) propose to replace the true gradient with a proxy that tries to satisfy constraints on a arg max operator via a projection.
Revised: (2018) propose to replace the true gradient with a proxy that tries to satisfy constraints on a arg max operator via a projection.

Original: However, their approach is computationally expensive, so they remove the tree constraint on dependencies during backpropagation.
Revised: However, their approach is computationally expensive, so they remove the tree constraint on dependencies during backpropagation.

Original: A parallel line of work focuses on sparse structures that are differentiable (Martins & Astudillo, 2016; Niculae et al., 2018).
Revised: A parallel line of work focuses on sparse structures that are differentiable (Martins & Astudillo, 2016; Niculae et al., 2018).

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: We presented a novel generative learning approach for semi-supervised dependency parsing.
Revised: We presented a novel generative learning approach for semi-supervised dependency parsing.

Original: We model the dependency structure of a sentence as a latent variable and build a Variational AutoEncoder.
Revised: We model the dependency structure of a sentence as a latent variable and build a VAE.

Original: We hope to motivate investigation of latent syntactic structures via differentiable dynamic programming in neural networks.
Revised: We hope to motivate investigation of latent syntactic structures via differentiable dynamic programming in neural networks.

Original: To effectively train the posterior distribution over dynamic programming latent variables, we build a continuous relaxation of the parsing algorithm.

Original: Future work includes research for an informative prior for the dependency tree distribution, for example by introducing linguistic knowledge (Naseem et al., 2010;
Revised: Future work includes research for an informative prior for the dependency tree distribution, for example by introducing linguistic knowledge (Naseem et al., 2010;

Original: Noji et al., 2016) or with an adversarial training criterion Makhzani et al. (2016).
Revised: Noji et al., 2016) or with an adversarial training criterion Makhzani et al. (2016).

Original: This work could also be extended to the unsupervised scenario.
Revised: This work could also be extended to the unsupervised scenario.

Revised: A CKNOWLEDGMENTS

Revised: We thank Diego Marcheggiani, Wilker Ferreira Aziz and Serhii Havrylov for their comments and suggestions.

Revised: We thank the anonymous reviewers for their comments.

Revised: The project was supported by the Dutch National Science Foundation (NWO VIDI 639.022.518) and European Research Council (ERC Starting Grant BroadSem 678254).

Original: A R EPARAMETRIZATION TRICK
Revised: A R EPARAMETRIZATION TRICK

Original: Sampling from a diagonal Gaussian random variable with mean vector m and variance vector v can be re-expressed as:
Revised: Sampling from a diagonal Gaussian random variable with mean vector m and variance vector v can be re-expressed as:

Original: [Equation]
Revised: [Equation]

Original: where z is the sample.
Revised: where z is the sample.

Original: As such, e ∼ N (0 , 1) is an input of the neural network for which we do not need to compute partial derivatives.
Revised: As such, e ∼ N (0 , 1) is an input of the neural network for which we do not need to compute partial derivatives.

Original: This technique is called the reparametrization trick (Kingma &
Revised: This technique is called the reparametrization trick (Kingma &

Original: Welling, 2013; Rezende et al., 2014).
Revised: Welling, 2013; Rezende et al., 2014).

Original: B G UMBEL -M AX TRICK
Revised: B G UMBEL -M AX TRICK

Original: Sampling from a categorical distributions can be achieved through the Gumbel-Max trick (Gumbel, 1954; Maddison et al., 2014).
Revised: Sampling from a categorical distributions can be achieved through the Gumbel-Max trick (Gumbel, 1954; Maddison et al., 2014).

Original: Randomly generated Gumbel noise is added to the log-probability of every element of the sample space.
Revised: Randomly generated Gumbel noise is added to the log-probability of every element of the sample space.

Original: Then, the sample is simply the element with maximum perturbed log-probability.
Revised: Then, the sample is simply the element with maximum perturbed log-probability.

Original: Let d ∈ (cid:52) k be a random variable taking values in the corner of the unit-simplex of dimension k with probability:
Revised: Let d ∈ (cid:52) k be a random variable taking values in the corner of the unit-simplex of dimension k with probability:

Original: [Equation]
Revised: [Equation]

Original: where w is a vector of weights.
Revised: where w is a vector of weights.

Original: Sampling d ∼ p ( d ) can be re-expressed as follows:
Revised: Sampling d ∼ p ( d ) can be re-expressed as follows:

Original: [Equation]
Revised: [Equation]

Original: where G (0 , 1) is the Gumbel distribution.
Revised: where G (0 , 1) is the Gumbel distribution.

Original: Sampling g ∼ G (0 , 1) is equivalent to setting g i =
Revised: Sampling g ∼ G (0 , 1) is equivalent to setting g i =

Original: − log( − log u i )) where u i ∼ Uniform (0 , 1) .
Revised: − log( − log u i )) where u i ∼ Uniform (0 , 1) .

Original: If w is computed by a neural network, the sampling process is outside the backpropagation path.
Revised: If w is computed by a neural network, the sampling process is outside the backpropagation path.

Original: C C ORPORA
Revised: C C ORPORA

Original: English We use the Stanford Dependency conversion (De Marneffe & Manning, 2008) of the Penn
Revised: English We use the Stanford Dependency conversion (De Marneffe & Manning, 2008) of the Penn

Original: Treebank (Marcus et al., 1993) with the usual section split: 02-21 for training, 22 for development and 23 for testing.
Revised: Treebank (Marcus et al., 1993) with the usual section split: 02-21 for training, 22 for development and 23 for testing.

Original: In order to simulate our framework under a low-resource setting, the annotation is kept for 10% of the training set only: a labeled sentence is the sentence which has an index (in the training set) modulo 10 equal to zero.
Revised: In order to simulate our framework under a low-resource setting, the annotation is kept for 10% of the training set only: a labeled sentence is the sentence which has an index (in the training set) modulo 10 equal to zero.

Original: French We use a similar setting with the French Treebank version distributed for the SPMRLshared task and the provided train/dev/test split (Abeill´e et al., 2000; Seddah et al., 2013).
Revised: French We use a similar setting with the French Treebank version distributed for the SPMRLshared task and the provided train/dev/test split (Abeill´e et al., 2000; Seddah et al., 2013).

Original: Swedish We use the Talbanken dataset (Nivre et al., 2006) which contains two written text parts: the professional prose part (P) and the high school students’ essays part (G).
Revised: Swedish We use the Talbanken dataset (Nivre et al., 2006) which contains two written text parts: the professional prose part (P) and the high school students’ essays part (G).

Original: We drop the annotation of (G) in order to use this section as unlabeled data.
Revised: We drop the annotation of (G) in order to use this section as unlabeled data.

Original: We split the (P) section in labeled train/dev/test using a pseudo-randomized scheme.
Revised: We split the (P) section in labeled train/dev/test using a pseudo-randomized scheme.

Original: We follow the splitting scheme of Hall et al.
Revised: We follow the splitting scheme of Hall et al.

Original: but ﬁx section 9 as development instead of k -fold cross-validation.
Revised: but ﬁx section 9 as development instead of k -fold cross-validation.

Original: Sentence i is allocated to section i mod 10 .
Revised: Sentence i is allocated to section i mod 10 .

Original: Then, section 1-8 are used for training, section 9 for dev and section 0 for test.
Revised: Then, section 1-8 are used for training, section 9 for dev and section 0 for test.

Original: D H YPER - PARAMETERS
Revised: D H YPER - PARAMETERS

Original: Encoder: word embeddings We concatenate trainable word embeddings of size 100 with external word embeddings.
Revised: Encoder: word embeddings We concatenate trainable word embeddings of size 100 with external word embeddings.

Original: 15 We use the word-dropout settings of Kiperwasser & Goldberg (2016).
Revised: 15 We use the word-dropout settings of Kiperwasser & Goldberg (2016).

Original: For
Revised: For

Original: English, external embeddings are pre-trained with the structured skip n-gram objective (Ling et al., 
Revised: English, external embeddings are pre-trained with the structured skip n-gram objective (Ling et al., 2015).

Revised: and we the Polyglot embeddings (Al-Rfou

Revised: 17 We stress out that no part-of-speech tag is used as input in any part of our network.

Original: Encoder: dependency parser
Revised: Encoder: dependency parser

Original: The dependency parser is built upon a two-stack BiLSTM with a hidden layer size of 125 (i.e. the output at each position is of size 250).
Revised: The dependency parser is built upon a two-stack BiLSTM with a hidden layer size of 125 (i.e. the output at each position is of size 250).

Original: Each dependency is then weighted using a single-layer perceptron with a tanh activation function.
Revised: Each dependency is then weighted using a single-layer perceptron with a tanh activation function.

Original: Arc label prediction rely on a similar setting, we refer to the reader to Kiperwasser & Goldberg (2016) for more information about the parser’s architecture.
Revised: Arc label prediction rely on a similar setting, we refer to the reader to Kiperwasser & Goldberg (2016) for more information about the parser’s architecture.

Original: Encoder: sentence embedding The sentence is encoded into a ﬁxed size vector with a simple leftto-right LSTM with an hidden size of 100.
Revised: Encoder: sentence embedding The sentence is encoded into a ﬁxed size vector with a simple leftto-right LSTM with an hidden size of 100.

Original: The hidden layer at the last position of the sentence is then fed to two distinct single-layer perceptrons, with an output size of 100 followed by a piecewise tanh activation function, that computes means and standard deviations of the diagonal Gaussian distribution.
Revised: The hidden layer at the last position of the sentence is then fed to two distinct single-layer perceptrons, with an output size of 100 followed by a piecewise tanh activation function, that computes means and standard deviations of the diagonal Gaussian distribution.

Original: Decoder The decoder use ﬁxed pre-trained embeddings only.
Revised: Decoder The decoder use ﬁxed pre-trained embeddings only.

Original: The recurrent layer of the decoder is a
Revised: The recurrent layer of the decoder is a

Original: LSTM with an hidden layer size of 100.
Revised: LSTM with an hidden layer size of 100.

Original: M LP (cid:121) , M LP (cid:120) and M LP (cid:35) are all single-layer perceptrons with an output size of 100 and without activation function.
Revised: M LP (cid:121) , M LP (cid:120) and M LP (cid:35) are all single-layer perceptrons with an output size of 100 and without activation function.

Original: Training We encourage the VAE to rely on latent structures close to the targeted ones by bootstrapping the training procedure with labeled data only.
Revised: Training We encourage the VAE to rely on latent structures close to the targeted ones by bootstrapping the training procedure with labeled data only.

Original: In the ﬁrst two epochs, we train the network with the discriminative loss only.
Revised: In the ﬁrst two epochs, we train the network with the discriminative loss only.

Original: Then, for the next two epochs, we add the supervised ELBO term (Equation 5).
Revised: Then, for the next two epochs, we add the supervised ELBO term (Equation 5).

Original: Finally, after the 6th epoch, we also add the unsupervised ELBO term (Equation 3).
Revised: Finally, after the 6th epoch, we also add the unsupervised ELBO term (Equation 3).

Original: We train our network using stochastic gradient descent for 30 epochs using Adadelta (Zeiler, 2012) with default parameters as provided by the Dynet library (Neubig et al., 2017).
Revised: We train our network using stochastic gradient descent for 30 epochs using Adadelta (Zeiler, 2012) with default parameters as provided by the Dynet library (Neubig et al., 2017).

Original: In the semisupervised scenario, we alternate between labeled and unlabeled instances.
Revised: In the semisupervised scenario, we alternate between labeled and unlabeled instances.

Original: The temperature of the
Revised: The temperature of the

Original: PEAKED - SOFTMAX operator is ﬁxed to τ = 1 .
Revised: PEAKED - SOFTMAX operator is ﬁxed to τ = 1 .

Original: E C OMPARISON WITH SEMIRING PARSING
Revised: E C OMPARISON WITH SEMIRING PARSING

Original: Dynamic programs for parsing have been studied as abstract algorithms that can be instantiated with different semirings (Goodman, 1999).
Revised: Dynamic programs for parsing have been studied as abstract algorithms that can be instantiated with different semirings (Goodman, 1999).

Original: For example, computing the weight of the best parse relies on the (cid:104) R , max , + (cid:105) semiring.
Revised: For example, computing the weight of the best parse relies on the (cid:104) R , max , + (cid:105) semiring.

Original: This semiring can be augmented with set-valued operations to retrieve the best derivation.
Revised: This semiring can be augmented with set-valued operations to retrieve the best derivation.

Original: However, a straightforward implementation would have a O ( n 5 ) space complexity: for each item in the chart, we also need to store the set of arcs.
Revised: However, a straightforward implementation would have a O ( n 5 ) space complexity: for each item in the chart, we also need to store the set of arcs.

Original: Under this formalism, the backpointer trick is a method to implicitly constructs these sets and maintain the optimal O ( n 3 ) complexity.
Revised: Under this formalism, the backpointer trick is a method to implicitly constructs these sets and maintain the optimal O ( n 3 ) complexity.

Original: Our continuous relaxation replaces the max operator with a smooth surrogate and the set values with an expectation over sets.
Revised: Our continuous relaxation replaces the max operator with a smooth surrogate and the set values with a soft-selection of sets.

Original: Unfortunately, (cid:104) R , PEAKED - SOFTMAX (cid:105) is not a commutative monoid, therefore the semiring analogy is not transposable.
Revised: Unfortunately, (cid:104) R , PEAKED - SOFTMAX (cid:105) is not a commutative monoid, therefore the semiring analogy is not transposable.

Original: F D IFFERENTIABLE DYNAMIC PROGRAMMING FOR PROJECTIVE DEPENDENCY PARSING
Revised: F D IFFERENTIABLE DYNAMIC PROGRAMMING FOR PROJECTIVE DEPENDENCY PARSING

Original: We describe how we can embed a continuous relaxation of projective dependency parsing as a node in a neural network.
Revised: We describe how we can embed a continuous relaxation of projective dependency parsing as a node in a neural network.

Original: During the forward pass, we are given arc weights W
Revised: During the forward pass, we are given arc weights W

Original: and we compute the relaxed projective dependency tree T that maximize the arc-factored weight (cid:80) h,m T h,m × W h,m .
Revised: and we compute the relaxed projective dependency tree T that maximize the arc-factored weight (cid:80) h,m T h,m × W h,m .

Original: Each output variable T h,m ∈ [0 , 1] is a soft selection of dependency with head-word s h and modiﬁer s m .
Revised: Each output variable T h,m ∈ [0 , 1] is a soft selection of dependency with head-word s h and modiﬁer s m .

Original: During back-propagation, we are given partial derivatives of the loss with respect to each arc and we compute the ones with respect to arc weights:
Revised: During back-propagation, we are given partial derivatives of the loss with respect to each arc and we compute the ones with respect to arc weights:

Original: [Equation]
Revised: [Equation]

Original: Note that the Jacobian matrix has O ( n 4 ) values but we do need to explicitly compute it.
Revised: Note that the Jacobian matrix has O ( n 4 ) values but we do need to explicitly compute it.

Original: The space and time complexity of the forward and backward passes are both cubic, similar to Eisner’s algorithm.
Revised: The space and time complexity of the forward and backward passes are both cubic, similar to Eisner’s algorithm.

Original: F. F ORWARD PASS
Revised: F. F ORWARD PASS

Original: The forward pass is a two step algorithm:
Revised: The forward pass is a two step algorithm:

Original: First, we compute the cumulative weight of each item and store soft backpointers to keep track of contribution of antecedents.
Revised: First, we compute the cumulative weight of each item and store soft backpointers to keep track of contribution of antecedents.

Original: This step is commonly called to inside algorithm.
Revised: This step is commonly called to inside algorithm.

Original: Then, we compute the contribution of each arc thanks to the backpointers.
Revised: Then, we compute the contribution of each arc thanks to the backpointers.

Original: This step is somewhat similar to the arg max reconstruction algorithm.
Revised: This step is somewhat similar to the arg max reconstruction algorithm.

Original: The outline of the algorithm is given in Algorithm 3.
Revised: The outline of the algorithm is given in Algorithm 3.

Original: The inside algorithm computes the following variables:
Revised: The inside algorithm computes the following variables:

Original: • a [ i j ][ k ] is the weight of item [ i j ] if we split its antecedent at k .
Revised: • a [ i j ][ k ] is the weight of item [ i j ] if we split its antecedent at k .

Original: • b [ i j ][ k ] is the soft backpointer to antecedents of item [ i j ] with split at k .
Revised: • b [ i j ][ k ] is the soft backpointer to antecedents of item [ i j ] with split at k .

Original: • c [ i j ] is the cumulative weight of item [ i j ] .
Revised: • c [ i j ] is the cumulative weight of item [ i j ] .

Original: and similarly for the other chart values.
Revised: and similarly for the other chart values.

Original: The algorithm is given in Algorithm 5.
Revised: The algorithm is given in Algorithm 5.

Original: The backpointer reconstruction algorithm compute the contribution of each arc.
Revised: The backpointer reconstruction algorithm compute the contribution of each arc.

Original: We follow backpointers in reverse order in order to compute the contribution of each item ˜
Revised: We follow backpointers in reverse order in order to compute the contribution of each item ˜

Original: c
Revised: c

Original: [ i j ] .
Revised: [ i j ] .

Original: The algorithm is given in Algorithm 6.
Revised: The algorithm is given in Algorithm 6.

Original: F. B ACKWARD PASS
Revised: F. B ACKWARD PASS

Original: During the backward pass, we compute the partial derivatives of variables using the chain rule, i.e. in the reverse order of their creation: we ﬁrst run backpropagation through the backpointer reconstruction algorithm
Revised: During the backward pass, we compute the partial derivatives of variables using the chain rule, i.e. in the reverse order of their creation: we ﬁrst run backpropagation through the backpointer reconstruction algorithm

Original: and then through the inside algorithm (see Algorithm 4).
Revised: and then through the inside algorithm (see Algorithm 4).

Original: Given the partial derivatives in Figure 3, backpropagation through the backpointer reconstruction algorithm is straighforward to compute, see Algorithm 7.
Revised: Given the partial derivatives in Figure 3, backpropagation through the backpointer reconstruction algorithm is straighforward to compute, see Algorithm 7.

Original: Partial derivatives of the inside algorithm’s variables are given in Figure 4.
Revised: Partial derivatives of the inside algorithm’s variables are given in Figure 4.

Original: function R ELAXED -E ISNER ()
Revised: function R ELAXED -E ISNER ()

Original: I NSIDE ( )
Revised: I NSIDE ( )

Original: B ACKPTR () for i = 0 . . .
Revised: B ACKPTR () for i = 0 . . .

Original: n do for j = 1 . . .
Revised: n do for j = 1 . . .

Original: n do if i < j then
Revised: n do if i < j then

Original: T i,j ← ˜ c
Revised: T i,j ← ˜ c

Original: [ i j ] else if j < i then
Revised: [ i j ] else if j < i then

Original: T i,j ← ˜ c
Revised: T i,j ← ˜ c

Original: [ i j ] function B ACKPROP -R ELAXED -E ISNER ()
Revised: [ i j ] function B ACKPROP -R ELAXED -E ISNER ()

Original: B ACKPROP -B ACKPTR ()
Revised: B ACKPROP -B ACKPTR ()

Original: B ACKPROP -I NSIDE () for i = 0 . . .
Revised: B ACKPROP -I NSIDE () for i = 0 . . .

Original: n do for j = 1 . . .
Revised: n do for j = 1 . . .

Original: n do if i < j
Revised: n do if i < j

Original: then ∂
Revised: then ∂

Original: L ∂W i,j ← ∂ L ∂c [ i j ] else if j < i then ∂
Revised: L ∂W i,j ← ∂ L ∂c [ i j ] else if j < i then ∂

Original: L ∂W i,j ← ∂ L ∂c [ j i ] function I NSIDE ( n ) for i ← 0 . . .
Revised: L ∂W i,j ← ∂ L ∂c [ j i ] function I NSIDE ( n ) for i ← 0 . . .

Original: n do c
Revised: n do c

Original: [ i i ] ← 0 , c [ i i ] ← 0 , c
Revised: [ i i ] ← 0 , c [ i i ] ← 0 , c

Original: [ i i ] ← 0 , c
Revised: [ i i ] ← 0 , c

Original: [ i i ] ←for l ← 1 . . .
Revised: [ i i ] ←for l ← 1 . . .

Original: n do for i ← 0 . . .
Revised: n do for i ← 0 . . .

Original: n − l do j ← i + l for k = i . . .
Revised: n − l do j ← i + l for k = i . . .

Original: j − 1 do a
Revised: j − 1 do a

Original: [ i j ][ k ] ← c [ i k ] + c [ k +j ] b [ i j ] ← softmax( a [ i j ]) c
Revised: [ i j ][ k ] ← c [ i k ] + c [ k +j ] b [ i j ] ← softmax( a [ i j ]) c

Original: [ i j ] ← W i,j + (cid:80) k
Revised: [ i j ] ← W i,j + (cid:80) k

Original: = i...
Revised: = i...

Original: j − 1 b [ i j ][ k ] × a [ i j ][ k ] for k
Revised: j − 1 b [ i j ][ k ] × a [ i j ][ k ] for k

Original: = i . . .
Revised: = i . . .

Original: j − 1 do a
Revised: j − 1 do a

Original: [ i j ][ k ] ← c [ i k ] + c [ k +j ] b [ i j ] ← softmax( a [ i j ]) c
Revised: [ i j ][ k ] ← c [ i k ] + c [ k +j ] b [ i j ] ← softmax( a [ i j ]) c

Original: [ i j ] ← W j,i + (cid:80) k
Revised: [ i j ] ← W j,i + (cid:80) k

Original: = i...
Revised: = i...

Original: j − 1 b [ i j ][ k ] × a [ i j ][ k ] for k
Revised: j − 1 b [ i j ][ k ] × a [ i j ][ k ] for k

Original: = i + 1 . . .
Revised: = i + 1 . . .

Original: j do a [ i j ][ k ] ← c
Revised: j do a [ i j ][ k ] ← c

Original: [ i k ]
Revised: [ i k ]

Original: + c [ k j ] b [ i j ] ← softmax( a [ i j ]) c
Revised: + c [ k j ] b [ i j ] ← softmax( a [ i j ]) c

Original: [ i j ] ← (cid:80) k
Revised: [ i j ] ← (cid:80) k

Original: = i +1 ...
Revised: = i +1 ...

Original: j b [ i j ][ k ] × a [ i j ][ k ] for k = i . . .
Revised: j b [ i j ][ k ] × a [ i j ][ k ] for k = i . . .

Original: j − 1 do a
Revised: j − 1 do a

Original: [ i j ][ k ] ← c [ i k ] + c [ k j ] b [ i j ] ← softmax( a [ i j ]) c
Revised: [ i j ][ k ] ← c [ i k ] + c [ k j ] b [ i j ] ← softmax( a [ i j ]) c

Original: [ i j ] ← (cid:80) k
Revised: [ i j ] ← (cid:80) k

Original: = i...
Revised: = i...

Original: j − 1 b [ i j ][ k ] × a [ i j ][ k ] function B ACKPTR () for i = 0 . . .
Revised: j − 1 b [ i j ][ k ] × a [ i j ][ k ] function B ACKPTR () for i = 0 . . .

Original: n do for j = i . . .
Revised: n do for j = i . . .

Original: n do ˜ c
Revised: n do ˜ c

Original: [ i j ] ← 0 , ˜ c [ i j ] ← 0 , ˜ c
Revised: [ i j ] ← 0 , ˜ c [ i j ] ← 0 , ˜ c

Original: [ i j ] ← 0 , ˜ c
Revised: [ i j ] ← 0 , ˜ c

Original: [ i j ] ←˜ c [ n ] ←for l = n . . .
Revised: [ i j ] ←˜ c [ n ] ←for l = n . . .

Original: 1 do for i = 0 . . .
Revised: 1 do for i = 0 . . .

Original: n − l do j ← i + l for k
Revised: n − l do j ← i + l for k

Original: = i + 1 . . .
Revised: = i + 1 . . .

Original: j do ˜
Revised: j do ˜

Original: c
Revised: c

Original: [ i k ] + ← ˜ c
Revised: [ i k ] + ← ˜ c

Original: [ i j ] × b [ i j ][ k ] ˜ c
Revised: [ i j ] × b [ i j ][ k ] ˜ c

Original: [ k j ] + ← ˜ c
Revised: [ k j ] + ← ˜ c

Original: [ i j ] × b [ i j ][ k ] for k
Revised: [ i j ] × b [ i j ][ k ] for k

Original: = i . . .
Revised: = i . . .

Original: j − 1 do ˜ c
Revised: j − 1 do ˜ c

Original: [ i k ] + ← ˜ c
Revised: [ i k ] + ← ˜ c

Original: [ i j ] × b [ i j ][ k ] ˜ c
Revised: [ i j ] × b [ i j ][ k ] ˜ c

Original: [ k j ] + ← ˜ c
Revised: [ k j ] + ← ˜ c

Original: [ i j ] × b [ i j ][ k ] for k
Revised: [ i j ] × b [ i j ][ k ] for k

Original: = i . . .
Revised: = i . . .

Original: j − 1 do ˜ c
Revised: j − 1 do ˜ c

Original: [ i k ] + ← ˜ c
Revised: [ i k ] + ← ˜ c

Original: [ i j ] × b [ i j ][ k ] ˜ c
Revised: [ i j ] × b [ i j ][ k ] ˜ c

Original: [ k +j ] +
Revised: [ k +j ] +

Original: ← ˜ c
Revised: ← ˜ c

Original: [ i j ] × b [ i j ][ k ] for k
Revised: [ i j ] × b [ i j ][ k ] for k

Original: = i . . .
Revised: = i . . .

Original: j − 1 do ˜ c
Revised: j − 1 do ˜ c

Original: [ i k ] + ← ˜ c
Revised: [ i k ] + ← ˜ c

Original: [ i j ] × b [ i j ][ k ] ˜ c
Revised: [ i j ] × b [ i j ][ k ] ˜ c

Original: [ k +j ] +
Revised: [ k +j ] +

Original: ← ˜ c
Revised: ← ˜ c

Original: [ i j ] × b [ i j ][ k ] ∀ i < k ≤ j : ∂
Revised: [ i j ] × b [ i j ][ k ] ∀ i < k ≤ j : ∂

Original: ˜ c
Revised: ˜ c

Original: [ i k ] ∂ ˜ c
Revised: [ i k ] ∂ ˜ c

Original: [ i j ] =
Revised: [ i j ] =

Original: b [ i j ][ k ] ∀ i < k ≤ j : ∂
Revised: b [ i j ][ k ] ∀ i < k ≤ j : ∂

Original: ˜ c
Revised: ˜ c

Original: [ i k ] ∂b [ i j ][ k ] =
Revised: [ i k ] ∂b [ i j ][ k ] =

Original: ˜ c
Revised: ˜ c

Original: [ i j ] ∀ i < k ≤ j : ∂ ˜ c [ k j ] ∂ ˜ c
Revised: [ i j ] ∀ i < k ≤ j : ∂ ˜ c [ k j ] ∂ ˜ c

Original: [ i j ] =
Revised: [ i j ] =

Original: b [ i j ][ k ] ∀ i < k ≤ j : ∂ ˜ c
Revised: b [ i j ][ k ] ∀ i < k ≤ j : ∂ ˜ c

Original: [ k j ] ∂b [ i j ][ k ] =
Revised: [ k j ] ∂b [ i j ][ k ] =

Original: ˜ c
Revised: ˜ c

Original: [ i j ] ∀ i ≤ k < j : ∂ ˜ c [ i k ] ∂ ˜ c
Revised: [ i j ] ∀ i ≤ k < j : ∂ ˜ c [ i k ] ∂ ˜ c

Original: [ i j ] =
Revised: [ i j ] =

Original: b [ i j ][ k ] ∀ i ≤ k < j : ∂
Revised: b [ i j ][ k ] ∀ i ≤ k < j : ∂

Original: ˜ c
Revised: ˜ c

Original: [ i k ] ∂b [ i j ][ k ] =
Revised: [ i k ] ∂b [ i j ][ k ] =

Original: ˜ c
Revised: ˜ c

Original: [ i j ] ∀ i ≤ k < j : ∂ ˜ c
Revised: [ i j ] ∀ i ≤ k < j : ∂ ˜ c

Original: [ k j ] ∂ ˜ c
Revised: [ k j ] ∂ ˜ c

Original: [ i j ] =
Revised: [ i j ] =

Original: b [ i j ][ k ] ∀ i ≤ k < j : ∂ ˜ c
Revised: b [ i j ][ k ] ∀ i ≤ k < j : ∂ ˜ c

Original: [ k j ] ∂b [ i j ][ k ] =
Revised: [ k j ] ∂b [ i j ][ k ] =

Original: ˜ c
Revised: ˜ c

Original: [ i j ] ∀ i ≤ k < j : ∂ ˜ c [ i k ] ∂ ˜ c
Revised: [ i j ] ∀ i ≤ k < j : ∂ ˜ c [ i k ] ∂ ˜ c

Original: [ i j ] =
Revised: [ i j ] =

Original: b [ i j ][ k ] ∀ i ≤ k < j : ∂
Revised: b [ i j ][ k ] ∀ i ≤ k < j : ∂

Original: ˜ c
Revised: ˜ c

Original: [ i k ] ∂b [ i j ][ k ] =
Revised: [ i k ] ∂b [ i j ][ k ] =

Original: ˜ c
Revised: ˜ c

Original: [ i j ] ∀ i ≤ k < j : ∂ ˜ c
Revised: [ i j ] ∀ i ≤ k < j : ∂ ˜ c

Original: [ k +j ] ∂ ˜ c
Revised: [ k +j ] ∂ ˜ c

Original: [ i j ] =
Revised: [ i j ] =

Original: b [ i j ][ k ] ∀ i ≤ k < j : ∂ ˜ c
Revised: b [ i j ][ k ] ∀ i ≤ k < j : ∂ ˜ c

Original: [ k +j ] ∂b [ i j ][ k ] =
Revised: [ k +j ] ∂b [ i j ][ k ] =

Original: ˜ c
Revised: ˜ c

Original: [ i j ] ∀ i ≤ k < j : ∂ ˜ c [ i k ] ∂ ˜ c
Revised: [ i j ] ∀ i ≤ k < j : ∂ ˜ c [ i k ] ∂ ˜ c

Original: [ i j ] =
Revised: [ i j ] =

Original: b [ i j ][ k ] ∀ i ≤ k < j : ∂
Revised: b [ i j ][ k ] ∀ i ≤ k < j : ∂

Original: ˜ c
Revised: ˜ c

Original: [ i k ] ∂b [ i j ][ k ] =
Revised: [ i k ] ∂b [ i j ][ k ] =

Original: ˜ c
Revised: ˜ c

Original: [ i j ] ∀ i ≤ k < j : ∂ ˜ c
Revised: [ i j ] ∀ i ≤ k < j : ∂ ˜ c

Original: [ k +j ] ∂ ˜ c
Revised: [ k +j ] ∂ ˜ c

Original: [ i j ] =
Revised: [ i j ] =

Original: b [ i j ][ k ] ∀ i ≤ k < j : ∂ ˜ c
Revised: b [ i j ][ k ] ∀ i ≤ k < j : ∂ ˜ c

Original: [ k +j ] ∂b [ i j ][ k ] =
Revised: [ k +j ] ∂b [ i j ][ k ] =

Original: ˜ c
Revised: ˜ c

Original: [ i j ]BACKPTR ( for l = 1 . . .
Revised: [ i j ] for l = 1 . . .

Original: n do for i = 0 . . .
Revised: n do for i = 0 . . .

Original: n − l do j ← i + l ∂ L ∂
Revised: n − l do j ← i + l ∂ L ∂

Original: ˜ c
Revised: ˜ c

Original: [ i j ] ← 0 , ∂ L ∂ ˜ c [ i j ] ← 0 , ∂ L ∂ ˜ c
Revised: [ i j ] ← 0 , ∂ L ∂ ˜ c [ i j ] ← 0 , ∂ L ∂ ˜ c

Original: [ i j ] ← 0 , ∂ L ∂ ˜ c
Revised: [ i j ] ← 0 , ∂ L ∂ ˜ c

Original: [ i j ] ←for k = i . . .
Revised: [ i j ] ←for k = i . . .

Original: j − 1 do ∂ L ∂ ˜ c
Revised: j − 1 do ∂ L ∂ ˜ c

Original: [ i j ] + ← ∂ L ∂ ˜
Revised: [ i j ] + ← ∂ L ∂ ˜

Original: c [ i k ] b [ i j ][ k ] + ∂ L ∂ ˜ c
Revised: c [ i k ] b [ i j ][ k ] + ∂ L ∂ ˜ c

Original: [ k +1 j ] b [ i j ][ k ] ∂ L ∂b [ i j ][ k ] ← ∂ L ∂ ˜ c
Revised: [ k +1 j ] b [ i j ][ k ] ∂ L ∂b [ i j ][ k ] ← ∂ L ∂ ˜ c

Original: [ i k ] ˜ c
Revised: [ i k ] ˜ c

Original: [ i j ] + ∂ L ∂ ˜ c
Revised: [ i j ] + ∂ L ∂ ˜ c

Original: [ k +1 j ] ˜ c
Revised: [ k +1 j ] ˜ c

Original: [ i j ] for k = i . . .
Revised: [ i j ] for k = i . . .

Original: j − 1 do ∂ L ∂ ˜ c
Revised: j − 1 do ∂ L ∂ ˜ c

Original: [ i j ] + ← ∂ L ∂ ˜
Revised: [ i j ] + ← ∂ L ∂ ˜

Original: c [ i k ] b [ i j ][ k ] + ∂ L ∂ ˜ c
Revised: c [ i k ] b [ i j ][ k ] + ∂ L ∂ ˜ c

Original: [ k +1 j ] b [ i j ][ k ] ∂ L ∂b [ i j ][ k ] ← ∂ L ∂ ˜ c
Revised: [ k +1 j ] b [ i j ][ k ] ∂ L ∂b [ i j ][ k ] ← ∂ L ∂ ˜ c

Original: [ i k ] ˜ c
Revised: [ i k ] ˜ c

Original: [ i j ] + ∂ L ∂ ˜ c
Revised: [ i j ] + ∂ L ∂ ˜ c

Original: [ k +1 j ] ˜ c
Revised: [ k +1 j ] ˜ c

Original: [ i j ] for = i . . .
Revised: [ i j ] for = i . . .

Original: j − 1 do ∂ L ∂ ˜ c
Revised: j − 1 do ∂ L ∂ ˜ c

Original: [ i j ] + ← ∂ L ∂ ˜
Revised: [ i j ] + ← ∂ L ∂ ˜

Original: c [ i k ] b [ i j ][ k ] + ∂ L ∂ ˜ c
Revised: c [ i k ] b [ i j ][ k ] + ∂ L ∂ ˜ c

Original: [ k j ] b [ i j ][ k ] ∂ L ∂b [ i j ][ k ] ← ∂ L ∂ ˜ c
Revised: [ k j ] b [ i j ][ k ] ∂ L ∂b [ i j ][ k ] ← ∂ L ∂ ˜ c

Original: [ i k ] ˜ c
Revised: [ i k ] ˜ c

Original: [ i j ] + ∂ L ∂ ˜ c [ k j ] ˜ c
Revised: [ i j ] + ∂ L ∂ ˜ c [ k j ] ˜ c

Original: [ i j ] for = i + 1 . . .
Revised: [ i j ] for = i + 1 . . .

Original: j do ∂ L ∂ ˜ c
Revised: j do ∂ L ∂ ˜ c

Original: [ i j ] + ← ∂ L ∂ ˜
Revised: [ i j ] + ← ∂ L ∂ ˜

Original: c [ i k ] b [ i j ][ k ] + ∂ L ∂ ˜ c
Revised: c [ i k ] b [ i j ][ k ] + ∂ L ∂ ˜ c

Original: [ k j ] b [ i
Revised: [ k j ] b [ i

Original: [Table]
Revised: [Table]

Original: ∀ i ≤ k < j : ∂a
Revised: ∀ i ≤ k < j : ∂a

Original: [ i j ][ k ] ∂c
Revised: [ i j ][ k ] ∂c

Original: [ i k ] = ∀ i ≤ k <
Revised: [ i k ] = ∀ i ≤ k <

Original: j : ∂a
Revised: j : ∂a

Original: [ i j ][ k ] ∂c
Revised: [ i j ][ k ] ∂c

Original: [ k +j ] = ∀ i ≤ k <
Revised: [ k +j ] = ∀ i ≤ k <

Original: j : ∂a
Revised: j : ∂a

Original: [ i j ][ k ] ∂c
Revised: [ i j ][ k ] ∂c

Original: [ i k ] = ∀ i ≤ k <
Revised: [ i k ] = ∀ i ≤ k <

Original: j : ∂a
Revised: j : ∂a

Original: [ i j ][ k ] ∂c
Revised: [ i j ][ k ] ∂c

Original: [ k +j ] = ∀
Revised: [ k +j ] = ∀

Original: i < k ≤ j : ∂a
Revised: i < k ≤ j : ∂a

Original: [ i j ][ k ] ∂c
Revised: [ i j ][ k ] ∂c

Original: [ i k ] = ∀
Revised: [ i k ] = ∀

Original: i < k ≤ j : ∂a
Revised: i < k ≤ j : ∂a

Original: [ i j ][ k ] ∂c
Revised: [ i j ][ k ] ∂c

Original: [ k j ] = ∀ i ≤ k <
Revised: [ k j ] = ∀ i ≤ k <

Original: j : ∂a
Revised: j : ∂a

Original: [ i j ][ k ] ∂c
Revised: [ i j ][ k ] ∂c

Original: [ i k ] = ∀ i ≤ k <
Revised: [ i k ] = ∀ i ≤ k <

Original: j : ∂a
Revised: j : ∂a

Original: [ i j ][ k ] ∂c
Revised: [ i j ][ k ] ∂c

Original: [ k j ] = ∀ i ≤ k, k (cid:48) <
Revised: [ k j ] = ∀ i ≤ k, k (cid:48) <

Original: j : ∂b [ i j ][ k ] ∂a
Revised: j : ∂b [ i j ][ k ] ∂a

Original: [ i j ][ k (cid:48) ]
Revised: [ i j ][ k (cid:48) ]

Original: =
Revised: =

Original: b [ i j ][ k ]( 1 [ k = k (cid:48) ] − b [ i j ][ k (cid:48) ]) ∀ i ≤ k, k (cid:48) <
Revised: b [ i j ][ k ]( 1 [ k = k (cid:48) ] − b [ i j ][ k (cid:48) ]) ∀ i ≤ k, k (cid:48) <

Original: j : ∂b [ i j ][ k ] ∂a
Revised: j : ∂b [ i j ][ k ] ∂a

Original: [ i j ][ k (cid:48) ]
Revised: [ i j ][ k (cid:48) ]

Original: =
Revised: =

Original: b [ i j ][ k ]( 1 [ k = k (cid:48) ] − b [ i j ][ k (cid:48) ]) ∀ i < k ≤ j : ∂b [ i j ]
Revised: b [ i j ][ k ]( 1 [ k = k (cid:48) ] − b [ i j ][ k (cid:48) ]) ∀ i < k ≤ j : ∂b [ i j ]

Original: [ k ] ∂a
Revised: [ k ] ∂a

Original: [ i j ][ k (cid:48) ]
Revised: [ i j ][ k (cid:48) ]

Original: =
Revised: =

Original: b [ i j ][ k ]( 1 [ k = k (cid:48) ] − b [ i j ][ k (cid:48) ]) ∀ i ≤ k < j : ∂b [ i j ]
Revised: b [ i j ][ k ]( 1 [ k = k (cid:48) ] − b [ i j ][ k (cid:48) ]) ∀ i ≤ k < j : ∂b [ i j ]

Original: [ k ] ∂a
Revised: [ k ] ∂a

Original: [ i j ][ k (cid:48) ]
Revised: [ i j ][ k (cid:48) ]

Original: =
Revised: =

Original: b [ i j ][ k ]( 1 [ k = k (cid:48) ] − b [ i j ][ k (cid:48) ]) ∀ i ≤ k < j : ∂c
Revised: b [ i j ][ k ]( 1 [ k = k (cid:48) ] − b [ i j ][ k (cid:48) ]) ∀ i ≤ k < j : ∂c

Original: [ i j ] ∂b [ i j ][ k ] = a [ i j ][ k ] ∀ i ≤ k < j : ∂c
Revised: [ i j ] ∂b [ i j ][ k ] = a [ i j ][ k ] ∀ i ≤ k < j : ∂c

Original: [ i j ] ∂a [ i j ][ k ] =
Revised: [ i j ] ∂a [ i j ][ k ] =

Original: b [ i j ][ k ] ∀ i ≤ k < j : ∂c
Revised: b [ i j ][ k ] ∀ i ≤ k < j : ∂c

Original: [ i j ] ∂b [ i j ][ k ] = a [ i j ][ k ] ∀ i ≤ k < j : ∂c
Revised: [ i j ] ∂b [ i j ][ k ] = a [ i j ][ k ] ∀ i ≤ k < j : ∂c

Original: [ i j ] ∂a [ i j ][ k ] =
Revised: [ i j ] ∂a [ i j ][ k ] =

Original: b [ i j ][ k ] ∀ i < k ≤ j : ∂c
Revised: b [ i j ][ k ] ∀ i < k ≤ j : ∂c

Original: [ i j ] ∂b [ i j ][ k ] = a [ i j ][ k ] ∀ i < k ≤ j : ∂c
Revised: [ i j ] ∂b [ i j ][ k ] = a [ i j ][ k ] ∀ i < k ≤ j : ∂c

Original: [ i j ] ∂a [ i j ][ k ] =
Revised: [ i j ] ∂a [ i j ][ k ] =

Original: b [ i j ][ k ] ∀ i ≤ k < j : ∂c
Revised: b [ i j ][ k ] ∀ i ≤ k < j : ∂c

Original: [ i j ] ∂b [ i j ][ k ] = a [ i j ][ k ] ∀ i ≤ k < j : ∂c
Revised: [ i j ] ∂b [ i j ][ k ] = a [ i j ][ k ] ∀ i ≤ k < j : ∂c

Original: [ i j ] ∂a [ i j ][ k ] =
Revised: [ i j ] ∂a [ i j ][ k ] =

Original: b [ i j ][ k ] ∀ i <
Revised: b [ i j ][ k ] ∀ i <

Original: j : ∂c
Revised: j : ∂c

Original: [ i j ] ∂W i,j = ∀ i < j : ∂c
Revised: [ i j ] ∂W i,j = ∀ i < j : ∂c

Original: [ i j ] ∂W j,i = function B ACKPROP -I NSIDE ( n ) for i = 0 . . .
Revised: [ i j ] ∂W j,i = function B ACKPROP -I NSIDE ( n ) for i = 0 . . .

Original: n do for j = i . . .
Revised: n do for j = i . . .

Original: n do ∂ L ∂c
Revised: n do ∂ L ∂c

Original: [ i j ] ← 0 , ∂ L ∂c
Revised: [ i j ] ← 0 , ∂ L ∂c

Original: [ i j ] ← 0 , ∂ L ∂c [ i j ] ← 0 , ∂ L ∂c
Revised: [ i j ] ← 0 , ∂ L ∂c [ i j ] ← 0 , ∂ L ∂c

Original: [ i j ] ←for l = n . . .
Revised: [ i j ] ←for l = n . . .

Original: 1 do (cid:46) Backpropagation through the ”inside” algorithm for i = 0 . . .
Revised: 1 do (cid:46) Backpropagation through the ”inside” algorithm for i = 0 . . .

Original: n − l do j ← i + l for k = i . . .
Revised: n − l do j ← i + l for k = i . . .

Original: j − 1 do ∂ L ∂b [ i j ][ k ] ← ∂ L ∂c
Revised: j − 1 do ∂ L ∂b [ i j ][ k ] ← ∂ L ∂c

Original: [ i j ] a [ i j ][ k ] ∂ L ∂a [ i j ][ k ] ← ∂ L ∂c
Revised: [ i j ] a [ i j ][ k ] ∂ L ∂a [ i j ][ k ] ← ∂ L ∂c

Original: [ i j ] b [ i j ][ k ] s = (cid:80)
Revised: [ i j ] b [ i j ][ k ] s = (cid:80)

Original: k
Revised: k

Original: = i...
Revised: = i...

Original: j −∂ L ∂b [ i j ][ k ] b
Revised: j −∂ L ∂b [ i j ][ k ] b

Original: [ i j ][ k ] (cid:46) Backpropagate through the softmax function for k
Revised: [ i j ][ k ] (cid:46) Backpropagate through the softmax function for k

Original: = i . . .
Revised: = i . . .

Original: j − 1 do ∂ L ∂a
Revised: j − 1 do ∂ L ∂a

Original: [ i j ][ k ] ← b [ i j ][ k ] (cid:16)
Revised: [ i j ][ k ] ← b [ i j ][ k ] (cid:16)

Original: ∂ L ∂b [ i j ][ k ] − s (cid:17) for k
Revised: ∂ L ∂b [ i j ][ k ] − s (cid:17) for k

Original: = i . . .
Revised: = i . . .

Original: j − 1 do ∂ L ∂c
Revised: j − 1 do ∂ L ∂c

Original: [ i k ] + ← ∂ L ∂a
Revised: [ i k ] + ← ∂ L ∂a

Original: [ i j ][ k ] ∂ L ∂c [ k j ] + ← ∂ L ∂a
Revised: [ i j ][ k ] ∂ L ∂c [ k j ] + ← ∂ L ∂a

Original: [ i j ][ k ] for k
Revised: [ i j ][ k ] for k

Original: = i + 1 . . .
Revised: = i + 1 . . .

Original: j do ∂ L ∂b [ i j ][ k ] ← ∂ L ∂c [ i j ] a [ i j ][ k ] ∂
Revised: j do ∂ L ∂b [ i j ][ k ] ← ∂ L ∂c [ i j ] a [ i j ][ k ] ∂

Original: L a [ i j ][ k ] ← ∂ L ∂c
Revised: L a [ i j ][ k ] ← ∂ L ∂c

Original: [ i j ] ∂b [ i j ][ k ] s = (cid:80) k
Revised: [ i j ] ∂b [ i j ][ k ] s = (cid:80) k

Original: = i +1 ...
Revised: = i +1 ...

Original: j ∂ L ∂b [ i j ][ k ] b
Revised: j ∂ L ∂b [ i j ][ k ] b

Original: [ i j ][ k ] (cid:46) Backpropagate through the softmax function for k
Revised: [ i j ][ k ] (cid:46) Backpropagate through the softmax function for k

Original: = i + 1 . . .
Revised: = i + 1 . . .

Original: j do ∂ L ∂a
Revised: j do ∂ L ∂a

Original: [ i j ][ k ] ← b [ i j ][ k ] (cid:16)
Revised: [ i j ][ k ] ← b [ i j ][ k ] (cid:16)

Original: ∂ L ∂b [ i j ][ k ] − s (cid:17) for k
Revised: ∂ L ∂b [ i j ][ k ] − s (cid:17) for k

Original: = i + 1 . . .
Revised: = i + 1 . . .

Original: j do ∂ L ∂c
Revised: j do ∂ L ∂c

Original: [ i k ] + ← ∂ L ∂a
Revised: [ i k ] + ← ∂ L ∂a

Original: [ i j ][ k ] ∂ L ∂c [ k j ] + ← ∂ L ∂a
Revised: [ i j ][ k ] ∂ L ∂c [ k j ] + ← ∂ L ∂a

Original: [ i j ][ k ] for k = i . . .
Revised: [ i j ][ k ] for k = i . . .

Original: j − 1 do ∂ L ∂b [ i j ][ k ] ← ∂ L ∂c
Revised: j − 1 do ∂ L ∂b [ i j ][ k ] ← ∂ L ∂c

Original: [ i j ] a [ i j ][ k ] ∂ L ∂a [ i j ][ k ] ← ∂ L ∂c
Revised: [ i j ] a [ i j ][ k ] ∂ L ∂a [ i j ][ k ] ← ∂ L ∂c

Original: [ i j ] b [ i j ][ k ] s = (cid:80)
Revised: [ i j ] b [ i j ][ k ] s = (cid:80)

Original: k
Revised: k

Original: = i...
Revised: = i...

Original: j −∂ L ∂b [ i j ][ k ] b
Revised: j −∂ L ∂b [ i j ][ k ] b

Original: [ i j ][ k ] (cid:46) Backpropagate through the softmax function for k
Revised: [ i j ][ k ] (cid:46) Backpropagate through the softmax function for k

Original: = i . . .
Revised: = i . . .

Original: j − 1 do ∂ L ∂a
Revised: j − 1 do ∂ L ∂a

Original: [ i j ][ k ] ← b [ i j ][ k ] (cid:16)
Revised: [ i j ][ k ] ← b [ i j ][ k ] (cid:16)

Original: ∂ L ∂b [ i j ][ k ] − s (cid:17) for k
Revised: ∂ L ∂b [ i j ][ k ] − s (cid:17) for k

Original: = i . . .
Revised: = i . . .

Original: j − 1 do ∂ L ∂c
Revised: j − 1 do ∂ L ∂c

Original: [ i k ] + ← ∂ L ∂a
Revised: [ i k ] + ← ∂ L ∂a

Original: [ i j ][ k ] ∂ L ∂c
Revised: [ i j ][ k ] ∂ L ∂c

Original: [ k +1 j ] + ← ∂ L ∂a
Revised: [ k +1 j ] + ← ∂ L ∂a

Original: [ i j ][ k ] for k = i . . .
Revised: [ i j ][ k ] for k = i . . .

Original: j − 1 do ∂ L ∂b [ i j ][ k ] ← ∂ L ∂c
Revised: j − 1 do ∂ L ∂b [ i j ][ k ] ← ∂ L ∂c

Original: [ i j ] a [ i j ][ k ] ∂ L ∂a [ i j ][ k ] ← ∂ L ∂c
Revised: [ i j ] a [ i j ][ k ] ∂ L ∂a [ i j ][ k ] ← ∂ L ∂c

Original: [ i j ] b [ i j ][ k ] s = (cid:80)
Revised: [ i j ] b [ i j ][ k ] s = (cid:80)

Original: k
Revised: k

Original: = i...
Revised: = i...

Original: j −∂ L ∂b [ i j ][ k ] b
Revised: j −∂ L ∂b [ i j ][ k ] b

Original: [ i j ][ k ] (cid:46) Backpropagate through the softmax function for k
Revised: [ i j ][ k ] (cid:46) Backpropagate through the softmax function for k

Original: = i . . .
Revised: = i . . .

Original: j − 1 do ∂ L ∂a
Revised: j − 1 do ∂ L ∂a

Original: [ i j ][ k ] ← b [ i j ][ k ] (cid:16)
Revised: [ i j ][ k ] ← b [ i j ][ k ] (cid:16)

Original: ∂ L ∂b [ i j ][ k ] − s (cid:17) for k
Revised: ∂ L ∂b [ i j ][ k ] − s (cid:17) for k

Original: = i . . .
Revised: = i . . .

Original: j − 1 do ∂ L ∂c
Revised: j − 1 do ∂ L ∂c

Original: [ i k ] + ← ∂ L ∂a
Revised: [ i k ] + ← ∂ L ∂a

Original: [ i j ][ k ] ∂ L ∂c
Revised: [ i j ][ k ] ∂ L ∂c

Original: [ k +1 j ] + ← ∂ L ∂a
Revised: [ k +1 j ] + ← ∂ L ∂a

Original: [ i j ][ k ]
Revised: [ i j ][ k ]