Original: E NABLING A RBITRARY T RANSLATION O BJECTIVES WITH A DAPTIVE T REE S EARCH
Revised: E NABLING A RBITRARY T RANSLATION O BJECTIVES WITH A DAPTIVE T REE S EARCH

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We introduce an adaptive tree search algorithm that can ﬁnd high-scoring outputs under translation models that make no assumptions about the form or structure of the search objective.
Revised: We introduce an adaptive tree search algorithm, which is a deterministic variant of Monte Carlo tree search, that can ﬁnd high-scoring outputs under translation models that make no assumptions about the form or structure of the search objective.

Original: This algorithm enables the exploration of new kinds of models that are unencumbered by constraints imposed to make decoding tractable, such as autoregressivity or conditional independence assumptions.
Revised: This algorithm enables the exploration of new kinds of models that are unencumbered by constraints imposed to make decoding tractable, such as autoregressivity or conditional independence assumptions.

Original: When applied to autoregressive models, our algorithm has different biases than beam search has, which enables a new analysis of the role of decoding bias in autoregressive models.
Revised: When applied to autoregressive models, our algorithm has different biases than beam search has, which enables a new analysis of the role of decoding bias in autoregressive models.

Original: Empirically, we show that our adaptive tree search algorithm ﬁnds outputs with substantially better model scores compared to beam search in autoregressive models, and compared to reranking techniques in non-autoregressive models.
Revised: Empirically, we show that our adaptive tree search algorithm ﬁnds outputs with substantially better model scores compared to beam search in autoregressive models and compared to reranking techniques in models whose scores do not decompose additively with respect to the words in the output.

Original: We also characterise the correlation of several translation model objectives with respect to BLEU.
Revised: We also characterise the correlation of several translation model objectives with respect to BLEU.

Original: We ﬁnd that while some standard models are poorly calibrated and beneﬁt from the beam search bias, other often more robust models (autoregressive models tuned to maximize expected automatic metric scores, the noisy channel model and a newly proposed objective) beneﬁt from increasing amounts  search using our proposed decoder, whereas the beam search bias limits the improvements obtained from such objectives.
Revised: We ﬁnd that while some standard models are poorly calibrated and beneﬁt from the beam search bias other often more robust models (autoregressive models tuned to maximize expected automatic metric scores, the noisy channel model and a newly proposed objective) beneﬁt from increasing amounts of search using our proposed decoder, whereas the beam search bias limits the improvements obtained from such objectives.

Original: Thus, we argue that as models improve, the improvements may be masked by over-reliance on beam search or reranking based methods
Revised: Thus, we argue that as models improve, the improvements may be masked by over-reliance on beam search or reranking based methods.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Conditional text generation tasks, such as machine translation, consist of two parts: a model that assigns scores to candidate outputs, and a search component that interacts with the model in order to ﬁnd an output that maximizes the score assigned by the model.
Revised: Conditional text generation tasks, such as machine translation, consist of two parts: a model that assigns scores to candidate outputs, and a search component that interacts with the model in order to ﬁnd an output that maximizes the score assigned by the model.

Original: This search problem is a hard combinatorial optimization problem, and as a result, constraints are frequently imposed on the structure of the model to make solving or approximating the search problem easier.
Revised: This search problem is a hard combinatorial optimization problem, and as a result, constraints are frequently imposed on the structure of the model to make solving or approximating the search problem easier.

Original: In neural machine translation, an autoregressive factorization of the output probability distribution is widely used (Kalchbrenner & Blunsom, 2013;
Revised: In neural machine translation, an autoregressive factorization of the output probability distribution is widely used (Kalchbrenner & Blunsom, 2013;

Original: Sutskever et al., 2014; Vaswani et al., 2017), and a variety of conditional independence assumptions are made in other model classes from statistical translation models (Brown et al., 1993; Koehn et al., 2003) to non-autoregressive neural models (Lee et al., 2018).
Revised: Sutskever et al., 2014; Vaswani et al., 2017), and a variety of conditional independence assumptions are made in other model classes from statistical translation models (Brown et al., 1993; Koehn et al., 2003) to non-autoregressive neural models (Lee et al., 2018).

Original: Although these assumptions enable fast and accurate approximations to the search problem with simple and efﬁcient algorithms (e.g., beam search), which can be crucial for efﬁcient production applications, they limit the form of the models and thereby restricting the kinds of architectures that can be used to address observed model failures.
Revised: Although these assumptions enable fast and accurate approximations to the search problem with simple and efﬁcient algorithms (e.g., beam search), which can be crucial for efﬁcient production applications, they limit the form of the models and thereby restricting the kinds of architectures that can be used to address observed model failures.

Original: Despite the algorithmic beneﬁts that beam search provides, we argue that it is a poor foundation for long-term scientiﬁc progress toward an accurate and reliable translation model whose scores adequately predict translation quality.
Revised: Despite the algorithmic beneﬁts that beam search provides, we argue that it is a poor foundation for long-term scientiﬁc progress toward an accurate and reliable translation model whose scores adequately predict translation quality.

Original: First, it can only be applied to autoregressive models, which have well-known length calibration problems due to a tendency to drop or “hallucinate” content, and this tendency has been remarkably resistant to remedy across a variety of different autoregressive architectures (Koehn & Knowles, 2017; Lin et al., 2020).
Revised: First, it can only be applied to autoregressive models, which have well-known length calibration problems due to a tendency to drop or “hallucinate” content, and this tendency has been remarkably resistant to remedy across a variety of different autoregressive architectures (Koehn & Knowles, 2017; Lin et al., 2020).

Original: The existing heuristic solutions—e.g., nonlocal corrections to the search objective at decoding time, or global statistics about the population of per-word probabilities (Wu et al., 2016; Meister et al., 2020)—point to non-autoregressive components in the search objective as necessary parts of the solution.
Revised: The existing heuristic solutions—e.g., nonlocal corrections to the search objective at decoding time, or global statistics about the population of per-word probabilities (Wu et al., 2016; Meister et al., 2020)—point to non-autoregressive components in the search objective as necessary parts of the solution.

Original:  We therefore would like to directly work with models classes that contain these solutions, rather  being dependent on limited heuristics that are imposed after the fact.
Revised: 1 We therefore would like to directly work with model classes that contain these solutions, rather than being dependent on limited heuristics that are imposed after the fact.

Original: Second, beam search is strongly biased towards translations that yield high initial scores due to the heuristic used to score partial translations (Stahlberg & Byrne, 2019;
Revised: Second, beam search is strongly biased towards translations that yield high initial scores due to the heuristic used to score partial translations (Stahlberg & Byrne, 2019;

Original: Meister et al., 2020).
Revised: Meister et al., 2020).

Original: While this feature provides short term translation quality gains as it addresses many of the calibration issues in autoregressive models, it is undesirable in the longer term as it masks many modeling issues that need addressing.
Revised: While this feature provides short term translation quality gains as it addresses many of the calibration issues in autoregressive models, it is undesirable in the longer term as it masks many modeling issues that need addressing.

Original: More importantly, as scientiﬁc progress drives better correlations between model scores and translation quality, search errors caused by this bias will inevitably have a negative impact on both model scores and translation quality.
Revised: More importantly, as scientiﬁc progress drives better correlations between model scores and translation quality, search errors caused by this bias will inevitably have a negative impact on both model scores and translation quality.

Original: To address these shortcomings, we introduce the beam adaptive tree search (BATS) algorithm, which is based on Monte Carlo tree search (Coulom, 2006;
Revised: To address these shortcomings, we introduce the beam adaptive tree search (BATS) algorithm, which is based on Monte Carlo tree search (Coulom, 2006;

Original: Browne et al., 2012).
Revised: Browne et al., 2012).

Original: Like MCTS, BATS estimates the value of internal nodes (i.e., partial translations) in the search tree with estimates from an expected-outcome model based on playouts from an auxiliary model, and these estimates are reﬁned as the search progresses.
Revised: Like MCTS, BATS estimates the value of internal nodes (i.e., partial translations) in the search tree with estimates from an expected-outcome model based on playouts from an auxiliary model, and these estimates are reﬁned as the search progresses.

Original: Because BATS is guided from the start by the true objective—whether autoregressive or not, and due to the reﬁnement of initial score estimates, the BATS decoder exhibits fewer biases as the search budget increases than beam search or reranking algorithms.
Revised: Because BATS is guided from the start by the true objective—whether autoregressive or not, and due to the reﬁnement of initial score estimates, the BATS decoder exhibits fewer biases as the search budget increases than beam search or reranking algorithms.

Original: Our experimental section aims to characterise the impact of decoding mechanisms on both nonautoregressive and autoregressive models.
Revised: Our experimental section aims to characterise the impact of decoding mechanisms on both nonautoregressive and autoregressive models.

Original: For autoregressive models, we show that the calibration issues in autoregressive models can be addressed by adding a non-autoregressive component that augments the autoregressive sequence score with the lowest scoring produced token in the autoregressive decomposition of the sequence, which we name max rank .
Revised: For autoregressive models, we show that the calibration issues in autoregressive models can be addressed by adding a non-autoregressive component that augments the autoregressive sequence score with the lowest scoring produced token in the autoregressive decomposition of the sequence, which we name max rank .

Original: We further show that existing non-autoregressive approaches, such as the “noisy channel” model (Yu et al., 2017; Yee et al., 2019;
Revised: We further show that existing non-autoregressive approaches, such as the “noisy channel” model (Yu et al., 2017; Yee et al., 2019;

Original: Ng et al., 2019; Yu et al., 2020b;a; Liu et al., 2021), which factorize the translation probability according to Bayes’ rule and have been argued to be better calibrated, are (1) poorly optimized by standard reranking-based approaches, and (2) ultimately have similar calibration failures as neural autoregressive models have.
Revised: Ng et al., 2019; Yu et al., 2020b;a; Liu et al., 2021), which factorize the translation probability according to Bayes’ rule and have been argued to be better calibrated, are (1) poorly optimized by standard reranking-based approaches, and (2) ultimately have similar calibration failures as neural autoregressive models have.

Original: Incorporating the max rank component to existing objectives beneﬁts both beam search and BATS, but the latter yields both the best translation and model scores.
Revised: Incorporating the max rank component to existing objectives beneﬁts both beam search and BATS, but the latter yields both the best translation and model scores.

Original: Crucially, we show that our decoder can search substantially longer and achieve higher model scores before
Revised: Crucially, we show that our decoder can search substantially longer and achieve higher model scores before

Original: BLEU starts to deteriorate, which suggests a negative impact of the search bias in beam search.
Revised: BLEU starts to deteriorate, which suggests a negative impact of the search bias in beam search.

Original: Finally, we show that once autoregressive models become robust enough to address the calibration issues, this bias has an equally negative impact on translation quality.
Revised: Finally, we show that once autoregressive models become robust enough to address the calibration issues, this bias has an equally negative impact on translation quality.

Original: By ﬁne-tuning an autoregressive model to better correlate with BLEU using minimum risk training (Shen et al., 2016), we show that
Revised: By ﬁne-tuning an autoregressive model to better correlate with BLEU using minimum risk training (Shen et al., 2016), we show that

Original: BATS can achieve higher translation quality and a better model score compared to beam search.
Revised: BATS can achieve higher translation quality and a better model score compared to beam search.

Original:  B ACKGROUND 2. B EAM S EARCH
Revised:  B ACKGROUND 2. B EAM S EARCH

Original: Translation tasks operate on the space of possible translations Y
Revised: Translation tasks operate on the space of possible translations Y

Original: = Σ ∗ ◦ { EOS } where Σ is a ﬁnite vocabulary, and EOS is a symbol represents the end of a sequence.
Revised: = Σ ∗ ◦ { EOS } where Σ is a ﬁnite vocabulary, and EOS is a symbol represents the end of a sequence.

Original: Elements in Y correspond to full sentences y = y 1 , . .
Revised: Elements in Y correspond to full sentences y = y 1 , . .

Original: , y n with n − 1 tokens, where y n = EOS (end of sentence).
Revised: , y n with n − 1 tokens, where y n = EOS (end of sentence).

Original: Each translation y is conditioned on a source sentence x and is assigned a score s ( x , y ) , which measures how well x translates to y .
Revised: Each translation y is conditioned on a source sentence x and is assigned a score s ( x , y ) , which measures how well x translates to y .

Original: Decoding algorithms aim to ﬁnd the best hypothesis y ∗ ∈ Y under the search objective s ( x , y ) .
Revised: Decoding algorithms aim to ﬁnd the best hypothesis y ∗ ∈ Y under the search objective s ( x , y ) .

Original: While autoregressive models may assign scores to preﬁxes of translations, we only assume that s is well deﬁned for full translations, y ∈ Y .
Revised: While autoregressive models may assign scores to preﬁxes of translations, we only assume that s is well deﬁned for full translations, y ∈ Y .

Original: In the particular case of machine translation under standard autoregressive models, the search objective is deﬁned as the conditional log probability:
Revised: In the particular case of machine translation under standard autoregressive models, the search objective is deﬁned as the conditional log probability:

Original: [Equation]
Revised: [Equation]

Original: As autoregressive models have probability emissions at the token level, the search problem can be cast into a shortest path problem for weighted graphs, where each node p is identiﬁed by a sequence y ( p ) .
Revised: As autoregressive models have probability emissions at the token level, the search problem can be cast into a shortest path problem for weighted graphs, where each node p is identiﬁed by a sequence y ( p ) .

Original: Nodes deﬁne an additional space Y ∗ including Y as well as partial translations in Σ ∗ that do not terminate with a EOS token.
Revised: Nodes deﬁne an additional space Y ∗ including Y as well as partial translations in Σ ∗ that do not terminate with a EOS token.

Original: Each node contains | Σ | + 1 edges, each appending a different word
Revised: Each node contains | Σ | + 1 edges, each appending a different word

Original: y i ∈ Σ ∪ { EOS } to y ( p ) generating a new node p ◦ y i .
Revised: y i ∈ Σ ∪ { EOS } to y ( p ) generating a new node p ◦ y i .

Original: Edge weights are given by the emission log probability log p ( y i | y ( p ) , x ) according to the autoregressive model.
Revised: Edge weights are given by the emission log probability log p ( y i | y ( p ) , x ) according to the autoregressive model.

Original: Search starts from the root node ε with an empty sequence and nodes that generate the EOS token have a single edged with weight 0 that lead to a single shared terminal node.
Revised: Search starts from the root node ε with an empty sequence and nodes that generate the EOS token have a single edged with weight 0 that lead to a single shared terminal node.

Original: One solution to this problem is A ∗ search (Hart et al., 1968), which is a best-ﬁrst algorithm that iteratively searches nodes p with the highest value ˆ v ( p ) =
Revised: One solution to this problem is A ∗ search (Hart et al., 1968), which is a best-ﬁrst algorithm that iteratively searches nodes p with the highest value ˆ v ( p ) =

Original: c ( p ) + f ( p ) , where c ( p ) is the sum of weights of all edges from ε to p , and f ( p ) is a heuristic function that attempts to estimate the sum of the edges on the highest scoring path from p to the terminal node.
Revised: c ( p ) + f ( p ) , where c ( p ) is the sum of weights of all edges from ε to p , and f ( p ) is a heuristic function that attempts to estimate the sum of the edges on the highest scoring path from p to the terminal node.

Original: Although theoretically appealing, good
Revised: Although theoretically appealing, good

Original: A ∗ heuristics are difﬁcult to obtain, and search can be extremely expensive with poor heuristics.
Revised: A ∗ heuristics are difﬁcult to obtain, and search can be extremely expensive with poor heuristics.

Original: To remedy this, beam search introduces approximations.
Revised: To remedy this, beam search introduces approximations.

Original: Rather than a best-ﬁrst traversal, beam search proceeds iteratively along a frontier at a certain depth.
Revised: Rather than a best-ﬁrst traversal, beam search proceeds iteratively along a frontier at a certain depth.

Original: Exploration is limited to b nodes at each depth, where b is selected as a hyperparameter that trades search accuracy (in terms of model score) for speed.
Revised: Exploration is limited to b nodes at each depth, where b is selected as a hyperparameter that trades search accuracy (in terms of model score) for speed.

Original: From depth 0 composed of just root node ε until a maximum depth limit Y is reached, or another stopping criteria is reached (Klein et al., 2017), beam search progressively scores all children of the nodes at the current depth and prunes all generated nodes except for the top scoring b nodes.
Revised: From depth 0 composed of just root node ε until a maximum depth limit Y is reached, or another stopping criteria is reached (Klein et al., 2017), beam search progressively scores all children of the nodes at the current depth and prunes all generated nodes except for the top scoring b nodes.

Original: Beam search scores each node with only the current cost ˆ v ( p ) =
Revised: Beam search scores each node with only the current cost ˆ v ( p ) =

Original: c ( p ) , and, in the case of neural machine translation, setting f ( p ) = 0 .
Revised: c ( p ) , and, in the case of neural machine translation, setting f ( p ) = 0 .

Original: By discarding the future cost biases search towards nodes with high scores without regard to whether they lead to a good path to the terminal state.
Revised: Discarding the future cost biases search towards nodes with high scores without regard to whether they lead to a good path to the terminal state.

Original: Thus, a large space of potentially good translations with low initial scores is never explored.
Revised: Thus, a large space of potentially good translations with low initial scores is never explored.

Original: Examples include re-orderings that place high-entropy words at the start of the sentence or shorter sentence constructions (e.g. “Help me” vs. “Lend me a hand”).
Revised: Examples include re-orderings that place high-entropy words at the start of the sentence or shorter sentence constructions (e.g. “Help me” vs. “Lend me a hand”).

Original: Interestingly, autoregressive models tend to overestimate the probability of short and ungrammatical translations that do not translate the entirety of the source sentence, which are pruned by this scoring heuristic (Stahlberg & Byrne, 2019; Holtzman et al., 2019).
Revised: Interestingly, autoregressive models tend to overestimate the probability of short and ungrammatical translations that do not translate the entirety of the source sentence, which are pruned by this scoring heuristic (Stahlberg & Byrne, 2019; Holtzman et al., 2019).

Original: Thus, while b may be set low to increase speed, it is often set low to improve translation quality .
Revised: Thus, while b may be set low to increase speed, it is often set low to improve translation quality .

Original: However, we believe that model changes evaluated with beam search’s biases may be obscured.
Revised: However, we believe that model changes evaluated with beam search’s biases may be obscured.

Original: We seek to proposing modeling improvements to mitigate degenerate solutions, such that search quality and model quality are aligned.
Revised: We seek to propose modeling improvements to mitigate degenerate solutions, such that search quality and model quality are aligned.

Original: 2. M ONTE C ARLO T REE S EARCH
Revised: 2. M ONTE C ARLO T REE S EARCH

Original: The Monte Carlo method replaces the heuristically driven measure of value ˆ v ( p ) for a node p with an expected-outcome model based on random game playouts.
Revised: In Monte Carlo tree search (Coulom, 2006, MCTS), the Monte Carlo method replaces the heuristically driven measure of value ˆ v ( p ) for a node p with an expected-outcome model based on random game playouts.

Original: For instance, given a node p representing a state of a game, one can assign a value to p by randomly playing from that state a certain number of times and compute the average score obtained from the playouts.
Revised: For instance, given a node p representing a state of a game, one can assign a value to p by randomly playing from that state a certain number of times and computing the average score obtained from the playouts.

Original: Thus, no burden is placed on the form of the objective function, allowing the deﬁnition  arbitrary complex objectives (e.g. winner of a chess game).
Revised: Thus, no burden is placed on the form of the objective function, allowing the deﬁnition of arbitrary complex objectives (e.g. winner of a chess game).

Original: Additionally, as each playout yields a possible terminal state, both current c ( p ) and future costs f ( p ) are naturally embedded within the obtained estimate.
Revised: Additionally, as each playout yields a possible terminal state, both current c ( p ) and future costs f ( p ) are naturally embedded within the obtained estimate.

Original: Monte-Carlo tree search (Coulom, 2006, MCTS) is an application of the Monte Carlo method for tree search in games.

Original: Unlike beam search and A ∗ search, where the search direction is determined by value ˆ v ( p ) , MCTS acknowledges the need to diversify the search space by allocating budget to less explored areas in the search space (Kocsis & Szepesv´ari, 2006). Another unique feature of MCTS is the continuous reﬁnement of the initial estimates obtained from playouts as search progresses.
Revised: Unlike beam search and A ∗ search, where the search direction is determined by value ˆ v ( p ) , MCTS diversiﬁes the search space by allocating budget to less explored areas in the search space (Kocsis & Szepesv´ari, 2006), and continually reﬁnes value estimates.

Original:  A DAPTIVE T REE S EARCH F OR T EXT G ENERATION
Revised:  A DAPTIVE T REE S EARCH F OR T EXT G ENERATION

Original: Whilethe MCTS (Coulom, 2006) can be directly applied  decodeof arbitrary translation objectives, the heuristics deﬁned in MCTS are optimised for environments where the computational cost of the scoring function s ( x , y ) is low.
Revised: While MCTS (Coulom, 2006) can be directly applied to decode arbitrary translation objectives, the heuristics deﬁned in MCTS are optimised for environments where the computational cost of the scoring function s ( x , y ) is low.

Original: For instance, the playout heuristic ˆ v in the game of go (Silver et al., 2016) runs hundreds of thousands of playouts, which can be computed in less than a second.
Revised: For instance, the playout heuristic ˆ v in the game of go (Silver et al., 2016) runs hundreds of thousands of playouts, which can be computed in less than a second.

Original: In neural text generation the scoring function frequently requires the computation of a neural network forward step, such as a log-probability computed using an autoregressive model, rendering these practices prohibitive.
Revised: In neural text generation the scoring function frequently requires the computation of a neural network forward step, such as a log-probability computed using an autoregressive model, rendering these practices prohibitive.

Original: One option is to rely on a heuristic to generate samples to train a neural network that estimates ˆ v (Leblond et al., 2021).
Revised: One option is to rely on a heuristic to generate samples to train a neural network that estimates ˆ v (Leblond et al., 2021).

Original: However this has been shown to be challenging as model scores are difﬁcult to estimate.
Revised: However this has been shown to be challenging as model scores are difﬁcult to estimate.

Original: Instead, we describe a variant of MCTS optimised  decoding text generation.
Revised: Instead, we describe a variant of MCTS optimised on decoding text generation.

Original: 3. D ETERMINISTIC P LAYOUT H EURISTIC
Revised: 3. D ETERMINISTIC P LAYOUT H EURISTIC

Original: We start by establishing our playout function ˆ v ( p ) , which is used as an initializer for node values.
Revised: We start by establishing our playout function ˆ v ( p ) , which is used as an initializer for node values.

Original: While the Monte Carlo method is effective at accurately estimating the value of a given sequence by performing multiple random playouts, such practice is infeasible as each of the playouts needs to be scored using the scoring function s ( x , y ) , which we expect will be prohibitively expensive.
Revised: While the Monte Carlo method is effective at accurately estimating the value of a given sequence by performing multiple random playouts, such practice is infeasible as each of the playouts needs to be scored using the scoring function s ( x , y ) , which we expect will be prohibitively expensive.

Original: Furthermore, chances that grammatical translations are sampled using this process are extremely low due to the sparsity of high-quality translations in Y .
Revised: Furthermore, chances that grammatical translations are sampled using this process are extremely low due to the sparsity of high-quality translations in Y .

Original: Thus, rather than multiple random playouts, we estimate the cost of a given node using a single informed playout, which is guided by greedy decoding using an autoregressive model.
Revised: Thus, rather than multiple random playouts, we estimate the cost of a given node using a single informed playout, which is guided by greedy decoding using an autoregressive model.

Original: Therefore, given a node p with preﬁx y ( p ) , we compute ˆ v ( p ) by recursively selecting the highest probability token y according to an autoregressive model p ( y | y
Revised: Therefore, given a node p with preﬁx y ( p ) , we compute ˆ v ( p ) by recursively selecting the highest probability token y according to an autoregressive model p ( y | y

Original: ( p ) 1 , . . .
Revised: ( p ) 1 , . . .

Original: , y ( p ) i , x ) , and scoring the translation using the objective function s ( x , y ) .
Revised: , y ( p ) i , x ) , and scoring the translation using the objective function s ( x , y ) .

Original: This also implies our approach does not employ the Monte Carlo estimates and the decoding method is fully deterministic.
Revised: This also implies our approach does not employ the Monte Carlo estimates and the decoding method is fully deterministic.

Original: The progression of the value estimates relies only on the reﬁnement of the initial value estimates performed as the tree expands introduced in MCTS.
Revised: The progression of the value estimates relies only on the reﬁnement of the initial value estimates performed as the tree expands introduced in MCTS.

Original: Therefore, we will refer to our algorithm as an adaptive tree search (ATS) algorithm.
Revised: Therefore, we will refer to our algorithm as an adaptive tree search (ATS) algorithm.

Original: 3. A DAPTIVE T REE S EARCH WITH A MODIFIED UCT C RITERIA
Revised: 3. A DAPTIVE T REE S EARCH WITH A MODIFIED UCT C RITERIA

Original: ATS operates on search trees instead of weighted graphs.
Revised: ATS operates on search trees instead of weighted graphs.

Original: A search tree covers the space of all possible full and partial translations Y ∗ , and each node encodes a particular sequence y ( p ) .
Revised: A search tree covers the space of all possible full and partial translations Y ∗ , and each node encodes a particular sequence y ( p ) .

Original: Nodes have | Σ | +1 children, each appending a new word y ∈ Σ ∪ { EOS } to the sequence y ( p ) .
Revised: Nodes have | Σ | + children, each appending a new word y ∈ Σ ∪ { EOS } to the sequence y ( p ) .

Original: We denote the child resulting from concatenating y to p as p ◦ y .
Revised: We denote the child resulting from concatenating y to p as p ◦ y .

Original: The child of a node that selects the EOS symbol is a terminal node, which is associated with a element in Σ ∗ , and therefore, can be scored using the objective s ( x , y ) .
Revised: The child of a node that selects the EOS symbol is a terminal node, which is associated with a element in Σ ∗ , and therefore, can be scored using the objective s ( x , y ) .

Original: Each node stores the number of visits n ( p ) and its current value estimate v ( p ) , which can be reassigned during search.
Revised: Each node stores the number of visits n ( p ) and its current value estimate v ( p ) , which can be reassigned during search.

Original: Nodes thatare have not been inserted in the tree have n ( p ) =0 and no estimate for v ( p ) .
Revised: Nodes that have not been inserted in the tree have n ( p ) = and no estimate for v ( p ) .

Original: Search starts with the root node ε with visit count n ( ε ) = 1 and value v ( ε )
Revised: Search starts with the root node ε with visit count n ( ε ) = 1 and value v ( ε )

Original: =
Revised: =

Original: ˆ v ( ε ) , which corresponds to the score obtained by translating x with greedy decoding.
Revised: ˆ v ( ε ) , which corresponds to the score obtained by translating x with greedy decoding.

Original: Afterwards the tree expands in a iterative manner, where each iteration expands the search tree and updates its statistics.
Revised: Afterwards the tree expands in an iterative manner, where each iteration expands the search tree and updates its statistics.

Original: Similar to the selection and expansion steps in ATS, we traverse the instantiated tree, starting from ε on the basis of the current estimated values v , together with conﬁdence about the quality of the estimates.
Revised: Similar to the selection and expansion steps in ATS, we traverse the instantiated tree, starting from ε on the basis of the current estimated values v , together with conﬁdence about the quality of the estimates.

Original: We recursively traverse the tree and select the child p ◦ y with the highest score according to the continuous upper conﬁdence tree criterion (Auger et al., 2013):
Revised: We recursively traverse the tree and select the child p ◦ y with the highest score according to the continuous upper conﬁdence tree criterion (Auger et al., 2013):

Original: [Equation]
Revised: [Equation]

Original: where C is a hyperparameter weighting two terms.
Revised: where C is a hyperparameter weighting two terms.

Original: The ﬁrst term ¯ v ( p ◦ y ) encourages exploitation of nodes with known high value.
Revised: The ﬁrst term ¯ v ( p ◦ y ) encourages exploitation of nodes with known high value.

Original: The second term √ n ( p ) 1+ n ( p ◦ y ) π ( y | y ( p ) ) encourages the algorithm to explore nodes with low visit counts more thoroughly.
Revised: The second term √ n ( p ) 1+ n ( p ◦ y ) π ( y | y ( p ) ) encourages the algorithm to explore nodes with low visit counts more thoroughly.

Original: Here, we specify the policy as the logprobability obtained from an autoregressive model π
Revised: Here, we specify the policy as the logprobability obtained from an autoregressive model π

Original: ( y | y ( p ) )
Revised: ( y | y ( p ) )

Original: =
Revised: =

Original: p ( y | y ( p ) , x ) .
Revised: p ( y | y ( p ) , x ) .

Original: The value of a node is determined by its current estimate ¯ v ( p ◦ y )
Revised: The value of a node is determined by its current estimate ¯ v ( p ◦ y )

Original: =
Revised: =

Original: v ( p ◦ y ) if n ( p ◦ y ) > 0 .
Revised: v ( p ◦ y ) if n ( p ◦ y ) > 0 .

Original: For nodes not yet inserted in the tree and have no value estimates, we compute an estimated value as:
Revised: For nodes not yet inserted in the tree, which have no value estimates, we compute an estimated value as:

Original: [Equation]
Revised: [Equation]

Original: where we estimate ¯ v ( p ◦ y ) by assuming that the ratio between policies π betweenthe p ◦ y and the highest value node p ◦ y ∗ is the same as that ratio between their values ¯ v .
Revised: where we estimate ¯ v ( p ◦ y ) by assuming that the ratio between policies π between p ◦ y and the highest value node p ◦ y ∗ is the same as the ratio between their values ¯ v .

Original: The traversal terminates when a node with n ( p ) = 0 or a terminal node is reached.
Revised: The traversal terminates when a node with n ( p ) = 0 or a terminal node is reached.

Original: In the former case, the node p is inserted into the tree, setting its visit count n ( p ) = 1 and estimating its value v ( p ) =
Revised: In the former case, the node p is inserted into the tree, setting its visit count n ( p ) = 1 and estimating its value v ( p ) =

Original: ˆ v ( p ) as done in the simulation step in MCTS.
Revised: ˆ v ( p ) as done in the simulation step in MCTS.

Original: We note here an important difference to many other formulations of MCTS, where selection terminates at leaf nodes (node where all Σ have n ( p ) > 0 ), which is followed by the expansion step that inserts a new child prior to simulation.
Revised: We note here an important difference to many other formulations of MCTS, where selection terminates at leaf nodes (node where all Σ have n ( p ) > 0 ), which is followed by the expansion step that inserts a new child prior to simulation.

Original: Expanding all children of a node is generally considered efﬁcient in domains with a small Σ and low playout cost ˆ v ( p ) , and the standard MCTS algorithm does not attempt to optimise the subset that needs to be expanded.
Revised: Expanding all children of a node is generally considered efﬁcient in domains with a small Σ and low playout cost ˆ v ( p ) , and the standard MCTS algorithm does not attempt to optimise the subset that needs to be expanded.

Original: In the text domain, most words in the vocabulary are not applicable as they do not ﬁt the context y ( p ) and correspond to the content in the source sentence x , and can be excluded using the value estimate described in Equation 2.
Revised: In the text domain, most words in the vocabulary are not applicable as they do not ﬁt the context y ( p ) and correspond to the content in the source sentence x , and can be excluded using the value estimate described in Equation 2.

Original: Next, we ascend from the selected node p ◦ y , updating visit counts and value estimates:
Revised: Next, we ascend from the selected node p ◦ y , updating visit counts and value estimates:

Original: [Equation]
Revised: [Equation]

Original: where each parent p increases its visit count n ( p ) and updates its value estimate v ( p ) to the child’s value  a new best translation is found.
Revised: where each parent p increases its visit count n ( p ) and updates its value estimate v ( p ) to the child’s value if a new best translation is found.

Original: Thus, v ( p ) represents the best translation obtained in the subtree represented by p .
Revised: Thus, v ( p ) represents the best translation obtained in the subtree represented by p .

Original: Starting from the score obtained using greedy decoding when v ( p ) is initialised, each new traversal that passes through p has a chance to reﬁne this initial estimate with the newly found translation.
Revised: Starting from the score obtained using greedy decoding when v ( p ) is initialised, each new traversal that passes through p has a chance to reﬁne this initial estimate with the newly found translation.

Original: 3. B EAM A DAPTIVE T REE S EARCH
Revised: 3. B EAM A DAPTIVE T REE S EARCH

Original: A standard way to guarantee progression in MCTS is to run an instance of MCTS per word.
Revised: A standard way to guarantee progression in MCTS is to run an instance of MCTS per word.

Original: Here, we would run an ATS instance ATS ( ε, k ) with k iterations starting from root ε .
Revised: Here, we would run an ATS instance ATS ( ε, k ) with k iterations starting from root ε .

Original: Then, we set ε ← ε ◦ y ∗ 1 , where y ∗ 1 = arg max y ∈ Σ v ( ε ◦ y ) is the child with the highest value estimate and repeat this process until y ∗ i = EOS .
Revised: Then, we set ε ← ε ◦ y ∗ 1 , where y ∗ 1 = arg max y ∈ Σ v ( ε ◦ y ) is the child with the highest value estimate and repeat this process until y ∗ i = EOS .

Original: This is the approach taken in turn-based competitive games (e.g. go) where players have an assigned time to choose a single action, which is not reversible.

Original: However, text generation more closely resembles puzzle solving games (e.g., Rubik’s cube), where decisions are subject to revision.

Original: In this type of game, it has been found that  restricting search to a set ofnodes high value nodes rather than a single one allows such games to be solved at a faster rate (Baier & Winands, 2012).
Revised: However, it has been found that in text generation tasks restricting search to a set of high value nodes rather than a single one allows such games to be solved at a faster rate (Baier & Winands, 2012).

Original: Thus, we modify our selection step as follows:
Revised: Thus, we modify our selection step as follows:

Original: [Equation]
Revised: [Equation]

Original: where d ( p ◦ y ) is initialised as (cid:96) ( p ◦ y ) , a function that counts the number of edges required to reach the root node from ( p ◦ y ) .
Revised: where d ( p ◦ y ) is initialised as (cid:96) ( p ◦ y ) , a function that counts the number of edges required to reach the root node from ( p ◦ y ) .

Original: Then, the following update rule is added to ensure that the d ( p ) stores that depth of the deepest node achievable from p :
Revised: Then, the following update rule is added to ensure that the d ( p ) stores that depth of the deepest node achievable from p :

Original: [Equation]
Revised: [Equation]

Original: where we update each node so that d ( p ) stores the value of the deepest node that is accessible from p .
Revised: where we update each node so that d ( p ) stores the value of the deepest node that is accessible from p .

Original: Thus, in Equation 3, condition d ( p ◦ y ) > d min tests whether p ◦ y contains a node deeper than d min .
Revised: Thus, in Equation 3, condition d ( p ◦ y ) > d min tests whether p ◦ y contains a node deeper than d min .

Original: We deﬁne BATS ( ε, k ) as a Beam ATS instance that runs ATS starting from the root node ε with the selection criteria UCT constrained with d min = 0 and gradually increasing d min by 1 every k iterations.
Revised: We deﬁne BATS ( ε, k ) as a Beam ATS instance that runs ATS starting from the root node ε with the selection criteria UCT constrained with d min = 0 and gradually increasing d min by 1 every k iterations.

Original: Search stops when no node satisﬁes d ( p ◦ y ) >
Revised: Search stops when no node satisﬁes d ( p ◦ y ) >

Original: d min or until a maximum depth d max .
Revised: d min or until a maximum depth d max .

Original:  O BJECTIVES
Revised:  O BJECTIVES

Original: As decoding with a decoder on vanilla autoregressive models is unlikely to yield translations with quality superior to beam search, as the beam search bias is essential to overcoming the calibration issues in these models, we propose modeling improvements in order to address these shortcomings.
Revised: As decoding with a decoder on vanilla autoregressive models is unlikely to yield translations with quality superior to beam search, as the beam search bias is essential to overcoming the calibration issues in these models, we propose modeling improvements in order to address these shortcomings.

Original: 4. M AX R ANK
Revised: 4. M AX R ANK

Original: Decoding in autoregressive models generally optimising a normalised log probability (Wu et al., 2016), (log p ( y | x )) ( 6 5+ | y | ) α , which combinesthe sum of the token level log-probabilities (when α = 0 ) and a length-based adjustment, which approximates the mean of the log-probabilities as the length | y | grows (when α = 1 ).
Revised: Decoding in autoregressive models generally optimises a normalised log probability (Wu et al., 2016), (log p ( y | x )) ( 6 5+ | y | ) α , which combines and a length-based adjustment, which approximates the mean of the log-probabilities as the length | y | grows (when α = 1 ).

Original: Similar to normalised log-probabilities, we consider a metric that characterizes translation by their minimum token level log-probability.
Revised: Similar to normalised log-probabilities, we consider a metric that characterizes translation by their minimum token level log-probability.

Original: The intuition here is that the quality of the translation is represented by the worst decision made in the sequence.
Revised: The intuition here is that the quality of the translation is represented by the worst decision made in the sequence.

Original: In practice, many degenerate cases in autoregressive models are created by making a single bad decision, such as generating a EOS token prematurely or omitting translations, which can be understood in terms of uniform information density (Meister et al., 2020).
Revised: In practice, many degenerate cases in autoregressive models are created by making a single bad decision, such as generating a EOS token prematurely or omitting translations, which can be understood in terms of uniform information density (Meister et al., 2020).

Original: However, the issue with the minimum of the token level log-probability is that log-probability ranges tend to vary depending on the context and number of translation options that are available.
Revised: However, the issue with the minimum of the token level log-probability is that log-probability ranges tend to vary depending on the context and number of translation options that are available.

Original: Thus, they are not very reﬂective on the quality of the choice made as the best choice at a given timestamp could still be the worst decision in the sequence.
Revised: Thus, they are not very reﬂective on the quality of the choice made as the best choice at a given timestamp could still be the worst decision in the sequence.

Original: Instead, we optimise the normalised rank:
Revised: Instead, we optimise the normalised rank:

Original: [Equation]
Revised: [Equation]

Original: where we count the number ofother actions  with lower log-probability than y i .
Revised: where we count the number of actions in Σ with lower log-probability than y i .

Original: By using the rank r instead of the log-probability p , we can compare values within the same range at the cost of a loss in relative precision.
Revised: By using the rank r instead of the log-probability p , we can compare values within the same range at the cost of a loss in relative precision.

Original: Thus, we name our metric max rank (MR), which is computed as follows:
Revised: Thus, we name our metric max rank (MR), which is computed as follows:

Original: [Equation]
Revised: [Equation]

Original: Finally, unlike the mean and sum of log-probabilities, the max of a sequence of log-probabilities is not autoregressive, so beam search is not applicable.
Revised: Finally, unlike the mean and sum of log-probabilities, the max of a sequence of log-probabilities is not autoregressive, so beam search is not applicable

Original:  4. N OISY C HANNEL M ODEL
Revised: . 4. N OISY C HANNEL M ODEL

Original: The noisy channel model uses the Bayes rule decomposition in order to decompose the probability of a sentence p ( y | x ) into p ( y | x ) = p ( x | y ) p ( y ) p ( x ) where the channel model p ( x | y ) can be trained as translation model trained in the reverse direction and p ( y ) is a language model.
Revised: The noisy channel model uses the Bayes rule decomposition in order to decompose the probability of a sentence p ( y | x ) into p ( y | x ) = p ( x | y ) p ( y ) p ( x ) where the channel model p ( x | y ) can be trained as translation model trained in the reverse direction and p ( y ) is a language model.

Original: Finally, the prior p ( x ) can be ignored in the context of a maximization problem.
Revised: Finally, the prior p ( x ) can be ignored in the context of a maximization problem.

Original: It has been pointed out that as the reverse model is not autoregressive in the space Σ ∗ , it allows the model to bypass many of the degenerative cases in autoregressive models.
Revised: Since the reverse model is not autoregressive in the space Σ ∗ , it can bypass many of the degenerative cases in autoregressive models.

Original: 4. M INIMUM R ISK T RAINED AUTOREGRESSIVE M ODELS
Revised: 4. M INIMUM R ISK T RAINED AUTOREGRESSIVE M ODELS

Original: In order to show that BATS can be an attractive alternative to beam search under autoregressive models, we need to improve the model so that the search bias is no longer as crucial to preserving the translation quality of the generated text.
Revised: In order to show that BATS can be an attractive alternative to beam search under autoregressive models, we need to improve the model so that the search bias is no longer as crucial to preserving the translation quality of the generated text.

Original: To this end, we ﬁne-tune our models using minimum risk training (Shen et al., 2016, MRT).
Revised: To this end, we ﬁne-tune our models using minimum risk training (Shen et al., 2016, MRT).

Original: The MRT training objective is designed to minimize the empirical risk r ( y , y (cid:48) ) is minimized in a subset of Σ ∗ obtained by sampling n translations.
Revised: The MRT training objective is designed to minimize the empirical risk r ( y , y (cid:48) ) by minimizing it in a subset of Σ ∗ obtained by sampling n translations.

Original: This allows the model to mitigate degenerate cases caused by optimising the likelihood objective by ﬁne-tuning the model on downstream metrics, such as BLEU (Papineni et al., 2002).
Revised: This allows the model to mitigate degenerate cases caused by optimising the likelihood objective by ﬁne-tuning the model on downstream metrics, such as BLEU (Papineni et al., 2002).

Original:  E XPERIMENTS 5. S ETUP
Revised:  E XPERIMENTS 5. S ETUP

Original: We conduct our experiments on the Chinese–English and Pashto–English tasks from WMT2020 (Barrault et al., 2020), and German–English from WMT2014 (Bojar et al., 2014), following the same training, development and test splits.
Revised: We conduct our experiments on the Chinese–English and Pashto–English tasks from WMT2020 (Barrault et al., 2020), and German–English from WMT2014 (Bojar et al., 2014), following the same training, development and test splits.

Original: Our autoregressive model transformer baseline uses the multiquery attention model (Shazeer, 2019).
Revised: Our autoregressive model transformer baseline uses the multiquery attention model (Shazeer, 2019).

Original: It uses the standard architecture with 6 encoder and decoder layers with 512 hidden units, 2048 sized tied embeddings for both source and target word projections and 8 attention heads.
Revised: It uses the standard architecture with 6 encoder and decoder layers with 512 hidden units, 2048 sized tied embeddings for both source and target word projections and 8 attention heads.

Original: We tokenize the data with byte-pair encoding (Sennrich et al., 2016) with 32K merges and set a maximum sentence size of Y = 128 .
Revised: We tokenize the data with byte-pair encoding (Sennrich et al., 2016) with 32K merges and set a maximum sentence size of Y = 128 .

Original: Translation quality evaluation is performed using sacreBLEU (Post, 2018).
Revised: Translation quality evaluation is performed using sacreBLEU (Post, 2018).

Original: We choose the checkpoint that yields the highest BLEU in the validation set using beam search with the normalisation constant α = 0 .
Revised: We choose the checkpoint that yields the highest BLEU in the validation set using beam search with the normalisation constant α = 0 .

Original: 8 and beam size 6 .
Revised: 8 and beam size 6 .

Original: We ﬁne tune our models using MRT by setting the sample size to8 , which are generated via temperature sampling with temperature 0 . 8 . Finally, we set the risk r ( y , y (cid:48) ) = − 12 BLEU ( y , y (cid:48) ) − 12 ChrF
Revised: We also compare a variant ﬁne-tuned using MRT according to the procedure described in Appendix A.3.

Original: ( y , y (cid:48) ) , where BLEU ( · ) is the sentence level sacreBLEU (Post, 2018) and ChrF ( · ) is the sentence level

Original: ChrF (Popovi ´ c, 2015).

Original: We used the average of BLEU and ChrF because BLEU was designed as a corpus level metric, and ChrF provides a better estimate of translation quality with sparser matches against a reference for a single sentence.

Original: For the noisy channel model, we train the channel model by simply swapping the translation direction of the autoregressive model with the same hyperparameters.
Revised: For the noisy channel model, we train the channel model by simply swapping the translation direction of the autoregressive model with the same hyperparameters.

Original: For the language model prior, we employ the TransformerXL architecture (Dai et al., 2019) trained on 1 billion words (Chelba et al., 2013).
Revised: For the language model prior, we employ the TransformerXL architecture (Dai et al., 2019) trained on 1 billion words (Chelba et al., 2013).

Original: On non-autoregressive objectives, we use beam search as a proxy to generate translations candidates which are rescored using the non-autoregressive metric (Yee et al., 2019; Yu et al., 2020a).
Revised: On non-autoregressive objectives, we use beam search as a proxy to generate translations candidates which are rescored using the non-autoregressive metric (Yee et al., 2019; Yu et al., 2020a).

Original: For BATS, we simply set hyperparameter C = 1 .
Revised: For BATS, we simply set hyperparameter C = 1 .

Original: While optimising C could lead to more efﬁcient optimisation of the model score, our goal is to study how model scores are correlated with translation quality in different objectives.
Revised: While optimising C could lead to more efﬁcient optimisation of the model score, our goal is to study how model scores are correlated with translation quality in different objectives.

Original: The translation budget (beam size for beam search and iterations for BATS) is swept by doubling its value starting from 1 to 256.
Revised: The translation budget (beam size for beam search and iterations for BATS) is swept by doubling its value starting from 1 to 256.

Original: We combine different objective components under a log-linear model, where the weights of the components are tuned with MERT (Och, 2003).
Revised: We combine different objective components under a log-linear model, where the weights of the components are tuned with MERT (Och, 2003).

Original: In order for model scores to be comparable, all weights are tuned on a pool of 256 translations using beam search.
Revised: In order for model scores to be comparable, all weights are tuned on a pool of 256 translations using beam search.

Original: 5. R ESULTS
Revised: 5. R ESULTS

Original: Table 1 illustrates the translation quality results using BATS and beam search on the normalised autoregressive model baseline and the optimal model (Column “System”) for each language pair (Column “Language Pair”).
Revised: Table 1 illustrates the translation quality results using BATS and beam search on the normalised autoregressive model baseline and the optimal model (Column “System”) for each language pair (Column “Language Pair”).

Original: We perform a grid search over the following decisions: (1) Whether to use Max Rank (Column “MR”), (2) Whether to use the Noisy Channel Model (Column “NC”), (3)
Revised: We perform a grid search over the following decisions: (1) Whether to use Max Rank (Column “MR”), (2) Whether to use the Noisy Channel Model (Column “NC”), (3)

Original: Whether to tune the autoregressive model using MRT (Column “MRT”), (4) Whether to use beam search or BATS (5) The translation budget of each decoder.
Revised: Whether to tune the autoregressive model using MRT (Column “MRT”), (4) Whether to use beam search or BATS (5) The translation budget of each decoder.

Original: The combination with the highest BLEU on the validation set is used to decode on the test set and the BLEU scores obtained using beam search and Bats are reported (Columns “Beam Search” and “BATS”) 2 .
Revised: The combination with the highest BLEU on the validation set is used to decode on the test set and the BLEU scores obtained using beam search and Bats are reported (Columns “Beam Search” and “BATS”).

Original: We observe that for all language decoding with BATS can yield gains over decoding with beam search when using the combination of objectives with the highest BLEU on the validation set.
Revised: We observe that for all pairs decoding with BATS can yield gains over decoding with beam search when using the combination of objectives with the highest BLEU on the validation set.

Original: As expected, due to the search bias in beam search, it does comparatively better on some language, such as Chinese-English and Pashto-English.
Revised: As expected, due to the search bias in beam search, it does comparatively better on some language pairs, such as Chinese-English and Pashto-English.

Original: In terms of modeling, we note that our proposed metric, Max Rank, and the MRT method combined yield the best results for Mandarin and German.
Revised: In terms of modeling, we note that our proposed metric, Max Rank, and the MRT method combined yield the best results for Mandarin and German.

Original: The noisy channel model only yields improvements in Pashto-English, when used with Max Rank and MRT as the training data is small (500k parallel sentences) and the model relies on the large sized language model to provide accurate predictions.
Revised: The noisy channel model only yields improvements in Pashto-English, when used with Max Rank and MRT as the training data is small (500k parallel sentences) and the model relies on the large sized language model to provide accurate predictions.

Original: [Table]
Revised: [Table]

Original: 5. B EAM S EARCH AND BATS
Revised: 5. B EAM S EARCH AND BATS

Original: We now provide a more in-depth analysis on the Chinese-English language, where we believe results are more informative, on the elements that make each method more attractive for decoding.
Revised: We now provide a more in-depth analysis on the Chinese-English language pair, where we believe results are more informative.

Original: Tableillustrates the results obtained using some models that were explored in our grid search (Column s ( x , y ) ).
Revised: Table 2 illustrates the results obtained using some models that were explored in our grid search (Column s ( x , y ) ).

Original: For each model, we illustrate the best BLEU obtained on the test set (Column “BLEU”), the beam size (Column “Beam”) or number of iterations (Column “Iter”), where the best BLEU was obtained on the validation set and the percentage improvement between results obtained using beam search and BATS (Column “Delta”).
Revised: For each model, we illustrate the best BLEU obtained on the test set (Column “BLEU”), the beam size (Column “Beam”) or number of iterations (Column “Iter”), where the best BLEU was obtained on the validation set and the percentage improvement between results obtained using beam search and BATS (Column “Delta”).

Original: Finally, autoregressive and non-autoregressive models are marked with ar with nar , respectively.
Revised: Finally, autoregressive and non-autoregressive models are marked with AR with NAR , respectively.

Original: It is also important to refer that the number of iterations1 Tuning λ using BATS to genererate candidates yields similar weights. The optimal models with the highest BLEU on the validation for Beam Search and BATS are the same. in BATS and beam in beam search are not comparable point-wise, instead we analyse the overall behavior of the model score and BLEU curves.
Revised: It is also important to refer that the number of iterations in BATS and beam in beam search are not comparable point-wise, instead we analyse the overall behavior of the model score and BLEU curves.

Original: When to use Beam Search?
Revised: When to use Beam Search?

Original: In the models where beam search outperforms BATS(Rows “Log
Revised: In the models where beam search outperforms BATS(Rows “Log

Original: Probability AR ( α = 0 )” and “Log Probability AR ( α = 0 . 8 )”), we notice that the search budget in both cases is always low.
Revised: Probability AR ( α = 0 )” and “Log Probability AR ( α = 0 . 8 )”), we notice that the search budget in both cases is always low.

Original: Figure 1 plots the evolution of the BLEU (Top) and model score (Bottom) of both decoders as translation budget is increased.
Revised: Figure 1 plots the evolution of the BLEU (Top) and model score (Bottom) of both decoders as translation budget is increased.

Original: For the normalised log-probability (Column “Log
Revised: For the normalised log-probability (Column “Log

Original: Probability ( α = 0 . 8 )”), the model scores obtained for BLEU search and BATS are very similar.
Revised: Probability ( α = 0 . 8 )”), the model scores obtained for BLEU search and BATS are very similar.

Original: However, we an observe a large gap between the BLEU scores obtained at the same model score.
Revised: However, we an observe a large gap between the BLEU scores obtained at the same model score.

Original: This shows that the beam search bias is ﬁltering the set of candidates in beam search so that they are of higher quality, even though the model score is failing to discriminate them.
Revised: This shows that the beam search bias is ﬁltering the set of candidates in beam search so that they are of higher quality, even though the model score is failing to discriminate them.

Original: Additionally,  the standard BLEU curve (Stahlberg & Byrne, 2019), where after a set of initial iterations BLEU deteriorates rapidly, which renders elaborate decoding mechanisms unneeded.
Revised: Additionally, we observe the standard BLEU curve (Stahlberg & Byrne, 2019), where after a set of initial iterations BLEU deteriorates rapidly, which renders elaborate decoding mechanisms unneeded.

Original: Additionally, BATS and other MCTS-based methods are not ideal for low search budget scenarios as they depend on high node visit counts to accumulate enough statistics to make informed decisions.
Revised: Furthermore, BATS and other MCTS-based methods are not ideal for low search budget scenarios as they depend on high node visit counts to accumulate enough statistics to make informed decisions.

Original: We see in all objectives that model scores for BATS do not outperform Beam Search until a large budget is allocated.
Revised: We see in all objectives that model scores for BATS do not outperform Beam Search until a large budget is allocated.

Original: In conclusion, if the model cannot support high search budgets, beam search is the preferred alternative, especially if the modeling issues can be addressed with the search bias.
Revised: In conclusion, if the model cannot support high search budgets, beam search is the preferred alternative, especially if the modeling issues can be addressed with the search bias.

Original: When to use BATS?
Revised: When to use BATS?

Original: With the addition of the noisy channel model (Row “Log Probability ( α = 0 . 8 ) + NC NAR ) and Max Rank (Row “Log Probability ( α = 0 . 8 ) + MR NAR ”), the search budget until BLEU deteriorates for both decoders is increased.
Revised: With the addition of the noisy channel model (Row “Log Probability ( α = 0 . 8 ) + NC NAR ) and Max Rank (Row “Log Probability ( α = 0 . 8 ) + MR NAR ”), the search budget until BLEU deteriorates for both decoders is increased.

Original: Here, we observe that BATS is the decoding option that yields higher BLEU and that the delta is higher when BATS can employ more iterations.
Revised: Here, we observe that BATS is the decoding option that yields higher BLEU and that the delta is higher when BATS can employ more iterations.

Original: We observe in Figure 1  that the evolution of BLEU is considerably more stable with the noisy channel model (Column “Log Probability ( α = 0 . 8 ) + NC”) and when Max Rank is applied (Column “Log
Revised: Figure 1 shows that the evolution of BLEU is considerably more stable with the noisy channel model (Column “Log Probability ( α = 0 . 8 ) + NC”) and when Max Rank is applied (Column “Log

Original: Probability ( α = 0 . 8 ) + MR”).
Revised: Probability ( α = 0 . 8 ) + MR”).

Original: In these non-autoregressive  BATS yields signiﬁcantly better model scores than reranking.
Revised: For these non-autoregressive models BATS yields signiﬁcantly better model scores than reranking.

Original: More interestingly, comparing the behavior of Beam Search and BATS, similar model scores between the two methods do not yield similar BLEU scores.
Revised: More interestingly, comparing the behavior of Beam Search and BATS, similar model scores between the two methods do not yield similar BLEU scores.

Original: In the noisy channel model, we observe that at 16 iterations and beam size 16 , both decoders yield similar model scores and BLEU scores.
Revised: In the noisy channel model, we observe that at 16 iterations and beam size 16 , both decoders yield similar model scores and BLEU scores.

Original: However, as the model score increases beyond that point, the degeneration in beam search as the score increases is signiﬁcant, while BATS observes almost no deterioration.
Revised: However, as the model score increases beyond that point, the degeneration in beam search as the score increases is signiﬁcant, while BATS observes almost no deterioration.

Original: This suggests that the beam search bias has a negative impact in translation quality at high values of b by ﬁltering good translations from the search space.
Revised: This suggests that the beam search bias has a negative impact in translation quality at high values of b by ﬁltering good translations from the search space.

Original: Using Max Rank, we observe that not only
Revised: Using Max Rank, we observe that not only

Original: BATS can achieve considerably higher model scores as it is optimising the non-autoregressive model directly, it also yields considerably higher BLEU scores.
Revised: BATS can achieve considerably higher model scores as it is optimising the non-autoregressive model directly, it also yields considerably higher BLEU scores.

Original: In beam search, BLEU stops improving at 32 iterations even though model score keeps increasing due to the search bias.
Revised: In beam search, BLEU stops improving at 32 iterations even though model score keeps increasing due to the search bias.

Original: Finally, wecan observe that the search bias issue is not restricted to reranking, by applying beam search to a ﬁned tuned autoregressive model using MRT, we observe in Table 2 that the same trend occurs (Row “Log Probability MRT ( α = 0 .8 )”).
Revised: Finally, we observe that the search bias issue is also present when using beam search to decode from an autoregressive model ﬁne-tuned with MRT ( Table 2, Row “Log Probability MRT ( α = 0 .

Original: By making the autoregressive model more robust to degenerate cases found in vanilla autoregressive models, we take notice that the search bias in beam search is no longer helpful but detrimental to translation quality.
Revised: 8) AR ”).

Original: If we observe Figure 1, we notice that once MRT is applied (Column “Log Probability MRT ( α = 0 . 8 )”), not only BATS can ﬁnd signiﬁcantly better model scores, but they are are associated with better BLEU scores.
Revised: Figure 1 shows that once MRT is applied (Column “Log Probability MRT ( α = 0 . 8 )”), not only BATS can ﬁnd signiﬁcantly better model scores, but they are are associated with better BLEU scores.

Original: On the other hand, beam search stagnates after 8 iterations.
Revised: Additionally, beam search stagnates after 8 iterations.

Original: In conclusion, as translation models become more robust, there is a growing need of better decoding mechanisms, such as BATS, in order to maximize translation quality.
Revised: In conclusion, as translation models become more robust, there is a growing need of better decoding mechanisms, such as BATS, in order to maximize translation quality.

Original: We perceive that in both autoregressive and non-autoregressive models, there is a limit to both model and BLEU scores that can be obtained using beam search, which is partially attributed to the fact that its search is strongly biased due to the lack of future costs.
Revised: We perceive that in both autoregressive and non-autoregressive models, there is a limit to both model and BLEU scores that can be obtained using beam search, which is partially attributed to the fact that its search is strongly biased due to the lack of future costs.

Original: We believe that future research will drive models to extents where translation quality nearly perfectly matches with model scores.
Revised: We believe that future research will drive models to extents where translation quality nearly perfectly matches with model scores.

Original: In an oracle setup, where the objective is the sentence level BLEU by peeking at the reference (Row “Oracle BLEU”), we hypothesise that the delta between beam search and BATS would grow vastly, and observe that a delta of 39 .  
Revised: In an oracle setup, where the objective is the sentence level BLEU by peeking at the reference (Row “Oracle BLEU”), we hypothesise that the delta between beam search and BATS would grow vastly, and observe that a delta of 39 .  C ONCLUSION

Revised: This paper proposes an adaptive tree search algorithm designed to optimise arbitrary metrics.

Revised: It uses rollouts from an auxiliary autoregressive model to obtain estimates for the value estimates of internal nodes.

Revised: This allows the decoder to optimise arbitrary objectives and avoid the search biases of

Original: [Table]
Revised: [Table]

Original: [Figure] 
Revised: [Figure] [Table] [Figure] [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Original:  [Table]
Revised: [Figure] [Table] [Figure] [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: C ONCLUSION
Revised: [Table] [Figure] [Table] [Figure]

Original: In this paper, we propose an adaptive tree search algorithm, which is designed to optimise arbitrary metrics.
Revised: [Table]

Original: It uses rollouts from an auxiliary autoregressive model to obtain estimates for the value estimates of internal nodes.
Revised: [Figure] [Table] [Figure] [Table]

Original: This allows the decoder to optimise arbitrary objectives and avoid the search biases of manually deﬁned heuristics, such as partial translation probability in autoregressive models.
Revised: [Figure] [Table] [Figure] manually deﬁned heuristics, such as partial translation probability in autoregressive models.

Original: BATS is particularly useful when models are robust enough to allow for a higher translation quality ceiling through large amounts of translation budget.
Revised: BATS is particularly useful when models are robust enough to allow for a higher search budget.

Original: As many existing objectives are found to be poorly correlated with translation quality, we propose a new metric named max rank, as well as use existing methods, such as the noisy channel model and minimum error rate training in order to address the degenerate cases that prevent vanilla autoregressive models from achieving this ceiling.
Revised: As many existing objectives are found to be poorly correlated with translation quality, we propose a new metric named max rank and use existing methods, such as the noisy channel model and minimum error rate training, to address the failure modes of vanilla autoregressive models.

Original: Results on three language pairs are favorable to BATS when  our proposed augmentations of the autoregressive model.
Revised: Results on three language pairs are favourable to BATS when using our proposed augmentations of the autoregressive model.

Original: Additionally, we ﬁnd that the gap in translation quality between beam search and BATS increases as more robust models are employed.
Revised: Additionally, we ﬁnd that the gap in translation quality between beam search and BATS increases as more robust models are employed.

Original: More importantly, we observe that the search bias prevents beam search from achieving high quality translations as it ﬁlters good translations that are unfavoured by the search heuristic.
Revised: More importantly, we observe that the search bias prevents beam search from achieving high-quality translations as it ﬁlters good translations that are unfavoured by the search heuristic.

Original: Thereby, the model score increases but BLEU decreases.
Revised: Thereby, the model score increases, but BLEU decreases.

Original: This shows that beam search limits the potential of many models by establishing translation quality ceilingsthat are unrelated to the robustness of the model but to the topology of the search space they establish.
Revised: This shows that beam search limits the potential of many models by establishing translation quality ceilings unrelated to the robustness of the model, but to the topology of the search space they establish.

Original: Thisshows suggests that  scientiﬁc progress drives more robust models, exploring more robust decoding methods, such as BATS, is fundamental for advancing the ﬁeld of text generation. 
Revised: This suggests that as scientiﬁc progress drives more robust models, exploring more robust decoding methods, such as BATS, is fundamental for advancing the ﬁeld of text generation. A CKNOWLEDGMENTS

Revised: We thank R´emi Leblond and Phil Blunsom for useful feedback throughout the different stages of this project.

Original: A A PPENDIX A. BATS VS . ATS
Revised: A A PPENDIX A. BATS VS . ATS

Original: The advantage of the beam variant of MCTS (Baier & Winands, 2012) is that the search algorithm does not have to commit to a single branch every k iterations.
Revised: The advantage of the beam variant of MCTS (Baier & Winands, 2012) is that the search algorithm does not have to commit to a single branch every k iterations.

Original: As often occurs during translations, multiple valid translation exist and correspond to different branches in the tree and only after furthering the search tree can the optimal translation be ﬁltered out.
Revised: As often occurs during translations, multiple valid translation exist and correspond to different branches in the tree and only after furthering the search tree can the optimal translation be ﬁltered out.

Original: In ATS, once the EOS is chosen, search ends immediately, with no chance for the decoder to explore other branches, which accentuate issues where degenerate solutions are chosen (e.g. prematurely ending a translation).
Revised: In ATS, once the EOS is chosen, search ends immediately, with no chance for the decoder to explore other branches, which accentuate issues where degenerate solutions are chosen (e.g. prematurely ending a translation).

Original: A comparison between
Revised: A comparison between

Original: ATS and BATS on our WMT2020 Chinese-English validation set using autoregressive models is shown in Table 3, where we observe that ATS has both a bias towards degenerate cases and yields worse model scores and BLEU.
Revised: ATS and BATS on our WMT2020 Chinese-English validation set using autoregressive models is shown in Table 3, where we observe that ATS has both a bias towards degenerate cases and yields worse model scores and BLEU.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table][Table]
Revised: [Table] [Table]

Original: [Table] [Table][Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: A. M IN P ROB VS . M AX R ANK
Revised: A. M IN P ROB VS . M AX R ANK

Original: The most straight-forward approach to select the worst decision in a translation is to select the lowest log-probability in the sentence.
Revised: The most straight-forward approach to select the worst decision in a translation is to select the lowest log-probability in the sentence.

Original: However, log-probabilities are not a good indicator of whether a decision is good or bad as some word translations are inherently low probability (words with many valid translations).
Revised: However, log-probabilities are not a good indicator of whether a decision is good or bad as some word translations are inherently low probability (words with many valid translations).

Original: Thus, we decided to use the maximum rank instead.
Revised: Thus, we decided to use the maximum rank instead.

Original: Table 4 provides a comparison between the Min Prob (Row “Log Probability ( α = 0 .
Revised: Table 4 provides a comparison between the Min Prob (Row “Log Probability ( α = 0 .

Original: + MP”)
Revised: + MP”)

Original: and the Max Rank objective (Row “Log Probability ( α = 0 . 8 ) + MR”).
Revised: and the Max Rank objective (Row “Log Probability ( α = 0 . 8 ) + MR”).

Original: We observe that while Min Prob yields a relatively small improvement, it is signiﬁcantly smaller than Max Rank’s improvement over the baseline (Row “Log
Revised: We observe that while Min Prob yields a relatively small improvement, it is signiﬁcantly smaller than Max Rank’s improvement over the baseline (Row “Log

Original: Probability( α = 0 . 8 )”).
Revised: Probability( α = 0 . 8 )”).

Original: Additionally, it clearly does not address the degenerate solutions problem in autoregressive models.
Revised: Additionally, it clearly does not address the degenerate solutions problem in autoregressive models.

Original: [Table]
Revised: [Table]

Original: Log Probability ( α = 0 . 8 ) + MP NAR
Revised: Log Probability ( α = 0 . 8 ) + MP NAR

Original: [Table]
Revised: [Table]

Original: Log Probability ( α = 0 . 8 ) + MR NAR
Revised: Log Probability ( α = 0 . 8 ) + MR NAR

Original: [Table]
Revised: [Table]

Original: The number of iterations is tuned on the validation set.
Revised: The number of iterations is tuned on the validation set.

Original: A. M INIMUM R ISK T RAINING WITH C HR F AND BLEU
Revised: A. M INIMUM R ISK T RAINING WITH C HR F AND BLEU

Original: While BLEU (Papineni et al., 2002) is the downstream translation metric used in most MT evaluations, its sentence level predictions tend to be sparse and inaccurate.
Revised: While BLEU (Papineni et al., 2002) is the downstream translation metric used in most MT evaluations, its sentence level predictions tend to be sparse and inaccurate.

Original: Thereby, training quickly overﬁts before optimal translation quality is reached.
Revised: Thereby, training quickly overﬁts before optimal translation quality is reached.

Original: With sentence level ChrF (Popovi´c, 2015) training is more stable, and a better optimal translation quality can be obtained.
Revised: With sentence level ChrF (Popovi´c, 2015) training is more stable, and a better optimal translation quality can be obtained.

Revised: We use a sample size of 8 translations per sentence, and these are generated via temperature sampling with temperature 0 . 8 .

Original: In our experimental setup (Section 5.1), we set the risk r ( y , y (cid:48) )
Revised: Finally, we set the risk r ( y , y (cid:48) )

Original: = − 12 BLEU
Revised: = − 12 BLEU

Original: ( y , y (cid:48) ) − 12 ChrF
Revised: ( y , y (cid:48) ) − 12 ChrF

Original: ( y , y (cid:48) ) , where BLEU ( · ) is the sentence level sacreBLEU (Post, 2018) and ChrF ( · ) is the sentence level ChrF (Popovi´c, 2015).
Revised: ( y , y (cid:48) ) , where BLEU ( · ) is the sentence level sacreBLEU (Post, 2018) and ChrF ( · ) is the sentence level ChrF (Popovi´c, 2015).

Revised: We used the average of BLEU and ChrF because BLEU was designed as a corpus level metric, and ChrF provides a better estimate of translation quality with sparser matches against a reference for a single sentence.

Original: Table 5 compares the results obtained using only bleu ( · ) , chrf ( · ) and their combinations, using Beam Search with beam 6, on the validation set in the WMT2020 Chinese-English dataset.
Revised: Table 5 compares the results obtained using only bleu ( · ) , chrf ( · ) and their combinations, using Beam Search with beam 6, on the validation set in the WMT2020 Chinese-English dataset.

Original: Here, we can observe that a combination of both scores yields the optimal translation quality.
Revised: Here, we can observe that a combination of both scores yields the optimal translation quality.

Original: [Table]
Revised: [Table]

Original: A. C OMPUTATIONAL C OST OF BATS 
Revised: A. C OMPUTATIONAL C OST OF BATS For Beam

Original: For Beam Search with beam size b and a max sentence length Y , beam search requires b × Y ×
Revised: Search with beam size b and a max sentence length Y , beam search requires b × Y ×

Original: A operations, where A is a transformer+softmax block.
Revised: A operations, where A is a transformer+softmax block.

Original: Additionally, for reranking, the model score s ( x , y ) needs to be computed for each of the b translations and has a computational cost of B .
Revised: Additionally, for reranking, the model score s ( x , y ) needs to be computed for each of the b translations and has a computational cost of B .

Original: Thus, the total cost is b × ( Y ×
Revised: Thus, the total cost is b × ( Y ×

Original: A + B ) .
Revised: A + B ) .

Original: In BATS, computation is proportional to the number of expanded nodes, when the playout heuristic is applied.
Revised: In BATS, computation is proportional to the number of expanded nodes, when the playout heuristic is applied.

Original: Each playout requires the computation of greedy decoding, followed by the scoring function s ( x , y ) .
Revised: Each playout requires the computation of greedy decoding, followed by the scoring function s ( x , y ) .

Original: Thus, each rollout requires Y ×
Revised: Thus, each rollout requires Y ×

Original: A + B computations.
Revised: A + B computations.

Original: Finally, all nodes that follow the path used in greedy decoding will have the same value v , which means that the ﬁrst node that is expanded, which corresponds to this path will have no cost, with the exception of the root node.
Revised: Finally, all nodes that follow the path used in greedy decoding will have the same value v , which means that the ﬁrst node that is expanded, which corresponds to this path will have no cost, with the exception of the root node.

Original: Thus, the cost of BATS is (1 + z ) ×
Revised: Thus, the cost of BATS is (1 + z ) ×

Original: ( Y × A + B ) , where z is the number of non-root nodes expanded more than once.
Revised: ( Y × A + B ) , where z is the number of non-root nodes expanded more than once.

Original: As Y × A + B is a common denominator for both methods, we deﬁne it as the a computational unit.
Revised: As Y × A + B is a common denominator for both methods, we deﬁne it as the a computational unit.

Original: Table 6 shows the results obtained for max rank objective (row “Log Probability ( α = 0 . 8 ) + MR ” in Table 2).
Revised: Table 6 shows the results obtained for max rank objective (row “Log Probability ( α = 0 . 8 ) + MR ” in Table 2).

Original: The “Cost” column represents the number of computational units, and we observe that
Revised: The “Cost” column represents the number of computational units, and we observe that

Original: BATS is considerably more expensive to run than Beam Search.
Revised: BATS is considerably more expensive to run than Beam Search.

Original: However, observe the model scores (Column “Score”), we notice that Beam Search gradually decreases the rate at which model score gains are observed (Column “Gain”) even though the cost doubles at each row.
Revised: However, observe the model scores (Column “Score”), we notice that Beam Search gradually decreases the rate at which model score gains are observed (Column “Gain”) even though the cost doubles at each row.

Original: For BATS, we observe that while the cost is extremely high initially (73.049 at 2 iterations), the cost increases at a linear rate with the number of iterations.
Revised: For BATS, we observe that while the cost is extremely high initially (73.049 at 2 iterations), the cost increases at a linear rate with the number of iterations.

Original: More importantly, we notice considerable gains even with high numbers of iterations (11.82 at 128 iterations).
Revised: More importantly, we notice considerable gains even with high numbers of iterations (11.82 at 128 iterations).

Original: Finally, at beam 256 , we notice that the cost of Beam
Revised: Finally, at beam 256 , we notice that the cost of Beam

Original: Search is comparable to the cost of running 16 BATS iterations, which achieves similar model scores.
Revised: Search is comparable to the cost of running 16 BATS iterations, which achieves similar model scores.

Original: It is important to also refer that cost efﬁciency is not the goal of this work, but to expose the need for better decoders that are devoid of the beam search bias.
Revised: It is important to also refer that cost efﬁciency is not the goal of this work, but to expose the need for better decoders that are devoid of the beam search bias.

Original: Many improvements to MCTS-based methods can be made to improve efﬁciency, such as training value and policy networks iteratively (Silver et al., 2016). 
Revised: Many improvements to MCTS-based methods can be made to improve efﬁciency, such as training value and policy networks iteratively (Silver et al., 2016). [Table]

Original: A. E XAMPLE T RANSLATIONS AND S EARCH E RRORS
Revised: A. E XAMPLE T RANSLATIONS AND S EARCH E RRORS

Original: We compare the translated sentences using an a MRT tuned autoregressive model for Chinese-English, where BATS and beam search yield similar BLEU but where BATS achieves signiﬁcantly lower model scores.
Revised: We compare the translated sentences using an a MRT tuned autoregressive model for Chinese-English, where BATS and beam search yield similar BLEU but where BATS achieves signiﬁcantly lower model scores.

Original: Table 7 provides three example translations from the test set obtained using beamfor beam search (row “Beam Search”) and 256 iterations for BATS (row “BATS”), which is the setup that obtained optimal results in the validation set.
Revised: Table 7 provides three example translations from the test set obtained using beamfor beam search (row “Beam Search”) and 256 iterations for BATS (row “BATS”), which is the setup that obtained optimal results in the validation set.

Original: The ﬁrst example shows that beam search tends to prolong sentences by using longer expressions (“many” vs. “there are also many”) in order to get[Table] short term value gains, but lower overall score, also slightly shifting tone of the sentence.
Revised: The ﬁrst example shows that beam search tends to prolong sentences by using longer expressions (“many” vs. “there are also many”) in order to get short term value gains, but lower overall score, also slightly shifting tone of the sentence.

Original: Figureillustrates this issue, we observe that by using the expression “there are also many”, it delays the generation of the word “technological” for three timestamps, leading to the higher score of − 0 .(left path) compared to the alternative 1 . 20077 (right path) in the same timestamp.
Revised: Figureillustrates this issue, we observe that by using the expression “there are also many”, it delays the generation of the word “technological” for three timestamps, leading to the higher score of − 0 .(left path) compared to the alternative 1 . 20077 (right path) in the same timestamp.

Original: While this score regularizes to − 1 .
Revised: While this score regularizes to − 1 .

Original: 58682 once the word “technological” is generated, its likely that the alternative translation is pruned by beam search.
Revised: 58682 once the word “technological” is generated, its likely that the alternative translation is pruned by beam search.

Original: In the second example, we observe that beam search prefers to reorder the original sentence, so that higher scoring terms (“U.S. destroyer USS Decatur”) in the sentence are inserted ﬁrst.
Revised: In the second example, we observe that beam search prefers to reorder the original sentence, so that higher scoring terms (“U.S. destroyer USS Decatur”) in the sentence are inserted ﬁrst.

Original: However, as one can observe from the ﬁnal score, this decision is only favorable in the short term as the ﬁnal score of the sentence is substantially lower than the translation found using BATS, which respects the order of the original sentence.
Revised: However, as one can observe from the ﬁnal score, this decision is only favorable in the short term as the ﬁnal score of the sentence is substantially lower than the translation found using BATS, which respects the order of the original sentence.

Original: In the last sentence, we observe that in the ﬁnal portion of the translation, the decoder makes a set of individually high scoring decisions that lead to an ungrammatical translation as all grammatical options have been ﬁltered from the beam.
Revised: In the last sentence, we observe that in the ﬁnal portion of the translation, the decoder makes a set of individually high scoring decisions that lead to an ungrammatical translation as all grammatical options have been ﬁltered from the beam.

Original: there many ,
Revised: there many ,

Original: Similarly are also many -0.
Revised: Similarly are also many -0.

Original: technological root -0.
Revised: technological root -0.

Original: technological -0.
Revised: technological -0.

Original: entreprises that have -0.
Revised: entreprises that have -0.

Original: [Figure]
Revised: [Figure]

Original: s
Revised: s

Original: [Table]
Revised: [Table]

Original: x , y )
Revised: , y )

Original: Source
Revised: Source

Original: [Table]
Revised: [Table]

Original: Reference
Revised: Reference

Original: Similarly, many technology companies have encountered numerous infringements crisis in patent technology licensing and patent technology use.
Revised: Similarly, many technology companies have encountered numerous infringements crisis in patent technology licensing and patent technology use.

Original: Beam Search
Revised: Beam Search

Original: Similarly, there are also many technological enterprises that have encountered great infringing waves in the licensing of patented technology and the use of patented technology.
Revised: Similarly, there are also many technological enterprises that have encountered great infringing waves in the licensing of patented technology and the use of patented technology.

Original: BATS
Revised: BATS

Original: Similarly, many technological enterprises have encountered great infringing waves in the licensing of patented technology and the use of patented technology.
Revised: Similarly, many technological enterprises have encountered great infringing waves in the licensing of patented technology and the use of patented technology.

Original: Source
Revised: Source

Original: [Table]
Revised: [Table]

Revised: 国

Revised: [Table]

Revised: Gaven

Revised: [Table]

Revised: （ Johnson Reef

Revised: [Table]

Original: Reference
Revised: Reference

Original: According to the comprehensive foreign reports of the Central News
Revised: According to the comprehensive foreign reports of the Central News

Original: Agency, the U.S. ofﬁcial who requested anonymity revealed that the
Revised: Agency, the U.S. ofﬁcial who requested anonymity revealed that the

Original: United States Navy destroyer, USS Decatur, cruised into the 12 nautical mile territorial limit of Gaven Reef and Johnson Reef of the Nansha
Revised: United States Navy destroyer, USS Decatur, cruised into the 12 nautical mile territorial limit of Gaven Reef and Johnson Reef of the Nansha

Original: Islands.
Revised: Islands.

Original: Beam Search
Revised: Beam Search

Original: U.S. destroyer USS Decatur sailed into the Gaven Reef and Johnson
Revised: U.S. destroyer USS Decatur sailed into the Gaven Reef and Johnson

Original: Reef 12 nautical miles ( 12 nautical miles ) of the Southern Sand Islands, according to Central Intelligence Agency’s Comprehensive Outreach
Revised: Reef 12 nautical miles ( 12 nautical miles ) of the Southern Sand Islands, according to Central Intelligence Agency’s Comprehensive Outreach

Original: News on 30.
Revised: News on 30.

Original: BATS
Revised: BATS

Original: According to Central News Agency’s comprehensive external telecommunications report on 30 June, U.S. ofﬁcials who requested anonymity, the USS Decatur sailed into the Gaven Reef and Johnson Reef of the
Revised: According to Central News Agency’s comprehensive external telecommunications report on 30 June, U.S. ofﬁcials who requested anonymity, the USS Decatur sailed into the Gaven Reef and Johnson Reef of the

Original: Southern Sand Islands within 12 nautical miles.
Revised: Southern Sand Islands within 12 nautical miles.

Original: Source 意 大 利 疑 欧 派 政 府 的目 标 是 未来 三 年 预 算 赤 字 相 当 于 国 内 生 产 总 值 (GDP)
Revised: Source 意 大 利 疑 欧 派 政 府 的目 标 是 未来 三 年 预 算 赤 字 相 当 于 国 内 生 产 总 值 (GDP)

Original: 的 2.4 % ， 这 表 明 仅 管 面 临 减 赤 要 求
Revised: 的 2.4 % ， 这 表 明 仅 管 面 临 减 赤 要 求

Original: 仍 未有 债 务削 减
Revised: 仍 未有 债 务削 减

Original: Reference
Revised: Reference

Original: The aim of Italian Eurosceptic government was that the budget deﬁcit was equivalent to 2.4% of gross domestic product (GDP) in the next three years, suggesting that there was no debt reduction despite deﬁcit reduction requirements.
Revised: The aim of Italian Eurosceptic government was that the budget deﬁcit was equivalent to 2.4% of gross domestic product (GDP) in the next three years, suggesting that there was no debt reduction despite deﬁcit reduction requirements.

Original: Beam search
Revised: Beam search

Original: The Italian Euroskeptic government’s goal is to have a budget deﬁcit equivalent to 2.4% of gross domestic product over the next three years, suggesting that only facing deﬁcit reduction requirements has not yet been debt reduction.
Revised: The Italian Euroskeptic government’s goal is to have a budget deﬁcit equivalent to 2.4% of gross domestic product over the next three years, suggesting that only facing deﬁcit reduction requirements has not yet been debt reduction.

Original: BATS
Revised: BATS

Original: The Italian Euroskeptic government’s goal is to have a budget deﬁcit equivalent to 2.4% of gross domestic product over the next three years, indicating only that there is no debt reduction required to reduce the deﬁcit.
Revised: The Italian Euroskeptic government’s goal is to have a budget deﬁcit equivalent to 2.4% of gross domestic product over the next three years, indicating only that there is no debt reduction required to reduce the deﬁcit.