Venue: ICLR 2017 conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes a method for iteratively improving the output of an existing machine translation by identifying potential mistakes and proposing a substitution, in this case using an attention-based model.  It is motivated by the method in which (it is assumed) human translators operate.

The paper is interesting and imaginative.  However, in general terms, I am somewhat sceptical of this kind of approach -- whereby a machine learning method is used to identify and correct the predictions of another method, or itself -- because in the first case, if the new method is better, why not use it from the outset in place of the other method?  And in the second case, since the method has no new information compared to previously, why is it more likely to identify more past mistakes and correct them, than identify past correct terms and turn them into new errors?  That is unless there is a specific reason that an iterative approach can be shown to converge to a better solution when run over several epochs.

This paper does not convince me on these points.  Indeed, unsurprisingly, the authors note that "the probability of correctly labelling a word as a mistake remains low (62%)" - this admittedly beats a random-chance baseline, but is not compared to something more meaningful, such as simply contrasting the existing system with a more powerful convolutional model and labelling all discrepancies as mistakes.  The oracle experiments are rather meaningless - they just serve to confirm that improving a translation is very easy when the existing mistakes have been identified, but much harder when they are not. 

Although I do like the paper on the whole, to really convince me that main objective -- ie. that **iterative** improvement is beneficial -- has been satifactorily demonstrated it would be necessary to include stronger baselines - and in particular, to show that an iterative refinement scheme can really improve over a system closely matched to the attention-based model, both when used in isolation and when used in system combination with a PBMT system, and to demonstrate that the PBMT system is not simply acting as a regulariser for the attention-based model.

Minor comments:

I find the notation excessively fiddly at times - eg F^i = (F^{i,1}, F^{i,|F^i|}) - why use |F^i| here when F is a matrix, so surely the length of the slice is not dependent on i?

In the discussion in section 4 - it seems that this still creates a mismatch between the training and test conditions - could anything be done about this?








 
review_title: Iterative translation improvement motivated by humans - interesting idea, but not fully convincing...

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes a model for iteratively refining translation hypotheses. This has several benefits, including enabling the translation model to condition not only on “left context”, but also on “right context”, and potentially enabling more rapid and/or accurate decoding. The motivation given is that often translators (and text generators generally) use a process of refinement in generating outputs.

This is an important idea that is not currently playing much of a role in neural net models, so this paper is a welcome contribution. However, while I think this is an important first step, I do feel that the lack of in depth analysis suggests this paper is not quite ready for a final publication version. For example, there are many possible connections to prior work in NLP, MT, and other parts of ML that could better contextualize this work (see specifics below). More substantively, the model in Section 3 could be interpreted as a globally normalized, undirected (~CRF) translation model trained using a pseudo-likelihood objective. In this analysis, the model squarely back in the context of traditional discriminative translation models which used “undirected” features, and the decoding algorithm then looks more like a standard greedy hill-climbing algorithm (albeit with an extra heuristic model for selecting which variable to update), which is also nothing unfamiliar.

My second criticism the limitations of the model are not well discussed. For example, the proposed editing procedure cannot obviously remove or insert a word from a translation. While I think this is a reasonable assumption than can be made for the sake of tractability, it is very unfortunate since missing or extra words (esp. function words) are a common problem in the baseline models that are being used. Second, the standard objections to absolute positional models (vs. relative positional models) seem particularly crucial to bring up in this work, especially since they might make some of the design decisions a bit more justifiable.

Overall, this is an initial step in an interesting direction, but it needs more thorough analysis to demonstrate its value. A more thorough analysis will also likely suggest some important model variants (for example: is a global translation model really the goal? or is a post-editing model that fixes outputs with more complex operations more ideal?)

Related work:
I think that more could be done to put this work in the context of what has come before and what is currently going on in other parts of ML. The idea of iterative refinement has been proposed in other problems that have complex output spaces, for example the DRAW model of Gregor et al. and the conditional adversarial network models used to refine images proposed recently by Isola et al. In NLP, there have been several (stochastic) hill climbing approaches that have been proposed, such as the work on parsing by Zhang and Lei et al. (2014) who use random initial guesses and then do greedy hill climbing using a series of local refinements, the structured prediction cascades of Weiss and Taskar (2009) (not to mention general coarse-to-fine modeling strategies). Finally, in MT, Arun et al. (2009) who use a Gibbs sampler to refine an initial guess to do decoding with a more complex model. The use of an explicit error model is rather novel in the context of correction, but I would point out that although the proposed architecture is different, the discriminative word lexicon models of Mauser et al. (2009) and the neural version of the same by Ha et al. (2014) are similar in spirit. There have also been a number of papers on “automatic post editing”, including the shared task at WMT2016, and there are not only standard test sets and baselines, but also datasets that could actually be used to train a post editing model with human-generated data. Minimally using the techniques they described could be a useful foil for the models presented in this paper.

“the target sentence is also embedded in distributional space via a lookup table” I think “distributional space” is a bit unclear. Maybe “the target sentence is represented in terms of distributed word representations via a lookup table” or something like that. “distributional” suggests that the representations are derived from how the words are distributed in the corpus, whereas you are learning these representations on this task which isn’t modeling their distribution except only very indirectly.

Section 3 Model: In Section 3, the model computes the distribution over target word types at an absolute position i in the output sentence, given the target language context and the source language context. It is introduced as the model that is used to refine an existing hypothesis, but it is not immediately clear that the training data for this model (at least in this section) are the gold standard translations- “training set” could be interpreted in variety of ways. This becomes clearer when reading later in the paper, but it’s a bit less clear when reading from the beginning for the first time.

The use of a fixed sized window for representing the target word in context also seems to make something like a model 1 assumption since only the lexical features (and not any “alignment” or “positional” features) determine the attention. This should be clarified since it will make the assumptions of the model more transparent (and also suggest possible refinements to the model, e.g., including (representations) of i and j as components of S^j and T^i, which would allow model 2/3-like responses to be learned- although by leaving them out, the model might behave a bit more like a relative positional model than an absolute positional model, which is probably attractive).

Finally, some discussion for why a fixed window is used to represent the target sentence is worth including (since a global context is apparently used to represent the source sentence).

The relationship between this training objective and pseudo likelihood (PL; Besag, 1975) might be worth mentioning. Since I believe this is just a PL objective for a certain global model, this suggests alternative decoding algorithms, or certainly a different analysis of the proposed decoding objective.

The section 4 model conditions on the true context of a position in the true target, the current target guess, and the source. I don’t completely understand the rationale for this model since at test time only two of these variables are available, and the replacement of y_ref with y_g seems hard to justify.
review_title: iterative refinement is an important modeling and inference idea

final_score: rating: 4: Ok but not good enough - rejection
review_content: This work proposes to iteratively improve a sentence that has been generated from another MT system (in this case, a phrase-based system). The authors use a neural net that takes in the source sentence and a window of (gold) words around the current target word, and predicts the current target word. During testing, the gold words are replaced with the generated words. While this is an interesting area of research, I am not convinced by the proposed approach, and experimental evidence is lacking.

Under the current framework, it is all but impossible for the model to do anything more than a rudimentary word replacement (e.g. it cannot change "I went to the fridge even though I was not hungry" to "Although I was not hungry, I went to the fridge"). The fact that only 0.6 words are edited on average supports this. 

Specific comments:
- It would be interesting to see what the improvements are if the baseline model is a neural system.
- It seems strange (to me at least) that T^i and L(y^{-i|k}) only look at a window of 2k words. It means that when making the decision to change the i-th word, the model does not know what was generated outside of the window? 
- Relatedly, the idea of changing individual words based on local (i.e. word-level) scores seems counterintuitive. Given that we have the full generated sentence, don't we want a global score? Scoring at the sentence-level could also make room for non-greedy search strategies, which could potentially facilitate richer edits.
- How does the approach compare to a model that simply re-ranks the k-best output?
- Instead of editing, did you consider learning an encoder-decoder that takes in x, y_g, and generates y_ref? When decoding you can attend to both x and y_g.

Minor comments:
- Iteratively improving a generated text was also explored in https://arxiv.org/pdf/1510.09202v1.pdf from a reinforcement learning angle.
- I don't understand footnote 1.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: Disclosure: I am not an expert in machine translation algorithms.

Summary: A human translator does not come up with the final translation right
away. Instead, (s)he uses an iterative process, starting with a rough draft
which is corrected little by little. The idea behind this paper is to
implement a similar framework for an automated system. 

This paper is generally well written. 

It is my opinion however that drawings illustrating the architectures would help
understanding how the different algorithms relate to one another.

I like a lot that you report on a preliminary experiment to give an
intuition of how difficult the task is. You should highlight the links
between the task of finding the errors in a guess translation and the task
of iterative refinement. Could you use post-edited text to have a more
solid ground-truth?

My main concern with this paper is that in the experimental section the 
iterative approach tries to improve upon only one type of machine translation. 
Which immediately prompts these questions:
- why did they choose that approach to improve on?
- what is the part of the improvement that comes from the choice of the
  initial draft (maybe it was a very bad draft)? 

Here are some minor typos:
- p.2: ... a lookup table that replace*S* each word... ?
- p.3: I might be mistanken but it seems to me that j is used for two
  different things. It is confusing.
- p.3: ...takes as input these representation*S* and outputs... ?

review_title: null

Meta-review: The paper is an interesting first step, but the reviewers found flaws in the overall argument. Further, the method is not contextualized well enough in relation to prior work.
Decision: Reject