Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: This paper introduces an encoder-decoder as a differentiable loss function for sequential autoregressive generation tasks and more specifically for summarization. 
This is done by adding a recorder network that that takes the decoded sequence from the summarizer as input and is trained to output the reference summary.

I see a fundamental issue with this work:

* During inference, authors decode from the probability distribution of the seq2seq model using beam search. 
* But for training (original seq2seq + recorder) authors backpropagate the NLL loss (which is fully differentiable) of the recorder on reference summaries through the softmax probabilities of outputs from the seq2seq model. 

>> This whole architecture can be seen as a traditional end-to-end seq2seq model with non-linearity and normalization (softmax) in the middle. 

Additionally: 
>> "backpropagating through the softmax weights during training and using the argmax during inference" falls into a long line of work for propagating non-differential objective functions through continuous relaxations 
 of categorical latent variables, more specifically the "straight through"  and "gumbel-softmax" (see refs.)
These methods have proven to be a strong alternative to reinforcement learning to train non-differential objectives and have been implemented quite a lot for sequence generation mainly for SeqGANs and even for text summarization connections to this line of work must be established in this paper. 

references 
- Estimating or propagating gradients through stochastic ´neurons for conditional computation. 
Bengio et al. 2013 arXiv preprint arXiv:1308.3432, 2013.
- CATEGORICAL REPARAMETERIZATION WITH GUMBEL-SOFTMAX
Jang et al. 2018 https://arxiv.org/pdf/1611.01144.pdf
- GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
https://arxiv.org/pdf/1810.05739.pdf
- Learning to Ask Questions in Open-domain Conversational Systems with Typed Decoders
https://arxiv.org/pdf/1805.04843.pdf
- MeanSum : A Neural Model for Unsupervised Multi-Document Abstractive Summarization
https://arxiv.org/pdf/1810.05739.pdf





review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: This paper proposed an encoder-decoder-based summarization network as a loss function within a similar encoder-decoder-based summarization framework to demonstrate that the proposed model obtains better automatic and human evaluation scores compared to the baseline model of See et al. (2017) with just traditional loss functions. Overall, the paper is well-written and the presented results, analyses, and comparisons appear to be reasonable. One notable advantage of the proposed model would be to circumvent the approaches that rely on the evaluation metric, ROUGE as a reward component e.g. in a reinforcement learning setting, although with the expense of additional memory and time complexity.   

Few comments:

- "A presents either a word ...." --> this sentence is not clear.

- "Embedded representations ... differ somewhat from w_i"--> Please clarify this aspect with more details.

- In Figure 1, the proposed model with recoder seems to be suffering from issues related to redundancy and referential clarity, as it repeats the name "malia" several times. Would you comment on why this is the case?

- It would be great if you could provide more details on the selection criteria/qualifications of the mechanical turk workers. Also, it is not clear why each example was given to only one worker and not to multiple workers. Wouldn't it be ideal to evaluate each example by multiple workers to get a sense of the inter-rater agreement? Please clarify. 
review_title: Official Blind Review #1

final_score: rating: 1: Reject
review_content: This paper proposes to use an additional component to the commonly used encoder-decoder approach for summarization, which is referred to as the recoder, which is an RNN-syle component that takes the output of the decoder. The intuition offered in the paper is that a good summary should produce itself via the recoder network, and in training it together with the original encoder-decoder it should improve its performance as it would be able to capture more than what the word-level loss does.

I have the following objections to this paper:
- I can't see why this extra component should improve the quality of the summary produced. If say our encoder-decoder architecture models the training data perfectly, then a recoder that does not do anything would be the right choice. Taking this further, a recoder could actually be fixing problems in the output of the decoder, and thus not providing a good training signal. It doesn't make sense to my that a summary should produce itself via a neural network, unless we are training an auto-encoder. The experiments validate this; the difference in ROUGE score are less than a point, which is the kind of fluctuation one expects due to random seed differences, etc.

- I find it odd that ROUGE score is dismissed as a loss to train against (referred to as a heuristic in the end of section 4.1), but then 1 ROUGE point difference is considered a "significant" improvement (making such claims without statistical significance testing is misleading). Sure it has flaws, if there is something better why not use it for evaluation? Furthermore, claiming that the recoder does a better job requires evidence. Why not train this extra function against human judgements? Assuming that what is wanted is to train a model that estimates the quality of the summary, it would make sense to look at the approaches used for the similar task of machine translation quality estimation: https://www.mitpressjournals.org/doi/pdf/10.1162/tacl_a_00056

- The approach is in my view a kind of actor-critic approach; the recoder could play that role, and in fact the Bahdanau et al. 2016 paper cited does this. However no comparison is offered, be it theoretical or experimental. Furthermore, the criticism that sequence-level training requires differentiable losses is incorrect; MIXER for example does train against scores such as ROUGE that are not differentiable. Furthermore, the BSO approach by Wiseman and Rush (2016) cited does give a continuous output to optimize for seq2seq that is asked for in the beginning of page 4.

- In the experiments, how was the length penalty determined to be the graduated curve mentioned in section 5? I would expect to have comparison against other approaches that try to train encoder-decoder to improve summarization, such as those mentioned in section 4.1.

- On the human evaluation experiments: the difference between the two models is quite small, especially given that the workers were not allowed to say that the models were equally good/bad. Furthermore, there is no inter-annotator agreement, as each comparison was done by a single crowd worker. Showing only the first 400 tokens of the original document would incorrectly disadvantage models selecting content from later in the articles. Finally, showing the reference summary creates another bias, since equally good summaries can disagree on what content to include. It might be helpful to look at some recent work on manual evaluation of summarization that tried to address these issues: https://arxiv.org/abs/1906.01361


review_title: Official Blind Review #2

Meta-review: This paper presents an encoder-decoder based architecture to generate summaries. The real contribution of the paper is to use  a recoder matrix which takes the output from an existing encoder-decoder network and tries to generate the reference summary again. The output here is basically the softmax layer produced by the first encoder-decoder network which then goes through a feed-forward layer before being fed as embeddings into the recoder. So, since there is no discretization, the whole model can be trained jointly. (the original loss of the first encoder-decoder model is used as well anyway).

I agree with the reviewers here, that this whole model can in fact be viewed as a large encoder-decoder model, its not really clear where the improvements come from. Can you just increase the number of parameters of the original encoder-decoder model and see if it performs as good as the encoder-decoder + recoder? The paper also does not achieve SOTA on the task as there are other RL based papers which have been shown to perform better, so the choice of the recorder model is also not empirically justified. I recommend rejection of the paper in its current form.
Decision: Reject