Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a novel approach [CASR] for sequence generation where multiple language models are connected in an iterative manner and predictions of a previous timestep are taken as an input to the model at future timestep [different from vanilla RNN]. The rationale is that sequence generation problems only specify “difficult” FINAL sequences which need to be solved, and the intermediate order to solve “simpler sequences” is defined by heuristics [Tan et Al]. 


strength_and_weaknesses: **Strengths**

- The authors demonstrate that a network can itself learn the simpler sequences it needs to solve, and that there is a natural order of progression from solving “simpler” sequences to complex ones. Furthermore, they empirically reveal that this progression is different from the simple left-to-right solving order that sequence generation usually takes. 

- The manuscript is well crafted, with convincing experiments and detailed ablations. Authors claim SOTA [70%] on KVRET F1 metric. 

**Weaknesses**

- The basic training process [Algo 2] has a limitation that the language model at each timestep needs to be trained fully [line 3] before moving on to the next timestep. This prevents the network to be E2E trainable, leaving CASR prone to parameter-effficient tuning. [3.3].

- Authors define the notion of density measuring the attention magnitude at each sequence token [5.2]. Increasing density at each recurrent step, seems to show that the prediction made by the model becomes more informative. I am curious as to whether the density becomes maximum when the network converges to optimal solution. Please clarify this.

- On sudoku, the restart strategy lags behind Continue strategy by a huge difference ~20% (table 4). But we don’t observe this trend in other three datasets, well recognized in the community (Table 3).  Could the authors please mention their intuitions on these observations?

clarity,_quality,_novelty_and_reproducibility: Authors have released the code for this work anonymously. 
summary_of_the_review: This work makes a SINGLE minor change to the mathematical formulation of Tan et Al [Figure 1], but still improves empirical results on only 1 out of 3 datasets studied. [Sudoku is a toy dataset from Kaggle which authors don’t show other methods against]. Due to lack of novelty in technical contribution, but still beating SOTA on 1 dataset, this work has incremental contributions.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper deals with tasks where left-to-right generation is not the best strategy. The authors propose a refinement framework named CASR – which models p(i-th-timestep output token at t-th refinement step | input to task, output of model at the (t-1)-th refinement step, the prefix/history of output tokens at t-th refinement step). So when training CASR, the input contains the models’ predictions at the previous refinement step as well as the input to the task, and the output is the reference answer. At test-time, iterative refinement is applied. 

Experiments are done on WebQSP, MTOP, KVRET, Sudoku (see Section 2.2 for details). The authors have also done ablation studies on architecture choices (e.g., how to encode the input to the CASR model, how to initialize weights of the model at each refinement step, fine-tuning vs. using adapters, etc.). CASR is effective, and achieves SoTA results on KVRET. 

strength_and_weaknesses: The comparison of the authors’ approach vs. iterative refinement and progressive generation is quite clear in Table 1. The table helps understanding greatly. 

Ablation studies are quite comprehensive and inspiring. The analysis on attention is interesting. 

Reproducibility is great, given that the code is provided. 



I’m not convinced at why using the previous refinement round’s output as input to the CASR model is necessary. Experiments on iterative refinement using a regular refinement model would be helpful. Regular refinement: train regular refinement model (inputs: generated output from original model; outputs: reference output); then, do one-round refinement after the original model and use these outputs as the input to the refinement model; then, do one-round refinement using this updated model (after the original model), and use these outputs as input to the refinement model; etc. Please correct me if I’m missing important details. (*)

The second paragraph on page 2 (starting with “however”) is lacking. The authors believe that if there are right-side dependencies “which together lead to multi-hop dependency chains,” then “left-to-right is not the best order for generation.” There needs to be citation or empirical proof. First, left-to-right generation with multiple refinement steps could lead to good results. Second, if we have a perfect model that scores each sequence, and a good decoding strategy (e.g., sampling many candidates and then reranking), then left-to-right generation can solve the tasks like Sudoku and WebQSP. The issue may be that our model is not perfect at scoring sequences, so we may need to rely on other strategies. Anyhow, the authors’ writing that “because there are multi-hop dependency chains, left-to-right is not the best order” should be more nuanced and in depth. 

Related to the above: Have the authors attempted better decoding strategies like sampling 100 candidates and rerank, or beam search with large beam size? It’s not clear whether the authors have definite proof that left-to-right decoding doesn’t work and refinement is necessary. (*)

clarity,_quality,_novelty_and_reproducibility: Line 5 of the third paragraph on page 2: Sun et al. should be cited using citet instead of citep.

The authors use Wu et al. (2016) as L2R’s citation, which is a bit weird because left-to-right generation existed many decades ago (e.g., Hochreiter, 1998). 

What do the authors mean by “additional exposure bias” at the end of the fourth paragraph on page 2? Additional to what? 

It’s not clear how hard/easy is defined on the second line of the first paragraph of page 3.   


Reproducibility is great, given that the code is provided. 
summary_of_the_review: The idea is interesting. Results on the chosen tasks (though quite specific) are great. Ablation studies are comprehensive and inspiring. I would want to see two more baselines -- labeled (*) above; or have the authors justify why the two baselines are not necessary. Currently I'm leaning toward weak accept.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes CASR: a framework that generates complex sequences by iteratively editing previously generated sequences. The problem is well-motivated: in complex scenarios, we might need multiple rounds of editing to generate correct sequences. CASR achieves this goal by feeding a transformer model with input X and prediction from last iteration. The model achieves good performance over both conventional AR and NAR baselines. Compared to template-based methods, CSAR is more flexible and avoids additional exposure bias 
strength_and_weaknesses: Strength:

1. Well-motivated problem and clear presentation.
2. The proposed CSAR is effective compared to vanilla AR and NAR models.
3. The evaluation and analysis are somewhat comprehensive. 

Weakness:

The major concern I have regarding this submission is that some critical related works are missing.

In particular, this line of related work, according to my knowledge, started from Deliberation Networks 2016[1], which is similar to CSAR except that it only focuses on the decoder (the previous prediction will go through the decoder multiple times to be refined). Another recent model that I found similar to CSAR is Draft and Edit 2020 [2], which is basically a CASR with T=2, and uses VAE as the backbone. Many other similar models can be found in Deliberation Networks' citation. 
In short, I believe this line of research is highly correlated with this submission and has to be discussed. I can see that there are still differences between CSAR and these works, but those have to be thoroughly discussed in the paper. 

Furthermore, although the existing evaluation is already quite comprehensive, compared to related works above is necessary. Such that we can perceive what is the advantage of CSAR over other iterative models. 
 
clarity,_quality,_novelty_and_reproducibility: The paper is clearly written.
The evaluation is solid.

The major novelty of CASR, as discussed in the Introduction is 1) we no longer need templates/rules to generate intermediate sequences 2) we will not introduce exposure bias because of (1).
However, if we consider those iterative methods(see Weakness), they have those advantages too. In terms of model structure, CASR is also quite similar to previous works. From this perspective, this submission's novelty is somewhat marginal. 
summary_of_the_review: Given those discussed in Weakness and Novelty, I think this submission is not ready for acceptance yet, but I am happy to raise my score if my concerns are addressed.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The reviewer consensus is to accept this paper. The paper is well-written and addresses an important problem. It is well-placed in the context of previous works on complex sequence generation. In the proposed CASR method, model-generated intermediate outputs are fed into the next step. For training, the model generated (possibly incorrect) output is fed into the next step, resulting in a matching training/inference procedure. Reviewers found the experimental results to be convincing, with a mix of real world and the Sudoku task especially suitable for this method.

Reviewers liked that the source code is available for their review as well.
justification_for_why_not_higher_score: no reviewers felt this was an especially strong paper. Given that the differences are usually not huge (except for Sudoku) and the tasks are not very competitive. 
justification_for_why_not_lower_score: Reviewer consensus
Decision: Accept: poster