Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposed a new framework for training sequence labeling models to optimize reward metrics, and a CML loss to help the model better understand the reward space. The experimental results show superiority compared with recent baselines. 
strength_and_weaknesses: ### Pros
1. The proposed GROOT is simple, effective, and easy to implement.
2. The whole framework is clear.
3. The experimental results show great improvement in reward function and precision.

### Cons
1. The relationship and difference with *Shu++* should be discussed more clearly.
2. Only one machine translation baseline *Shu++* was compared. It will be more convincing if it is compared with more baselines.
3. Sufficient experiments and figures are provided, but there are no consistent performance gains across different datasets. 
4. There should be more descriptions of the datasets.
4. Some statements are difficult to read and may contain some grammatical errors, e.g, *One may assume that just the above may be sufﬁcient for models to be learn the output space sufﬁciently as done in prior work (Shu et al., 2021)* on page 4, and *Correct erroneous candidate* paragraph on page 4, and *Works by replace incorrect tags in y with the correct ground-truth tags* on page 6. The authors should pay more attention to the writing.
clarity,_quality,_novelty_and_reproducibility: - The clarification can be clearer.
- The proposed method could help to improve the performance of the sequence labeling task.
- This work should be original.
- The paper presents experimental details for reproducibility.
summary_of_the_review: I'd like to assign 5 (marginally below the acceptance threshold). I hope the authors could make the clarification clearer and do more analysis on the influences of different modules/hyperparameters.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes GROOT,  a novel and general reward optimization framework for training sequence labeling models toward reward metrics, via construct corrective candidates as contrastive examples to guide the model learning.
strength_and_weaknesses: Strengths:
* The paper is well-written and easy to understand.
* The proposed approach is novel and general.

Weaknesses:
* Although the authors use 4 datasets and include languages more than English, they don't provide details of their specific domains and tasks, and looks all of the 4 datasets are for semantic parsing task in the spoken language domain. As the paper mentions, the sequence labeling tasks range from NER to QA. Therefore, the experiments are too narrow to support their claims generally on top of sequence labeling tasks.
* As a paper on sequence labeling tasks, the authors don't include the basic solution, token classification, as one of the baselines. Although this paper focuses on the generative manner, the performance of token classification should be still considered as important reference.
* Moreover, the token classification can also be performed using the same sequence-to-sequence backbone model, just like how BART (https://arxiv.org/pdf/1910.13461.pdf) does in the QA task (SQuAD), which should be also considered as an important baseline.
clarity,_quality,_novelty_and_reproducibility: Quality: 3/5
Clarity: 5/5
Originality: 5/5

summary_of_the_review: Overall, the proposed method is novel and general, and paper is well-written, but the experiments cannot support the general claim because they lack task and domain diversity, and important baselines are missing.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: Seq2seq based is a popular approach for generative sequence labelling. However, the popoular log likelihood training doesn't always model well the loss each practical problem speicfies. 

In this paper, the authors first pose such a setting, and then proposes a framework (GROOT) that allows to incoporate those information in generative sequence modelling frameworks. Authors propose two novel techniques to solve this issue: 1) correction function which incrementally improves over the current predicion model 2) Margin loss that can be applied to leverage current predicitons, gold annotations as well as the candidates generated by correction function. Through empirical studies, the authors show that their method improves over baseline as well as recently proposed methods.
strength_and_weaknesses: Strength:
- The paper exposes a clear motivation and proposes novel correction function to better explore the space.
- The paper shows strong empirical results
- The paper has done quite thorough ablation studies showing the effectivenss of their methods as well being stable and not very senstible to various settings

Weakness:
- The paper exposes results in one particular reward function setting (section 4.1) which shows convincing results. However, to know that the proposed algorithm is effective, one should at least know the performance of other settings; it is mentioned in the paper that other settings yielded similar empirical findings, however I can not find those findings including appendix. Without this, I can't know if the paper has proposed an effective algorithm or not.
- (Question) I don't' understand why the paper proposes to incorporate these methods only once the model is fined tuned. The method is general and can be applied as a fine tuning method. Not sure why there is no experiment as well as no mentioning on this.

clarity,_quality,_novelty_and_reproducibility: The paper has clear motivation and well exposes their method. The empirical results as well as the ablation studies show convincing results for their approach. The way to explore more candidates seem normal to me, although the proposed loss has no particular novelty. The margion loss needs to be adjusted in the approach so potentially hinders reproducibility; nevertheless, the approach is interesting and does seem to solve some practical issues.
summary_of_the_review: For generative sequence labelling, authors propose to model reward to better guide seq2seq learning. Authors propose two novel techniques to solve this issue: 1) correction function which incrementally improves over the current predicion model 2) Margin loss that can be applied to leverage current predicitons, gold annotations as well as the candidates generated by correction function. Through empirical studies, the authors show that their method improves over baseline as well as recently proposed methods.

My main concern is that I can't find support that the algorithms authors propose generalize to more scenarios than the ones posed in 4.1.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper proposes GROOT, a new framework for training sequence labeling models to optimize reward metrics. The approach constructs corrective candidates as contrastive examples and uses a margin loss. Experiments show improved performance on several datasets in different domains. Though the work is well motivated and the method is interesting, the reviewers had concerns on the empirical studies, including lack of comparison with relevant baselines, lack of diversity of the evaluation datasets, missing details of the experiments, etc.
justification_for_why_not_higher_score: Reviewers had concerns on the empirical studies, including lack of comparison with relevant baselines, lack of diversity of the evaluation datasets, missing details of the experiments, etc.
justification_for_why_not_lower_score: The work is well motivated and the method is interesting and reasonably sound.
Decision: Reject