Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The traditional selective rationalization consists of one rationale generator and one predictor. This cooperative rationalization structure reveals a major problem, i.e., model interlocking. To tackle this problem, in this paper, the authors propose a new rationalization framework, which introduces a third component into the architecture, a predictor driven by soft attention as opposed to selection. The additional predictor enables the framework to exhibit a better convexity property. 
main_review: Strength:
1. The research topic is interesting. In the traditional rationalization framework, the predictor only sees what the generator selects during training, and tends to overfit the selection of the generator. The authors found that the fundamental cause of the problem of interlocking is that the rationalization objective is undesirably concave with respect to the rationale generator’s policy, which leads to many sub-optimal corner solutions.
2. The proposed A2R method is inspiring and effective. It combines the advantages of both the attention model (convexity) and binarized rationalization (faithfulness) into one. The proposed rationalization scheme can overcome the concavity of the original setup, and thus can avoid being trapped in sub-optimal rationales. 
3. The claims are well justified and proved, and the experiments are well designed.

Weakness:
1. The paper is not easy to follow for readers not familiar with selective rationalization and NLP. I suggest using more real-world examples to explain the problem, the challenges, and the advantages of the proposed framework.
2. The experimental improvement seems trivial in some of the designed scenarios. E.g., Apperance and Aroma in Table 3.
limitations_and_societal_impact: I donot think this work has potential negative societal impact.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The paper uncovers an issue in training rationale extraction models, which the authors call _interlocking dynamics_. When training a rationalizer, it turns out the optimization problem is concave, which means that in certain conditions the model can converge into a sub-optimal local minimum. When the model relies instead on a soft-attention as a rationalization, the problem is convex but the global minimum is not faithful. To solve this, A2R is proposed. A2R uses both hard-choice and soft attention rationales to obtain a better loss landscape. A2R is compared to other rationale extractors and obtains state-of-the-art in two datasets.
main_review: _Originality_: The idea presented is novel and well-defined in relation to related work. However, I think [this paper](https://arxiv.org/pdf/2004.13876.pdf) should be cited (see comments below for more discussion about this).

_Quality_: This is a great submission. The contributions are technically sound and the experiments support them. The ablation shown in Table 2 is very interesting. I found no critical weakness to recommend rejection for this paper.

_Clarity_: This paper is very well-written. Explanations are thorough and it is well-organized.

_Significance_: The paper reveals an issue in the training of most rationale extraction models that the community should be aware of. The model proposed aims to solve the issue and advances the state-of-the-art on two datasets.

__Typos__

- Line 36 and 74: soft-max --> softmax
- Line 36: "introducing additional component to" --> introducing an additional component to (...) / introducing additional components to (...)
- Line 62: "During the training" --> During training

__Comments__

How do you think the [paper mentioned above](https://arxiv.org/pdf/2004.13876.pdf) handles the unfaithfulness of the attention-based models? Since the attention is sparse, I would expect this solution can lie near the grey line in your Figure 1c.
limitations_and_societal_impact: The authors have adequately addressed the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper analysis the problem of interlocking in cooperative rationalization, provide both theoretical and practical analysis for this problem, and derive a potential way to solve it:
- The paper first shows theoretically, that, when formulating selective rationalization as a two player cooperative game, where a rationale generator generates a mask, and a predictor uses the masked input to predict the output, a problem of interlocking can happen if both the predictor and the rationale generator initially have a correlated (but potentially invalid) bias as to which part of the input best explains the output. In such cases, as it has been mostly train on one, potentially suboptimal, subpart of the input, the predictor will incur greater errors when using other (potentially better) parts of the inputs, which, in turn, will reinforce the generator in selecting the suboptimal subpart. The paper analyses this problem theoretically, by showing that the prediction loss provided optimal predictor is concave in the policy, meaning that some vertices of the simplex might be local optima.
- In turn, the authors show that when using soft-attention as an explanation scheme instead of hard attention provides a convex prediction loss provided optimal predictor, but loses the property of being faithful.
- The authors propose a methods that combine the soft-attention and hard-attention explanation scheme to better optimize the hard attention one, by training both an attention predictor and a rationale predictor, using the attention predictor to train the rationale generator, while regularizing the attention predictor to keep it close to the rationale predictor.
- The authors validate the existence of the interlocking problem on a toy dataset (by manually favoring one of the subparts of the inputs by hand), and show that there approach partly alleviate interlocking. They proceed to demonstrate that their approach achieves state of the art on real world datasets.
main_review: Disclaimer: I am quite unfamiliar with the cooperative rationalization literature, and thus can hardly compare this work to potential related papers.

That being said, I found this paper to be quite an interesting read. The presentation is overall very clear:
- The general framework of Cooperative Rationalization is clearly formulated.
- A simple yet effective intuition of the interlocking problem is provided, and interlocking seems like a natural hypothesis to explain failure modes of cooperative rationalization.
- The subsequent formalization of interlocking in term of concavity of the prediction loss in the policy, provided optimal predictor, is natural. The derivations are, to the best of my understanding, correct.
- The proposed solution which boths regularize the soft-attention predictor toward the hard-attention one, while using the soft-attention predictor to train the generator is quite interesting.
- The experimental validation of the existence of the interlocking problem is convincing, as well as the more real world experiments.

My only concern is with the dependency of the algorithm on the \lambda parameter. One of the effects of lambda is to trade how much the soft-attention model is going to be regularized by the hard-attention model. With a low value of lambda, one falls back to the interlocking problem occurring with pure hard-attention models. With a high value of lambda, the attention model is only loosely regularized by the hard attention model, and the generator is then incited to pick sub-optimal rationales, since those rationale provide better explanation in a soft-attention setting, but not in a hard-attention one.
- As using a bad value of lambda could lead to potentially bad results, would it be possible to provide an analysis of the sensitivity of results to lambda?
- In the same vein, interlocking seems to be susceptible to happen mostly at the beginning of learning. Would it make sense to progressively anneal lambda (and flow gradients from the rationale predictor into the generator), to avoid convergence to a suboptimal solution?

# Post rebuttal edit:
The authors provided the additional ablation on lambda that I was interested in, and the results seems reasonably robust to a large range of lambdas. I am consequently raising my score from 6 to 7, given the clarity and interest of the proposed analysis and of the provided experimental results.
limitations_and_societal_impact: Both limitations and societal impacts are adequately addressed.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper describes "interlocking dynamics" that lead to local optima in selective rationalization techniques. The idea is simple: if the rational generator selects suboptimal features, you risk the predictor overfitting to these features, thereby removing a path to the optimal solution (since the generator cannot improve the predictor's loss by selecting better features if the predictor has overfit to suboptimal features). The paper builds intuition through a simple coordination game example, and describes sufficient conditions for their attention-augmented approach to overcome the coordination problem.
main_review: The paper presents well-written description of an intuitive problem with nice simple examples to make the problem clear, but it's let down by the description of their solution. The toy example (section 3.2) and theorems (section 3.3) do a good job of explaining what can go wrong in selective rationalization approaches, and these are supported with synthetic examples (section 5.3) that show that these issues can occur in (sufficiently adverse) practical settings, but then there is very little in the presentation of their method that explains why their solution achieves the desired loss landscape shown in figure 1 (c), or why that particular loss was chosen?

It seems like with different choices of $\lambda$, you could interpolate between the situation in fig 1 (a) and (b) and the paper provides no guidance on how to choose $\lambda$? The explanation given in section 4.3 says, 

>If the $\lambda$ in Equation (12) is zero, then the loss for the rationale generator would be exactly the lowest curve. As $\lambda$ increases, the attention-based loss curve would shift upward towards the rationale-based loss. As a result, the actual loss curve for the generator will resemble the gray curve in the middle [of fig 1 (c)], which addresses the concavity problem and thus the interlocking problem, without introducing unfaithful solutions.

But the conclusion that "the actual loss curve for the generator will resemble the gray curve in the middle" is not supported by the the experiments in 5.3 which clearly show that their method is still sensitive to interlocking under sufficient skew. Of course, this is unsurprising given that there's no a priori method for selecting $\lambda$, but it shouldn't be presented as though it solves the problem.

Additionally:
 - I would have like to see more discussion on why the particular loss was chosen: why do we need to constrain the two predictors to output the same distribution? Why do we use JS divergence to enforce this (and not some other metric / divergence)? 
 - The discussion on the feasibility of the assumptions for theorem 2 should really be in the main text. Having read that discussion, it seems like theorem 2 is more a statement about nice conditions under which $\mathcal{L}_a$ is convex, rather than an argument that it is generally convex. E.g. you state "as long as the strong convexity of the loss function dominates the non-convexity of $f_a$"- fine, but we have no way of checking that. 
limitations_and_societal_impact: I've described the limitations of the work above; I have no concerns about societal impact - generally methods such as these make machine learning more transparent which is positive.
review_title: null

Meta-review: This paper addresses the problem of selective rationalization, revealing that the usual cooperative rationalization paradigm suffers from model interlocking — this arises when the predictor overfits to the features selected by the generator thus reinforcing the generator's selection even if the selected rationales are sub-optimal. To sidestep this, the paper proposes a new rationalization framework (A2R), which introduces a third component into the architecture, a predictor driven by soft attention as opposed to selection. Experiments on two synthetic benchmarks and two real datasets demonstrate that A2R can significantly alleviate the interlock problem.

This is a solid paper that proposes a simple strategy to solve an important practical problem with rationalizers. While reviewers pointed out some weaknesses (lack of clarity and need for better motivation, no sensitivity analysis for the lambda coefficient), some of these concerns have been successfully alleviated in the rebuttal, with new results for dependency on lambda. Therefore I recommend acceptance. I urge the authors to take into account the reviewers’ comments when preparing the final version. 
Decision: Accept (Poster)