Paper: Towards Faithful Explanations: Boosting Rationalization with Shortcuts Discovery (ICLR-2024)

Reviewer comment (Writing):
the clarity of the paper could be improved, especially section 3

Author response:
> **Comment3**: the clarity of the paper could be improved, especially section 3

Thank you for your suggestion, we will clarify the details of the method again. Besides, we have added a illustration of the algorithm in the revised version (Appendix A.1 and A.2).

**Injecting Shortcuts into Prediction.**

Based on the identified shortcuts, we hope that the model can disentangle the shortcuts features from the input ones during training, thus alleviating the problem of using shortcuts in the data for task prediction.Specifically.

1.The predictor $q_{\psi}(y|x)$ adopts the input to yield task results.

2.We employ predictor $q_{\psi}(y|z_{s})$ adopts the shortcut tokens to algin a uniform class distribution:

$L_{unif}=E_{\substack{ z_{s}  \sim D_{sup}}}\left[\operatorname{KL}\left( \mathcal{U}(0, |N|) \|  q_{\psi}(y|z_{s})\right)\right],$

where KL represents the Kullback–Leibler divergence, $|N|$ is the total number of classes, and $\mathcal{U}(0, |N|)$ denotes the uniform class distribution.

Based on the original loss  $L_{sup}$,  we add a “uniform” constraint to ensure the predictor $q_{\psi}(\cdot)$ identifies the shortcuts features as meaningless features and disentangles the shortcuts features from the input ones.

3.The learned features information in  the  supervised phase can be transferred into the unsupervised phase method through the shared encoder.

4.We denote SSR with this  strategy as  ${SSR}_{unif}$, and

the objective of  ${SSR}_{unif}$ can be defined as the sum of the losses:

$L_{ssr_{unif}} = L_{un} + L_{sup} + \lambda_{unif} L_{unif}$.

**Virtual Shortcuts Representations.** 

Since it is difficult for us to annotate the rationales with $D_{un}$ and further obtain shortcut tokens to improve the performance, we propose a virtual shortcuts representations strategy (${SSR}\_{virt}$) with transferred shortcuts knowledge from $\mathcal{D}_{sup}$ as guidance.

Specifically, ${SSR}\_{virt}$ contains two phases (i.e., the supervised and unsupervised phase).

In the supervised phase:

1.We first adopt another predictor $q_{\eta}(y|z_{s})$ to predict task results based on the shortcuts $z_{s}$, and ensure the encoder $f_{p_{\eta}}(\cdot)$ in $q_{\eta}$ captures sufficient shortcuts representations $f_{p_{\eta}}(z_{s}) \in \mathbb{R}^{d}$ by minimizing :

​    $L_{s} = E_{\substack{z_{s} \sim D_{sup}}}\left[-\log q_{\eta}(y|z_{s})\right] = E_{\substack{z_{s} \sim D_{sup}}}\left[-\log softmax(W_{\eta}f_{p_{\eta}}(z_{s}))\right].$

2.Then, we employ an additional shortcut imitator $f_{a}(x_{sup})$ which takes $x$ in $D_{sup}$ as the input (denoted by $x_{sup}$ for clarity) to align and mimic $f_{p_{\eta}}(z_{s})$ by minimizing the squared euclidean distance of these two representations:

​    $L_{virt}=E_{\substack{ x_{sup},z_{s}  \sim D_{sup}}}\left[ \left\|f_{p_{\eta}}(z_{s})-f_{a}(x_{sup})\right\|^{2} \right],$

where $f_{a}(\cdot)$ and $f_{p_{\eta}}(\cdot)$ share~parameters:

In the unsupervised phase:

1.We keep $f_{a}(\cdot)$ frozen and employ it to generate virtual shortcuts representations $f_{a}(x_{un})$ by taking $x$ in $\mathcal{D}_{un}$ as the input.

2.After that, to encourage the model to remove the effect of shortcuts on task predictions, we  adopt $f_{a}(x_{un})$ to match a uniform distribution by calculating:

$L_{diff} =  E_{\substack{ x \sim D_{un}}}\left[\operatorname{KL}\left( \mathcal{U}(0, |N|)  \| q_{\sigma}(y|x_{un}) \right)\right],$

where $q_{\sigma}(y|x_{un}) = softmax(W_{a}f_{a}(x_{un}))$.

3.We set $W_{a}$ and $W_{p}$ share parameters to transfer the shortcut information into the predictor $f_{p}$. 

Formally, the final objective of ${SSR}\_{virt}$ is $L_{{SSR}\_{virt}} = L_{un} + L_{sup}  + L_{s} + \lambda_{virt} L_{virt}+ \lambda_{diff} L_{diff} $.

Reviewer comment (Evaluation):
lack of human evaluation

Author response:
> **Comment7**:Lack of human evaluation

Specifically, we randomly select 100 samples from the Movie dataset by comparing ${SSR}\_{virt}$ with Inter_RAT and WSEE.  From the observation in Table, we can ﬁnd ${SSR}\_{virt}$ outperforms Inter_RAT and WSEE in all metrics, illustrating the effectiveness of ${SSR}\_{virt}$.

Besides, in our original manuscript, we used ChatGPT for the subjective evaluation (Appendix C.3).

From the results, we find that our model achieves a similar partial order relationship between human evaluation and ChatGPT evaluation (Appendix C.3). This further illustrates the effectiveness of our method.

|                | Usefulness | Completeness | Fluency |
| -------------- | ---------- | ------------ | ------- |
| Inter_RAT      | 3.69       | 3.53         | 3.88    |
| WSEE           | 3.82       | 3.78         | 4.05    |
| ${SSR}_{virt}$ | 3.90       | 3.88         | 4.20    |

Detailed standards for human evaluation：

Usefulness

Q: Do you think the selected rationales can be useful for explaining the predicted labels?

• 5: Exactly. Selected rationales are useful for me to get the correct label.

• 4: Highly useful. Although several tokens have no relevance to correct label, most selected tokens are useful to explain the labels.

• 3: Half of them are useful. About half of the tokens are useful for getting labels.

• 2: Almost useless. Almost all of the tokens are useless.

• 1: No Use. The selected rationales are useless for identifying labels.

Completeness

Q: Do you think the selected rationales are enough for explaining the predicted labels?

• 5: Exactly. Selected rationales are enough for me to get the correct label.

• 4: Highly complete. Several tokens related to the label are missing.

• 3: Half complete. There are still some important tokens that have not been selected, and they are in nearly the same number as the selected tokens.

• 2: Somewhat complete. The selected tokens are not enough.

• 1: Nonsense. None of the important tokens is selected

Fluency

Q: Do you think the selected rationales are fluent?

• 5: Very fluent.

• 4: Highly fluent.

• 3: Partial fluent.

• 2: Very unfluent.

• 1: Nonsense.

Reviewer comment (Experiments):
How would perform sup-rat with data augmentation?

Author response:
> **Comment2**:The relationship between the number of augmented data vs task/rational performance is currently unclear. I would appreciate having a graph showing how the performance evolve according the number of added data

Here, we compare ${SSR}_{unif}$ with different percentages (0-25%) of semantic DA on the Evidence Inference dataset.

From the experimental results, we can observe that both task and rationale performance are improving as the number of added data increases. Among them, the rationale performance improvement is more obvious and has a significant improvement when the number of added data reaches 15% of the total data.

| Method             | 0          |            | 5        |          | 10       |          | 15       |          | 20       |           | 25         |            |
| ------------------ | ---------- | ---------- | -------- | -------- | -------- | -------- | -------- | -------- | -------- | --------- | ---------- | ---------- |
| ${SSR}_{unif}$     | Task       | Token-F1   | Task     | Token-F1 | Task     | Token-F1 | Task     | Token-F1 | Task     | Token-F1  | Task       | Token-F1   |
| + $k$% random DA   | 46.8 ± 0.3 | 26.8 ± 0.2 | 47.3±0.5 | 28.5±0.3 | 47.7±0.4 | 29.0±0.5 | 48.5±0.6 | 30.3±0.2 | 49.2±0.4 | 30.7±0.7  | 46.0±0.1   | 33.1±0.2   |
| + $k$% semantic DA | 46.8 ± 0.3 | 26.8 ± 0.2 | 46.6±0.1 | 28.9±0.4 | 47.1±0.3 | 29.3±0.5 | 47.6±0.7 | 31.9±0.4 | 48.0±0.7 | 32.1± 0.5 | 48.7 ± 0.2 | 33.5 ± 0.4 |

Reviewer comment (Evaluation):
Could you also report metrics regarding comprehensiveness and sufficiency to assess the improvement of the proposed approach to decrease spurious correlation?

Author response:
> **Comment6**:Could you also report metrics regarding comprehensiveness and sufficiency to assess the improvement of the proposed approach to decrease spurious correlation?

Thank you for bringing new metrics to us! We use the  comprehensiveness and sufficiency metrics from ERASER [3], and report the corresponding results in the table. As shown in table, ${SSR}\_{unif}$ and  ${SSR}\_{virt}$ still performs  better than baselines.

|                  | IMDB                |                  | SST-2               |                  |
| ---------------- | ------------------- | ---------------- | :------------------ | ---------------- |
| Model            | suff ($\downarrow$) | com ($\uparrow$) | suff ($\downarrow$) | com ($\uparrow$) |
| Vanilla Un-RAT   | 0.22                | 0.33             | 0.38                | 0.18             |
| Vanilla Semi-RAT | 0.15                | 0.40             | 0.24                | 0.23             |
| WSEE             | 0.13                | 0.42             | 0.20                | 0.26             |
| ${SSR}_{unif}$   | 0.10                | 0.41             | 0.14                | 0.32             |
| ${SSR}_{virt}$   | 0.07                | 0.45             | 0.11                | 0.35             |

Reviewer comment (Theory):
The concept of a shortcut itself is very vague, and the definition and proposed selection algorithm (as described above by discarding the undefined) is too simplistic. The authors use one example of such a spurious correlation that does not describe much of the decision much, throughout all the article.

Author response:
> **Comment1**: The definition and suggested algorithm for selection is too simplistic. 

Thank you for your comments, below we present the details of the definition and suggested algorithm for selection.
Definition: Given the text input $x={x_{1}, x_{2}, \ldots, x_{n}}$, the goal of selection is to first generate a mask variable $m=\{m_{1}, m_{2}, \ldots, m_{n}\}$, where $m_{j} \in\{0,1\} $ indicates whether the $i$-th token is a part of the rationale. Finally, the selection function selects the rationales as $z = m \odot x= \{m_{1}\cdot x_{1}, m_{2} \cdot x_{2}, \ldots, m_{n} \cdot x_{n} \}$.
Briefly, the goal of the selection is to extract a subsequence from the original text to be utilized as rationales to support the prediction results.

Algorithm: In order to select whether each token is rationale or not, we reduce the rationale extraction algorithm to a token-level binary classification task, i.e., predicting the probability that a token is a rationale token. Specifically, we map each token $x_{i}$ to the probability of being selected as part of rationale: 
  $p_{\theta}(m_{i}|x_{i}) = softmax(W_{s_{un}}f_{s_{un}}(x_{i}))$.
Among them, $f_{s_{un}}(\cdot)$ represents an encoder encoding $x_{i}$ into a $d$-dimensional vector, and $W_{s_{un}} \in {R}^{2 \times d}$.

Then, in order to train the model to be able to extract rationales, there are different training approaches for unsupervised and supervised rationalization:

For unsupervised rationalization, due to the lack of labeled rationales, the selection function must be cascaded with the predictor, where the rationale learning signal for unsupervised rationalization must rely on comparing the prediction of the predictor with the ground-truth task label.

For supervised rationalization, since the labeled rationale is known, we can use the token-level binary cross-entropy (BCE) cost function to train supervised rationalization.

Reviewer comment (Writing):
There is no clearly stated hypothesis at the beginning of the text. The approach may be obvious to the author, but readers will not understand and will abandon reading before the end.

Author response:
> **Comment3**: Could you clearly state your hypothesis in the introduction?

Thank you very much for your suggestion, putting our assumptions in the introduction is really beneficial for readers to read. Therefore, we have revised the introduction (the blue text) in the revised version and uploaded it to ICLR. Please review it, and if there are any problems, please let us know.

Reviewer comment (Writing):
The text introduces a great deal of patterning, both when describing existing methods and the author's own proposal. This does not make it easy to read, as many of them do not explain the next steps in any way, such as the definition of Gumbel-softmax on page 3.

Author response:
> **Comment9**: The text introduces a great deal of patterning, both when describing existing methods and the author's own proposal. This does not make it easy to read, as many of them do not explain the next steps in any way, such as the definition of Gumbel-softmax on page 3.

Gumbel-Softmax is a commonly used technique for handling discrete data in generative models and optimization problems. It combines the Gumbel distribution and the Softmax function to sample from a discrete probability distribution. The key idea is to introduce noise from the Gumbel distribution and then transform this noise into a sample from a discrete distribution using the Softmax function.

The process of Gumbel-Softmax can be summarized as follows:

Sampling noise from the Gumbel distribution: First, a noise vector $g$ is sampled from the Gumbel(0, 1) distribution, where $0$ is the location parameter and $1$ is the scale parameter. This noise vector $g$ introduces randomness into the sampling process.

Computing the Gumbel-Softmax sample: Next, the noise vector $g$ is added to the logarithmic values of the discrete probability distribution $p$. Then, the Softmax function is applied to obtain a sample from the discrete distribution. Specifically, for the logarithmic value $z_i$ of the discrete probability distribution, the Gumbel-Softmax sample is calculated as follows:

$\text{Gumbel-Softmax}(z_i) = \frac{\exp((\log(p_i) + g_i)/\tau)}{\sum_{j=1}^{K} \exp((\log(p_j) + g_j)/\tau)},$

where $\tau$ is the temperature parameter that controls the smoothness of the sample. Higher temperature values result in smoother samples, while lower temperature values tend to produce more one-hot vectors that represent discrete values. By adjusting the temperature parameter $\tau$, the randomness and smoothness of the Gumbel-Softmax samples can be controlled. As $\tau$ approaches $0$, the sample tends to be a one-hot vector, where only one element is $1$ and the rest are $0$, resembling maximum likelihood estimation. As $\tau$ approaches positive infinity, the sample tends to approach a uniform distribution, where all elements have equal probabilities.

The benefit of Gumbel-Softmax is that it provides a differentiable approximation for sampling discrete variables, making it compatible with optimization algorithms such as gradient descent. This property makes it applicable in unsupervised rationalization to sample rationale tokens.

Reviewer comment (Novelty):
The definition and suggested algorithm for selection (as described above by rejecting the undefined) is too simplistic. The authors use only one example ('received a lukewarm approach' in the film review) throughout the article. Doesn't such a solution reduce the proposal to a supervised approach? Please use another example of a shortcut.

Author response:
> **Comment2**: The authors use only one example ('received a lukewarm approach' in the film review) throughout the article. Doesn't such a solution reduce the proposal to a supervised approach? Please use another example of a shortcut.

First, the shortcut does not reduce the proposal to a supervised approach. Specifically, the example we give in the paper is a real example from the Movie dataset, where the shortcut 'received a lukewarm approach' is obtained by our shortcut discovery strategy, not by a priori human annotation. Besides, it is noted that our shortcut discovery strategy relies on the rationale of the true annotation. However, since the rationale annotation is difficult to obtain for most tasks, shortcuts can not reduce the proposal to a supervised approach in most cases. Therefore, in our SSR, we use a semi-supervised approach.
Finally, we also visualize some of the shortcuts obtained by our shortcut discovery strategy in  Appendix  C.5, which we further illustrate one of them here:

We visualize another example on Movies (the label is positive),  where the italic tokens represent the real rationales, and the bolded ones are the predicted rationales.:

| Model    | Visualized example  | Predicted label |
| - | - | --|
| Vanilla Un-RAT  | **Mozart is a famous musician** and amadeus is a biographical film about him , ***amadeus is a true work of art*** . it is one of those few movies of the 80 ' s that *will be known for its class , its style , and its intelligence. why is this **such a good film***... | positive        |
| ${SSR}\_{unif}$ | Mozart is a famous musician and amadeus is a biographical film about him , ***amadeus is a true work of art*** . it is one of those few movies of the 80 ' s that ***will be known for its class*** , *its style , and its intelligence. why is this **such a good film***... | positive  |

From the results, we can find  although both Vanilla Un-RAT and ${SSR}\_{unif}$ predict the label as positive correctly, Vanilla Un-RAT still extracts shortcuts as rationales. Specifically, "Mozart is a famous musician" is the shortcuts. Although Mozart was a great musician, it has no relevance to how good his biographical film is. ${SSR}\_{unif}$ avoids these shortcuts, but Vanilla Un-RAT extracts these as rationales.

Reviewer comment (Writing):
The entire article is written in a way that is difficult to understand. There is no clearly stated hypothesis at the beginning of the text. The approach may be obvious to the author, but readers will not understand and will abandon reading before the end. Could you clearly state your hypothesis in the introduction?

Author response:
> **Comment3**: Could you clearly state your hypothesis in the introduction?

Thank you very much for your suggestion, putting our assumptions in the introduction is really beneficial for readers to read. Therefore, we have revised the introduction (the blue text) in the revised version and uploaded it to ICLR. Please review it, and if there are any problems, please let us know.

Reviewer comment (Writing):
Is the whole of section 2 your proposition, or the definition of possible solutions used now? It is not clear.

Author response:
> **Comment4**: Is the whole of section 2 your proposition, or the definition of possible solutions used now? It is not clear.

Thank you again for your suggestions. Based on your suggestions, we have organized section 2 and updated it in the revised version.

Reviewer comment (Writing):
A number of cost functions are given that are to be utilized in different configurations. Since the background models are quite complex (Transformer, both as encoder and encoder/predictor, as well as other generative models) these loss functions tend to be complex too. Some might be removed with more intuition on the more important in exchange.

Author response:
> **Comment5**: A number of cost functions are given that are to be utilized in different configurations. Since the background models are quite complex (Transformer, both as encoder and encoder/predictor, as well as other generative models) these loss functions tend to be complex too. Some might be removed with more intuition on the more important in exchange.

In rationalization studies, several deep learning models such as bert and other Transformer based methods are widely used in order to get better performance. In order to make a fair comparison with these methods, we also use bert as the encoder/predictor in our paper. Although this will increase the complexity of the model, performance improvement can be obtained. In the future, we will also further investigate how to reduce the complexity of the model while maintaining the performance.

In addition, our loss functions can be divided into two parts. The first part is the traditional supervised\semi-supervised\unsupervised losses, such as Eqs. 2, 3, and 4. These losses cannot be simplified.

The second part is the newly added loss functions for SSR. Specifically, for Strategy 1, compared to the traditional loss, we only add Eq. 6 to ensure that the predictor identifies the shortcuts features as meaningless features, thus mitigating the effect of shortcut on the model's prediction. KL divergence in Eq. 6 is very easy to implement and does not increase the complexity of the loss function.



For Strategy 2, we introduce a shortcut imitator, which assumes the role of learning shortcuts (in the supervised phase) and shortcut simulation (in the unsupervised phase). First, in the supervised phase, we use Eq.7 to capture the features of the shortcut. Essentially, Eq.7 is a cross-entropy loss function like  $\mathcal{L}_{sup\_task}$. Afterward, in order to learn the shortcut knowledge, we introduce the shortcut imitator to fit the distribution of shortcuts by using the original text as input with Eq. 8 (a simple MSE loss). Finally, in the unsupervised phase, since the shortcuts are unknown, we can use the shortcut imitator to obtain a potential shortcut representation and map it to a uniform distribution using Eq. 9 to identify the shortcuts features as meaningless ones, where the operation is similar to Eq. 6.
Compared with the traditional method, we have added three loss functions, but our results are also significantly improved, which shows that all these loss functions are important for SSR. In the future, we will also explore new methods to reduce the number of loss functions to simplify the model.

Thank you for your suggestions!

Reviewer comment (Evaluation):
In the results tables, these with the best value are bold-faced as the best. But the mean differences may be as low as 0.1%, which statistically are not, in any way, significant. The authors should perform some statistical analysis and group the algorithms into groups of statistical equivalence (see e.g. Demsar, Statistical comparisons of classifiers over multiple data sets, JMLR 7, 2006; open software for that approach is available).

Author response:
> **Comment8**: In the results tables, these with the best value are bold-faced as the best. But the mean differences may be as low as 0.1%, which statistically are not, in any way, significant. The authors should perform some statistical analysis and group the algorithms into groups of statistical equivalence (see e.g. Demsar, Statistical comparisons of classifiers over multiple data sets, JMLR 7, 2006; open software for that approach is available).

From a statistical point of view, there is no significant difference between SSR and its data augmented variants. However, in this work, we are more interested in how effective the models are in practice. For example, SSR+random DA outperforms SSR on most metrics, but the statistical difference is not significant. In practical applications, we prefer SSR+random to SSR. Because in some scenarios where decision making is important and the amount of data is large, even a minor improvement may bring huge economic returns. 
Therefore, we prefer models with the excellent effectiveness of extraction and prediction in practice. In the future, we will also explore new counterfactual data augmentation methods to further improve the performance of SSR+DA.

Reviewer comment (Presentation):
Instead of very many formulas, drawing diagrams of the methods should be shown, which would explain much more. Also, nothing is contributed by the detailed descriptions of all cost functions in the description of the general methods as well as the own proposal.

Author response:
> **Comment10**: Difficulty in reading may come from poor English. I suggest the involvement of a native speaker. & Instead of very many formulas, drawing diagrams of the methods should be shown, which would explain much more. Also, nothing is contributed by the detailed descriptions of all cost functions in the description of the general methods as well as the own proposal.

Thank you very much for your suggestion, we will invite native speakers to help us embellish the paper in the future, and will further draw diagrams of the methods to explain the process of SSR.

Reviewer comment (Presentation):
In table 3 the result 90.7 for SSR_unif with DA is chosen as best, even though that of WSEE with DA of 91.0 mean seems better. That is perhaps a typing error, or is it?

Author response:
> **Comment11**: In table 3 the result 90.7 for SSR_unif with DA is chosen as best, even though that of WSEE with DA of 91.0 mean seems better. That is perhaps a typing error, or is it?

Thank you, this is a typing error. We have modified in the revised version.

Reviewer comment (Theory):
The task of finding shortcuts remains ambiguous. It's unclear whether shortcuts always exist in the text. If they aren’t, what would the model predict? The usage of this system seems limited.

Author response:
>**Comment 1:**  It's unclear whether shortcuts always exist in the text. If they aren’t, what would the model predict?

We argue that shortcuts should always exist in texts such as  entity bias [1,2]  and statistical bias [3]. Previous studies [4,5,6,7] that study shortcuts in texts also support our viewpoint. Moreover, from the causal graph we constructed in Appendix D, we observe that since there always exists a backdoor path S↔Z→Y which makes shortcut Z and the label Y spuriously correlated, we conclude shortcuts always exist in the text. Finally, we assume that some datasets have been processed without significant shortcuts, enabling the model to learn the true causal relationships within the data and function as a debiased model. However, this assumption is overly idealistic when applied to real-world datasets.

Reviewer comment (Evaluation):
There's a need for a more robust analysis of prior methodologies, particularly the unsupervised ones. The author asserts that unsupervised methods frequently identify shortcuts as inefficient rationales and provides examples. Yet, the reader would be curious about how often that happens, and if the model already yields good results, why do we have to give up shortcuts? I suggest a stronger argument for why finding a good rationale is significant, such as it can be helpful for other reasoning tasks.

Author response:
> **Comment2:**  The author asserts that unsupervised methods frequently identify shortcuts as inefficient rationales and provides examples. Yet, the reader would be curious about how often that happens.

As in the answer to comment 1, shortcuts commonly occur in text. Meanwhile, [5] has shown that the vanilla rationalization criterion in unsupervised methods is prone to highlighting spurious correlations (shortcuts) between the input features and the output as valid explanations. Therefore, we argue that unsupervised methods will frequently identify shortcuts as inefficient rationales.

> **Comment3:**  If the model already yields good results, why do we have to give up shortcuts? I suggest a stronger argument for why finding a good rationale is significant, such as it can be helpful for other reasoning tasks.

This is a valuable question! If the shortcut helps us get better results, do we still need to remove the shortcut? Our answer is yes. Specifically, when faced with test data that is distributed with the same distribution as the training data, the shortcut facilitates us to obtain accurate results. However, since shortcuts do not reflect the real causal relationship between the data and labels, when the test data is distributed differently from the training data (i.e., the shortcut does not exist in the test data), a model that relies on shortcuts will have poor prediction results. For example, in the following table (Table 3 in the original manuscript), Vanilla Un-RAT achieves promising results when we test it on an identically distributed dataset IMDB, but it achieves poor predictive results on out-of-distribution data SST-2. Thus, we can conclude that when the model relies on the shortcut in the data for training and prediction, the model fails in the face of new data distributions, further limiting the application of the model.

| Methods|IMDB(ID)|SST-2(OOD)|
|-|-|-|
|Vanilla Un-RAT|85.3±0.2|45.3±8.1|
|SSR\_unif|90.3±0.2|79.4±0.3|
|SSR_virt|89.9±0.2|79.9±0.4|

Furthermore, finding a good rationale is significant:

1. Extracting a good rationale can help us avoid extracting shortcuts as rationales, which can help the model focus on the real causal relationship between the data and labels during training and prediction. It can increase the accuracy and robustness of the model when facing the test data with different distributions.
2. Extracting a good rationale can improve the interpretability of the prediction results, which can be applied in some high-risk domains, such as law [4].
3. Rationalization can help other reasoning tasks. For example, rationalization is used as a knowledge extractor in [8] to extract the most responsible features for the predictions. After that, [8] expands the extractive rationales using commonsense resources to generate natural language explanations and give the final prediction.

Reviewer comment (Writing):
The manuscript's writing style can be perplexing in several sections, notably in Methodology (Section 3). Mathematical expressions should be consistent, straightforward, and clear, and should only be included when indispensable. For example, instead of writing an algorithm for the semantic data augmentation, the reader might be more interested in seeing how specifically you do the semanticly similar word retrieval in Appendix A.

Author response:
> **Comment4:**  Instead of writing an algorithm for the semantic data augmentation, the reader might be more interested in seeing how specifically you do the semanticly similar word retrieval in Appendix A. 

Given a word $x_i$ and its semantic representation $f_{p_{un}}(x_i)$, and a database $D_{local}={(f_{p_{un}}(x_{i}^{r}), x_{i}^{r} ), \forall  x_{i}^{r} \in x^{r}}$
 to be retrieved, our goal is to retrieve the word $x_{i}^{r}$ in the database that is semantically closest to the word $x_i$.

In the retrieval process, we first calculate the L2 distance between the semantic representation of the retrieved word $x_i$ and each word $x_{i}^{r}$ in the database. We consider the smaller the L2 distance, the closer an input semantically to another. Finally, we choose the word with the closest L2 distance to the retrieved word $x_i$ as the semantically similar word. In the specific code implementation, we employ FAISS [1] to achieve this retrieval goal.

Reviewer comment (Experiments):
Elaborating on data augmentation might be beneficial. For instance, details about the number of augmented instances introduced and the ideal quantity would be insightful.

Author response:
> **Comment7:**  Elaborating on data augmentation might be beneficial. For instance, details about the number of augmented instances introduced and the ideal quantity would be insightful.

In Table 1, for random data augmentation, we augment 25% of the data in the original dataset. For example, in Movie, there are 1600 training data. Based on this, we augment 400 instances (i.e., 1600*0.25=400).
Similarly, for semantic data augmentation, we augment the data with 25% of the original dataset.
In mixed data augmentation, we mix the data obtained from random DA with the data obtained from semantic DA, which is equivalent to adding 50% of the data of the original dataset.

In Figure 4, we test the performance of SSR with data augmentation on the MultiRC dataset. In this experiment, we set the percentage of data augmentation to 5%, 10%, 15% and 25% respectively.
We find that SSR with data augmentation performs best when the data augmentation percentage is 15%.

Then, we additionally investigate SSR with data augmentation on the Evidence Inference dataset. From the experimental results, we observe that SSR with data augmentation performs best on the Evidence Inference dataset when the data augmentation percentage is 25%.

| Method             | 0          |            | 5        |          | 10       |          | 15       |          | 20       |           | 25         |            |
| ------------------ | ---------- | ---------- | -------- | -------- | -------- | -------- | -------- | -------- | -------- | --------- | ---------- | ---------- |
| ${SSR}_{unif}$     | Task       | Token-F1   | Task     | Token-F1 | Task     | Token-F1 | Task     | Token-F1 | Task     | Token-F1  | Task       | Token-F1   |
| + $k$% random DA   | 46.8 ± 0.3 | 26.8 ± 0.2 | 47.3±0.5 | 28.5±0.3 | 47.7±0.4 | 29.0±0.5 | 48.5±0.6 | 30.3±0.2 | 49.2±0.4 | 30.7±0.7  | 46.0±0.1   | 33.1±0.2   |
| + $k$% semantic DA | 46.8 ± 0.3 | 26.8 ± 0.2 | 46.6±0.1 | 28.9±0.4 | 47.1±0.3 | 29.3±0.5 | 47.6±0.7 | 31.9±0.4 | 48.0±0.7 | 32.1± 0.5 | 48.7 ± 0.2 | 33.5 ± 0.4 |

From the above experiments, it is found that the optimal data augmentation percentage is different for different datasets. Since our data augmentation method relies on human labeled rationales and most semi-supervised methods assume that the labeled rationales percentage is 25%, we also use 25% as our percentage in our practical implementation.

Reviewer comment (Writing):
The term datastore can be confusing when discussing random data augmentation since this word is also used when describing the semantic key-value pair in semantic data augmentation.

Author response:
> **Comment5:**  The term datastore can be confusing when discussing random data augmentation since this word is also used when describing the semantic key-value pair in semantic data augmentation

Thanks for your suggestion! We revise the description of the Random Data Augmentation method to the following:

As we have identified the potential shortcuts $z_{s}$ in $D_{sup}$, we can replace these shortcuts tokens with other tokens which are sampled randomly from the datastore $D_{random}$. Among them, the database $D_{random}$ contains all tokens of $D_{un}$ and $D_{sup}$ (i.e., $D_{random}={x_{j}, \forall x_{j} \in D_{un}  \vee  \forall x_{j} \in D_{sup}}$).