Paper: Quantifying the Plausibility of Context Reliance in Neural Machine Translation (ICLR-2024)

Reviewer comment (Experiments):
Experiments are not very systematic in that OpenMT was mainly employed in section 4 for evaluating each step, but section 5 used mBART-50 in an end-to-end evaluation setting. I'd expect two settings in the end-to-end experiments for completeness.

Author response:
Dear Reviewer GSke,

Thank you for taking the time to actively engage with our submission! We are glad you found our method “simple but sound, with easy-to-interpret outputs,” and appreciate your comment about our work addressing “an important problem in context-aware machine translation.”

**Usage of mBART in Section 5 end-to-end experiments**

Our initial decision to use mBART for our end-to-end analysis in Section 5 was aimed at showcasing the applicability of our approach to other models and target languages. However, we agree with your comment suggesting that missing OpusMT examples might undermine the systematicity and completeness of our analysis since the system was used in previous sections. For this reason, in the revised version of our paper, we extend our end-to-end analysis to the OpusMT Large model, adding two examples of OpusMT results in Section 5 and several more in Appendix I (Table 7). We observe that OpusMT results present several issues with lexical cohesion that closely match those previously shown for mBART, making the new result consistent with our original analysis.

Reviewer comment (Experiments):
I'd like to know the reason for not running OpenMT for the end-to-end experiment in section 5.

Author response:
Dear Reviewer GSke,

Thank you for taking the time to actively engage with our submission! We are glad you found our method “simple but sound, with easy-to-interpret outputs,” and appreciate your comment about our work addressing “an important problem in context-aware machine translation.”

**Usage of mBART in Section 5 end-to-end experiments**

Our initial decision to use mBART for our end-to-end analysis in Section 5 was aimed at showcasing the applicability of our approach to other models and target languages. However, we agree with your comment suggesting that missing OpusMT examples might undermine the systematicity and completeness of our analysis since the system was used in previous sections. For this reason, in the revised version of our paper, we extend our end-to-end analysis to the OpusMT Large model, adding two examples of OpusMT results in Section 5 and several more in Appendix I (Table 7). We observe that OpusMT results present several issues with lexical cohesion that closely match those previously shown for mBART, making the new result consistent with our original analysis.

Reviewer comment (Experiments):
Lack of comparation. The proposed method are not compared with other SOTA methods for plausibility.

Author response:
**Lack of comparisons to established practices in plausibility evaluation**

The established practice for plausibility evaluation is to verify the overlap between model rationales and human annotation for pre-defined target tokens of interest. Our framework, instead, proposes a more comprehensive evaluation framework that includes the identification of those targets. The comparison between the two is performed in Section 4.5. The CCI results from gold context-sensitive tokens in Figure 4 reflect the traditional hypothesis-driven procedure for plausibility evaluation. Additional background and a concrete example of plausibility evaluation are now provided in Appendix A to avoid any confusion.

Instead, end-to-end medians in the same Figure represent the outcome of our proposed framework, leading to more conservative plausibility estimates that reflect a broader notion of model contextual reliance, incorporating the detection of context-sensitive target tokens in the evaluation. 

Importantly, the goal of our work is not to adopt the traditional “SOTA” procedure for plausibility evaluation to improve results on some metrics of interest (e.g. Macro F1 between human and model rationales in Figure 4), but to address the shortcomings of this procedure by proposing a shift from hypothesis-driven to data-driven plausibility evaluation. This shift ultimately enables its usage in real-world settings where references and annotations are unavailable. We added a sentence in the revised Section 4.5 to clarify this aspect.

In terms of our choice of methods, the P-CXMI method can be seen as the “previous SOTA” for CTI, as it was used by [1] for context sensitivity detection, while for CCI, the Attention Mean method by [2]  was the best one that does not require pre-hoc knowledge (as in the case of Attention Best) before conducting the evaluation.

Reviewer comment (Experiments):
Although your method is for generative language models, it seems that it is more suitable for seq2seq generation tasks, such as machine translation, rather than pure generation tasks, such as story generation. Also, it seems only works for encoder-decoder structure, rather than currently widely used decoder-only language model. Can your method work in such scenarios?

Author response:
**Narrow scope, encoder-decoder specific method**

The PECoRe framework is not architecture-specific and can be applied to modern decoder-only large language models. We apologize for not making this clearer in the initial version of the paper, and we have added a clarifying remark in the revised version of the introduction. Moreover, we have added some examples in Appendix J to further highlight the broad applicability of our approach to study the plausibility of a state-of-the-art decoder-only large language model to quantify context reliance in question answering, story generation tasks and information extraction. Our preliminary evidence suggests that PECoRe can be useful to ensure a trustworthy context reliance in these domains.

Reviewer comment (Experiments):
Where is the comparison between your method and other SOTA method? Correct me if I am wrong, it seems that you only evaluate the performance of different metrics on different models.

Author response:
**Lack of comparisons to established practices in plausibility evaluation**

The established practice for plausibility evaluation is to verify the overlap between model rationales and human annotation for pre-defined target tokens of interest. Our framework, instead, proposes a more comprehensive evaluation framework that includes the identification of those targets. The comparison between the two is performed in Section 4.5. The CCI results from gold context-sensitive tokens in Figure 4 reflect the traditional hypothesis-driven procedure for plausibility evaluation. Additional background and a concrete example of plausibility evaluation are now provided in Appendix A to avoid any confusion.

Instead, end-to-end medians in the same Figure represent the outcome of our proposed framework, leading to more conservative plausibility estimates that reflect a broader notion of model contextual reliance, incorporating the detection of context-sensitive target tokens in the evaluation. 

Importantly, the goal of our work is not to adopt the traditional “SOTA” procedure for plausibility evaluation to improve results on some metrics of interest (e.g. Macro F1 between human and model rationales in Figure 4), but to address the shortcomings of this procedure by proposing a shift from hypothesis-driven to data-driven plausibility evaluation. This shift ultimately enables its usage in real-world settings where references and annotations are unavailable. We added a sentence in the revised Section 4.5 to clarify this aspect.

In terms of our choice of methods, the P-CXMI method can be seen as the “previous SOTA” for CTI, as it was used by [1] for context sensitivity detection, while for CCI, the Attention Mean method by [2]  was the best one that does not require pre-hoc knowledge (as in the case of Attention Best) before conducting the evaluation.

Reviewer comment (Experiments):
The main thing that could make this paper have a significantly bigger impact would be testing the use of PECORE for tasks other than contextual MT: as authors point out, PECORE is general enough to be applicable to any (contextual) LM, and some experiments on other tasks (like summarization or even other modalities) could make this even more suitable to conference like ICLR (doing something like what was done for FLORES if human rationales don’t exist).

Author response:
**Application of PECoRe to other tasks/modalities**

As you correctly state, our framework can be applied to other tasks or modalities to study the impact of context dependence beyond context-aware MT. We agree that showcasing such applications would strengthen the presentation of our method, and for this reason, we include a new appendix section in the revised submission (Appendix J). This new section applies PECoRe to a state-of-the-art decoder-only large language model to quantify context reliance in question answering, story generation tasks and information extraction. Our preliminary evidence suggests that PECoRe can be useful to ensure a trustworthy context reliance in these domains.

While such applications are undoubtedly exciting, we opt to include those as appendix materials since a more thorough evaluation of PECoRe for these tasks would, in our opinion, be more fitting for future studies leveraging task and modality-specific expertise to extend our investigation.

Reviewer comment (Writing):
While I think the authors did well in side-stepping the issues with the faithfulness/plausibility argument, I think some parts of the writing still hint that these explanations reflect the model’s internal behaviour (e.g. “leaving space for improvement through the adoption of more faithful attribution” this implies that more faithful atribbution will lead to more plausible explanations, which might not be the case). Some extra care about this sort of statements could be useful (but overall I found that the authors were already quite careful).

Author response:
**Statement conflating attribution plausibility with faithfulness**

We agree that the sentence you report assumes a relation between faithfulness and plausibility when these two evaluation dimensions were shown to be largely independent in previous interpretability work. We have adjusted our formulation at the end of Section 4.5 to clarify that we are specifically interested in attribution methods that better align with human rationales.

Reviewer comment (Theory):
Is it possible that $\tilde{y}^\star$ ends up being the same as the contextual prediction $\hat{y}$ (in remark 3.1)? Or does a high selector score imply that the highest probability token is gonna be different?

Author response:
**Question regarding $\tilde y^{*}$ and $\hat y$**

Even for cases where $\tilde y^{\*}$ is identical to $\hat y$, the probabilities associated with the tokens of the two sequences are likely to be different. Indeed, while both are produced from target prefix $\hat y_{<t}$, $\tilde y^{*}$ use non-contextual source inputs, while $\hat y$ uses contextual source inputs. If the probabilities of the same next token $\hat y_t$ are different, the result of the contrastive target function $f_{tgt}$ will also be different, leading to non-zero attribution scores over the context.

The only case in which these scores would be equal to zero is when the probability of the next token (or the distribution over the vocabulary, for KL-Divergence) is exactly the same with and without input context, i.e. if context is not affecting the generation process. We have added a new remark (3.2) to clarify this aspect.

Reviewer comment (Evaluation):
The authors acknowledge limited performance, as indicated by the data presented in Table 1 and Figure 3. They attribute the suboptimal performance to domain shift issues affecting lexical choice. This suggests that the proposed methodology predominantly excels in handling anaphora cases. However, from my perspective, it's worth noting that lexical choice in context-aware translation is a significant aspect, and it's not entirely clear if the approach can effectively diagnose and interpret the model in practical context-aware translation research.

Author response:
**Limited performance and unclear generalizability to other contextual phenomena**

Your statement, “the proposed methodology predominantly excels in handling anaphora cases,” implies that the weaker CTI scores on lexical choice in Figure 3 could be attributed to issues with our proposed PECoRe framework. However, this is not the case: Table 1 reports the performance of the MT model we use as a “test subject” and not the results of our proposed plausibility evaluation framework (Figures 3 and 4). These two aspects should not be conflated, as our method aims to detect cases of implausible model behavior and not to directly improve the performance of the underlying model.

Indeed, from Table 1, it is evident that the MT model can less proficiently use contextual information for lexical choice, likely due to the limited presence of such phenomena in the training data. If the model cannot use contextual information to disambiguate lexical choices in several cases (Table 1, column “OK”), it is expected that CTI would identify this setting as implausible from a context-sensitivity perspective (Figure 3, Left). This motivates us to include results for the OK-CS split (Figure 3, right), the data subset for which we know context availability leads to the correct disambiguation. On this split, we can see that the proposed KL-Divergence metric fares competitively, suggesting the robustness of PECoRe despite the domain shift.

As additional evidence of PECoRe’s generalizability to unseen phenomena, Appendix G (Table 5) in the revised draft presents some SCAT+ examples for whose a context sensitivity in the choice of grammatical formality (“you” → “toi/vous”) was correctly identified by the KL-Divergence CTI metric. However, those examples were considered wrong in the evaluation of Section 4.3 since it did not match the gender disambiguation phenomena (“it, they” → “il/elle, ils/elles”) used to annotate SCAT+.