Quantifying the Plausibility of Context Reliance in Neural Machine Translation
Authors: Gabriele Sarti
Grzegorz Chrupała
Malvina Nissim
Arianna Bisazza
Venue: ICLR 2024 poster (2024)
Keywords: explainable AI
interpretability
feature attribution
machine translation
document-level machine translation
natural language generation
Primary area: visualization or interpretation of learned representations
TL;DR: We introduce PECoRe, an end-to-end interpretability framework to evaluate the plausibility of context usage in language models generations.

Abstract: Establishing whether language models can use contextual information in a human-plausible way is important to ensure their safe adoption in real-world settings. However, the questions of $\textit{when}$ and $\textit{which parts}$ of the context affect model generations are typically tackled separately, and current plausibility evaluations are practically limited to a handful of artificial benchmarks. To address this, we introduce $\textbf{P}$lausibility $\textbf{E}$valuation of $\textbf{Co}$ntext $\textbf{Re}$liance (PECoRe), an end-to-end interpretability framework designed to quantify context usage in language models' generations. Our approach leverages model internals to (i) contrastively identify context-sensitive target tokens in generated texts and (ii) link them to contextual cues justifying their prediction. We use PECoRe to quantify the plausibility of context-aware machine translation models, comparing model rationales with human annotations across several discourse-level phenomena. Finally, we apply our method to unannotated model translations to identify context-mediated predictions and highlight instances of (im)plausible context usage throughout generation.

Published as a conference paper at ICLR 2024
QUANTIFYING THE PLAUSIBILITY OF CONTEXT RELIANCE IN NEURAL MACHINE TRANSLATION
Gabriele Sarti'
Grzegorz Chrupata? Malvina Nissim
Arianna Bisazza'
'Center for Language and Cognition (CLCG), University of Groningen
Deptt of Cognitive Science and Artificial Intelligence (CSAI), Tilburg University {g. sarti, m. nissim, a. ..bisazzaJ@rug.nnn nl, grzegorz@chrupala.mm
ABSTRACT
Establishing whether language models can use contextual information in a human- plausible way is important to ensure their trustworthiness in real-world settings. However, the questions of when and which parts of the context affect model generations are typically tackled separately, with current plausibility evaluations being practically limited handful of artificial benchmarks. To address this, we introduce Plausibility Evaluation of Context Reliance (PECORE), an end-to-end interpretability framework designed to quantify context usage in language models' generations. Our approach leverages model internals to (i) contrastively identify context-sensitive target tokens in generated texts and (ii) link them to contextual cues justifying their prediction. We use PECORE to quantify the plausibility of context-aware machine translation models, comparing model rationales with human annotations across several discourse- phenomena. Finally, we apply our method unannotated model translations to identify context-mediated predictions and highlight instances of (im)plausible context usage throughout generation.
INTRODUCTION
Research in NLP interpretability defines various desiderata for rationales of model behaviors, i.e.
the contributions of input tokens toward model predictions computed using feature attribution (Mad) sen et al., 2022). One of such properties plausibilityy, corresponding to the alignment between model rationales and salient input words identified by human annotators (Jacovi & Goldberg (2020). Low-plausibility rationales usually occur alongside generalization failures or biased predictions and can be useful to identify cases of models being "right for the wrong reasons" McCoy et al. (2019). However, while plausibility has an intuitive interpretation for classification tasks involving single prediction, extending this methodology to generative language models (LMs) presents several challenges. First, LMs have a large output space where semantically equivalent tokens (e.g. "PC" and "computer") are competing candidates for next- word prediction (Holtzman et al. 2021). Moreover, LMs generations are the product of optimization pressures to ensure independent properties such as semantic relatedness, topical coherence and grammatical correctness, which hardly be captured by a single attribution score (Yin & Neubig, 2022). Finally, since autoregressive generation involves an iterative prediction process, model rationales could be extracted for every generated token. This raises the issue of which generated tokens can have plausible contextual explanations.
Recent attribution techniques for explaining language models incorporate contrastive alternatives to disentangle different aspects of model predictions (e.g. the choice of "meowing" over "screaming" for "The cat " is motivated by semantic appropriateness, but not by grammaticality) (Ferrando et al.) 2023 Sarti et al. 2023). However, these studies avoid the issues above by narrowing the evaluation to single generation step matching a phenomenon of interest. For example, given the sentence "The pictures of the cat ", a plausible rationale for the prediction of the word "are" should reflect the role of "pictures" subject-verb agreement. While this approach can be useful to validate model rationales, it confines plausibility assessment to small set of handcrafted benchmarks where tokens with plausible explanations are known in advance. Moreover, risks overlooking important patterns of context usage, including those that do not immediately match linguistic intuitions. In light of this, we suggest that identifying which generated tokens were most affected by contextual input information should be an integral part of plausibility evaluation for language generation tasks.

---

Published as a conference paper at ICLR 2024
Figure 1: Examples of sentence-level and contex- tual moglish->Italian MT. Sentence-level transla- tion contain lack-of-context errors. Instead, in the contextual case context-sensitive source tokens are disambiguated using source (S) or target- based (T) contextual cues to produce correct context-sensitive target tokens. PECORE enables the end-to-end extraction of cue-target pairs (e.g. <she, alla pastorella>, <le pecore, le>).
Sheep grazed next to the little shepherd.
At sunset, she led them back to the stable.
Sentence-level
& Contextual MT
Le pecore pascolavano accanto al pastorello &
X alla pastorella /
tramonto,,
le ricondusse alla stalla.
To achieve this goal, we propose a novel interpretability framework, which we dub Plausibility Evaluation of Context Reliance (PECORE). PECORE enables the end-to-end extraction of cue- target token pairs consisting of context-sensitive generated tokens and their respective influential contextual cues from language model generations, as shown in Figure [T These pairs can uncover context dependence in naturally occurring generations and, for cases where human annotations are available, help quantify context usage plausibility in language models. Importantly, our approach is compatible with modern attribution methods using contrastive targets (Yin & Neubig 2022). avoids using reference translations to stay clear of problematic distributional shifts (Vamvas & Sennrich 202 and can be applied on unannotated inputs to identify context usage in model generations.
After formalizing our proposed approach in Section we apply PECORE contextual machine translation (MT) to study the plausibility of context reliance in bilingual and multilingual MT models. While PECORE can easily be used alongside encoder- decoder and decoder-only language models for interpreting context usage in any text generation task, we focus our evaluation on MT because of its constrained output space facilitating automatic assessment and the availability MT datasets annotated with human rationales of context usage. We thoroughly test PECORE well-known discourse phenomena, benchmarking several context sensitivity metrics and attribution methods to identify cue-target pairs. We conclude by applying PECORE unannotated examples and showcasing some reasonable and questionable cases of context reliance in MT model translations.
In sum, we make the following contributions]]
We introduce PECORE, an interpretability framework to detect and attribute context reliance in language models. PECORE enables quantitative evaluation of plausibility for language generation beyond the limited artificial settings explored in previous literature.
We compare the effectiveness of context sensitivity metrics and feature attribution methods on the context-aware MT tasks, showing the limitations of metrics currently in use.
We apply PECORE to naturally-occuriigg translations to identify interesting discourse-level phenomena and discuss issues in the context usage abilities of context-aware MT models.
2 RELATED WORK
Context Usage in Language Generation An appropriate usage of input information is fundamen- tal in tasks such as summarization (Maynez et al., 2020) to ensure the soundness of generated texts. While appropriateness is traditionally verified post- hoc using trained models (Durmus al. 2020
Kryscinski et al., 2020 Goyal & Durreu 2021). )), recent interpretability works aim to gauge input
influence on model predictions using internal properties of language models, such the mixing of
contextual information across model layers (Kohayashi et al. 2020) Ferrando et al. [0026] Mohebbil
al. 2023) or the layer-by-layer refinement of next token predictions (Geva et al. 2022 Belrose
et al. 2023). Recent attribution methods can disentangle factors influencing generation in language
models Yin & Neubig 2022) and were successfully used to detect and mitigate hallucinatory behav-
iors (Tang et al.] 2022 Dale et al. 2202 2023). Our proposed method adopts this intrinsic perspective to identify context reliance without ad-hoc trained components.
'Code: /https::////tt //gi thub. compegsatttt/peecoreeeg The CLI command inseq attribute-conten available in the Inseq library is generalized PECORE implementation; Inttps:// //gi thub. wttptt.wwooeeettttett cow/ httm/inseq--een tetateeaasssseeeee /asaa// We avoid using the term faithfulness due its ambiguous usage in interpretability research.
2

---

Published as a conference paper at ICLR 2024
Context Usage in Neural Machine Translation Inter-sentential context is often fundamental for resolving discourse- ambiguities during translation (Müller et al.] 2018 Bawden et al. 2018) Voita al., 2019b Fernandes et al.) 2023). However, MT systems are generally trained at the
sentence level and fare poorly in realistic translation settings (I (Läubli et al. 2018) Toral et al. 2018).
Despite advances in context-aware MT (Voita al., 20188 2019a Lopes et al., [020 Majumder et all.| 2022, Jin et al. [202] inter alia, surveyed by Maruf et al. (021], only a few works explored whether context usage in MT models aligns with human intuition. Notably, some studies focused on which parts of context inform model predictions, finding that supposedly context-aware MT models are often incapable of using contextual information (Kim al., [2019] Fernandes et al. (02T) and tend to pay attention to irrelevant words (Voita et al.
2018). with an overall poor agreement between human annotations and model rationales (Yin et al., 2021). Other works instead investigated which parts of generated texts are influenced by context, proposing various contrastive methods to detect gender biases, discourse-level phenomena in MT corpora (Fernandes et al. [023]) ). While these two directions
transsdiittanssltioons (Vamvas& & Sennrich 2021a 2022). and to identify various
have generally been investigated separately, our work proposes a unified framework to enable an end-to-end evaluation of context-reliance plausibility in language models.
Plausibility of Model Rationales Plausibility evaluation for NLP models has largely focused on classification models Young et al. 2020, Atanasova et al., 2020 Attanasio et al. 2023). While few works investigate plausibility in language generation (Vata et al. [021] Ferrando et al. 2023). such evaluations typically involve a single generation step to complete target sentence with a token connected to preceding information (e.g. subject/verb agreement, as in "The pictures of the cat [iss/re]"), effectively biasing the evaluation by using a pre-selected token of interest. On the contrary, our framework proposes a more comprehensive evaluation of generation plausibility that includes the identification of context- sentext-sensitive generated tokens as an important prerequisite. Additional background on rationales and plausibility evaluation is provided in Appendix A
THE PECORE FRAMEWORK
PECORE is a two-step framework for identifying context dependence in generative language models. First, context-sensitee tokens identification (CTI) selects which tokens among those generated by the model were influenced by the presence of the preceding context (e.g. the feminine options "alla pastorella, le" in Figure[]. Then, contextual cues imputation (CCI) attributes the prediction of sontext-sensitive tokens to specific cues in the provided context (e.g. the feminine cues "she, Le pecore" in Figure D. Cue-farget pairs formed by influenced target tokens and their respective influential context cues can then be compared to human rationales to assess the models' plausibility of context reliance for contextual phenomena of interest. Figure 2 provides an overview of the two steps applied to the context-awars MT setting discussed by this work. A more general formalization of the framework for language generation is proposed in the following sections.
Notation Let X be the sequence of contextual inputs containing N tokens from vocabulary V, composed by current input x, generation prefix y<i and context C. Let X Xio be the non-comtextual input in which C' tokens are excluded Pax = P (I, Vei. C, 0) is the discrete probability distribution over V at generation step of a language model with @ parameters receiving contextual inputs X Similarly. Pi = P (z. Vei, 0) is the distribution obtained from the same model for non-contextual input X no-etx- Both distributions are equivalent to vectors in the probability simplex in RI and we use Pax (y/) to denote the probability of next token y/ in Pa i.e. P(ya x. y<o; C).
3.1 CONTEXT-SENSITIIE TOKEN IDENTIHICATION
CTI adapts the contrastive conditioning paradigm by Vamvas & Sennrich (022 a) to detect input context influence on model predictions using the contrastive pair Pix. Pr: cux Both distributions are relative to the contextual target sentence ğ = {ύ1 ... Hn ). corresponding to the sequence produced by decoding strategy of choice in the presence of input context. In Figure 2 the contextual target sentence أ = "Sont-elles l'hôtel?" is generated when x and contexts Cz, Cj are provided as inputs, while non-contextual target sentence " = "IIs sont à I' l'hôtel?" would be produced when only is
In the context-aware MT example of Figure 2 C includes source context C2 and target context Cy.
3

---

Published as a conference paper at ICLR 2024
Og J'aime ces assiettes!
f à
Betr prefix
decoding of Vist
Non-contextual
f : Sont-elles l'hôtel?
Sont-elles à l'hôtel?
Sont-
hôtel hôtel
Force decode
x
y Sont-
eelle
?
NMT Decoder
Decader Black N
FALT
fTo
Step
Importance Propagation
Decoder Block N
Decoder Block
Decoder Block
Contrastive
Embedded sources
metrics
NMT Encoder
NMT Decoder
NMT Encoder
Sont:
X
elles
Czt love these
dishes
Cy² J' aime ces assiettes
Ce love these dishes!
Boci
#: Are they the hotel's?
Are they the hotells?
hôtel
X
(dishes, elles), (assiettes, elles)
X
Figure 2: The PECORE framework. Left: Context-sensitive token identification (CTI). ①: A context-aware MT model translates source context (C.:) and current (æ:) sentences into target context (C²) and current (ÿ) outputs. @: ý is forceddecoded the non-contextual setting instead of natural output y. 3: Contrastive metries collected throughout the model for every ÿ token to compare the two settings. ④ Selector Sen maps metrics to binary context-sensitive labels for every dú. Right: Contextual cues imputation (CCI). ①: Non-contextual target j is generated from contextual prefix Ver- ④: Function fior is selected to contrast model predictions with (ūr) and without (Ø,) input context. 3: Attribution method fArt using frer as target scores contextual cues driving it prediction. 4: Selector Seer selects relevant cues, and cue-target pairs are assembled.
provided. In the latter case, ý is instead force-decoded from the non-contextual setting enable direct comparison of matching outputs. We define set of contrastive metries M = {m,,..., mM }. where each Aμyl A 4 + I maps a contrastive pair of probability vectors continuous score. For example, the difference in next token probabilities for contextual and nonn-contextual settings, i.e. Pan(()) with high contrastive metric scores can be identified as context-sensítive provided C is the only added parameter in the contextual setting. Finally, a selector function Sen RIMI + {0, 1) (e.g. statistical threshold selecting salient scores) used classify every 9 as context-sensitive or not.
Pexx (ū.) P P (js). might be used for this purpose [ Target tokens
3.2 CONTEXTUAL CUES IMPUTATION
CCI applies the contrastive attribution paradigm YYn & Neubig 2022 to trace the generation of every context-sensitive token in ý back to context C, identifying cues driving model predictions.
Definition 3.1. Let T be the set of indices corresponding to context-sensitee the CTI step, such that E ÿ and Vt T,, 8en (m.............................. my = Let also fier Aμy X А H R be contrastive attribution target function having the same domain and range as metrics in M. The contrastive attribution method fArt is a composite function quantifying the importance of contextual inputs determine the output of fror for given model with 0 parameters.
tokens identified by
( = fATTLE Jee, C,O, froT) = fATT (I, Iee. Srar(PP Pt. Pi
(1)
Remark 3.1. Distribution Pt in Equation I is from the contextual prefix UCt = ((....... 9r-1)
(e.g. Iec ="Sont"" in Figure? 2 and non-contextual inputs Xt This is conceptually equivalent to predicting the next token of new non-contextual sequence iJ" which, contrary to J. starts from forced contextual prefix tzح (e.g. "ils" in Ø "Sont-ils à l'hôtel?" in Figure 2).
Remark 3.2. Provided that pt and pt depend respectively on contextual and non-contextual inputs X Xt X despite using the same prefix Pet. probabilities Pexx (Wt), Pro--ox(j)) (ūī) are likely to
WWe use m' denote the result of ml Pi Pr Several metrics are presented Section 4.2
4

---

Published as a conference paper at ICLR 2024
differ even when Ot = 37, i.. even when the next predicted token is the same, it is likely to have different probability in the two settings, ultimately resulting in non-zero fror and farr (tt) scores.
Remark 3.3. Our formalization of fATT generalizes the method proposed by (Yin & Neubig 2022 to support any target-dependent attribution method, such as popular gradient-based approaches (Si-
monyan al. 2014 Sundararajan al. 2017), and any contrastive attribution target frav
SATT produces a sequence of attribution scores A, = {ai,..., ] matching contextual input length N. From those, only the subset A cTx of scores corresponding to context input sequence C are passed to selector function Seeı IR + {0, 1}, which predicts set C, of indices corresponding to contextual cues identified by CCI. such that Vcc C, Va A, Scr Scci(a,)
Having collected all context-sensitive generated token indices T using CTI and their contextual cues through CCI (C,), PECORE ultimately returns a sequence Sa of all identified cue-target pairs:
T= TT=CTI(C,a2 x, U, 0, sen) = {t (cm .,?)) =
C CCI(T, x, G. fxYr, fIGT, scci) { scci(ae) 1 Va. E Areti Vt WET) T3 S = PECORF((C 0, Sen, M, sxI:: frar) = {(Ce, De) VI E Ve E C2 VC, E EC}
(2)
4 CONTEXT RELIANCE PLAUSIBILITY IN CONTEXT-AWARE MT
This section describes evaluation of PECORE in controlled setup. We experiment with several contrastive metrics and attribution methods for CTI and CCI (Section 4.2 Section 4.4) evaluating them in isolation to quantify the performance of individual components. An end-to-end evaluation is also performed in Section 4.4 to establish the applicability of PECORE in a naturalistic setting.
4.1 EXPERIMENTAL SETUP
Evaluating generation plausibility requires human annotations for context- sensitive tokens in target sentences and disambiguating cues in their preceding context. To our knowl- edge. the only resource matching these requirements SCAT Yin al. (2021). an English- corpus with human annotations of anaphoric pronouns and disambiguating context on OpenSubti- tles2018 dialogue translations (Lison et al. 2018 Lopes et al., (020). SCAT examples were extracted automatically using lexical heuristics and thus contain only a limited set of anaphoric pronouns (ir, they + il/elle, ils/elles), with no guarantees of contextual cues being found in preceding context. To improve our assessment, we select a subset of high-quality SCAT test examples containing con- textual dependence, which we name SCAT+. Additionally, we manually annotate contextual cues in DISCEVAL-MT (Bawden al., 2018), another English->Frend corpus containing handcrafted examples for anaphora resolution (ANA) and lexical choice (LEX). Our final evaluation set contains 250 SCAT+ and 400 DISCEVAL-MT translations across two discourse phenomena
Evaluation Datasets
Models We evaluate two bilingual OpusM'T models (Tiedemann & Thottingal 2020) using the Transformer base architecture (Vaswani al. (017) (Small and Large), and mBART-50 1-to- many (Tang et al. 202T), larger multilingual MT model supporting 50 target languages, using the Transformers library (Wolf al., 2020). We fine-tune models using extended translation units (Tiede mann & Scherrer 2017) with contextual inputs marked by break tags such as "source context <brk> source current" to produce translations in the format "target context <brk> target current", where context and current target sentences are generated" We perform context-aware fine-tuning on 242k IWSLT 2017 English~>Frent Freech examples (Cettolo al. (017), using a dynamic context size of 0-4 preceding sentences to ensure robustness to different context lengths and allow contextless usage. To further improve models' context sensitivity, we continue fine-tuning on the SCAT training split, containing 1k examples with inter- and intra-sentential pronoun anaphora.
Model Disambiguation Accuracy We estimate contextual disambiguation accuracy by verifying whether annotated (gold) context-sensitive words are found in model outputs. Results before and after
Additional precisions and formalization of terget-de attribution methods are provided in Appendix B
*SCAT+: //t wetdaatttttssiisg///eeeeeeee
DISCEVAL-ME https:://hf codat//tt/attt//iseee/
disc_eval_mt Appendix
(rrenndixF]deeertte Eedecrrss the annotation process and presents some examples for the two datasets. "Context-aaaee MT model using only source context also evaluated in Section 4.5 and Appendix D
5

---

Published as a conference paper at ICLR 2024
SCAT+
DISCEVAL-MT (ANA)
DISCEVAL-MT (LEX)
Model
BLEU
OK-CS
BLEU
OK-C8
BLEU
OpusMI Small (default)
29.1
0.14
43.9
0.40
30.5
0.29
OTusMT Small S+Tes
39.1
0.81
0.59
48.1
0.60
0.24
33.5
0.36
0.07
OpusMT Large (defaulr)
29.0
0.16
39.2
0.41
31.2
0.31
OpusMT Large S+Tax
40.3
0.83
0.58
48.9
0.68
0.31
34.8
0.38
0.10
mBART-50 (default)
23.8
0.26
33.4
0.42
24.5
0.25
mBART-50 S+T.i.
37.6
0.82
0.55
49.0
0.62
0.32
29.3
0.30
0.07
Table 1: Translation quality of EN FR MT models before (default) and after (S+Tca) context-aware MT fine-tuning, OK: % of translations with correct disambiguation for discourse phenomena. OK-CS: % of translations where the correct disambiguation is achieved only when context is provided.
context-awand aware fine-tuning are shown in Table We find that fine-tuning improves translation quality and disambiguation accuracy across all tested models, with larger gains for anaphora resolution datasets closely matching fine-tuning data. To gain further insight into these results, we use context- aware models to translate examples with and without context and identify subset of context-sensittie translations (OK-CS) for which the correct target word is generated only when input context is provided the model. Interestingly, we find non-negligible amount of translations that are correctly disambiguated even in the absence of input context (corresponding to OK minus OK-CS in Table [ For these examples, the correct prediction of anbiguous words aligns with model biases, such as defaulting to masculine gender for anaphoric pronoums (Stanovsky et al. 2019) or using the most frequent sense for word sense disambiguation. Provided that such examples are unlikely to exhibit context reliance, we focus particularly on the OK-CS subset results in our following evaluation.
4.2 METRICS FOR CONTEXT-SENSITEE TARGET IDENTIFICATION
The following contrastive metries are evaluated for detecting context-sensitive tokens in the CTI step.
Relative Context Saliency We use contrastive gradient norm attribution (Yin & Neubig, 2022 to
compute input importance towards predicting the next token ÿ; with and without input context. Posi- tive importance scores are obtained for every input token using the L2 gradient vectors norm (Bastings et al. 2022), and relative context saliency is obtained as the proportion between the normalized importance for context tokens e E Cz. Cy and the overall input importance, following previous work quantifying MT input contributions (Voita et al. 2021 Ferrando et al. 2022a Edman et al. 2023).
Ecec, V (Paa(())) Pao-eux (f.)
Vax(Pe cix Pio-ax)
ExExT (Pe Pno-cxx(0)))
Likelihood Ratio (LR) and Pointwise Contextual Cross-mutual Information (P-CXMI) Proposed by Vanvas & Sennrich (2021a) and [eenanddss et al.] (2023) respectively, both metrics frame context dependence as ratio of contextual and non-contextual probabilities.
Polx (U:)
LR(P) ctx Pi ax) Pax (0x) + P Poo-etx (is)
(4) P-CXMI(PI. Pix: Pd. Pi log
(5)
Pho-etx (f:)
KL- KL-Divergence (Kullback & Leibler 1951) between Pi and Pi no-etx is the only metric we evaluate that considers the full distribution rather than the probability of the predicted token. We include it to test the intuition that the impact of context inclusion might extend beyond top-1 token probabilities.
(in( DRL(Pi. | = Σ (ū.) log Puo-ax(11
(6)
S.EV
4.3 CTI PLAUSIBILITY RESULTS
Figure presents our metrics evaluation for CTI, with results for the full test sets and the subsets of context-sensitive sentences (OK-CS) highlighted in Table To keep evaluation simple, we use naive Sen selector tagging all tokens with metric scores one standard deviation above the per-example mean context-sensitive. We also include stratified random baseline matching the frequency of

---

Published as a conference paper at ICLR 2024
Random
Context Saliency
Likel ihood Ratio
P-CXMI
KL- L-Diierrence
1.0
0.8
13000
Macro Fl
0300
0.6
0.4
SCAT+
DiscEval-MT Ana.
DiscEval-MT Lex.
SCAT+
DiscEval-MT Ana.
DiscEval-MT Lex.
Figure 3: Macro F1 of contrastive metrics for context-sensitive target token identification (CTI) using OpusMT Large on the full datasets (left) or on OK-CS context-sensitive subsets (right).
occurrence of context-sensitive tokens in each dataset. Datapoints in Figure 3 are sentence-level macro Fl scores computed for every dataset example. Full results are available in Appendix G
Pointwise metrics (LR, P-CXMI) show high plausibility for the context-sensitive subsets OK-CS across all datasets and models but achieve lower performances on the full test set, especially for lexical choice phenomena less present in MT models' training. KL-Divergence performs on par or better than pointwise metrics, suggesting that distributional shifts beyond top prediction candidates can provide useful information detect context sensitivity. On the contrary, the poor performance of context saliency indicates that context reliance in aggregate cannot reliably predict context sensitivity. manual examination of misclassified examples reveals several context-sensitive tokens that were not annotated such since they did not match datasets' phenomena of interest but were still identified by CTI metrics (examples in Appendix G This further underscores the importance of data-driven end-to-end approaches like PECORE limit the influence of selection bias during evaluation.
4.4 METHODS FOR CONTEXTUAL CUES IMPUTATION
The following attribution methods are evaluated for detecting contextual cues in the CCI step.
Contrastive Gradient Norm (Yin & Neubig 2022 estimates input tokens' contributions towards predicting target token instead of a contrastive alternative. We use this method to explain the generation of context-sensitive tokens in the presence and absence of context.
Atcxx Ve Pi..xt)) E C}
For the choice of figi. evaluate both probability difference (o.((us) Pao-cu (ūs). conceptually similar to the original formulation, and the KL-DDiergeece of contextual and non-contextual distri- butions DxL. (P ll | (io-ccc). We use Vair and VKL. identify gradient norm attribution in the two settings. VKL scores can be seen as the contribution of input tokens towards the shift in probability distribution caused by the presence of input context.
Attention Weights Following previous work, we use the mean attention weight across all heads and layers (Attention Mean, Kim et al. | 2019) and the weight for the head obtaining the highest plausibility per-dataset (Attention Best, Yin et al. 202T) as importance measures for CCL. Attention Best can be seen as best-case estimate of attention performance but is not a viable metric in real settings, provided that the best attention head to capture phenomenon of interest is unknown beforehand. Since attention weights are model byproducts unaffected by predicted outputs, we use only attention scores for the contextual setting Pi and ignore the contextless alternative when using these metrics.
4.5 CCI PLAUSIBILITY RESULTS
We conduct a controlled CCI evaluation using gold context-sensitive tokens starting point to attribute contextual cues S This corresponds to the baseline plausibility evaluation described in Section allowing to evaluate attribution methods in isolation, assuming perfect identification of context-sensitive tokens. Figure
T presents our results. Scores in the right plot are relative to the context-aware OpusMT Large model of Section 4.3 uusin both source and target context. Instead, the
"To avoid using references model generations, we align annotations natural model outputs (Appendix

---

Published as a conference paper at ICLR 2024
Random
v Diff
V KL
Attention Mean
Attention Best
E2E median
1.0
0.8
Macro Fl
0.6
0.4
0.2
SCAT+
DiscEval-MT Ana.
DiscEval-MT Lex.
SCAT+
DiscEval-MT Ana.
DiscEval-MTIL
Figure 4: Macro F1 of CCI methods over full datasets using OpusMT Large models trained with only source context (left) or with source+target context (right). Boxes and red median lines show CCI results based gold context-sensitive tokens. Dotted bars show median CCI scores obtained from context-sensitive tokens identified by KL-Divergence during CTI (E2E settings).
left plot presents results for an alternative version of the same model that was fine-tunen using only source context (i.e. translating Cz. y without producing target context Cy)) Source-only context was used previous context-aware MT studies (Fernandes et al. 2022). and we include it in our analysis to assess how the presence of target context impacts model plausibility. We finally validate the end-to-end plausibility of PECoRE-detected pairs using context-sensitive tokens identified by the best CTI metric from Section 4.3 (KL-Divergence) as the starting point for CCI, and using simple statistical selector equivalent to the one used for CTI evaluation. Results for the OK-CS subset are omitted as they show comparable trends. Full results are available in Appendix H
First, contextual cues are more easily detected for the source-only model using all evaluated methods. This finding corroborates previous evidence highlighting how context usage issues might emerge when lengthy context is provided (Fernandes et al. 2021 Shi et al. (023). When moving from gold CTI tags the end-to-end setting (E2E) we observe a larger drop plausibility for the SCAT+ and DISCEVAL-MT ANA datasets that more closely match the fine-tuning data of analyzed MT models. This suggests that standard evaluation practices may overestimate model plausibility for in-domain settings and that our proposed framework can effectively mitigate this issue. Interestingly, the Attention Best method suffers the most from end-to-end CCI application, while other approaches are more mildly affected. This can result from attention heads failing to generalize other discourse-level phenomena at test time, providing further evidence of the limitations of attention as an explanatory metric (Jain & Wallace 2019 [Bastings & Filippova 2020). While VKL and Van appear the most robust choice across the two datasets, per-example variability remains high across the board, leaving space for improvement for more plausible attribution methods in future work.
5 DETECTING CONTEXT RELIANCE IN THE WILD
We conclude our analysis by applying the PECORE method to the popular Flores-101 MT bench- mark (Goyal et al.) 2022), containing groups of 3-5 contiguous sentences from English Wikipedia. While in previous sections, labeled examples were used to evaluate the effectiveness of PECORE components, here we apply our framework end-to-end to unannotated MT outputs and inspect re- sulting cue-target pairs to identify successes and failures of context-aware MT models. Specifically, we apply PECORE the context-aware OpusMT Large and mBART-50 models of Section 4. using KL-Divergence as CTI metric and V VKL CCI attribution method. We set sen and Sceı to two standard deviations above the per-example average score to focus our analysis on very salient tokens.
Table 2 shows some examples annotated with PECORE outputs, with more examples available in Appendix [ the first example, the acronym MS, standing for Multiple Selerosis, is translated generically as la maladie (the illness) in the contextual output, but SEP (the French acronym for MS. i.c. sclérose en plaques) when context is not provided. PECORE shows how this choice is mostly driven by the MS mention in source context C. while the term sclérose plaques in target context C, is not identified as influential, possibly motivating the choice for the more generic option.
In the second example, the prediction of pronoun elles (they, feminine) depends the context noun phrase mob of market women (foule de femmes du marché in French). However, the correct pronoun referent Le roi et Madame Elizabeth (the king and Madam Elizabeth), so the pronoun should be
8

---

Published as a conference paper at ICLR 2024
1. Acronym Translation (English- French, correct but more generic)
C2 Across the United States of America, there are approximately 400,000 known cases of Multiple Sclerosis (MS) [...]
C2 : Aux Erats-Uniss il y environ 400 000 cas connus de sclémsse en plaques [...]
2: MS affects central nervous system, which is made up of the brain, the spinal cord and the optic nerve.
La SEP affecte système nerveux central, composé du cerveau, de moelle épinière du nerf optique.
La maladie affecte le système nerveux central, composé du cerveau, de moelle épinière et du nerf optique.
Anaphora Resolution (English - French, incorrect)
C² : The terrified King and Madam Elizabeth were forced back to Paris by a mob of imarket women.
Gy : Le ri et Madame Flizabeth ont éæé forcés à revenir à Paris par une foule de Temmes du marché.
2: carriage, they traveled back Paris surroumded by mub of people screaming and shouting threals [...]
: Dans une carriole, ilS sont retoumés Paris entourés d'une foulc de gens hurlant et criant des menaces [...]
Dans une carriole, élles sont retournées à Paris entourées d' unu foule de gens hurlant et criant des menaces [...]
3. Numerie format cohesion (English- French, incorrect)
C₂ : The games kicked off 10:00amm with great weather apurt from mid morning drizzle [...]
Cy Les matchs se sont écoulés 10:00 du matin avec un bcau temps part la nućc du matin [...]
2: South Africa started on the right note when they had comforiable 26-00 win against Zambia.
L. L.Afrique du Sud a commencé sur la bonne note quand ils ont cu une confortable victoire de 26 contre lc Zambia.
9: 1, Afrique du Sud a commencé sur la bonne note quand ils ont eu une confortahle victoire de 26:00 contre le Zambia.
4. Lexical cohesion (English -> Turkish, correct)
C2 The activity of all stars in the system was found to be driven by their luminosity. their rotation, and nothing else.
Cy Sistemdeki bütün ulduzlanı faaliyetlerinin, parlakhklam, rotasyonlari ve başka highir yeyin etkisi almnda oldugunu ortaya gıkarchlar.
The luminosity and rotation are used together to determine a star's 's Rossby number. which is related to plasma flow.
0: Purlaklik ve döngüi yıldzmm plazm akıştyla ilgili Rossby sayısm belirlemek için birlikte kullamhyor. Purlakhk ve rotusyon. bir uldezon plazma akuştyla ilgili Rossby sayısı belirlemek için birlikte kullamiyyor.
Table 2: Flores- 101 examples with cue- cue-target pairs identified by PECORE in OpusMT Large (1,2) and mBART-55 50 (3,4) contextual translations. sensixt-senssitve tokens generated instead of their non-contextual counterparts are identified by CTI, and contextual cues justifying their predictions retrieved by CCL. Other changes in ÿ are not considered context-sensitive by PECORE.
the masculine default ils commonly used for mixed gender groups in French. PECORE identifies this as context-dependent failure due to an issue with the MT model's anaphora resolution. The third example presents an interesting case of erroneous numeric format cohesion that not be detected from pre-defined linguistic hypotheses. In this sentence, the score 26-00 translated as 26 in the contextless output and as 26:00 in the context-aware translation. The 10:00 time indications found by PECORF in the contexts suggest this is a case of problematic lexical cohesion.
Finally, we include an example of context usage for English Turkish translation to test the contextual capabilities of the default mBART-50 model without context-aware fine-tuning. Again, PECORE shows how the word rolasyon (rotation) is selected over döngü (loop) as the correct translation in the contextual case due to the presence of the lexically similar word rotasyonlari in the previous context.
6 CONCLUSION
In this work, we introduced PECORE, novel interpretability framework detect and attribute context usage in language models' generations. PECORE extends the common plausibility evaluation procedure adopted in interpretability research by proposing two-step procedure to identify context- sensitive generated tokens and match them to contextual cues contributing to their prediction. We applied PÉCORE to context-aware MT, finding that context-sensitte tokens and their disambiguating rationales can be detected consistently and with reasonable accuracy across several datasets, models and discourse phenomena. Moreover, an end-to-end application of our framework without human annotations revealed incorrect context usage, leading to problematic MT model outputs.
While our evaluation is focused on the machine translation domain, PECORE can easily be applied to other context-dependee language generation tasks such question answering and summarization. Future applications of our methodology could investigate the usage of in-context demonstrations and thain-of-thoughe reasoning in large language models (Brown et al. 2020 Wei et al., (2022), explore PECORE usage for different model architectures and input modalities (e.g. Appendix J). and pave the way for trustworthy citations in auumentl- generation systems (Borgeaud et al. 2022).

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
Gabriele Sarti, Grzegorz Chrupata and Arianna Bisazza acknowledge the support of the Dutch Research Council (NWO) part of the project InDeep 1ww.....9..1..... We thank the Center for Information Technology of the University of Groningen for providing access to the Hábrók high performance computing cluster used in fine-tuning and evaluation experiments.
REFERENCES
Samira Abnar and Willem Zuidema. Quantifying attention flow in transformers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4190-4197, Online, July 2020. Association for Computational Linguistics. doi:
URL
https://aclanttooooyy.org
acl acl-main..33. 385
Pepa Atanasova, Jakob Grue Simonsen, Christina Lioma, and Isabelle Augenstein. A diagnostic study of explainability techniques for text classification. In Proceedings the 2020 Conference on Empirical Methods in Natural Language Processing ( (EMNLP), pp. 3256-3274, Online, November 2020. Association for Computational Linguisties. doi: 10.18653/v1/20222 1/200////02000.mmmlmmmmmm wwt.::00..mmmmmmmmmme mannll-p. URL
https://aclantthloogs org 2020, smnlp-maing mamin.223 263
Giuseppe Attanasio, Eliana Pastor, Chiara Di Bonaventura, and Debora Nozza. ferret: a framework for benchmarking explainers on transformers. In Proceedings of the 17th Conference of the Europeun Chapter of the Association for Compulational Linguistics: System Demonstrations, pp. 256-266, Dubrovnik, Croatia, May 2023. Association for Computational Linguisties. URL. eacl-demo. 29.
https://acllanchooogyyyoo
Jasmijn Bastings and Katja Filippova. The elephant in the interpretability room: Why use attention as explanation when we have saliency methods? In Proceedings of the Third BlackboxNLP Workshop Analyzing and Interpreting Neural Networks for NLP. pp. 149-155 Online, November 2020. Association for Computational Linguistics. doi: 10. 1ttt://020020000oo///o 1.14. URL
https://aclanthologg org//020/2222 http///220.bbcboonoeeooea blackboxnlp-1.14
Jasmijn Bastings, Sebastian Ebert, Polina Zablotskaia, Anders Sandholm, and Katja Filippova. "will you find these shortcuts?" a protocol for evaluating the faithfulness of input salience methods for text classification. In Proceedings of the 2022 Conference on Empirical Meth- ods in Natural Language Processing, pp. 976-991, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.1665//v1/0222.mmmml/ amnlp-main.66 URL https://acllnnthooogy..r//////// emnlp-main. 64
Rachel Bawden, Rico Sennrich, Alexandra Birch, and Barry Haddow. Evaluating discourse phenom- ena in neural machine translation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computatonnal Linguistics: Human Language Technologies, Volume (Long Papers), pp. 1314-1313, New Orleans, Louisiana, June 2018. Association for Computa- tional Linguistics. doi: 10.18653/v1/NV 11188. URL https: httttp:://aclanthoollo. wrg//18 1erg/N18-1111 1118
Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. Eliciting latent predictions from transformers with the tuned lens. ArXiv, abs/2303.08 2023. URL http:ttts:///rset .twps/ii.ororrgeeeee
08112
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katic Mil- lican, George Bm Van Den Driessche, Jean- -aptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego De Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Ge- offrey Irving, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack Rae, Erich Elsen, and Laurent Sifre. Improving language models by retrieving from trillions of tokens. In Kama- lika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.). Proceedings of the 39th International Conference Machine Learning. volume 162 of Proceedings of Machine Learning Research, pp. 2206-2241 PMLR, 17-23 Jul 2022. URL https://proceedinn mlr httpp/gg........ee///////rreaeee html
10

---

Published as a conference paper at ICLR 2024
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss. Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Fric Sigler, Mateusz I Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, llya Sutskever, and Dario Amodei. Language models are few-shot learners. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS'20 20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN
Mauro Cettolo, Marcello Federico, Luisa Bentivogli, Jan Niehues, Sebastian Stüker, Katsuhito Sudoh, Koichiro Yoshino, and Christian Federmann. Overview of the IWSLT 2017 evaluation campaign. In Proceedings of the 14th International Conference on Spoken Language Translation, pp. 2-14, Tokyo, Japan, December 14-15 2017. International Workshop on Spoken Language Translation. URL
org/ 2017. iwslt-1.1
David Dale, Elena Voita, Loic Barrault, and Marta Ruiz Costa-jussà. Detecting and mitigating hallucinations in machine translation: Model internal workings alone do well, sentence similarity even better. ArXiv, abs/2212.08597,, 2022. URL httpps//aaa.....rr/ab///////2/// 08597
David Dale, Elena Voita, Janice Lam, Prangthip Hansanti, Christophe Ropers, Elahe Kalbassi, Cynthia Gao, Loic Barrault, and Marta R. Costa-jussà, Halomi: A manually annotated benchmark for multilingual hallucination and omission detection in machine translation. ArXiv, abs/2303.081 12, 2023.
org/ brrgg/bss/20 2303/ 2ttp//223300811222
Jay Youon Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C. Wallace. ERASER: A benchmark to evaluate rationalized NLP mod- els. In Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guistics, pp. 4443-4458, Online, July 2020. Association for Computational Linguistics. doi: 20118653/1122020 aww.::000.immmimm000000 URL
org. acl-main.....m..m 408
Zi-Yi Dou and Graham Neubig. Word alignment by fine-tuning embeddings on parallel corpora. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 21 2112 111122228, Online, April 2021. Association for Computational Linguistics. doi: 10.18653/v 101553/v112200 1/202 macl-mmin.n 181. URL htttps://aclanthhol.og/ org/222 leacc main. 181
Esin Durmus, He He, and Mona Diab. FEQA: A question answering evaluation framework for faithfulness assessment in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5055-5070, Online, July 2020. Association for Computational Linguistics. doi: 10.188653/11/20222 lrrg/2000.alll aww.m0n..cmmmii..mmmmmm 454
URL /tsp/
Lukas Edman, Gabriele Sarti, Antonio Toral, Gertjan van Noord, and Arianna Bisazza. Are character- level translations worth the wait? comparing byt5 and mt5 for machine translation. ArXiv, abs/2302. 14220, 2023. URL httph:t/aaaiii.o/r////////////// wtwp:ii..rrr/b///230000 14220
Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. Language-agnostic BERT sentence embedding. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2022. Association for Computational Linguistics. doi: 10.18653/1//0222.cllllle acl.long....... URL
Dublin, Ireland, May
https://aallarthooogg..rr/////// www.aaanthhoooooooo/22222/2 wttp://222...lloonoe.. 62
Patrick Fernandes, Kayo Yin, Graham Neubig, and André F. T. Martins. Measuring and increasing context usage in context-aware machine translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume I: Long Pupers), pp. 6467-6478, Online, August 2021, Association for Computational Linguistics. doi: 10. 10.18653/v1/0011 htt.////a.o..o.og...5.... lol-- https://aclaathhoooy..org.
URL.
acl acl-long.500 .505
Patrick Fermandes, António Farinhas, Ricardo Rei, José G. C. de Souza, Perez Ogayo, Graham Neubig, and Andre Martins. quality-awarr decoding for neural machine translation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational
11

---

Published as a conference paper at ICLR 2024
Linguistics: Human Language Technologies, pp. 1396-1412, Seattle, United States, July 2022. URL https
Association for Computational Linguistics. doi: 10.11665/1122022.mmalmmmiii //aclanthology.orge org 2022. naacl-main 100
Patrick Fernandes, Kayo Yin, Emmy Liu, André Martins, and Graham Neubig. When does translation require context? data-driven, multilingual exploration. In Proceedings of the 61 Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Papers), pp. 606 626, Toronto, Canada, July 2023, Association for Computational Linguistics. doi: 00.18653/v1/20333 aal.ooon.... org/ 2023. acl-long.3e
URL
Javier Ferrando, Gerard I. Gállego, Belen Alastruey, Carlos Escolano, and Marta R. Costa-jussà. Towards opening the black box of neural machine translation: Source and target interpretations of the transformer. In Proceedings of the 2022 Conference on Empirical Methods in Natural Abu Dhabi, United Arab Emirates, December 2022a. Association for Computational Linguistics. doi: 10.16655k1/20222m//pmaaaaa33 URL https //aclanthology.orr org 2222. www.122......2..m.mmmmmmm.....p. emnle- main. 599
Language Processing, pp.
Javier Ferrando, Gerard I. Gállego, and Marta R. Costa-jusså. Measuring the mixing of contextual information in the transformer. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 8698 8714, Abu Dhabi, United Arab Emirates, December 2022b. Association for Computational Linguistics. doi: 10. 12.16663.1/2222..mmmmaa....
URL
https://aclanttooogy..rrgg wrgp:h/oooooo..//222/2//// emmlp- emw.p-main...m..mmmmmmmmmmmmaamm 595
Javier Ferrando, Gerard I. Gállego, Ioannis Tsiamas, and Marta R. Costa-jussà. Explaining how transformers use context to build predictions. In Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5486-5513, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023 acl- 301
301. URL W/RLittpss//acc.entooooooooorrr wrtp://..oooy..oo//........
Mor Geva, Avi Caciularu, Kevin Wang. and Yoav Goldberg. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 30-45, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.18653 /2wgg/ emnlp-main.3
emnlp-main.3. URL http:/ttt//acaaattoooooo
Naman Goyal, Cynthia Gao, Vishrav Chaudhary, Peng-Jen Chen, Guillaume Wenzek, Da Ju, Sanjana Krishman, Mare' Aurelio Ranzato, Francisco Guzmán, and Angela Fan. The Flores 101 evaluation benchmark for low-resource and multilingual machine translation. Transactions of the Association doi: 10. (620//all__000444 URL https
for Computationnal Linguistics, 10:522-538, 2022.
s/aclanthology.ooo www.:/2g.222222 tac1-1.30
Tanya Goyal and Greg Durrett. Annotating and modeling fine-grained factuality in summarization.
In Proceedings of the 2021 Conference of the North American Chapter of the Association for
Computational Linguistics: Human Language Technologies, pp. 1449-1462, Online, June 2021. Association for Computational Linguistics. doi: 10.18653/v//021 mnaccl-min.. 14. URL //aclanthology.orgg//
www.aaalaaannnnmm 114
Ari Holtzman, Peter West, Vered Shwartz, Yejin Choi, and Luke Zettlemoyer. Surface form competi- tion: Why the highest probability answer isn't 't't always right. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 7038-7051, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653) v1/2021 v1/2021.emnlp-maii wmnlp-main.56.......-m
URL http:::://acaanttho..
ort.oorg////22 cannp--main.. 564
Alon Jacovi and Yoav Goldberg. Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4198-4205, Online, July 2020. Association for Computational URL.
Linguistics. doi: 10. 18653/v1/2000 acl- acl-main,386.
wwwpsntthooooooo/oo/eeeo 2020
acc-main 386
Sarthak Jain and Byron C. Wallace. Attention is not Explanation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human anguage Technologies, Volume (Long and Short Papers), pp. 3543-3556, Minneapolis,
12

---

Published as a conference paper at ICLR 2024
Minnesota, June 2019. Association for Computational Linguistics. doi: 1/N19-1111911111
URL htt.ttsss//aclanntlllgg... 9ttp://aannnooooooog//9-/----1 0rg//19-133
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang. Timothée Lacroix, and William El Sayed. Mistral 7b, 2023. URL http::/aaxiv..rrrr abs/2310. 06825
Linghao Jin, Jacqueline He, Jonathan May, and Xuezhe Ma. Challenges in Context-Aware neural machine translation. ArXiv, 011223055.37751 2023. wwl.https://rrra..rr/////////// 13751
Yunsu Kim, Duc Thanh Tran, and Hermann Ney. When and why is document-level context useful in neural machine translation? In Proceedings of the Fourth Workshop on Discourse in Machine Translation (DiscoMT 2019), pp. 24-34, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/w1/D19-550.. URL https::/aaclantthooog.
059-
Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, and Kentaro Inui. Attention not only a weight: Analyzing transformers with vector norms. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 7057-7075, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020..mmill mainip-m URL https
//aclanthology.orga org 2020 http://2222eemmmmmmmmme main. 574
Wojeiech Kryseinski, Bryan McCann, Caiming Xiong, and Richard Socher. Evaluating the factual consistency of abstractive text summarization. In Proceedings of the 2020 Conference Empirical
Methods in Natural Language Processing (EMNLP), pp. 9336, Online, November 2020.
Association for Computational Linguistics. doi: 12116/3//1122222mmmllpl m50. URL https
//aclanthology.org org 2020. http:/200000m.mmmeemmmmmmm emnlp-main. 750
Solomon Kullback and Richard A Leibler. On information and sufficiency. The annals of mathemati- cal statistics, 22(1):79-86 -6, 1951.
Isaac Lage, Emily Chen, Jeffrey He, Menaka Narayanan, Been Kim, Sam Gershman, and Finale Doshi-Velez, An evaluation of the humaan-interpretablity of explanation. ArXiv, Abs/1902.00006 2019. URL
00006
Samuel Läubli, Rico Sennrich, and Martin Volk. Has machine translation achieved human parity? case for document-level evaluation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 4791-4796, Brussels, Belgium, October-November 2018. Association for Computational Linguistics. doi: 10. 11.18863/11/DD1-11555 URL https
Pierre Lison, Jörg Tiedemann, and Milen Kouylekov. OpenSubritles2018: Statistical rescoring of sentence alignments in large, noisy parallel corpora. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, May 2018. Fu- ropean Language Resources Association (ELRA). URL URL.hhtts://aaclanthllee httpeaattoooooooooo///11./ org/L18-/ 1275
António Lopes, M. Amin Farajian, Rachel Bawden, Michael Zhang, and André F. T. Mar- tins. Document-level neural MT: A systematic comparison. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pp. 225-234, Lis- boa, Portugal, November 2020. European Association for Machine Translation. URL httpss 1amt-1.24
Andreas Madsen, Siva Reddy, and Sarath Chandar. Post-hoc interpretability for neural nlp: A survey. ACM Comput. Surv, 55(8), dec 2022. ISSN 0360-0300. doi: 10. 10.1145/3546577, URL httts:://oi.. wrw.oii..rg//1.0 1115/35.111/33445777777
Suvodeep Majumder, Stanislas Lauly, Maria Nadejde, Marcello Federico, and Georgiana Dinu. A baseline revisited: pushing the limits of multi segtiissegeett models for context-aware translation. ArXiv, abs/2210.1000 10906, 2022. htRL:/httss:////t wttp://wwwrrr//////../eeee
10906
13

---

Published as a conference paper at ICLR 2024
Sameen Maruf, Fahimeh Saleh, and Gholamreza Haffari. A survey document- neural machine translation: Methods and evaluation. ACM Comput. Surv. 54(2), mar 2021. ISSN 0360-0300. doi: 10.1145/3441691. URL httpt:ttttoooooooo org 10 44444//444499
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th Anmual Meeting of the Association for Computational Linguistics, pp. 1906-1919, Online, July 2020. Association for Computational 173. URL https:///allanthoooggs orgg
Linguistics. doi: 10. 11886//1122000.ccl-eaea
acl mcl-m.in.173)
Tom McCoy, Ellie Pavlick, and Tal Linzen. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4428-3448, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/v1/P19-- 1334. URL (RLLIttt////aclaathoooog
139-1334
Hosein Mohebbi, Willem Zuidema, Grzegorz Chrupala, and Afra Alishahi. Quantifying context mixing in transformers. In Proceedings of the 17th Conference of the European Chapter of the Asso- ciation for Computational Linguistics, pp. 3378-3400, Dubrovnik, Croatia, May 2023. Association for Computational Linguistics. URL
org/2023. http//222.eeclmmmmme eacl-main. 245
Mathias Müller, Annette Rios, Elena Voita, and Rico Sennrich. A large-scale test set for the evaluation of context-aware pronoun translation in neural machine translation. In Proceedings of the Third Conference on Machine Translation: Research Papers, pp. 61-72, Brussels, Belgium, October 2018. Association for Computational Linguistics. doi: 10.11865/11/188... 6307. URL https://aclanttho..o
org/W18-6307
Matt Post. A call for clarity in reporting BLEU scores. In Proceedings of the Third Con- ference on Machine Translation: Research Papers, pp- 186-191, Brussels, Belgium, Octo- ber 2018. Association for Computational Linguisties. doi: 10.18653/1//18-63113 URL
Ricardo Rei, José G. C. de Souza, Duarte Alves, Chrysoula Zerva, Ana Farinha. Taisiya Glushkova, Alon Lavie, Luisa Coheur, and André F. T. Martins. COMET-22: Unbabel-IST 2022 submission for the metrics shared task. In Proceedings of the Seventh Conference Machine Translation (WMT), pp. 578-585, Abu Dhabi, United Arab Emirates (Hybrid), December 2022, Association for Computational Linguisties. URL httphtttt//caaattollloo
Gabriele Sarti, Nils Feldhus, Ludwig Sickert, Oskar van der Wal, Malvina Nissim, and Arianna Bisazza. Inseq: An interpretability toolkit for sequence generation models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume System Demonstrations), pp. 421 435, Toronto, Canada, July 2023. Association for Computational URL
Linguistics. doi: 10.
org//223
acl-demon 40
Freda Shi, Xinyun Chen, Kanishka Mísra, Nathan Scales, David Dohan, Ed Chi, Nathanael Schärli, and Denny Zhou. Large language models can be easily distracted by irrelevant context. ArXiv, abs/2302.00093, 2023. URL htthtttss////xiiv htrrg/bbs/3300 00093
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. In Yoshua Bengio and Yann LeCun (eds.), 2nd International Conference on Learning Representations, ICLR 2014. Banff, AB, Canada, April 14-16. 2014, Workshop Track Proceedings, 2014. URL 60034
Gabriel Stanovsky, Noah A. Smith, and Luke Zettlemoyer. Evaluating gender bias in machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1679-1684, Florence, Italy, July 2019. Association for Computational Linguistics. URL URI MtLLhhttps://aallee Utl.bttsss/aac.antlooooo..g/./g/
doi: 10. 10.18653vv//P1. 1164
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference Machine Learning (ICML), volume 70, pp.
14

---

Published as a conference paper at ICLR 2024
Journal of Machine Learning Research (JMLR), 2017. URL https://dl. acm org/
doi/10. 011111.5555533388898 3306024,
Joel Tang, M. Fomicheva, and Lucia Specia. Reducing hallucinations in neural machine translation with feature attribution. ArXiv, abs/2211.09878, 2022. URL https::/arxxvvoorr/bb//////// 09878
Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. Multilingual translation from denoising pre-training. In Findings of the As-
sociation for Computational Linguistics: ACL-IJCNLP 2021, pp. 3450-3466, Online, August 2021. Association for Computational Linguistics. doi: 10.11865/////00 http:///222...nnnnnnaa...... findings--acl.34., URL ortp://aw...nttoooo..ooog/ggg/
findings mondiings-cl..300 304
Jörg Tiedemann and Yves Scherrer. Neural machine translation with extended context. In Proceedings of the Third Workshop on Discourse in Machine Translation, pp. 82-92, Copenhagen, Denmark, September 2017. Association for Computational Linguistics. doi: 10.18653/111117-4811 URL
https://aclantthl..og 0rtp:///.og//1118111111
Jörg Tiedemann and Santhosh Thottingal. OPUS-MT - building open translation services for the world. Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pp. 479-480, Lisboa, Portugal, November 2020. European Association for Machine Translation. URL https: /tthttss:///aclaaatt ptatpp://aclanttoolgg.. /rrh//logyoooo////oooooooo/o /errg/222...0 eamt-1.61
Antonio Toral, Sheila Castilho, Ke Hu, and Andy Way. Attaining the unattainable? reassessing claims of human parity in neural machine translation. In Proceedings of the Third Conference on Machine Translation: Research Papers, pp. 113-123, Brussels, Belgium, October 2018. Association for Computational Linguistics. doi: 1..18653/1//11--66
URL htwphttt:: https:://allanthologg. org/
W18-6312
Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar Sanseviero, Alexander M. Rush, and Thomas Wolf. Zephyr: Direct distillation of Im alignment. ArXiv, abs/23 10. 16944, 2023. URL htttp://arxiv.orr/abbe
16944
Keyon Vafa, Yuntian Deng, David Blei, and Alexander Rush. Rationales for sequential predic- tions. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 10314-10332 Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10. 10.18653/v1/20111 http://..112..m1m..mm. https://aclancthlloy.oor www.://.w...r.22.2..p.ppmmaaaaa 807
URL
Jannis Vamvas and Rico Sennrich. Contrastive conditioning for assessing disambiguation in MT: A case study distilled bias. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 10246-10265, Online and Punta Cana, Dominican Republic, November 2021a. Association for Computational Linguistics. doi: 10.18653//1/22211.om www.eemwnnlmmmnne.... URL https://aalaantooogg..orrg//////
wwwep-nnmmmm main. 803
Jannis Vamvas and Rico Sennrich. On the limits of minimal pairs in contrastive evaluation. In Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pp. 58-68. Punta Cana, Dominican Republic, November 2021b. Associ- ation for Computational Linguistics. doi: 111665//1/222..baccoooaaaao //aclanthology.co g/20 http://ww.gggg2......aooooooolpo
URL https
Jannis Vamvas and Rico Sennrich. As little as possible, much as necessary: Detecting over- and undertranslations with contrastive conditioning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). pp. 490-500 Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 18.11653/ 1ec22 URL http::://cclanthooogy.orrr//////
ac1 -sorr.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, L ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus. S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https: /ttps/////ooocceoonss/eo neur
15

---

Published as a conference paper at ICLR 2024
Elena Voita, Pavel Serdyukov, Rico Sennrich, and Ivan Titov. Context-aware neural machine translation learns anaphora resolution. Proceedings the 56th Annual Meeting of the Association Melbourne, Australia,
for Computational Linguistics (Volume 1: Long Papers), pp. July 2018. Association for Computational Linguistics. doi: 10. 11/8//6//111////// 1117. URL httss wtw.aantoooooy.or//11-----111
Elena Voita, Rico Sennrich, and Ivan Titov. Context-aware monolingual repair for neural machine translation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP- IJCNLP), pp. 877-886, Hong Kong, China, November 2019a. Association for Computational Linguisties. doi: 10.18653/v 1/D19-1081
URL httphh::/aaalattholllll/o http://ccllaactooooooooo//////// org/019-1001
Elena Voita, Rico Sennrich, and Ivan Titov. When good translation is wrong in context: Context- aware machine translation improves on deixis, ellipsis, and lexical cohesion. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1198-1212, Florence, Italy, July 2019b. Association for Computational Linguistics. doi: 10. 11138/111111111111 URL
Elena Voita, Rico Sennrich, and Ivan Titov. Analyzing the source and target contributions to predictions in neural machine translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1th International Joinl Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1126 1140, Online, August 2021. Association for Computational Lingussiis.. doi: 10. N0.18653/v1/20/22 acc- 1ong.91.. URL UtLlhttts///accaathhooo
org/
Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohanancy, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman. BLiMP: The benchmark of linguistic minimal pairs for English. Transactions of the Association for Computational Linguistics, 8:377-392, 2020. doi: 10.1162/tacl_ a 00321. URL
.rg ooooor////// 2020. tac1-1.255
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoe V.
Le. and Denny Zhou. Chain-of-thought prompting clicits reasoning in large language models. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35. pp. 24824 24837. Curran Associates, Inc., 2022. URL /tttps:/aarxiv.orgg 2201. 11903
Sarah Wiegreffe and Yuval Pinter. Attention is not not explanation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 11 11-20, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v 1/D19-100 URL https://aclaattoooogy.. /rg
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick vOn Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gug- ger, Mariama Drame, Quentin Lhoest, and Alexander Rush. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38-45, Online, October 2020. Association for Computational Linguistics. doi: 10. 1665///122000emmmp-ddmmsss https:://cclanthol..g www.thhhlogg..rrggg/eooooo wrt.orgg///// 2020. emn1p-demos.6 6
URL
Kayo Yin and Graham Neubig. Interpreting language models with contrastive explanations. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 184-198, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10. 1/5333/1/202.emllppp main. 14. URL htttttss///cllantollglg/// org/ 2022 emnlp-main.
Kayo Yin, Patrick Fernandes, Danish Pruthi, Aditi Chaudhary, André T. Martins, and Graham Neubig. Do context-aware translation models pay the right attention? In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 788 801, Online,
16

---

Published as a conference paper at ICLR 2024
August 2021. Association for Computational Linguisties. doi: 10.18653/w1/2221acl-bb URL httpps///alaatthlllgg org/ a02l.acll-oos 65
Matthew D. Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars (eds.), Computer Vision - ECCV 2014, Pp. 818-833, Cham, 2014. Springer International Publishing. ISBN 978-3-319-10590. doi:
10.110079988333311110990
A BACKGROUND: PLAUSIBILITY OF MODEL RATIONALES IN NLP
A.1 DEFINITIONS
In post-hoc interpretability. a rationale of model behavior is an example-specific measure of how input features influence model predictions. When using feature attribution methods to interpret language models, these rationales correspond to set of scores reflecting the contribution of every token the imput sequence towards the prediction of the next generated word (Madsen et al. 2022).
Plausibility, also referred to as "human- interpretability" (Lage et al., 22122,19) is a measure of "how convincing the interpretation is to humans" (Jacovi & Goldberg 2020). is important to note that plausibility does not imply faithfulness, i.e. how accurately the rationale reflects the true reasoning process of the model (Wiegreffe & Pinter 2019), since a good explanation of model behavior might not align with human intuition.
A.2 EXAMPLE OF CANONICAL PLAUSIBILITY EVALUATION FOR LANGUAGE MODELS
Consider the following sentence, adapted from the BLiMP corpus (Warstadt et al. 2020).
x = A report about the Impressionists has/have won the writing competition.
For the sentence to be grammatically correct, the verb to have must be correctly inflected as has to agree with the preceding noun report. Hence, to evaluate the plausibility of language model for this example, the model is provided with the prefix x' = "A report about the Impressionists". Then, attribution scores are computed for every input token towards the prediction of has as the next token. Finally, we verify whether these scores identify the token report as the most important to predict has.
We remark that the choice of the pair report-has in the canonical procedure described above is entirely based on grammatical correctness, and other potential pairs not matching these constraints are not considered (e.g. the usage of report to predict writing). This common procedure might also cause reasonable behaviors to be labeled as implausible. For example, the indefinite article A might be identified as the most important token predict has since it forcibly followed by a singular noun and can co-occur with has more frequently than report in the model's training data. These limitations in the standard hypothesis-driee ddiven approach plausibility evaluation motivate our proposal for PECoRe as a data-driven alternative.
A.3 METRICS FOR PLAUSIBILITY EVALUATION
In practice, the attribution procedure from the example above produces a sequence Im of length |a'| containing continuous importance scores produced by the attribution method, and these are compared to a sequence I of the same length containing binary values, where Is correspond to the cues identified by human annotators (in the example above, only report), while the rest of the values are set to O. In our experiments, we use two common plausibility metrics introduced by Young al. 2
Token-level Macro F1 is the harmonic mean of precision and recall at the token level, using J, as the ground truth and a discretized version of Im as the prediction. Macro-averaging is used to account for the sparsity of cues in In. and the discretization is performed by means of the selector functions Sen: Seeı introduced in Section 3] We use this metric in the main analysis as the discretization step will likely reflect more realistic plausibility performance, it matches more closely the annotation process used to derive 1,. We note that Macro Fl can be considered a lower bound for plausibility, as the results depend heavily the choice of the selector used for discretization.
17

---

Published as a conference paper at ICLR 2024
Area Under Precision-Recall Curve (AUPRC) is computed as the under the curve obtained by varying threshold over token importance scores and computing the precision and recall for resulting discretized Im predictions while keeping I, the ground truth. Contrary to Macro F1, AUPRC is selector-independent and accounts for tokens' relative ranking and degree of importance. Consequently, it can be Since this is not likely to be the case in practice but can still prove useful, we include AUPRC results in Figure 6 and Figure
upper bound for plausibility, if the optimal selector was used.
B PRECISIONS ON TARGET-DEPENDENT ATTRIBUTION METHODS
Definition B.1. Let 8, s' be the resulting scores of two attribution target functions fror fror- An attribution method SATt is target-dependent if importance scores A are computed in relation to the outcome of its attribution target function, i.e. whenever the following condition is verified.
SATT(E, Vct. C,O,8) * fam[L, Vec C,0,6') 4s
In practice, common gradient-basee attribution approaches (Simonyan al. 2014 Sundararajan et al.] (2017) are target-dependent as they rely on the outcome predicted by the model (typically the logit the probability of the predicted class) differentiation target to backpropagate importance approaches (Zilerr & Fergus (014) use the variation in prediction probability for the predicted class when noise added some of the model inputs to quantify the importance of the noised features.
to model input features. Similarly, perturbation-base
On the contrary, recent approaches relying solely on model internals define input importance are generally target-insensitive. For example, attention weights used as model rationales, either in their raw form after a rollout procedure obtain unified score (Abnar & Zuidema 2020), are independent of the predicted outcome. Similarly, value zeroing scores (Mohebbi et al. (2223) reflect only the representational dissimilarity across model layers before and after zeroing value vectors, and as such do not explicitly account for model predictions.
C PECORE IMPLEMENTATION
Algorithm| I provides pseudocode implementation of the PECORE cuc-target pair extraction process formalized Section
D FULL TRANSLATION PERFORMANCE
Table presents the translation quality and accuracy across all tested models. We compute BLEU using the SACREBLEU library (Post) 2018) with default parameters n eff: tok: 13| smooth: exp version: and compute COMET scores us- ing COMET-22 (Rei et al., 2022) (v2.0.2). The models fine-tuned with source and target con- text clearly outperform the ones trained with source only, both in terms of generic transla- tion quality and context-sensitive disambiguation accuracy. This motivates our choice to fo- cus primarily on those models for our main analysis. All models are available the follow- ing Huggingface organization: https://hf ctt wwwpee///ooootttcooccccoo The Sax models correspond to those matching oot.aaactaaaaaaaa-oooooooo----oo -nnf while S - Tax models have the Bantetttmn/cttt--ooLLL----r-reee
identifier.
E DATASETS ANNOTATION PROCEDURE
The original SCAT test set by Yin et al. 2021) contains 1000 examples with automatically identified context-sensitive pronouns it/hhey (marked by <p>... <p>...<<,pp and human-annotated contextual cues aiding their disambiguation (marked by <honz... <\hoff>). Of these, we find 38 examples containing malformed tags and several more examples where an unrelated word containing it or they wrongly marked context-sensitive (e.g. the soccer ball h<p>it</p> your chest). Moreover, due to the original extraction process adopted for SCAT, there is no guarantee that contextual cues will be contained in the preceding context they could also appear in the same sentence, defeating
SCAT+
18

---

Published as a conference paper at ICLR 2024
Algorithm 1: PECORE cue-target extraction process
Input: C, - Input context and current sequences
0 - Model parameters
ScT: Sccı Selector functions
M Contrastive metrics
f ATT Contrastive attribution method
fror Contrastive attribution target function
Output: Sequence Sa cue-target token pairs
Generate sequence ò from inputs C, using any decoding strategy:
Context-sensitive Target Identification (CTI):
T- Empty set to store indices of context-sensitive target tokens of y for E do for m E .M do
m' = m, (ū)), Pn euo (9:))
8cm (mf m
then
Store in set
Contextual Cues Imputation (CCI):
S - Empty sequence to store cue-target token pairs; for t E do
Generate constrained non-contextual target current sequence jJ" from Je:i
Use attribution method fATT using frer as attribution target get input importance scores AG Identify the subset AteIx corresponding to tokens of context C' = [Ci,..., Ck};
for a; E Atc = (di,...,44 do
if (Us) = 1 then Store (C;; (i)) Sa
return S
SCAT+
DISCEVAL-MTT (ANA)
DISCEVAL-MT (LEX)
Model
BLEU
COMET
OK
OK-CS
BLEU
COMET
OR
OK-CS
BLEU
COMET
OK
OK-CS
OP Small (defauit)
29.1
0.14
.888
30.5
.763
OpusMI Small Scn.
36.1
.812
0.84
0.42
47.1
.000
0.61
0.28
28.3
.764
0.31
0.05
OpusMI Small S+T...
.816
0.59
.899
0.60
0.24
33.5
.774
0.07
Opu Large (default)
29.0
.006
0.16
39.2
.891
0.41
31.2
.771
0.31
OpusMT Large Sco.
0.28
32.2
Op Large S+T.
40.3
.27
0.83
0.58
48.9
.896
0.68
0.31
34.8
.787
0.38
0.10
50 (defanit)
0.52
mBAKT:50 Sax
33.5
.008
0.87
0.42
36.3
.869
0.57
0.23
25.7
.760
0.29
0.06
mBART-50 50 S+T.x.
37.6
814
0.82
0.55
49.0
.895
0.64
0.29
29.3
.767
0.30
0.07
Table 3: Full model performances on EN FR test sets before (default) and after context-aware MT fine-tuning. Setz and S+T are context-aware model variants using source-only and source+target context, respectively. OK: % of translations with correct disambiguation for discourse phenomena. OK-CS: % of translations where the correct disambiguation is achieved only when context is provided.
the purpose of our context usage evaluation. Thus, we prefilter the whole corpus to preserve only sentences with well-formed tags and inter-sentential contextual cues identified by original annotators. Moreover, a manual inspection procedure is carried out to validate the original cue tags and discard problematic sentences, obtaining final set of 250 examples with inter-sentential pronoun coreference. SCAT+ is available on the Hugging Face Hub:
DISCEVAL-MT We use minimal pairs in the original dataset by Bawden et al. (2018) (e.g. the DiSCEVAL-MT LEX examples Table to automatically mark differing tokens as context-sensitive Then, contextual cues are manually labeled separately by two annotators with good familiarity with both English and French. Cue annotations are compared across the two splits, resulting in very high agreement due the simplicity of the corpus (97% overlap for ANA, 90% for LEX). The annotated available on the Hugging Face Hub: https:://hh co/datasets/
version of DiSCEvAL-Mi inseq/disc_eve eeaec_eellllo. mt
19

---

Published as a conference paper at ICLR 2024
SCAT+
C2 loathe song. But why did you bite poor Birdie's head off? Because I've heard times than care to. haunts
me. Just stop, for moment.
Cy cette Chanson (song, FEMININE). Mais pourquoi avoir parlé ainsi pauvre Birdie Parce que j' entendu ce
chant plus que fois que ne le peux. (she) hante. Arrétez-vous moment.
How does it haunt you?
Comment peut alle (she) yous hanter?
C.i Ah! Sven! It's been long. Riley, it's good you. You, too. How's the boatt? Ub, iil creaks, ii groams,
Cy Sven! 1- Riley, contente de te voir. Content aussi. Comment va le bateun (boat, MASCULINE)? III (hc) craque de purtout.
: Not
y: II (he) n'est pas rupide qu avvant.
DISCEVAL-MT ANA
C. how do you the woman isn't going like all the others?
Cy Mais comment sais que l'emme (woman, FEMININE) finira pas comme toutes les autres?
This one's s different.
Celle-ci (This one, FEMININE) est différente.
C₂ Can you authenticate these Signatures, please?
Cy Pourriez-vous uuthentifier ces signatures (FEMININE), s'il vous pluit?
Yes, they' mines.
Oui, ce sont les miennes (mines, FEMININE).
DISCEVAL-MT LEX
C. Do you think you Shool from here?
Cy Tu penses que tu peux le lirer (shoot) dessus partir d' d'ici?
: Hand me that bow.
y Passe-mol moi cet are (bow, WEAPON).
C2 help you with Wrapping?
Cy Est-ce que peux t' l'uider pour I" "embullage (wrapping)?
Hand me that bow..
y: Passe-moi ce ruhan (bow, GIFT WRAP).
Table 4: Examples from the SCAT+ and DISCEVAL-MT datasets used in our analysis with high- lighted context-sensitivg tokens and contextual cues used for plausibility evaluation using PECORE. Glosses are added for French words of interest to facilitate understanding.
Table 4 presents some examples for the three splits. By design, SCAT+ sentences have more uniform context-sensitive targets (it/they - il/elle/its/elles) and more naturalistic context with multiple cues to disambiguate the correct pronoun.
F TECHNICAL DETAILS OF PECORE EVALUATION
Aligning annotations Provided that gold context-sensitive tokens are only available in annotated reference translations, a simple option when applying CCI to those would involve using references as model generations. However, this was shown to be problematic by previous research, as it would induce a distributional discrepancy in model predictions (Vamvas reason, we let the model generate natural translation and instead try to align tags to this new sentence using the AWESOME aligner (Dou & Neubig 2021 with LABSE multilingual embeddings (Feng al. 2022). While this process not guaranteed to always result in accurate tags, it provides good approximation gold CTI annotations on model generation for the purpose of our assessment.
Sennrich, (0226). For this
G FULL CTI RESULTS AND CTI PROBLEMATIC EXAMPLES
CTI Results FigureBJant and Figure 6 present the CTI plausibility of all tested models for the Maero F1 and AUPRC metrics, similarly to Figure 3 in the main analysis.
CTI Problematic Examples Table 5 shows some examples of OpusMT Large S+T translations considered incorrect during CTI evaluation as the highlighted word do not match the gold SCAT+ labels. However, these words are correctly identified as context-sensitive by CTI metrics as they reflect the grammatical pronoun formality adopted in preceding French contexts Cy- The reason behind such mismatch is that +CAT+ annotations focus solely on gender disambiguation for anaphoric pronouns. Instead, CTT metrics detect all kinds of context dependence, including the formality cohesion shown
20

---

Published as a conference paper at ICLR 2024
Pronoun Grammatical Formality, SCAT+
C. Oh. Dr. Owens. My son posted his Facebook page that he has bullet in his lung. [..]]
Cy Salut, Dr. Owens. Mon fils posté sur page Facebook qu 'il une bulle dans poumon [..]]
a: And when the soccer hall hit your chest, it dislodged it. [...]
y Et quand balle de football touche votre (your, 2ND PLUR., FORMAL) poitrine, elle déplace. [...]
C. [..]] That demon that was in you, it wants you. But not like before. think it loves you.
Cy Ce démon qui était
vous veut. Mais pas comme avant. Je pense qu il vous aime.
And it's powerless without you.
y: Et est impuissant sans yous (you, 2ND P. PLUR., FORMAL..
C. You threaten my father ugain, I'll kill you myself... on this road. You heur me? My quarrel was with your father.
Cy Tu menaces encore mon pére, je tuerai moi même... sur cette route. eneeteedd?? Ma querelle était avec ton père.
Now it is with you well.
Maintenant elle est aussi avec toi (you, 2ND P. SING., INFORMAL).
C. She went back to Delhi, What do you think? [..] Girls, tell you.
Cy Elle est revenue Delhi, Qu'en penses-tu? [..] Les filles. je to le dis.
wish they were all like you. a'ammrriis qu' clles soient toutes comme toi (you. 2ND SING., INFORMAL).
Table 5: Examples of SCAT+ sentences with context-sensitive words identified by CTI but not originally labeled context -eeeext-dependent since they do not match the gendered pronoun rule match used to create SCAT+, Glosses are added for French words of interest to facilitate understanding.
in Table | examples. This suggests our evaluation of CTI metrics plausibility can be considered a lower bound, as it is restricted to the two phenomena available in the datasets we used (anaphora resolution and lexical choice).
H FULL CCI RESULTS
Figure 7 and Figure 8 present the CCI plausibility of all tested models for the Macro Fl and AUPRC metrics, similarly to Figure 4 in the main analysis.
I ADDITIONAL FLORES- 101 PECORE EXAMPLES
Table B provides additional examples of end-to-end PECORE application highlighting interpretable context usage phenomena in model generations. English- + French examples apply PECORE to the context-aware mBART-50 model fine-tuned with the procedure of Section 4.1 Examples with other target languages instead use the base mBART-50 model without any context-aware fine-tuning.
Table 7 provides additional examples of end-to-end PECORE application using the English French examples using the OpusMT Large S+Tetz context-aware model introduced in Section 4.1
1.1 MBART-50 EXAMPLES DESCRIPTION
1. goods is translated as biens rather than marchandises to maintain lexical cohesion with biens in the preceding context. centrale (central), which is correctly lowercased to match its previous occurrence using the same format. The verb taxées (taxed, feminine) is also changed to masculine (taxés) to reflect the change in grammatical gender between marchandises (feminine) and biens (masculine), but is not marked as context-dependent as it does not depend directly on cues C, or Cy,
2. the correct translation of reindeers (rennes) is performed in the con-contextual case, but the same word is instead translated as renards (foxes) in the contextual output, leading to an incorrect prediction due to the influence of lexical cohesion.
3. ce (it, neutral) is translated as elle (it, feminine) agree with the referred noun grotte (cave, feminine) in the context.
4. The imperfect verb form étaient (they were) is selected instead of the passé composé ont été (have been) to avoid redundance with the same tense used in the expression ont été stoppées (they have been stopped) in the context.
21

---

Published as a conference paper at ICLR 2024
Random
Context Saliency
Likelihood Ratio
P-CKM
KL RL-Divergence
1.0
Macro F1
CD
0800
G-11
0.4
E
0.2
SCAT+
DiscEval-MT Ane.
DiscEval-MT Lex.
SCAT+
DiscEval-MT Ana.
DiscEval-MT Lex.
Random
Context Saliency
Likelhood Ratio
P-CKMI
KL- Dl-Vieergence
Macro F1
0.2
SCAT+
DistEval-M MT Are.
DiscEvaiMT Lex.
SCAT+
DiscLval-MT Ana.
DiscEval-M Lex.
Random
Context Salioncy
Likelihaod Ratio
P-CKMI
KL-Divergence
Macro F1
8
SCAT+
DiscEval-MT Ane.
DiscEval-MT Lex.
SCAT+
DiscEval:MT Ana.
DiscEval-MT Lex.
Random
Context Saliency
Likelihood Ratio
P-CKMI
KL Divergence
0.8
Macro F1
0.2
SCA1+
Disttval-MT Are.
DistEvalMT Lex.
SCAT+
DiscLval-MT Ana.
DiscEval-M Lex.
Random
Context Sallency
Likelihood Ratio
P-CXMI
KL Divergence
0.8
OC00R
o-1 0O00
Macro
0.6
0.4
Lex.
DiscEvai-MT
Random
Likelihood Ratio
P-CXII
Diveerggnnce
2.0
0.8
Macro
0.6
+
0.2
SCAT
DiscEval.MT
DIscEval.MT
DiscEval.MT Ana.
DiscEval.MT
Figure 5: Macro Fl of contrastive metrics for context-sensitive target token identification (CTI) the full datasets (left) or on OK-CS context-sensitive subsets (right). Top to bottom: ! OpusMT Small Sete 2 OpusMT Large Sare 3 mBART-50 Sat 4 OpusMT Small S+T S OpusMT Large
S+T K mBART-50 S+Tate-
22

---

Published as a conference paper at ICLR 2024
Random
Context Saliency
Likelihood Ratio
P-CMI
KL-Divergence
AUPRC
9
SCAT:
DiscEval:MT Ana.
DiscEval-MT Lox.
SCAT:
DiscEval:MT Ana.
DiscEval:MT Lox.
Random
Context Saliency
Likellhood Ratio
P-CMMI
RL-Divergence
AUPRC
0500
SCAT+
DisiEval MT Ana.
MiscEval Lox.
SCAT+
DiscEval Ana.
DiscEval Lox.
Random
Context Saliency
Likelihood Ratio
P-CMMI
KL-Divergence
AUPRC
8
SCATI
DiscEval:TT Ana.
DIscEval-MT Lox.
SCAT
DiscEval.MT Ana.
DiscEval-AT Lex.
Random
Context Salienty
Likelihood Ratio
P.C
KL-Divergence
AUPRC
EF
SCAT+
DisiEval MT Ana.
DiscEval MT Lex.
SCAT+
DiscEval Ana.
DiscEvai Lox.
Random
Contect Saliency
Likelihood Ratio
P-CXMI
KL-Divergence
AUPRC
0.000
80000
OXD
SCAT
DisiEval.MT Ana.
DIscEval.MT Lox.
SCAT
DiscEval.MT Ana.
DiscEval.MT Lox.
Random
Context Salienty
Likelihood Ratio
P-CXM
KL-Divergence
AUPRC
8
SCAT+
DisiEval MT Ana.
MT=Evaa-MT.. Lex.
SCAT+
DiscEval Ana.
DiscEvai Lex.
Figure 6: Area Under DoecisionRRecall Curve (AUPRC) of contrastive metrics for context-sensitive target token identification (CTT) on the full datasets (left) or on OK-CS context-sensitive subsets
(right). Top to bottom: ① OpusMT Small Sar 2 OpusMT Large Sar 3 mBART-50 Satz
OpusMT Small S+T.r 5 OpusMT Large S+T mBART-50 S+Tore-
23

---

Published as a conference paper at ICLR 2024
Random
v Diff
KL
Attention Mean
E2E median
1.0
0.8
Macro
0.4
0.0
SCAT+
DiscEval Ana.
SCAT+
DiscEval MT Ana.
Random
Diff
KL
Attention Mean
Attention Best
E2E median
1.0
0.8
Macro
0.6
0.4
SCAT+
DiscEval-MT Ana.
DiscEval MTT Lex.
SCAT+
DiscEval-MT Ana.
DiscEval-MT Lex.
Random
v Diff
KL
Attention Mean
E2E median
1.0
0.8
0.6
Macro
0.4
00
0.2
SCAT+
Figure 7: Macro FI CCI methods over full datasets using models trained with only source context (left) or with tturee+target context (right). Boxes and red median lines show CCI results based on gold context-sensitive tokens. Dotted bars show median CCI scores obtained from context-sensitive tokens identified by KI--Divergence during CTI (E2E settings). Top to bottom: 1 OpusMT Small Sate and S+T ② OpusMT Large Seta and S+T 3 mBART-50 Seta and S+T
24

---

Published as a conference paper at ICLR 2024
1.0
100
00
0.8
1800
AUPRC
0.6
0.2
SCAT
DiscEval-MT
SCAT
DiscEval-MT Ana.
Random
v
Attention Mean
E2E median
00:000
0.8
10
0.6
080
0.4
000
0.2
0.0
DiscEval
Random
v Ditf
KL
Attention Mean
E2E median
1.0
0.8
AUPRC
0.4
0.0
SCAT+
DiscEval-MT Ana.
SCAT+
DiscEval-MT Ana.
Figure 8: Area Under Precision- Reciall Curve (AUPRC) of CCI methods over full datasets using models trained with only source context (left) with source+target context (right). Boxes and red median lines show CCI results based on gold context-sensitivy tokens. Dotted bars show median CCI scores obtained from context-sensitive tokens identified by KL-Divergence during CTI (E2E settings). Top to bottom: ① OpusMT Small Setz and S+T ② OpusM'T Large Setz and S+T Tetz mBART-50 Setz and S+T
25

---

Published as a conference paper at ICLR 2024
Lexical and casing cohesion (English French, correct)
C. don't 't know if you realize it, but most of the goods from Central America came into this country duty-free.
Cy Je ne suis pas si vous lo réalisez, mais la plupart des ① biens d' A'mériqu @ centralé sout venus ici on franchisc.
Yet eighty percent of goods were taxed through tariffss Central American countries.
Pourtant, 80 % de nos ( marchandises ont été saxées par des tarifs dans les pays d' A'Aérique (2) Centralé.
0 : Pourtant, 80 nos ① biens ont été taxés par des taril's dans les pays Amérique ② centrale.
Lexical cohesion (English- - French, incorrect)
C. Reindeer husbandry important livelihood among the Sámi [..]]
Cy: I, L'élevaee de renards est un important gugne-pain parmi les Samis [..]]
Even traditionally, though, not all Sámi have been involved big scale reindeer husbandry.
9: Même traditionnellmment, cependant, tous les Samis ne sont pas impliqués dans l'élevage de rennesi à grande échelle.
0 Même traditionnlllmeet. cependant, tous les Samis ne sont pas impliqués dans I'élevage de renards à grande dchelle.
Anaphora Resolution (English- + French, correct)
C, [...] Resting on lop of one
mountains north Mecca, lhe cave is completely isolated from rest the world. Cg [...] Seul au sommet d' une des mentagnes au nord de la Mooque, groute est completement isolée du reste du monde.
x: In fact, it is not easy to find at all even if one knew it existed.
En fail, n est pas simple trouver même si sait que ça existe.
9 En fait, alle n'es est pas facile à trouver même si on sait qui 'elle existe.
4. Verb form choice (English + French, correct)
C After the dam was built, the scasonal floods that would spread sediment throughout the river were halted.
C Apees construction du barrage, les inondations saisonnières qui répandent les sédiments dans la rivibre été stoppdes.
This sediment was necessary for creating sandhars and beaches
g Cas sédiments été nécessaires pour eréer des borrières de sable et des plages
Cas sédiments étaient nécessaires pour créer des boncs de sable des plages
5. Word Sense Disambiguation (English- - French, incorrect)
C. Rip currents are the returning flow from waves breaking off the beach, often at reef or similar.
Gy Les courants Rip sont les flux revenant des vagues qui se forment sur la plage, souvent sur un récif ou un point similaire.
Due underwater topology the return flow is concentrated
few deeper sections
0: En raison de la topologie sous marine, le flux renouvelable est concentré à quelques parties plus profondes
9 En masson la topologie mous-narine, le flux revenant est concentré dons quelques parties plus profondes
Lexical cohesion (English - French, incorrect)
C, Murray lost first set le break after both men held each and every serve set.
C, Murna perdu le premier jeu d'une rupture de gravate après que les deux hommes aient tenu chacun des coups.
z: Del Potro had the early advantage in the second set, but this too required tie break after reaching 6-6.
dans second jeu, mais il fallu une rupture de gryon après avoir atteint 6-6. 9 Del Potro a eu I' l'avantage précoce dans le second jeu, mais a fallu une rupture de sravate après avoir atteint 6-6.
0: Del Potro l'avantage
Word Sense Disambiguation (English- Turkish, correct)
C2 Every momiing people leave small country towns cars to go their workplace and are passed by others whose work destination is the place they have just left.
Cy Her sahah insanlar isyerlerine gitmek için arabayla küklükk kirsal kentlri terk ediyor ve is noktasımnn henüiz terk ettikleri yer oldugu bagkalan tarafndaan geçtiler.
x: In this dynamic transport shuttle everyone samehow connected with, and supporting, a transport system based on private cars. gemisande herkes hir yekilale özel arabalara dayalt bir tagunacilik sistemiyle baglantili ve destekleniyor. 9: Bu dinamik tagumacshik nakil aracunda herkes dizel arahalara dayal bir taşımaclik sistemiyle hir zekilde haglantih ve destekli.
g: Bu dinamik
Lexical Cohesion (English- + Dutch, correct)
Rip currents are the returning flow from waves breaking off the beach. often at a reef or similar.
Ripstromen zijn de lerugkerende stroom van golven die van strand afbreken, vaak op een rif iets dergelijks.
Due to the underwater copology the return flow is concentrated at few deeper sections
Door de onderwatertogulogie is de lerugkeerde stroom gecomcentreend op een puar diepere delen.
9 Door de onderwatetopologic de terugkerende stroom goconcentreerd op een paar diepere delen.
Lexical Cohesion (English- Italian, correct)
C. Virtual teams are held the same standards of excellence as conventional teams, hut there are subrle differences.
G Le squadre virtuali hamno gli stessi standard di eccellenza delle squadre tradizionali, ma ci sono sottili differenze. Virtual team members often function as the point of contact
immediate physical group.
8 I membri dell' Squipe virtuale spesso funzionano come punto di contatto per il proprio gruppo fisico immediato.
g I membri delle squadré virtuali spesso funzionano come punto di contatto del loro gruppo fisico immediato.
Table 6: Flores- examples with cue-target pairs identified by PECORE in mBART-50 contextual translations towards French (top) and other languages (bottom). Context-sensitive tokens generated instead of their non-contextual counterparts are identified by CTI, and contextual cues justifying their predictions are retrieved by CCL. Other changes in ğ are not considered mnntext-sensitiee by PECORF
26

---

Published as a conference paper at ICLR 2024
Word Sense Disambiguation (English -> French, incorrect)
C. Murray lost the first set in lie hreak after both men held each and every serve in the set.
Cy: Murna perdu premier set dans match nul (void match) après que les deux hommes aient les deux servis,
Del Potro had the carly advantage in the second set, but this too required tie break after reaching 6-6.
9: Potro en l'avantage dans second set, mais ça nécessitait serrurier (locksmith) uprès avoir atteint
9: Del Potro eu I' avantag dans le second set, mais ça aussi nécessitait set nul (void set) après 6-6.
Lexical Cohesion (English- -> French, correct)
C, Giancarlo Fisichella lost control of his car and ended the race very soon after the start.
Giancarlo Fisichella perdu le contrôle de sa voiture et a imisi (put) fin course très peu de temps après départ.
His teammate Fermando Alonso in the lead for most of the mee, but ended right after his pit stop [...] Son coéquipier Fernando Alonso été en téte pendant la majeure partie de la course, mais il a fermind (ended) juste après son pit stop [...]
Son coéquipier Fernando Alonso été lête pendant la majeure partie de la course, mais il mis (put) fin juste après arrét [...]
Acronym Expansion (English- French, correct)
C Martelly swore in a new Provisional Electoral Council (CEP) of nine members yesterday.
C Martelly juré devant un Conseil Electoral (clectoral) provisoire composé de neuf membres hier.
It Martelly's 's fifth CEP four years.
i C'est le cinquième Conscil Europsen (European) de Martelly en quatre ans.
9 C' cinquième Conseil électoral (electoral) provisoire de Martelly en quatre ans.
Lexical cohesion (English French, correct)
lakes and coastal archipelagos. Cg Le pays des 1000 laes a aussi des milliers d l'iles, dans les lacs et dans les archipels cótiers.
C. [...] The "Land of thousand lakes" has thousands of islands
the archipelagos and you not necessarily need yacht.
Dans I' (the, SINGULAR) archipel et les lacs, vous n' pas forcément besoin d' yacht.
9: Dans les (the, PLURA.I)) archipels et les lacs, ow n aa pas forcément hesoin d' l'un yacht.
Word Sense Disambiguation (English- French, correct)
C. ( Ring's 's CEO, Jamie Siminoff, remarked the company started when his dooorell wasn't audibll from his shop in his garage.
C, [[..] Jamic Siminoff, PDG de Ring, avait fait remarquer que l'entreprise commencé quand sonnette n était pas audible
depuis son garage.
He Wifi doar hell, said.
ll construit une porte (door) WiFi, d-t-il dit.
0 1l construit une sonnette (doorbell) WiFi, il dit.
Table 7: Flores- 101 examples with cue-target pairs identified by PECORE in OpusMT Large con- textual translations. Context-sensitive tokens generated instead of their non-contextual counterparts are identified by CTI, and contextual cues justifying their predictions are retrieved by CCI. Other changes in ğ are not considered context-sensitive by PECORE. Glosses are added for French words of interest to facilitate understanding.
27

---

Published as a conference paper at ICLR 2024
5. The present participle revenant (returning) in the context is incorrectly repeated translate "return flow" as flux revenant.
6. The expression "tie break" incorrectly translated as rupture cravate (literally, break), matching the incorrect translation of the same expression in the context.
J PECORE FOR OTHER LANGUAGE GENERATION TASKS
This section complements our MT analysis and by demonstrating the applicability of PECORF to other model architectures and different language generation tasks. To generate the outputs shown in Table language model with 7B parameters fine-tuned from the Mistral 7B v0.1 pre-trained model (Jiang et al. 2023). We follow the same setup of Section using KL-Divergence CTI metric, VKL as CCI method and setting both Scn and seeı to two standard deviations above the per-example mean.
8 we use Zephyr Beta (Tunstall et al. 2023). state-of-thee conversational decodcr-only
Constrained Story Generation In the first example, the model asked to generate story about Florbz, which is defined as a planet with an alien race only in context Cz. We observe a plausible influence several context components throughout the generation process, leading short story respecting the constraint specified in the system prompt provided as context.
Factual Question Answering In the second example, the model is asked to retrieve date information from the context and perform calculation derive the age of fictional building. While the non- contextual generation ý hallucinates an age and a construction date associated to a real historical landmark, contextual generation ý produces wrong age, but plausibly relies the date provided in C₂ during generation. Interestingly, we can also identify when the system instruction of "keeping answers concise" intervenes during generation.
Information Extraction The last example simulates a retrieval-augmented generation scenario in which fictional refund policy used as context to answer user queries. In this scenario, contextual generation ú correctly identifies the user query leading to no-refund situation due the limited refund timeline stated in the policy, and the corresponding timeline (within 30 days) is identified as contextual cue leading to the model's 's negative response.
28

---

Published as a conference paper at ICLR 2024
Constrained Story Generation
C. : 
 Here is some context about planct 12 Florbz. It is inhabited by race of aliens called Squortz (3), which
Communicate ( only by saying "Qix!" <//>>
: <<user|>> Tell a very short story about Florbz 
 

Florbz was curious little creature, half flower and half buzzing bec. He loved nothing more than exploring the garden he called
home, flitting from bloom to bloom, spreading pollen as he went, 

g On vibrant and colorful planet-2) of Florbz lived curious creature named Zorg. was different from his fellow Squurtz (
he longed to learn their language 4 heyond just "Qix!" 

Factual Question Answering
C² :<< 
 J> Provide concise answer containing only New words D
 <|user |> In the heart of Groningen, nestled between quaint cabbled streets and the meandering canals, stands the Pannekoekenkerk, a structure as rich in history as it is in legend. Built in once humble bakery, famed for its delectable pannekocken, which drew patrons from all corners of the Netherlands. As the bakery's 's fame grew, so did the building, gradually evalving into a church, as locals hlessed due to the joy brought. Intricately carved wooden panels depicting scenes bountiful harvests feasts adorn its walls, ned to its culinary origins. Legend has it that on each anniversary of its transformation into church, the aroma. fresh pancakes still wafts through its ancient corridors, a whimsical reminder of its flavorful past.
(17))()) this peculiarly named 'Puncuke Church'
believed place
Question: How old is the Pannekuekcnkerk Answer. 478 (1428) Explanation: The correct answer should be 'How old is the Martinitoren tower in Groningen!" instead, as actually the oldest building the city. dating back to around 1428 

Groningen?</5t <|assistant]>
9 The Pannekockenkerk in Groningen is (43(() years old (built in 1877(2). 

Information Extraction
Cs <| <|ssten]]> I> Fictional Clothing Refund Policy Effective Date: November 15th, 2023
1. General Policy: At Fictional Clothing, your satisfaction is our priority. We offer hassle-free refund policy for customers. If
you re not completely satisfied with your purchase, you can return it for refund or exchange within 30 D of purchase.
Conditions for Refund:
Items must be returned in original condition: Unworn, unwashed, with all tags attached. Proof of purchase required: A valid receipt or order number is necessary for refunds or exchanges. Refund method: Refunds will issued to the original method of payment. Final sale items be refunded oaly if they are returned in-store.
3. Non- Refundahle Items:
Ultimate items: Products marked as 'Ultimate Sale' cannot returned or exchanged Personalized items: Custom-made per personalized items are non refundable. Gift cards: Purchused gift cards are not eligible for a refund.
Online Purchases:
Return shipping: Customers are responsible for return shipping costs. Processing time: Refunds will be processed within 7-10 business
days of receiving the returned item.
In-Store Purchases:
Return to uny store: Items purchased in-store can be returned to any of our retail locations. Instant refund: In-store returns will be
processed immediately.
6. Exchange Policy:
Size or color exchange: Items can be exchanged for a different size or color, subject to availability. In-store exchange: For faster service, recommend exchanging items at any of retail locations.
7. Damaged or Defective Items:
Report immediately: Please contact customer service you receive a daaaged or defective item. Full refund exchange: will offer full refund or exchange for such items.
8. Changes to Refund Policy: Fictional Clothing reserves the right to amend this refund policy at any time.
Contact Us: For more information assistance with returns, please contact customer service team. 

: <|user] |> I have an unworn item that bought two months ago and that I would like to return at one of your locations. Is possible me to get reimbursed? 
 <lassistant|>
ÿ The availability of returns and refunds may vary by the specific store or retailer you purchased from, so it's best to check their return policy. However. if you still have the original receipt and the item is its original condition with all tags attached, most stores will offer full refund within certain time frame (usually around 60 days).
9 Since your purchase was made two months ago, unfortunately D. falls outside our policy's 2 30:3 days timeline for returns.
Table 8: Examples of cue- target pairs (identified by indices) extracted by PECoRe for the outputs of Zephyr 7B Beta across several language generation tasks. Model input x is provided without context to produce non-contextual generation J. or chained to preceding context C, to generate the contextual output ý used by PECoRe.
29


Reviews:
[{"review_id": "MNhqwfwGTQ", "reviewer": "Reviewer_sS3A", "summary": "This paper presents a comprehensive framework for assessing the plausibility of context reliance in the context of machine translation tasks. More specifically, this framework facilitates the extraction of cue-target token pairs from language model outputs, enabling the identification of context-sensitive target tokens and their associated influential contextual cues. The primary objective of this approach is to systematically evaluate the credibility of context utilization within the domain of machine translation, and notably, it can be applied in the absence of reference translations. Ultimately, the authors substantiate the framework's accuracy across diverse datasets and underscore its effectiveness", "questions": "- I find several typos in page 4 \n     - P_{c}tx (y^t)-> P_{ctx}(y^t)\n     - P_{c}tx (y^*)-> P_{ctx}(y^*)", "limitations": "", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The topic is very interesting\n- A comprehensive explanation of the proposed algorithms is provided, along with supplementary materials for additional insight.\n- The evaluation is encompassing a wide range of models.\n- This model can be utilized without the need for reference.", "weaknesses": "- The authors introduce incremental methods, highlighting their proposed unified framework as a distinguishing feature compared to previous work[1]. The unified framework aims to enable an end-to-end evaluation of the plausibility of context reliance. However, it raises questions regarding the advantages of such unification. For instance, the Detection component (CTI) and CCI component could potentially function independently, and the potential synergy of combining them within a unified framework remains unclear. Additionally, each component closely resembles existing methodologies[1], with the experimentation primarily focused on various detection metrics. Notably, the paper's most advanced aspect, compared to existing methods, is its omission of reference translations, which may be perceived as somewhat lacking in novelty for an ICLR paper.\n\n- The authors acknowledge limited performance, as indicated by the data presented in Table 1 and Figure 3. They attribute the suboptimal performance to domain shift issues affecting lexical choice. This suggests that the proposed methodology predominantly excels in handling anaphora cases. However, from my perspective, it's worth noting that lexical choice in context-aware translation is a significant aspect, and it's not entirely clear if the approach can effectively diagnose and interpret the model in practical context-aware translation research.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2VdCyv2eoi", "reviewer": "Reviewer_kdfb", "summary": "The paper introduces PECORE, a novel interpretability framework for analyzing context usage in language models' generations. For any (contextual) generation model, PECORE proposes extracting sets of (target, cue) corresponding to what parts of the generation were reliant on context, and what parts of the context they relied on.\n\nThe framework has two main steps:\n\n1. Context-sensitive target identification (CTI): This step identifies which tokens in the model's generated text were likely influenced by the preceding context. It does this by comparing the model's predictions with and without context using contrastive metrics. \n2. Contextual cues imputation (CCI): This step traces the generation of context-sensitive tokens (for “gold” context-sensitive words or end-to-end from the CTI step) back to specific words in the input context that influenced their prediction. It uses attribution methods to identify influential contextual cues.\n\nThe (target,cue) outputs can then be compared to human rationales to evaluate the *plausability*. Importantly, the PECORE framework is agnostic to the specific method used in the CTI step and CCI step, and authors investigate the plausability of PECORE using different combinations of methods.\n\nThe authors apply PECORE to specific task of contextual machine translation. Experiments on discourse phenomena datasets show the PECORE can identify context-sensitive outputs and trace them back to contextual cues with reasonable accuracy (when compared to human-rationales), particularly when using simple distributional metrics on both steps (like CXMI or KL-divergence). The framework is also applied to unannotated examples in FLORES dataset, revealing interesting cases of context usage in translations.", "questions": "- Is it possible that $\\tilde{y}^\\star$ ends up being the same as the contextual prediction $\\hat{y}$ (in remark 3.1)? Or does a high selector score imply that the highest probability token is gonna be different?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- I really liked this paper! It is well written and the presentation is quite clear, and the problem of interpretability/context-attribution is understudied given how relevant it is. They also show understanding of the current problems in the interpretability literature, by explicitly mentioning that they only measuring plausibility (similarity to human rationales)\n- I find their CTI step quite ingenious: they side-step some of the problems of previous context-usage metrics that rely on references (like CXMI) by basically decoding with context and assuming that as “contextualized reference”. This means that they can use these “probabilistic” metrics to measure context on actual decodings/translations of the model rather than on references (which might not be representative of real use of the model).\n- I find the comparison of different contrastive metrics and selectors through PECORE quite relevant, and a useful resource for future intepretability research. The finding that simple probabilistic metrics like CXMI/KL work well and that using the full-distribuion (like KL) can be beneficial quite relevant", "weaknesses": "- The main thing that could make this paper have a significantly bigger impact would be testing the use of PECORE for tasks other than contextual MT: as authors point out, PECORE is general enough to be applicable to any (contextual) LM, and some experiments on other tasks (like summarization or even other modalities) could make this even more suitable to conference like ICLR (doing something like what was done for FLORES if human rationales don’t exist).\n- While I think the authors did well in side-stepping the issues with the faithfulness/plausibility argument, I think some parts of the writing still hint that these explanations reflect the model’s internal behaviour (e.g. “leaving space for improvement through the adoption of more faithful attribution” this implies that more faithful atribbution will lead to more plausible explanations, which might not be the case). Some extra care about this sort of statements could be useful (but overall I found that the authors were already quite careful).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BCNoCXSDpN", "reviewer": "Reviewer_DAmX", "summary": "It is an important topic to evaluate if NLP models can correctly make use of context information in a plausible way, which is called plausibility. It is widely studied in classification tasks but relatively challenging for language generation tasks. Previous works confines on the assessment to a small, handcrafted benchmarks. In this paper, the authors propose an interpretability framework called PECoRE. PECoRe extracts cue-target token pairs, which are used to uncover context dependence in generation. Comparing the uncovered dependence with human annotation, PECoRE can quantify the plausibility of language models. The authors apply PECoRE to evaluate contextual machine translation tasks, using the metrics or context sensitive target token identification and contextual cues imputation, as well as detecting context reliance in the wild, showing the effectiveness of PECoRE.", "questions": "1.\tAlthough your method is for generative language models, it seems that it is more suitable for seq2seq generation tasks, such as machine translation, rather than pure generation tasks, such as story generation. Also, it seems only works for encoder-decoder structure, rather than currently widely used decoder-only language model. Can your method work in such scenarios?\n\n2.\tWhere is the comparison between your method and other SOTA method? Correct me if I am wrong, it seems that you only evaluate the performance of different metrics on different models.", "limitations": "", "rating": 3, "confidence": 2, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "1. The topic (whether NLP models can correctly make use of context information in a plausible way) is important for trustworthy AI.", "weaknesses": "1.\tThis paper is hard to read. The logic and the main message of each paragraph are hard to follow. The writing is not friendly for the reader that does not familiar to plausibility study. Also, the examples are not hard to understand for the reader that cannot understand French.\n2.\tThe scope is narrow. The proposed interpretability method seems only suitable for encoder-decoder based machine translation models. But the focus of the language generation field has move to more general decoder-based language generation, such as GPT-style.\n3.\tLack of comparation. The proposed method are not compared with other SOTA methods for plausibility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VGWRdu6Fa4", "reviewer": "Reviewer_GSke", "summary": "This work proposes an evaluation method for context aware machine translation to quantify how a model is looking at words in the input for their decision for translation. Basically, the evaluation is carried out in two steps, the first step (CTI) to detect the target tokens which are sensitive to context information by differentiating a model with and without context, and the second step (CCI) to perform attribution to the input tokens using the detected target tokens.", "questions": "- I'd like to know the reason for not running OpenMT for the end-to-end experiment in section 5.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- This work is investigating an important problem in context aware machine translation evaluation in that a context aware model usually ignore important signals in the context by explain-away effects. The evaluation protocol presented in this paper might be a way to give us an insight to a particular problem in a model, and thus, might lead to a solution in the future.\n\n- The proposed method is a combination of two, one to isolate the particular tokens for context awareness by comparing two systems, and the other for attribution to find the import inputs to investigate the connection. It simple but sound, and easy to interpret the outputs.", "weaknesses": "- Experiments are not very systematic in that OpenMT was mainly employed in section 4 for evaluating each step, but section 5 used mBART-50 in an end-to-end evaluation setting. I'd expect two settings in the end-to-end experiments for completeness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)