Explaining Interactions Between Text Spans
Authors: Sagnik Ray Choudhury
Pepa Atanasova
Isabelle Augenstein
Venue: EMNLP 2023 Main (2023)
Keywords: explainability
interactions
spans
TL;DR: A novel dataset and a novel unsupervised method to investigate model and human reasoning over textual spans located in distinct input parts.

Abstract: Reasoning over spans of tokens from different parts of the input is essential for natural language understanding (NLU) tasks such as fact-checking (FC), machine reading comprehension (MRC) or natural language inference (NLI). However, existing highlight-based explanations primarily focus on identifying individual important features or interactions only between adjacent tokens or tuples of tokens. Most notably, there is a lack of annotations capturing the human decision-making process with respect to the necessary interactions for informed decision-making in such tasks. To bridge this gap, we introduce SpanEx, a multi-annotator dataset of human span interaction explanations for two NLU tasks: NLI and FC. We then investigate the decision-making processes of multiple fine-tuned large language models in terms of the employed connections between spans in separate parts of the input and compare them to the human reasoning processes. Finally, we present a novel community detection based unsupervised method to extract such interaction explanations. We make the code and the dataset available on [Github](https://github.com/copenlu/spanex). The dataset is also available on [Huggingface datasets](https://huggingface.co/datasets/copenlu/spanex).

Explaining Interactions Between Text Spans
Sagnik Ray Choudhury',, Pepa Atanasova Isabelle Augenstein 'University of Michigan, UUniversity of Copenhagen sagnikrayc@ com, pepa@di. ku.dk, augenstein@di ku. dk
Abstract
Claim:
Stephen King
has made
54 video games.
A
Reasoning over spans of tokens from differ- ent parts of the input is essential for natural language understanding (NLU) tasks such as fact-checking (FC), machine reading compre- hension (MRC) natural language inference (NIJ). However, existing highlight-based ex- planations primarily focus on identifying in- dividual important tokens or interactions only between adjacent tokens or tuples of tokens. Most notably, there is lack of annotations capturing the human decision- decision-making process w.r.t. necessary interactions for informed decision-nin decision-making making in such tasks. To bridge this gap, we introduce SpanEx, dataset human span interaction explanations for two NLU tasks: NLI and FC. We then inves- tigate the decision- Decision-making processes of multi- ple fine-tuned large language models in terms of the employed connections between spans in separate parts the input and compare them the human reasoning processes. Finally, present novel community detection based un- supervised method extract such interaction explanations from model's 's inner workings.'
High-Level Synonym
High.Lvvel Hypenyy
High-Leve! Antorym
Low-Level Antonym
Evidence:
King
has published
54 novels.
including 7 under the pen name Richard Bachman,
Dangier
Label: REFUTES
Figure Human annotations explaining interactions between text spans on an instance from our SpunEx dataset, FEVER part. The presence of antonym interactions (high low- level) between the corresponding spans in claim and the evidence leads to label REFUTES.
annti--nnotator
whether the evidence supports the claim; in natu- ral language inference (NLI), we verify whether the premise entails the hypothesis. We conjecture that the model explanations for such tasks should capture these token interactions as well.
Rationale extraction methods to capture feature interactions used by model have been proposed before. However, these primarily capture interac- tions between neighboring tokens (Sikdar et al., 2021) or between tuples of tokens at arbitrary po- sitions in the input (Dhamdhere et al., 2020; Ye et al., 2021). They do not necessarily consider to- kens from distinct parts of the input, such claim and evidence documents, and the token tuples may not necessarily bear meaning on their own. Cur- rently, there lack of explainability techniques that unveil interactions among spans belonging to different parts of the input, where the spans are comprised semantically coherent phrases. More importantly, the development of interaction expla- nation techniques has not been accompanied by studies of the human decision-makine processes employed for multi-part inpul tasks. Before ex- tracting interaction explanations, we would want ensure that such explanations are indeed valid from a human perspective, i.c., a competent reader could also identify such interactions as an explicit reason
1 Introduction
Large language models (LLMs) employed for nat- ural language understanding (NLU) tasks are in- herently opaque. This has necessitated the de- velopment of explanation methods to unveil their mecision-making processes. Highlight-basee expla- nations (Atanasova et al., 2020; Ding and Koehn, 2021) are common: they produce importance scores for each token of the input, indicating its con- tribution to the model's prediction. In many NLU problems, however, the correct label depends in- teractions between tokens from separate parts of the input. For instance, in fact checking (FC), we verify
The first authors contributed equally. Q We make the code and the dataset available at: ht https: /ggtttb..h.....//gg cono httm/copenll/seee The dataset is also available at https:// huggi aace.ce...... wtt.atasttt///eaeaooooocoo coppnnll///////////////c sepoppenllpppeex spanex

---

behind their decision-making process. Moreover, the lack of such annotations impedes the compari- son between extracted model explanations and hu- man decision making.. We address these research gaps by answering three core research questions.
(the hypothesis contradicts the premise), or neu- tral (neither entailment nor contradiction holds). FEVER contains instances consisting of claim- evidence pairs, where one has to predict whether the evidence supports the claim, refutes the claim or there is not enough information (NET). From here on, we will use 'Entailment' to denote both 'entailment" and 'supports' labels, 'Contradiction" to denote both 'contradiction' and 'refutes' labels, and 'Neutral' to denote both 'neutral' and 'NEI' la- bels. We will also use 'Part I' to denote the premise for NLE or the evidence for FC; 'Part 2' to denote the hypothesis for NLI or the claim for FC.
RQI: What is the human decision-making pro- cess for tasks that involve connecting spans from different parts of the input? To study this, we col- lect the dataset SpanEx consisting of 7071 instances annotated for span interactions (described in $2; see Fig. for an example annotation). SpanEx is the first dataset with human phrase-level interac- tion explanations with explicit labels for interaction types. Moreover, SpanEx is annotated by three an- notators, which opens new avenues for studies of human explanation agreement- area in the explainability literature. Our study re- veals that while human annotators often agree on span interactions, they also offer complementary reasons for a prediction, collectively providing a more comprehensive set of reasons for prediction.
The FC task involves retrieving evidence sen- tences from Wikipedia articles as the initial step, followed by label prediction. As are interested only in the interaction between the claim and the ev- idence parts, we focus only on the second task and use the evidence sentences provided as gold anno- tations in the original FEVER dataset. For claims with no supporting evidence sentences (NEI class), we employ the well-performing system by Malon (2016 8) to collect sentences close to the claims. We collect annotations for random subset of the test splits of both datasets. While our analysis necessitates the collection of annotations for test instances, we also collect annotations for a random subset of 1 100 training instances from each dataset. The latter opens new avenues for studies incl including span interactions in the training processes of L.Mss. Interaction Spans. We introduce the notion of span interactions, where the spans are contiguous parts of the input sufficient to bear meaning. For an interaction, one span is selected from Part and one from Part 2 of the input. We annotate the spans at both high and low levels. A high-level span is the largest contiguous se- quence of tokens that i) is not the whole part, (i.c., not the entire premise hypothesis) ii) bears mean- ing in itself, and iii) can be associated with a span from another part using one of the defined relations. As an example, consider the premise "Two women are running" and the hypothesis "Two men are walking". The label is "contradiction", which has to be justified through the interactions of the con- stituents in the sentences. We see that the subjects, made out of noun phrases antonyms: "two men" (premise) and "two women" (hypothesis), and
understudied
RQ2: Do fine-tune fine-tuned LLMs follow the same maniin as the human annotators tasks with multi- part inputs? SpanEx enables an investi- gation of the alignment between LLMs and human maacii..-making.... We evaluate the sufficiency and comprehensiveness of the human explanations (see $3) for six LLMs. We find that the models rely on interactions that are consistent with the human decision- makinn process. Interestingly, the models depend more on the interactions where the inter- annotator agreement is high, indicating an induc- tive bias similar that observed in humans.
RQ3: Can one generate semantically coherent span interaction explanations? We propose a novel approach for generating interaction explanations that connect textual spans from different parts of the input (see 84). The generated explanations can contain spans in addition to single tokens, as an explanation consisting of groups of arbitrary tokens would lack meaning for end users (Chen, 2021).
2 Span Interaction Dataset
2.1 Manual Annotation Task
Datasets. We collect explanations of span interac- tions for NLI on the SNLI dataset (Bowman et al., 2015) and for FC on the FEVER dataset (Thorne et al., 2018). SNLI contains instances consisting of remise-hhypothsst pairs, where model has to pre- dict whether they are in a relationship of entailment (the premise entails the hypothesis), contradiction
216.8% of the FEVER instances that contain more than one evidence sentence for a claim are discarded as they require interaction explanation annotations hetween the evidence sen- tences themselves. We leave this for future work.

---

are the predicates, made out of verb phrases: "are running" and "are walking". These are the largest meaningful constituents where such relationships can be established. Therefore, they are consid- ered "high-level" spans. A low-level span is the smallest wweaning-bearing text span that still holds a relation. For the given example, these would be "man"""woman" "running"""walling
the same as the "negation" and "alternation" rela- tions in MacCartney and Manning (2014). For the Entailment label, there should be Synonym interac- tions ("equivalence" in MacCartney and Manning (2014)), or Part should be more specific. For example, consider an instance with Part 1 as "All workers joined for a French dinner." and Part 2 as "All workers joined for dinner.". "French din- ner" is true description of "dinner" (but not the other way) because "dinner" is more generic, so Part entails Part 2. In other words, this upward (Yanaka et al., 2019) or forward (MacCartney and Manning, 2014) entailment ("French dinner" + "dinner") should only happen from Part 2 to Part for our case, Hypernym-P2-PI interaction should exist. This also implies that Hypernym- P1-P2 interaction (downward (Yanaka et al., 2019) or backward (MacCartney and Manning, 2014) en- tailment) would make the label Neutral. However, one can also create a neutral hypothesis by creating text that has no synonym, hypernym, or antonym relation with a premise span. As described below, these are called Dangler-SYS-PP interactions ("in- dependence" relations in MacCartney and Manning (2014)) in our setup. In summary, Antonym inter- actions are important for the Contradiction labels, Hypernym-P22PP and Synonyms are important for the Entailment label, and Hypernym-P -P-P2 and Dangler-SYS--a are important for the Neutral la- bel. The same interactions are used for FEVER as the SNLI labels can be easily mapped to them.
For annotating high-level span boundaries, the annotators were shown the constituency parse tree a suggestion, but it was not enforced that the boundaries must adhere to the constituents, as the semantic segmentation of a sentence does not al- ways adhere to the syntactic one. Annotators first annotated spans at high-level and if smaller spans inside the high-level ones could still hold an inter- action with coherent semantics, they proceeded to annotate low-level span interaction (see exam- ple from the annotation platform in Fig. 6).
Interactions. We introduce three types of inter- actions: 'Synonym', 'Hypernym', and 'Antonym', A span is a 'synonym' for another one when both denote the same concept, c.g.. "two young children" and "two kids". 'Antonym' denotes the opposite, e.g., "one tan girl" and "a boy". 'Hypernym' indi- cates superordinate interactions, e.g., "a couple" is hypernym of "two married people" as people can be couple without getting married but the reverse is not truc. While 'Synonym' and 'Antonym' in- teractions are symmetric, 'Hypernym' interactions have a directional aspect, hence, we use two dis- tinct types: 'Hypemym-P2-P1'' and 'Hypernym-P1- P2' depending on whether the hypernym appears within Part 1 or Part 2.
To reduce the annotation load, we asked the an- notators not to annotate synonyms (both at low and high levels) where there is surface-level match (e.g., 'King' appearing in both the claim and the evidence in the example on Fig. 1). We automat- ically add them to the final version of the dataset (Synonym-Si)
The interaction types defined above are well sit- uated in previous work. The ideal approach to NLI and FC would be to translate Part and Part 2 into formal meaning representations such as first-order logic, but often such full semantic interpretation is unnecessary as pointed by MacCartney and Man- ning (2014). Consequently, the authors developed a calculus of natural logic based on an inventory of entailment relations between phrases entailment labels can be inferred based on these relations in- stead of producing a full semantic parse. Similarly, Yanaka et al. (2019) used the concept of upward and downward entailment.
interaction). 3 We also add spans that have not been annotated by an annotator as Dangler-SYS- and Dangler-SYS- interactions, depending on their location in Part 1 or Part 2. These are spans that cannot be matched with any span in the other part. They are particularly impor- tant if found in Part 2 as they reveal spans that are not supported/entailed by spans in Part 1, leading to the Neutral class.
"To reduce false positive surface matches, we removed those where both spans include only stopwords. In addition, we took care of false negative surface matches as we found that the original datasets have few instances with morphological errors (c.g., spelling mistakes, and missing apostrophes). We instructed unnotators to include those in their annotations.
can be easily seen that for the Contradiction label, there has to be at least one Antonym inter- action: a span must appear in Part that directly contradicts a span in Part 1. This interaction is

---

Dataset
Entailment
Neutral
Contradiction
Total
SNLI
FEVER
Interaction
SNLI FEVER
1298
1287
1280
Low
High
Low
High
945
1270
991
3206
Synonym
2719
8408
1033
19785
Hypemym-Pl--P
1594
2711
127
572
Table 1: Overview of the number of annotated instances from SNLI and FEVER in our SpanEx dataset per instance label "Entailment", "Neutral", "Contradiction".
Hypernym-P2PPI
2633
6351
1337
2724
Antonym
3941
4407
3325
3029
Synonym-SYS
17872
1866
37109
9939
Dangler-SYS-P.
4186
10151
10922
14374
Annotation Task. Each annotator is provided with an instance from FEVER or SNLI, together with its gold label. The gold label is provided so the annotators can find span interactions in accor- dance with the label hand. For example, Syn- onym interactions can be found in instances of all labels. Instances of the Entailment class should have all spans in Part be entailed by spans in Part 1. Hence, all tokens of Part 2 should be part of a Synonym or a Hypernym-P2-P) interaction with tokens in Part 1. Antonym interactions can be anno- tated only in instances with label Contradiction and at least one Antonym interaction has to be anno- tated for those. In instances with label Neutral, at least one Hyperaym-P1-PP interaction or a dangler in Part has to found. The above rules are also used for quality control of the annotations where instances that do not contain the necessary or al- lowed interactions are returned for correction. For detailed annotation guidelines see App. B.
Dungler-SYS-P2 7225
2542
5517
4211
Table 2: Annotated interactions for SNLI and FEVER test splits in SpanEx, for low- and high-level spans. Table 6 in App- presents detailed breakdown by instance label.
level, the Antonym interaction annotations are the most frequent. We conjecture this is because there are no exact matches or danglers we can annotate automatically for this relation.
Table 3 presents the length of the interaction spans. At high-level, the span length varies from 2.49 6.48 tokens on average. For high-level in- teractions, the Antonym interaction requires the longest spans, while the Synonym interaction the shortest. We see that the spans usually have a length of one token at the low level.
Finally, Table 4 presents information about the inter-annotator agreement (IAA) in annotating the spans and interactions for SpanEx. When consid- ering exact matches of span tuples constituting interaction annotated by the different annotators, observe numerous instances where the span ples annotated by one annotator do match with those of other annotators (#Span Agree 1) due to small differences in tokens included. Therefore, we also compute Relaxed Span Match agreement, where we consider span tuples as matching if both the Part 1 span and the span Part 2 have at least one matching token. With the relaxed matching, we find that most spans annotated by three annota- tors the high level and at least by two at the low level. We conjecture that the Relaxed Span Match- ing is less applicable the low level, where the annotated span interactions of the different anno- tators are rather complementary. Finally, the IAA for the span interaction type is significant up to 91.89 Fleiss' for low-level FEVER annotations. The IAA for span interaction type resulting from the Relaxed Spans Match remains high, indicating that a large number of the matched interactions are indeed the same spans but with minor token differ- ences. Table 7 in the Appendix presents a more detailed breakdown of the IAA
Each instance annotated by three profes- sional annotators* with university education and fluent English language skills, one being a native English speaker. The annotators were trained on 200 instances from each dataset, with two feed- back sessions, before annotating the main batch. The annotations were done using the brat tool (see example screenshots in App., Fig.6).
2.2 Dataset Analysis
Table shows the instance distribution across la- bels in SpanEx. In total, the dataset consists of 7071 instances, roughly equally distributed between the two tasks and in turn the three labels each. Table 2 gives an overview of the annotated span interac- tions. Interestingly, we find higher frequency of annotations for high-level interactions than for low- level ones. This is because spans smaller than the annotated high-level ones are not always semanti- cally coherent. At the high level, the number of Synonym interactions is the highest because they appear in instances of any label. At the low
'https://www http://.daaabbbeee..... net/ wtw.oaa.t..lllab.rro...o.o.

---

SNLI
FEVER
3.1
Evaluation Protocol
Interaction
Low
High
Low
High
Following Chen et al. (2021), we use Area Over the Perturbation Curve (AOPC) and Post-hoc Accu- racy (PHA) to evaluate how faithful model and hu- man explanations are to model's inner-workings.
Synonym
1.06
2.80
1.13
Hypemym-P1--P
1.12
4.48
1.23
4.19
Hypemymm-P-PPI
1.12
3.77
1.29
6.21
Antonym
1.09
4.43
1,42
6.48
Synonym-SYS
1.0
1.0
1.0
1.0
AOPC and PHA measure the utility of expla- nation e2 for instance æ; by first removing/adding the most important k spans from xy as per ej. This results in a perturbed instance x x
Danpler-SYS-PI
3.35
7.32
3.68
10.72
Dangler-SYS-P2 3.85
9.48
4.56
19.20
Table 3: Span token length per interaction type SNLI FEVER test splits in SpanFx for low and high-level spans.
(ss.s
(1)
x (2r,, Etop(es.k))
(2)
Anno- Intors
Span Level
Exact Spans Match Relaxed Spans Match
Interaction type Pleiss' x
where top(e,, k) is a function selecting set of the top most important spans according to 6₁-
actions
ations
type Fleiss"
SNLI
Area Over the Perturbation Curve. Following instance perturbation, AOPC measures the util- ity of the explanation as the difference between the probability for the originally predicted class Yi given the original instance 2; and the probabil- ity for the originally predicted class yi given the perturbed instance 2.7 /
Low
High 6889
989
Low 2560
2
High
3050
2065
Low
86.45
1334
3
High
3225
84.56
5617
72.22
FEVER
Low
2396
2099
1
High
6715
1788
Low
972
1064
High
2743
1584
(3)
Low 589
91.89
1581
87.52
(4)
High
4730
70.93
7142
74.65
The function r estimates the effect of removing most important spans from 2;. Intuitively, it measures the comprehensiveness (AOPC-Comp) of the top-k most important spans. If the list of most important spans is comprehensive, it should significantly decrease the predicted probability for the originally predicted class ÿi when removed. Alternatively, the function estimates the effect of preserving only the most important spans in the instance Tg. It measures the sufficiency (AOPC-Suff) of the most important k spans in preserving the probability for the originally pre- dicted class ú; Finally, AOPC measures the over- all utility of the explanation by iteratively increas- ing the number spans. The results for the sepurate it values are summarised by a single measure that estimates the area over the curve defined by the results for each < ki r r/a(2;, C5 k) > pair.
Table 4: Annotator agrecment for SpanFxx* Annotators' indi- cates the number of annotators that have annotated the interac- tion. either do an exact or relaxed match of the spans (sce $2.2). ## Interactions' indicates the number of interactions that have been annotated by each corresponding number of annotators. 'Interaction type Fleiss x' indicates the IAA for interactions annotated by all three annotators.
3 Model and Human Explanations Agreement
In $2, we discussed how the annotators modeled the spans and their interactions that led to the classi- fication decision. We next investigate if fine-tuned LLMs use the same decision-making process by comparing the human annotations with two base- lines. The Random Phrase baseline randomly samples both spans of the interaction from each of the two parts of the input. The Part Phrase baseline selects one span from the human annota- tions, and samples the other one at random from the remaining part. If the models follow the same rea- soning the annotators, the human explanations will have significantly higher score than the base- lines. However, if the annotated interactions are not important for the model, or only one part of the input is sufficient, we will see no such difference.
[0, K] of occluded or inserted
Post-hoc Accuracy. Post-hoc accuracy selects one or multiple values for k. and computes the preserved accuracy of model for the perturbed
instances in the dataset X' (x.²*) This results in top k-accuracy score (or curve in the case of multiple values) for one explanation method.
Adaptation for Span Interactions. A better explanation will have a higher AOPC-Comp and

---

PHI score and lower AOPC-Suff score. How- ever, the annotations do not have an importance score for each span interaction. Hence, the span pairs cannot be ranked, and, in turn, a top-k estima- tion is not possible. This naturally creates longer explanations, i.c., a higher number of tokens are perturbed. As AOPC and PHA scores are posi- tively correlated with the number of changed to- kens, the baselines can not be fairly compared with the human explanations. Therefore, we normal- ize the scores by the number of perturbed tokens. Morcover, in the baseline explanations, both the number of span pairs and the number of tokens in each span are sampled from the same distributions as in the annotations.
important than the baselines too. An exception is the Entailment class, where we would expect both Hypernym-P2-PI and Synonym interactions to have higher scores than the baselines and the other interactions, but the Part Phrase baseline has higher score for both of them. The Hypernym in- teractions show large variance as we average over both models and annotators. For the high-level interactions, similar trend can be observed for the Contradiction and Neutral instances in SNLI. However, for FEVER, we do not observe this; in fact, the baseline scores are mostly higher than the relevant interactions. We hypothesize that the high- level span annotations in the FEVER instances have significantly more extraneous information and pos- sibly can be heuristically shortened in future work. Evaluation summary of all metries. Explain- ability evaluation metries often disagree with each other (Atanasova et al., 2020). Therefore, in Ta- ble 5, we summarize (see App. &C for details) how different metrics vary in terms of ranking the relevant interactions. Ideally, the most relevant interactions should be ranked the highest by the AOPC-Comp and PHA scores, and the lowest by the AOPC-Suff scores. For example, for the Entail- ment instances in SNLI the low-level Synonym or Hypernym-P2-P1 interactions are found the most important by PHA (indicated by green ). None of these interactions is the most important according to the AOPC-Comp metric, but it finds at least one of them to be the second most important yellow ). AOPC-Suff, on the other hand, finds them to be the least important (green) ) as expected. In summary, in 64% cases, the relevant interactions are found to be the most (by AOPC-Comp or PHA, or least, by AOPC-Suff) important, in 31 % cases they are in the upper (lower) 50th percentile of all interac- tions, and in 5% cases they are not found relevant. AOPC-Comp and AOPC-Suff provide comple- mentary evaluations, and they both align well: they differ strongly (indicated by red green in the same rows in Table 5) in 4% cases, and moderately in 13% cases (yellow vs green).
3.2 Experiments, Results & Discussions
Both NLI and FC are multi-class classification tasks. We use linear classifiers on top of pre- trained LLM encoders and fine-tune the models. Six models of the BERT family (Devlin et al., 2019; Liu et al., 2019) varying in model size, tokenization, and pre-training objectives are used: BERTease-cased BERTbasa-uncaeet BaRge uncasedd ROBERTàrase: and RoBERTâlarge- For each model type, we train three models with standard training configurations but different number of epochs and minimize the Cross-Entropy loss. As the models show little difference in the test data, we choose the best-performing model from each type for the subsequent experiments.
BERTLarge-cased
AOPC-Comp results (Fig. 2). We merge the Synonym-Systent and Synonym categories as the annotators would have labeled them the same. In general, high-level interactions have lower scores than low-level ones. This is expected as they are more semantically coherent but they may con- tain extrancous tokens. The models find the rel- evant lowelevel annotated interactions, i.e., the ones correlated with human reasoning, more im- portant than: the Random Phrase baseline in all cases; the Part Phrase baseline in all but one case CONLI-Entailmente and non-relevant interactions in 66% ( of the cases. Humans would, e.g., find the Antonym interactions most important for the Contradiction instances, and so do the models. Similarly, for the Neutral ones, the most impor- tant interaction found by the models is Hypernym- P1-P2. Morcover, for SNLI, the Dangler-SY? P2 interactions in the Neutral instances are more
Do all models follow the human decision- making process? We analyze this in Fig. 3 by comparing the AOPC-Comp scores for the three most relevant low-level interactions for three classes: Antonyms for Contradiction, Synonyms
'In Figure 2, the Part-Phrase baseline uses the random tokens from Part 2. App. figures 9, 10, and 11 show the results when the random tokens are chosen from Part 1. The trends we observe there are similar.

---

Refutes. Contradiction.
Fever: EntaElient, Low
Fever SML1 Neutral, Low
Anturye
Dongler San
Myacroga PP)
part shroee
Nysornee
9.15
0.35
0.30
-0.85
SMLT-APP--CMM
Tever-A6RC-rers
9M.I-4RC--ro
Feeer-MP--car
SMLT-MPA CLL--MP--COM
Tever Wver-ApPE-cors
Fever Bu Tutes. SALL Conttrittiiii, odiction, High
Fever Susporrs,
SNLI- Entallntt,,,,,, High
Fover-NEI, SW.I Neutral, High
0.10
9.86
8.82
0.30
SAL1 APPS CORP
Fover ADPC COMS
JNLL ADMC COFD
Pever APCC coma
SALL APPS COFD
hover A0PS comp
Figure AOPC-Comp scores averaged (error bars: standard deviation) over annotators and models. A higher AOPC value relevant interaction (e.g.. Antonym Contradiction) indicates better human-model explanation alignment.
method of modeling the human decision process has strong correlation with the models' reasoning. Similarly, we investigate whether the models de- pend more on the interactions where the annotators agree. As before, compute AOPC-Comp the most relevant interactions but split them into: inter- actions where a) all three annotators agree, b) annotators agree and c) all disagree. Fig. 4 shows that IAA has a strong correlation with the AOPC scores, indicating again that the models have the same inductive biases humans.
AOPC- Comp
AOPC- Suff
Dataset
Level
PHA
SNLI-
Low
1/6
6/6
3/6
Contradict
High
1/8
8/8
2/8
SNLI-
Low
2/4
1/4
Entailment
High
5/6
6/6
1/6
SNLI-
Low
1/5
5/5
1/5
Neutral
High
1/7
7/7
1/7
FEVER-
Low
1/6
5/6
5/6
Refutes
High
3/8
5/8
3/8
FEVER
Low
1/4
Supports
High
3/6
6/6
2/6
FEVER-
Low
1/5
5/5
1/5
4 Extracting Interactive Explanations
NEI
High
2/7
1/7
1/7
An explanation method should output interac- tion pairs of sets tokens from the parts the input pd {(xxfart)) (apart u)li E [1, |part]]],j E [1, |part2]]] where each pair is further assigned significance value v depending on its influence on the prediction of the model. Interactions between features, e.g., tokens, in ML models are most commonly learned with an attention mechanism (Vaswani al., 2017). Hence, we generate directed bipartite interaction graph G (V, E) with tokens from two parts of the input. The weights for the edges come from the attention matrix. We keep only the attention weights be- tween tokens that belong to different parts of the in- put, thus, creating two vertex partitions. While we create the interaction graph using attention weights, can be built using other explanation techniques producing token interaction scores.
Table 5: The rank the most relevant interactions according different metrics. The colors green yellow and red indicate whether one of the most relevant interactions for label is the first (AOPC-Comp) and PHA, last for AOPC- Suff). in the top 50%. the bottom 50% (reverse for AOPC- Suff) of all interactions.
for Entailment, and Hypemyy- interactions for Neutral. For SNLI, we not see sig- and BERTiarge-cased models pay the least attention to the relevant interactions in the FEVER Refute and NET instances. These two models have good F1- scores the entire test dataset (86.2% and 87.2%, respectively) but very low scores on annotation instances 68.9% and 46.8% whereas all oth- ers have > 83% (except 81.1% for RoBERTabase: which again has a poor AOPC-Comp score for the NEI instances). This further shows that our
nificant difference, but the BERT
We use the top layer attention scores as they dictate how the final representation before the clas-

---

Fever SNLI Contradiction.
Low
SNLT Entailment,
Fever-NEt, SALE Neutral, Hypernya P1-P2,
kent soncc
tert-lar snuueee
kent uncased
reberts 2are
Larga
reberts
6.2
0.1
SAL1 HOPC CaRp
CORp
comp
COBB
40PC cong.
Figure 3: AOPC-Comp for different models in the most relevant interactions.
Fever-Btffttes SML1 Contraadiiii,nn, Antonse, Low Fuver-Supgorts, SML1-E M.Ll-Ettailunn, Synenya, Low
Faver-NEl, SML1-Meutre Moutral, typernse-P1-P2, Low
Zogreement
0.25
0.20
Figure 4: AOPC-Comp scores for the most relevant interactions split by inter-amnotator agreement.
sification layer is generated. This still leaves us with three choices: a) aggregating the attention trices from different heads; b) producing a multi- dimensional interaction graph (Tang al., 2010) where each head will present a different type of interaction; and c) finding the most important head for the classification task. Attention heads should not be aggregated as they are designed to provide different views of the data and capture different se- mantic and syntactic relations (Rogers et al., 2020). Therefore, choose to find the most important head using the two methods described next, and we leave the multi- paali-graphhae approach for future work. Classifier Weight. All our models use a lin- ear classifier (W, (n x m), m is the number of classes on top of an n-dimensional CLS vector de- noted c.? For input instance with predicted class index k, the logit score for k is dot product of (W2) and c, i.c, Sk E W/e, For all 4 > 0, the higher is w;, the higher 8k, and con- versely, for all & < 0, a higher value of w; makes a higher negative contribution. In summary, the logit score is proportional to Em sign(c;).up; We can write the CLS vector as (C1 ① C2.. @ ca] and w [W1 W2.. wal where is the number of attention heads and ()) denotes concatenation. Then Sy can be written Ej WjCj. The Wj with the
highest E sign(cy.)).ur (1 = n i... the dimension of each attention head) makes the high- contribution towards the classification and j is chosen as the most important attention head.
In another approach, Scalar Mix, we freeze the parameters of the encoders and train new mod- els with set of parameters [Ai on top of the frozen CLS representations. The resulting lin- classifier (W', (I m)) in this model uses the scalar mixed (Peters al., 2018) CLS tor Σi= argmar, (A,) determines the most important attention head.
We use community structure detection algo- rithms (Fang et al., 2020) on G to find groups of nodes (tokens) with dense inter-group and sparse intra-group connections. These algorithms computationally optimized for large social (Gu et al., 2019) and biological networks (Yanrui al., 2015) and hence overcome the limitation of exist- ing perturbation and simplification- based explana- tions that rely on the occlusion of groups of input tokens, which leads to combinatorial explosion when considering span interactions. We use the Louvain algorithm (Blondel al. (2008), see App. $D) which has been used in directed graphs such as ours. The bipartite nature of G ensures that the explanation tokens are from two parts of the input, which are then combined to produce spans based on their positions. Finally, a list of span pairs is
WWo use boldface upper case for matrices and boldface lower case for vectors.

---

AOPC CoFp
Several approaches also extend highlight-based explanations detect interactions between tuples of tokens. One line of work (Tsai et al., 2023; Sundararajan et al., 2020; Grabisch and Roubens, 1999), introduces axioms and methods to obtain interaction Shapley scores. Janizek et al. (2021) extend IG by assuming that the IG value for a dif- ferentiable model is itself differentiable function, thus, can be applied to itself. Masoomi al. (2022) extend univariate explanations to produce bivariate Shapley value explanations. Additionally, Chen et al. (2021) find groups of correlated tokens from different input parts, but the tokens are found at ar- bitrary positions and the produced explanations are not necessarily semantically coherent. In contrast, we investigate interactions between token spans that bear sufficient meaning and are semantically coherent and thus plausible to end users.
0.00
8.00
6.60
EST v01
Kandon phrane Part-a117u
0.09
Fever MAPC-ccag
6.40
Fever SoNer-AOnC-aar
S1L2 ABPC walf
PHA
0.00
9.86
6.42
6.00
Poner-PA
S1L2
Finally, there is a stream of work on explainabil- ity methods for constructing hierarchical interac- tion explanations. Sikdar et al. (2021) compute importance scores in bottom-up manner starting from the individual embedding dimensions, work- ing way up to tokens, words, phrases, and fi- nally the sentence. Zhang et (2021) build inter- pretable interaction trees, where the interaction is again defined based on Shapley values. While these methods produce spans of tokens that are part of an interaction, the hierarchical nature of the expla- nation limits the interactions only to neighboring spans. In contrast, are interested in spans that can appear in the different parts of the input for NLU tasks and are not necessarily neighboring.
Figure 5: Top-3 evaluation scores for Louvain community detection over two types of attention graphs, along with the Part Phrase and Random Phrase baselines. AOPC-Comp PHA: higher is better, AOPC-SufT: lower is better.
generated by a cartesian product of the generated spans. The score for each span pair is the sum of the edge weights between the nodes in them. The ranked list of span pairs constitutes the explanation. Results. We evaluate the explanations with the same metrics as before (see $3.1) )) but the 'top-k" versions. Fig. 5 shows the top-3 AOPC-Comp. AOPC-Suff, and PHA scores for the proposed methods and the baselines (App. ŞE, Table 10 shows t and top-5 results and some generated explanations). The PHA scores are significantly higher in the proposed methods vs. the baselines, and the AOPC-Suff scores are lower (as expected) but not greatly so. The baselines do much better in terms of AOPC-Comp. which means that the explanations produced by our methods are often sufficient, but not always comprehensive.
Conclusion
We introduce SpanEx, multi-annotator explana- tion dataset that captures the interactions between semantically coherent spans from different inputs in pairwise NLU tasks, here, NLI and FC. SpanEx maps the implicit human decision-mmainn process for these tasks to explicit lexical units and their interactions, opening up new research directions in explainability. Using this dataset, we show that fine-tuned LLMs share the human inductive bias as evidenced by their relatively higher scores on established explainability metrics compared to ran- dom baselines. We also propose novel community intection-based methods extract such explana- tions with modest success. We hope this work will pave the way for further research in the nascent area of interaction explanations.
5 Related Work
Existing work mainly explores interactions be- tween tuples of tokens. Tsang et al. (2020) pro- pose method to detect grouped pairwise token interactions, where the only interactions occur be- tween tokens in the same group. Hao al. (2021) creates attention attribution scores using integrated gradient (IG) over attention matrices and then con- structs self-attention attribution trees. The latter is extended by Ye et al. (2021) to multiple layers.

---

Limitations
Ethics Statement
In this work, we study explanations of span inter- actions for explaining decisions for NLU tasks. To accomplish this, we have introduced dataset of human annotations of span interactions for two ex- isting datasets for NLU tasks - fact checking and natural language inference. It is worth noting that there are other NLU tasks such as question answer- ing that necessitate reasoning involving interactions among multiple parts of the input. These tasks may involve different types of interactions, which could be investigated in future work. Furthermore, our dataset consists of interactions between spans from two separate parts of the input. Interactions of more than two spans and from more than two parts of the input are also possible for example in fact check- ing where interactions between several evidence sentences are possible well.
The primary objective of our work is to offer span interactive explanations for NLU tasks. The ex- planations provided by unsupervised commu- nity detection method can be utilized by both ma- chine learning practitioners and non-expert users. important to acknowledge the potential risks associated with overreliance on our span interac- tive explanations as the sole explanation method. Other explanation types, such as free-text expla- nations (Camburu et al., 2018; Wang et al., 2020; Rajani et al., 2019) can offer complementary in- formation, but their faithfulness could be hard to estimate (Atanasova al., 2023). Despite these limitations, we believe that our work is an impor- tant stepping stone in the area of interactive expla- nation generation.
Acknowledgements
In our model analysis, we studied the most pop- ular bidirectional Transformer models. With our dataset and the performed analysis, we have set the ground and only scratched the surface of the prospects to inspect the inner workings of multi- tude of different architectures for span interactions, such as auto-regressive Transformer models. The implementation can be easily adapted to perform the following studies in future work.
This research was co-funded by the Euro- pean Union (ERC, ExplainYoursell: 101077481) 1), by the Pioneer Centre for AI, DNRF grant number Pl, as well as by The Villum Synergy Programme. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union the European Re- search Council. Neither the European Union nor the granting authority can be held responsible for them. We thank the anonymous reviewers for their helpful suggestions.
We have introduced unsupervised community detection approach for explaining interactions be- tween spans of text, which serves foundational step for future research in interactive explanations. However, it crucial to address the limitations these initial advancements. Firstly, the explana- tions produced by the community detection method may consist of spans that lack semantic coherence, the start or end of span might not align pre- cisely with the tokens of an exact phrase. Ensur- ing better semantic coherence within the generated spans is an important aspect to consider for further improvement. Secondly, the current approach does not provide explanations at both the high and low levels, in accordance with the human annotations. Expanding the approach to incorporate explana- tions at both levels would enhance its completeness and alignment with human annotations. Finally, the method does not explicitly indicate the type of span interaction, such as Antonym, Synonym, Hypernym. Incorporating the identification of span interaction types would provide valuable in- formation and improve the interpretability of the generated explanations.
References
Pepa Atanasova, Oana-Maria Camburu, Christina oma, Thomas Lukasiewicz, Jakob Grue Simonsen, and Isabelle Augenstein. 2023. Faithfulness tests for natural language explanations. of the ōl Anmual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 283-294, Toronto, Canada. Association for Computational Linguistics. 10
Proceedings
Pepa Atanasova, Jakob Grue Simonsen, Christina Li- oma, and Isabelle Augenstein. 2020. A diagnostic study of explainability techniques for text classifi- cation. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (FMNLP), pages 3256-3274, Online. Association for Computational Linguistics. 1.
Vincent Blondel, Jean-Loup Guillaume. Renaud Lam- biotte, and Etienne Lefebvre. 2008. Fast unfold- ing of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment. 2008( 10): 8, 16

---

Samuel R. Bowman, Gabor Angeli. Christopher Potts, and Christopher D. Manning. 2015. A large anno- tated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empiri- cal Methods in Natural Language Processing, pages 632-642,, Lishon, Portugal. Association for Compu- tational Linguistics. 2
Ke Gu, Dianxing Liu, and Keming Wang. 2019. So- cial Community Detection Scheme Based on Social- Aware in Mobile Social Networks. IEFE Access, 7:173407-173418.
Yaru Hao, Li Dong, Furu Wei, and Ke Xu. 2021. Self- attention attribution: Interpreting information interac- tions inside transformer. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 12963-12971. 9
Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewiez, and Phil Blunsom. 2018. e-SNLI: Nat- ural Language Inference with Natural Language Ex- planations. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa aasa-Bianchi, and R. Garnett, ed- itors, Advances in Neural Information Processing Curran Associates,
Joseph D Janizek, Pascal Sturmfels, and Su-In Lee. 2021. Explaining explanations: Axiomatic feature interactions for deep networks. The Journal of Ma- chine Learning Research, 22(1):4687-47400 9
Systems 31, pages 9539-9544 Inc. 10
Yinhan Liu, Myle Ou, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. CoRR, abs/1900..1692. 6
Hanjie Chen, Song Feng, Jatin Ganhotra, Hui Wan, Chulaka Gunasekara, Sachindra Joshi, and Yangfeng Ji. 2021. Explaining neural network predictions on sentence pairs via learning word-group masks. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 3917-3930, Online. Association for Computa- tional Linguistics. 5.9
Bill MacCartney and Christopher D. Manning. 2014. Natural Logic and Natural Language Inference, pages 129-147. Springer Netherlands, Dordrecht. 3
Zeming Chen. 2021. Attentive tree-structured network for monotonicity reasoning. In Proceedings of the Ist and 2nd Workshops on Natural Logic Meets Machine Learning (NALOMA), pages 12-21, Groningen, the Netherlands (online). Association for Computational Linguisties.
Christopher Malon. 2018. Team papelo: Trans- former networks FEVER. In Proceedings of the First Workshop on Fact Extraction and VERification (FEVER), pages 109-113, Brussels, Belgium, Asso- ciation for Computational Linguistics. 2
Aria Masoomi, Davin Hill, Zhonghui Xu, Craig P Hersh, Edwin K. Silverman, Peter J. Castaldi, Stratis Ioan- nidis, and Jennifer Dy. 2022. Explanations of black- box models based on directional feature interactions. In International Conference on Learning Representa- tions.
Jacob Devlin, Ming-Wei Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume 1 (Long and Short Pupers), J, pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics. 6
Mark EJ Newman and Michelle Girvan. 2004. Find- ing and evaluating community structure in networks. Physical review E, 69(2):026113. 16
Kedar Dhamdhere, Ashish Agarwal, and Mukund Sun- dararajan. 2020. The shapley taylor interaction index.
Matthew E. Peters, Mark Neumann, Mohit Tyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018 Deep contextualized word repre- sentations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume I (Long Papers), pages 2227-2237, New Orleans, Louisiana. Association for Computa- tional Linguistics. 8
In Proceedings of the 37th International Conference on Machine Learning, ICML'20 20. JMLR. 1
Shuoyang Ding and Philipp Koehn. 2021. Evaluating saliency methods for neural language models. In Pro- ceedings of the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies. pages 5034-5052, Online. Association for Computational Linguistics.
Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. Explain your- self! leveraging language models for commonsense reasoning. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics, pages
Yixiang Fang, Xin Huang, Lu Qin, Ying Zhang, Wenjie Zhang, Reynold Cheng, and Xuemin Lin. 2020. A survey of community search over big graphs. The VLDB Journal, 29(1):33-33 8
Florence, Italy. Association for Computational Linguistics. 10
Michel Grabisch and Marc Roubens. 1999. An ax- iomatic approach to the concept of interaction among players in cooperative games. International Journal of game theory, 28:547-565.
Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2020. A primer in bertology: What we know about how bert works. CoRR, abs/2002. 12327. 8

---

Sandipan Sikdar, Parantapa Bhattacharya, and Kieran Heese. 2021. Integrated directional gradients: Fea- ture interaction attribution for neural NLP models. In Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguistics and the 11th International Joint Conference Natural Langwage Processing (Volume I: Long Papers), pages 865-878, Online. Association for Computational Linguistics. 1,9
Ding Yanrui. Zhang Zhen, Wang Wenchao, and Cai Yujie. 2013. Identifying the Communities in the Metabolic Network Using 'Component' Definition and Girvan-Newman Algorithm. In 2015 14th Inter- national Symposium on Distributed Computing and Applications for Business Engineering and Science (DCABES), pages 42-45. 8
Xi Ye, Rohan Nair, and Greg Durrett. 2021. Connect- ing attributions and QA model behavior on realistic counterfactuals. In Proceedings of the 2021 Confer- ence on Empirical Methods in Natural Language Pro- cessing, pages 5496-5512, Online and Punta Cana, Dominican Republic. Association for Computational Linguisties. 1,9
Mukund Sundararajan, Kedar Dhamdhere, and Ashish Agarwal. 2020. The shapley taylor interaction in- dex. In International conference on machine learn- ing, pages 9259-9268. PMLR.
Lei Tang, Xufei Wang, and Huan Liu. 2010. Commu- nity detection in multi- dimensinaaani networks. Tech- nical report, Arizona State Univ Tempe Dept of Com- puter Science and Engineering. 8
Die Zhang, Hao Zhang, Huilin Zhou, Xiaoyi Bao, Da Huo, Ruizhao Chen, Xu Cheng. Mengyue Wu, and Quanshi Zhang. 2021. Building interpretable interaction trees for deep nlp models. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 35, pages 14328-14337.
James Thome, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: large-scale dataset for fact extraction and VERification.
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computationnl Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 809-819, New Orleans, Louisiana. Association for Computational Linguistics.
Che-Ping Tsai, Chih-Kau Kuan Yeh, and Pradeep Raviku- mar. 2023. Faith-shap: The faithful shapley interac- tion index. Journal of Machine Learning Research, 24(94):1-42. 9
Michael Tsang, Sirisha Rambhatla, and Yan Liu. 2020. How does this interaction affect me? interpretable attribution for feature interactions. In Proceedings of the 34th International Conference on Neural In- formation Processing Systems, N1,S"20, Red Hook, NY, USA. Curran Associates Inc. 9
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, &. ukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. In Advances Neural Information Pro- cessing Systems, volume 30. Curran Associates, Inc. 7
Cunxiang Wang, Shuailong Liang, Yili Jin, Yilong Wang, Xiaodan Zhu, and Yue Zhang. 2020. SemEval- 2020 task 4: Commonsense validation and explana- tion. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 307-321, Barcelona (online). International Committee for Computational Linguistics. 10
Hitomi Yanaka. Koji Mineshima, Daisuke Bekki, Ken- taro Inui, Satoshi Sekine, Lasha Abzianidze, and Johan Bos. 2019. Can neural networks understand monotonicity reasoning? Proceedings of the 2019 ACI Workshop BlackboxNLP: Analyzing and Inter- preting Neural Networks for NLP, pages 31-40, Flo- rence, Italy. Association for Computational Linguis- tics. 3

---

Interaction
Eatall
Neutral
Contradict
Total
3. Contradiction: the hypothesis is definitely a false description of the image: "Two dogs are running through a field.". "The pets are sitting on a couch." " it's impossible for the dogs to be both running and sitting.
SNLI Low-Lov
Synonym
735
447
Hypenny--b-iop
1771)
446
417
2633
1529
65
1594
Amaayym
Synonym-YYSTEM
7807
5773
4292
17872
Dingler SYSTEM-IYPOHHESSS
1913
2937
7225
Dungle
11402
SNLI High-Level
Synonуn
3064
1954
B.2
General Description of the Fact Checking Task
Hyypemym---liep
3887
1392
1072
6351
Hypernyy-g--t-h
0
2526
185
2711
Amtonym
Synonym SYSTEM
404
1004
458
1866
You will be provided with <label evidence claim
81
1903
558
2542
SIISEESYSTEMAPREMEEE
1015]
>. The evidence comes from Wikipedia pages and the title of the page is prepended to the sentence (e.g. [source: Islamabad]). The pair can have one of the three possible labels: supports, refutes, not enough info.
FEVER Low-Lerel
Synongm
651
141
241
1033
Hyperay----lohh
117
127
Hypereym---iip
1123
112
102
1857
3325
Symonyns SYSTEM
13549
11482
12079
37109
Dingle SYSTEM- Claim
2020
1118
2379
5517
Dungles SYSTEM Evidence
4490
10922
FEVER High-Level
Synonym
6630
7081
6024
19785
B.3
Overall Description of the Labeling Task
Hypemym----oo-
554
572
Hypernym-b--oor
2259
339
126
2724
You will be provided with 1) the premise/evidence and the hypothesis/claim and 2) the label for the pair. You will have to find corresponding spans in the premise and the hypothesis and annotate the interaction between them. Our goal is see how humans determine NLI or FC labels using the interactions between the parts of the premise and hypothesis. There can be different types of interactions, which we define further down below. You will have to find these parts (spans) and label these interaction types.
Antorim
0
5027
3029
Synonyms
2130
Dungles-SYS Songlee-SYSTEAA SYSTEM-Clum
4082
120
4211
Dongler BYideme-ride
3942
6118
4011
14374
Table 6: Overview of the number of annotated interactions our SpanEx dataset per instance label.
A Detailed Overview of SpanEx
Table 6 presents a detailed overview of the an- notated interactions. Table 7 presents a detailed overview of the annotated spans.
B.4 Interaction types
B Annotation Guidelines
Two corresponding spans E premise and B E hypothesis can have one of the following interac- tions:
Fig. 6 presents screenshot from the annotation platform with three example annotations.
B.1 General Description of the NLI Task
Synonym - a denotes the same as B. Example: pretty and attractive.
You will be provided with <label premise hypoth- esis >, where premise and hypothesis are sentences, which can have one of the three possible labels: entailment, neutral, and contradiction, depending on whether the hypothesis entails the premise. The premise is caption of an image. The hypothesis was written given the premise, but not the image.
2. Antonym a denotes the opposite of B. Ex- ample: dead and alive; parent and child.
3. Hypernym a is superordinate to 3 In other words, 3 is more specific than a which can also be due to new details introduced in Ex- ample: an animal is a hypernym of mammal; red is a hypernym of scarlet; to cut is hyper- nym of trim and to slice; "wash their hands' is a hypernym of "wash their hands in a sink'.
1. Entailment: the hypothesis is definitely a true description of the image: "Two dogs are run- ning through a field.". "There are animals out- doors."
2. Neutral: the hypothesis might be true de- scription of the image: "Two dogs are running through a field", "Some puppies are running to catch a stick." " the dogs are not necessarily puppies.
4. Hypermy----to--- is in the hypothesis, & is in the premise
*We use 'premise" to denote both premise (NLI) and ev- idence (FC), we use "hypothesis" to denote both hypothesis (NLI) and claim (FC).

---

Dataset
Num. Ann.
Synonym
Antonym
Hypernynn-p-to-h
Hypernym-h--topp
Total
SNLI-Low
1
661
1413
919
1053
4046
SNLI-High
1
1614
1953
1429
1893
6889
SNLI-Low
2
640
808
369
743
2560
SNLI-High
2
1035
631
425
959
3050
SNLI-LoW
3
354
329
23
125
831
SNLI-High
3
1648
420
179
978
3225
SNLI-Low
14243
1655
2550
1311
1921
7447
SNLI-High
1+2+3
4297
3004
2033
3830
13164
SNLI-Low+HHig
1+2+3
5952
5554
3344
5751
20601
FEVER-LoW
397
1181
100
718
2396
FEVER-High
1
2991
1622
443
1659
6715
FEVER-LoW
2
232
483
15
242
972
FEVER-High
2
1898
414
63
368
2743
FEVER-LoW
3
102
406
2
79
589
FEVER-High
3
4412
195
5
118
4730
FEVER-LoW
1+2+3
731
2070
117
1039
3957
FEVER-High 1+2+3
9301
2231
511
2145
14188
FEVER-Low+High
1+243
10032
4301
628
3184
18145
Table Number of annotations type (Synonym, Antonym, Elypemym-P1-P2, Hypemym-P2-PI) and by the number of annotators (1, 3, 142+3) that have annotated it.
5. Hypernym-p-to-h a is in the premise, is in the hypothesis
Can have synonyms, danglers in the premise, Hypemyy--h-to-p.
Note: there can be spans in either part of the instance that do not have a corresponding span in the other part, for short danglers. You can leave these without annotations.
Contradiction/refute::
Have at least one antonym interaction;
Can have hypernym, synonym, dangler inter- actions;
Note: there can be spans with danglers low- level, with a danglers contained both in premise and hypothesis. this case, at high-level annotate the two corresponding spans as both Hypernym h-toop and Hypemmym--tto-h
B.6 High-Level Text Spans
Annotate interactions between high-level match- ing text spans in the premise and the hypothesis. Choose text span a in the premise that can be mapped to text span B in the hypothesis by one of the interactions: If there's 's no corresponding high-level span, mark them Dangler. The spans should be selected at the highest level of the syntax tree, i.e. longest possible chunks that can hold one such interaction. Annotate the two text spans a and | as premise and hypothesis. Connect the chunks the interac- tion with an arrow. For antonyms and synonyms, which are symmetric interactions, you can draw the interaction arrow starting from the hypothesis. For hypernyms, if the hypernym is located in the premise, start drawing the arrow from the premise, otherwise from the hypothesis. Annotate the type of the interaction.
B.5
How labels define which interactions can be used
synonym, antonym, hypernym.
Entailment/suppors::
(mainly synonyms, but premise can be more specific)
Have synonym interactions;
Can have danglers (additional information) in the premise;
Can have Hypernym-h-to-p interactions (more-specific premise).
Neutral/not enough info: (hypothesis has more info/is more specific)
Have least one Hypernym---to-h OR at least one dangler (when there are only syn- onym interactions between the premise and the hypothesis) in the hypothesis;
Note: If the high-level surface forms match, they still need to be annotated. At high-level span an- notation, do not annotate only the dangling parts

---

Premise
Hypemmym-hhhe po-p
anli-test-20 neutral
biond-haired doctor and her African american assistant looking threw new medical manuals.
Hyporym Hypothesi
doctor studyyng | 6 A b bland haired ad doctor ,,) and (. la her African american assistantt z) (a
looking ,) a) 2) 6, threw ( new medical manuals ,) a)... G doctor Cg is (, studying ,)
Antonym
Antonem Jntonym
Premise
ipleat
Bllent
Premise
enli-test-21
contradiction
One tan girl with a wool hat running and leaning over an object, while another person in wool
Antorom Amtoryy
Hstothesss
nieaf
Hypethests]
A boy runs into wail I G, One tan girl w) with wool hat C, (, running and C. leaning (. (, an object , J.. while (, Cu (, anather person ,,) wool hat .n) C., sitting (,, the ground
hat is sitting on the ground. |
) 60 boy 2) b
G, wall a
Synorym
Synonyt
Premiie
Synonym
Premise
pleaat
pplcat
snli-test-22
entailment
wool running and leaning over object, while another person
Syronum
Synony
Hypothesis
myyoethesl
sitting ground. running and (y leaning (, over (g an object ,) a a) (, while (, Le ( another person ad (, (y a wool hat w a al (. is (, sitting (a on (,, the ground ,,) ,) ...... lg A tan girl ,) (, nuns (,, leans .. (, over (_ an abject ,) al
girl runs
an object | C. (. One girl _)) (g with G, wool hat ..) e) a C. G,
Figure 6: Screenshot of the annotation tool.
(those that do not have a corresponding span in the other text part).
Hernandez' and 'Oscar Isaac Hernandez', Note: Always make sure that the annotated relations are coherent with the label provided for the instance.
Note FEVER: For the FEVER dataset, anno- tate interactions to the evidence document's title prepended to the evidence in square brackets Treat the evidence title and the evidence text one textual part, i.e. there should be no interactions be- tween the evidence title and the evidence text, but only between the evidence title and the claim and the evidence text and the claim. In such cases, for the same named entity in the claim there will be in- teractions both to the evidence and to the evidence title.
B.7 Low-level Text Spans
Annotate interactions between low-level match- ing text spans, inside the high-level spans, in the premise and the hypothesis. Choose a text span a in the premise that can be mapped to a text span ,3 in the hypothesis by one of the interactions: synonym, antonym, hypernym. The spans should be selected at the lowest possible level of the syntax tree, i.e. shortest possible chunks that can hold one such interaction. Do not annotate exact surface forms synonyms, e.g. in the high-level synonym interac- tion "while holding to go packages" the premise and "while holding to go packages" in the hypoth- esis, do not annotate the matching separate words as synonyms. Annotate as synonyms only spans that do not match in surface form. In high-level hy- pernym interactions, if additional details are being added in either part, leave these parts unannotated at the low level. E.g., 'holding to go packages' and 'holding packages', there is an additional mod- ifier 'to go' added to the premise, making it more specific, thus contributing for Hypemym-h--tu in-
Note: Annotate all possible interactions of one span to spans in the other part of the text. For example, if one entity can have corresponding span in the evidence and in the title, annotate both. These include pronouns as well, e.g. Example in Examples FEVER below 'Sabbir Khan' in the claim an interaction to both 'Sabbir Khan" in the document title and to 'he' in the document itself.
Note: If two spans refer to the same object/entity but with different surface forms, assign a synonym interaction between them, e.g. Example in Ex- amples FEVER 'Oscar Isaac' is related to 'Oscar

---

teraction. Do annotate articles, Annotate the text spans a and B pleaf and hleaf. Con- nect the chunks in the interaction with an arrow as above. Annotate the type the interaction.
Top
Relevant relation rank
Level
Metric
Relation
SNLI- ENLI-Contradiction
AOPC Comp
Antonym
1/6
Note: It is possible the high-level and low-level spans overlap in one part of the input. In such cases, annotate it both high-level and low-level (leaf) span.
Low
AOPC Suff
Antonym
6/6
PHA
Dangler-System-PI
3/6
AOPC Comp
Antonym
1/8
High
AOPC Suff
Antonym
8/8
PHA
Dangler-System-P2
2/8
C Detailed Results for Human and Model Explanation
SNLI-Entailment
AOPC Comp
Dangler-System-PP
2/4
AOPC Suff
Synonym
4/4
Table 8 shows the most (least for AOPC-Suff) im- portant interactions according to different metrics and the rank of the most relevant (according to humans) interactions. The colors green, yellow
PHA
Synonym
1/4
AOPC Comp
Part-phrase
5/6
High
AOPC Suff
Hypernym-P2-P1
6/6
PHIA
Hypernym-P-PP
1/6
and red indicate whether the most relevant interac- tion is the first (last for AOPC-Suff), in top (bottom for AOPC-Suff) 50% or bottom (AOPC-Suff) 50% of all interactions. Table is summary of Table and Table in $2 a summary of Table 8. AOPC- Suff and PHA scores for different interactions are shown in Fig. 7 and Fig. 8, respectively.
SNLI-Neutral
AOPC Comp
Hypernym-Ple PI-PP
1/5
Low
AOPC Suff
Hypernym-
5/5
PHIA
Dangler-System-P2
1/5
AOPC Comp
Hypernym-Ple PP--
1/7
High
AOPC Suff
Hyperyym-P1-PP
7/7
PHA
Dangler-System-P2
1/7
Gever-Refute
D Louvain Community Detection
AOPC Comp
Antonym
1/6
AOPC Suff
Hypemym-P1-a
5/6
an unweighted undirected graph, the number communities is known apriori, the minimum cut approach tells us to divide the vertices such that the number of edges between the partitions is min- imized. However, that number is often unknown, and without any such constraint, this minimization would simply produce the entire graph as single community which is not desirable.
PHA
Hypemym-P1-PP
5/6
AOPC Comp
Part-phrase
3/8
High
AOPC Suff
Hypernym-PI-PP
5/8
PHIA
Dangler-System--2
3/8
Fever-Support
AOPC Comp
Hypernym-P2-Pi
1/4
Low
AOPC Suff
Synonym
4/4
PHIA
Synonym
1/4
An effective way to partition a network into com- munities not just characterized by having a low number of connections between set of vertices but rather determined by a lower number of inter- community (equivalently, higher number of intra- community) connections than what would be pected. The concept that the genuine community structure in network aligns with a statistically unexpected distribution of connections can be mea- sured through metric called modularity (Newman and Girvan, 2004). Modularity, with a scaling fac- tor, represents the difference between the number of edges within groups and the expected number of edges in comparable network where connections are randomly distributed.
AOPC Comp
Part-phrase
3/6
High
AOPC Suff
Synonym
6/6
PHA
Dangler-System-P2
2/6
Fever-NEI
AOPC Comp Hypernym-P1- PP-r2
1/5
AOPC Suff
Dangler-System-P
5/5
PHA
Dangler-System-P2
1/5
AOPC Comp
Part-phrase
2/7
Hlgh
AOPC Suff
Random-phras
5/7
PHIA
Dangler-Systemm/
1/7
Table Most (least AOPC-Suff) important relations according to different metrics rank most relevant (according humans) relations. The colors green yellow
and red indicate whether the most relevant relation is the first, in top (bottom for AOPC-Suff) 50% or bottom (top for AOPC-Suff) 50% of all relations.
Louvain Community Detection algorithm (Blon- del al., 2008) uses modularity optimization to generate communities in directed graphs such as ours. The algorithm starts with each node in its

---

Fever-Retee Refutes, SMLI Contradiction, Lov
Rever-Supports, SNLI Entaztient, Low
Fever-NEl, SALI-Noutral, Low
9.67
Antwryn
Bangler: SP5 Be
hypemyhh PL Syreeyn
sort afrrse
rarcos a1/ose
Sargler-SYST0--12
Ryperrge-rb-Ft
0.04
9.01
ALLI-AAPC-OAO
Swvrr-APCC-suTT
Fover-ASPC-suff Supports, SNLI Entailient,, High
Fever A0PC
Fever Rofute,,,,,s Contradiction, High
NEI, SILI Neutral, High
0.175
0.150
0.188
8.675
0.825
SUL.-40PC-SUTT
Fever-J0RC-soff
SKLI SUFT
Fever-AlPC-suft
SML1-A0PC
Fever Fover-A0PC-su A0PC-suff
Figure 7: AOPC-Suff scores for different interactions: averaged (etror bars: standard deviation) annotators and models. A lower AOPC value for relevant interaction (Antonym for Contradiction) indicates better human model explanation alignment.
Rever-Suports,
0.05
nean
0.84
0.05
SHLI-IHA
Fever-Pht -HA
SALI PHA
Fever-PPA
SALE-PHK
Faver- PMAA
Fever-Refutes, SWLI Centradictiion
Tever-Supports SWLI-Em Imeett
Fever-NEI. SMLI-Neutrat. Hiigh
0.3
0.3
SNLT- PHA
Tever FIAK
SULT PIA
Fever PiAk
SXLT- F-
Fover- PIA
Figure PHA scores different interactions: averaged (error bars: standard deviation) over annotators and models. higher PHA value relevant interaction (Antonym for Contradiction) indicates better comman-model explanation alignment.

---

Fever- Beeer-Befute, SNLI-Contral iinn Low
lever-Suppert, SUL1 Intailment, Ltw
Fever-MLI, SMLI-feutral. Low
9.25
Antorgo
Dompler:
hoperyyn
part afrroc
1ypersye-2--F1
raan
0.10
-2.05
comp
comp
comp
comp
Fever-Refutes SM.I Contradiction, High
Tever-Supports SMLI Entad inent. HAghh
Fever-NEI, SKLI Neutral, Hügh
9.10
0.06
9.04
-2.02
comp
comp
comp
ADPC, comp
Figure AOPC-Comp scores for different interactions: averaged hars: standard deviation) and modcls. higher AOPC value relevant interaction (Antonym for Contradiction) indicates better human-model explanation alignment. For the Part Phrase baseline, random tokens are chosen Part 1.
Retutes, SNLI- Contradiction,
Fever- Supports, Entasiment,
SALI-Neutra Low
Antoeys
Dangtter: Mypernyw-2-91
Hypermmn
8.65
0.02
0.00
SALI AOPC suTY
Fever ADPC suff
SHLI ADPC suft
Fever ADPC suft
SMLT ADPC sUff
Fever AGPC suff
Fever Refutes,
Contradiction,
SuverrSussorts, SNLI Entad
SNLI Neutrat,
0.175
0.1225
0.188
0.459
0.005
SML1 ADPC suff
Fever ADPC suff
SHLI APPC suft
Fever APPC suft
SMLI 4OPC suff
Fever 60PC suff
Figure 10: AOPC-Suff scores for different interactions: averaged (error bars: standard deviation) over annotators and models.
A lower AOPC value for relevant interaction (Antonym Contradiction) indicates better human model explanation alignment. For the Part Phrase baseline, the random tokens are chosen from Part 1.

---

Dalaset
Label
Relation
Level
AOPC Comp
AOPC SuiT
PHA
mxan
mean
std
std
0
FEVER NEI
Dangler-Sysnn Syste---yttmm-- P2
High
0.003
0.004
0.01
0.005
0.317
0.057
FEVER NEI
Dangler-Syysten-- System-
Dungler
Low
0.013
0.006
0.005
0.005
0,023
0.005
2
FEVER NEI
Dangler System P1
High
0.004
0.004
0,005
0.002
0.11
0.013
3
FEVER
NEI
Danglor-Systom-P
Low
0.011
0.005
0.004
0.004
0,023
0.004
FEVER
Hypernym P2.
High
0.007
0.01
0.007
0,02
0.007
5
FEVER
NEI
Hypenaym mn PP-PPP1
Low
0.04
0.041
0.006
0.006
0.016
0.009
PEVER
Hypennym- PPP-P
High
0.018
0.009
0,007
0.004
0,077
0.005
7
FEVER NEI
Hypernym Pl P2
Low
0.11
0.078
0.007
0.006
0.018
0.007
8
FEVER NEI
Purt-phrase
High
0.019
0.011
0,005
0.006
0,021
0.007
9
FEVER
NET
Ramdou-phrase
High
0:015
0.907
0.004
0.006
0.021
0.006
10
FEVER
NEI
Synonym
High
0.006
0.002
0.008
0.006
0,023
0.006
FEVER
Synonym
Low
0.007
0,008
0.007
0.007
12
FEVER
Refutes
Antonym
High
0.045
0.018
0015
0.004
0.021
0.004
13
FEVER
Refutes
Antonym
Low
0.173
0.051
0015
0.004
0.01
0.004
14
FEVER
Refules
Dangler System- P2
High
0.011
0.006
0,045
0.022
0.103
0.051
15
FEVER
Rofutes
Dangler-Systom-P
Low
0.043
0.015
00188
0.005
0,014
0.005
FEVER
Refutes
Dangler System
High
0.024
0,027
0.01
0,029
17
FEVER
Refutes
Dangler-Syee Syster-- PI
Low
0.049
0.017
0.017
0.005
0,016
0.005
FEVER
Refules
Hypernyns un P2-P1
High
0.025
0016
0.004
0,012
0.006
19
PEVER
Refures
Hyparnym-P22
Low
0.087
0.085
0.016
0.004
0.007
0.004
20
FEVER
Refutes
Hypernym m. Pl P2
High
0.033
0.062
0011
0.011
0,014
0.009
21
FEVER
Refutes
Hypernyme P-P1m1
Low
0.115
0.16
0.007
0.007
0.017
0.008
22
FEVER
Refutes
Part- part-phrate
High
0.08
0.019
0,015
0.005
0.009
0.008
FEVER
Refules
Ramdonn phruse
High
0.065
0015
0.005
0.009
0.005
24
FEVER
Refutes
Synonym
High
0.039
0.011
0.018
0.006
0.011
0.007
FEVER
Relutes
Synonym
Low
0.039
0.006
0.006
26
FEVER
Supports
Dangler-Systen System- P2
High
0.023
0.016
0.097
0.09
0.097
0.
27
FEVER
Supports
Dungler System-PP
Low
0.061
0,015
0.21
0.007
0,011
0.007
28
PEVER
Supports
Danglon Syrt-yyt
High
0:033
0.011
0.038
0.013
0.019
0.012
29
FEVER
Supports
Dangler System Pl
Low
0.069
0.02
0.022
0.006
0.011
0.006
FEVER
Supports
Hypernyma un P2-P1
High
0017
0.009
0.009
31
FEVER
Supports
Hypernym-P2 PP-PP1
Low
0.123
0.039
0.016
0.006
0.01
0.007
FEVER
Supports
Part-phrase
0.095
0.004
0018
0.007
0.007
33
FEVER
Supports
Random phrase
High
0.07
0.008
0.019
0.007
0.008
0.007
34
FEVER
Supports
Synonym
High
0.047
0.009
0.117
0.007
0,018
0.007
FEVER
Supports
Synonym
Low
0.061
0.013
0,016
0.006
0.02
0.006
36
SNLI
Contradiction
Antoeym
ligh
0.088
0.009
0.008
0.004
0.109
0.018
SNLI
Contradicioon
Antonym m
Low
0.215
0014
0.009
0.04
0.011
38
SNLI
Contradiction
Donglor-Syste--2
High
0.025
0.008
0.069
0.008
0.13
0.01
39
SNLI
Conradiction
Dangler- Syste-yste-m-
Low
0.05
0.009
0,028
0.007
0,054
40
SNLI
Contradiction
Dangler System- Pl
Hlghh
0.042
0.013
0,031
0.005
0.106
0.008
41
SNLI
Contradiction
Danglee-System-P
Low
0.064
0.01
0.024
0.004
0,064
0.017
SNLI
Contrdiction
Hypernym: P2.
High
0,052
0.003
0,007
0.004
43
SNLI
Contradiction
Hypenyym m- P=-P-P1
Low
0.019
0.007
0.038
0.004
0.007
0.005
SNLI
Conradiction
Hypernym-B PPP-P
High
0.011
0.008
0.043
0.005
0.005
45
SNLI
Contrdiction
Hyparnym-P1-P
Low
0.02
0.002
0.048
0.026
0.008
0.008
SNLI
Conradiction
Part phrase
High
0.049
0.006
0.037
0.005
0.01
0.008
47
SNLI
Contradiction
Ramdon-phrase
High
0.043
0.004
0,036
0.005
0.023
0.007
48
SNLI
Conradiction
Synonym
High
0.008
0.008
0.083
0.003
0.004
0.003
SNLI
Contradiction
Synonym
Low
0.009
0,057
0.003
0.003
50
SNLI
batailment
Donglor-Ssste--2
High
0.053
0.008
0.082
0.031
0.143
0.058
SNLI
Entailment
Dang Dangler-Syp SysteSytem--
Low
0.074
0.006
0.03
0.003
0.052
0.009
52
SNLI
Eniuilment
Dangler System-PP P1
High
0.046
0.01
0,093
0.006
0.034
0.007
SNLI
Entailment
Dangler System: Pl
Low
0.046
0.011
0.043
0.004
0,039
0.009
54
SNLI
Estailment
Hypernym- PPPPP1
High
0:031
0.008
0.01
0.003
0,149
0.019
55
SNLI
Entailment
Hypernym P2.P1
Low
0.058
0.012
0.018
0.006
0.039
0.008
SNLI
Entailment
Part-phrase
0.076
0.029
0.005
57
SNLI
Fotailment
Random-phrase
High
0.042
0.008
0.032
0.005
0.031
0.01
58
SNLI
Entailment
Synonym
High
0.029
0.009
0013
0.005
0,103
0.007
59
SNLI
Eniuilment
Synonym
Low
0.043
0.006
0,011
0.004
0.084
0.004
60
SNLI
Neutral
Dangler-System--
High
0.029
0.013
0.028
0.008
0.256
0.033
SNLI
Neurral
Sangler-System-PP
Law
0.049
0.009
0.023
0.007
0.047
0.009
62
SNLI
Neutral
Dangler System Pl
High
0.036
0.01
0.029
0.007
0.183
0.045
SNLI
Neutral
Dangler-Syssem-P
Low
0.024
0.008
64
SNLI
Neurral
Hypernym- P2.P1
High
0.015
0.005
0.034
0.006
0.926
0.009
SNLI
Neutral
Hypernym m P2-P1
Low
0.024
0.011
0.026
0.005
0.02
0.009
SNLI
Neutral
Hyypenym-Pt -P2
High
0.06
0.012
0.015
0.007
0.079
0.008
67
SNLI
Neutral
Hypernym-P1-aa
Low
0.105
0.018
0.019
0.008
0.032
0.009
SNLI
Neutral
Part phrase
High
0.036
0.028
0.009
0.012
69
SNLI
Neutral
Ramdom--hhase
High
0.032
0.004
0.028
0.009
0.029
0.013
SNLI
Neurral
Synonym
High
0.016
0.005
0.037
0.008
0.022
0.011
71
SNLI
Neutral
Synonym
Low
0.019
0.005
0.042
0.006
0.021
0.008
Table 9: AOPC comprehensiveness, AOPC sufficiency and PHA scores for FEVER and SNLI across different labels, relations, and levels.
community and iteratively moves them to the neigh- boring communities if that contributes to a positive modularity gain. The modularity gain of moving a
node into a community C' can be summarized as AQ An.i
Din +k.n gout where kaut, ki"
m m² are the outer and inner weighted degrees of node

---

Fever-Support,
0.03
SYSTEMORS
1opemyo-F1-12
SYSTgM
0.01
0.03
SNLT-PHA
Tevnr-FIK
SALT-PHA
Fever-PIA
SKLT-PIK
Feuer-PIA PIIA
Fever-Refuttee SMLT Contradictioon High
Tever-Support. SMLT- MuIL--Eniaa Imeett High
Fever-NEE, SMLI-Neutral. Hiigh
9.3
0.3
SHLT-PHA
Tever-FII
PHA
Fever-Pilk
T-P
Faver-PilA
Figure scores different interactions: averaged (error standard deviation) over annotators models. A higher PHA value for relevant interaction (Antonym for Contradiction) indicates better human-model explanation alignment. For the Part Phrase baseline, random tokens are chosen from Part 1.
Fin Dout are the sum of in-going and out-going i, -ott links incident to nodes in C. The algorithm termi- nates when no such gain can be achieved.
APPC -oop
0.08
E Explanation Extraction Results
Top-1 and Top-3 evaluation results are shown in Fig. 12 and Fig. 13, respectively. See Table 10 for some examples of top-1 explanations generated by our method.
6.84
acalan san
Part-strase
6:33
Peeer ADPC- comp
SNLL: CarC-app
ACPC-Suff
0.033
0.039
0.023
6.620
6.015
6.013
Feuer JOPC
SHLT MPC suff
0.35
6.34
6.93
8.92
0.31
0.33
SOLE- PHA
Figure 12: Top-1 evaluation scores for Louvain community detection over two types of attention graphs. along with the Part Phrase and Random Phrase baselines. AOPC-Comp & PHA: higher is better, AOPC-Suff: lower is better.

---

Label
Part 1
Part
Top-1 explanation
Comment
Contradiction
One tan girl with wool hat is running and lcaning over an object, while another person in wool hat sitting the ground.
A boy runs into a wall (One tan girl, boy)
correct, tam girl is antonym boy
young family enjoys feeling ocean waves lap their feet.
A family is out at (feeling, is) restaurant.
incorrect, these are not antonym relations.
Entailment
young family enjoys feeling ocean waves lap at their feet.
A family is at the (ocean waves, heach) beach.
correct, ocean waves indicate beach, i.c., synonym relation
couple walk hand hand down
A couple walking together.
(hand, together)
incorrect, relation.
Neutral
couple walk hand in hand down street.
The couple married.
(A couple, married)
correct, hypemym-PI- P2 as "a couple" does not necessarily imply married, but the re- verse true,
One girl with wool hat is running and leaning over object, while another person in wool hat sitting the ground.
A watches his daughter leap
(girl, daughter)
Correct. hypernym- P1-P2 "girl" does necessarily imply daughter, but the
One tan girl with wool hat is running and lcaning over an object, while another person in wool hat sitting the ground.
A man watches his (while, watches) daughter leap
Incorrect, there is relation.
Table 10: Example top-1 explanations generated on SNLI by the Classifer- method.

---

0.80
9.42
wedg?s
Pansen- phrane
4.80
Fever A33CC-CPC
5NL1-A37S-cong
ADPE-SSUf
6.885
8.
Fecer-Pll
7-PHA
Figure 13: Top-5 evaluation sonres for Louvain community detection over two types of attention graphs, along with the Part Phrase and Random Phrase baselines. AOPC-Comp & PHA: higher's is better, AOPC-Suff: lower better.


Reviews:
[{"review_id": "CaX71zFAFq", "reviewer": "Reviewer_hKS7", "summary": "", "questions": "1. On Page 3, Footnote 3, You mention that the datasets are highly curated which should reduce false positives of surface matched synonyms. I am not sure I understand in what you meant the datasets are curated? Can you expand on this?\n\n2. In the part phrase baseline, do you randomly select which part of the input the human explanation would be used and which one random explanation will come from?\n\n3. On line 468, you introduce the symbol c_i but it is not clear where is comes from. Is it ith dimension of CLS vector?\n\n4. When constructing G_I, do you only keep the attention weights between tokens that belong to different parts of the input to get the bipartite graph? Are the attention scores renormalized?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for appreciating the usefulness of the dataset for investigations of the models' inner workings, for guiding them to be right for the right reasons, and for future explainability research in general. Please find the answers to the questions raised below.\n\n-  _On Page 3, Footnote 3, You mention that the datasets are highly curated which should reduce false positives of surface matched synonyms. I am not sure I understand in what you meant the datasets are curated? Can you expand on this?_\n\nTo reduce false positive surface matches, we removed those where both spans include only stopwords. In addition, we took care of false negative surface matches as we found that the original datasets (FEVER and SNLI) have few instances with morphological errors (e.g., spelling mistakes, missing apostrophes). We instructed annotators to include those in their annotations. The implementation of the detailed data curation will be included upon release of the dataset, accompanied by a more meticulous description of the aforementioned steps in the manuscript.\n\n-  _In the part phrase baseline, do you randomly select which part of the input the human explanation would be used and which one random explanation will come from?_\n\nWe chose to report the results for the random tokens from the hypothesis part but we do have results where the random tokens were chosen from the premise part. They were not included as the results showed similar trends. We will add the new set of results in the appendix. \n\n-  _On line 468, you introduce the symbol c_i but it is not clear where is comes from. Is it ith dimension of CLS vector?_\n\nThanks for pointing this out. Yes, $c$ refers to the `CLS` vector, and $c_i$ refers to the $i^{th}$ element of the `CLS` vector. We will be sure to clarify this.\n \n-  _When constructing G_I, do you only keep the attention weights between tokens that belong to different parts of the input to get the bipartite graph? Are the attention scores renormalized?_\n\nYes, we keep the attention weights between the tokens that belong to different parts to create the bipartite graph. No, they are not re-normalized.\n\n-  _While not necessary, one additional experiment that may strengthen the paper is to train a model with provided explanations to see if it returns superior accuracy compared to current models. This training may take a simple form of an additional loss supervising the attention matrix, etc._\n\nThank you for this valuable suggestion. We will certainly explore this in future work.\n\nAs for the reproducibility of the paper, the complete implementation with all hyper-parameters will be released along with the proposed dataset.\n\nWe appreciate your efforts in a careful reading of the paper as reflected in the clarification questions. We hope that our responses and suggested revisions have bolstered the __soundness__ of our study as we made an effort to provide \"extra support or details\" for the \"minor points\" suggested and reflected in your soundness score.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The main strengths of this paper are:\n1. Collection of useful corpus of explanations that would provide research opportunities on whether the model follows similar reasoning processes as humans when making predictions on multi-part inputs.\n2. Experiments show that some current models do not yet use similar reasoning mechanisms to reach their conclusions, allowing the community to further investigate what mechanisms are in fact being used by the models and how to guide them to be right for the right reasons.\n3. Paper is well written and understandable by a technical reader.", "reasons_to_reject": "I have not found specific risks that should prevent this paper from appearing at a conference.", "questions_to_authors": "1. On Page 3, Footnote 3, You mention that the datasets are highly curated which should reduce false positives of surface matched synonyms. I am not sure I understand in what you meant the datasets are curated? Can you expand on this?\n\n2. In the part phrase baseline, do you randomly select which part of the input the human explanation would be used and which one random explanation will come from?\n\n3. On line 468, you introduce the symbol c_i but it is not clear where is comes from. Is it ith dimension of CLS vector?\n\n4. When constructing G_I, do you only keep the attention weights between tokens that belong to different parts of the input to get the bipartite graph? Are the attention scores renormalized?", "paper_topic_and_main_contributions": "This paper provides a corpus of span interaction annotations for two datasets (SNLI and FEVER) that can provide insight into reasoning process that humans may use to solve the associated tasks. The main goal of this corpus is to check whether the NLP models also use similar reasoning processes (or atleast focus on similar span interactions) when reaching their own predictions for these tasks.\n\nThe corpus provides span interactions of three types between token spans of two parts of the inputs - Synonyms, Antonyms and Hypernym relations. Each example is annotated by 3 annotators and the authors report a high Inter-Annotator Agreement.\n\nThe authors then test whether existing BERT based models also use similar reasoning mechanisms to reach their conclusion. They measure the AOPC based metrics (comprehensiveness and sufficiency) as the explanation tokens are either removed or kept within the inputs. The human annotated explanations are compared to two simple baselines: random selection of token spans, and keeping human explanation for one part of the input while choosing random span for other. Overall, the results are decidedly mixed where the model follows the intuitive behavior in some cases (for example, focusing on antonyms for contradiction) but not others. The authors do a good job teasing out these behaviors.\n\nThe authors also propose an algorithm to extract span interactions that the model might be using by extracting important span pairs based on final layer attention matrices. They use community detection algorithms on bipartite graphs of tokens using attention weights as edge weights. Overall, they observe high post hoc accuracy using only the extracting span pairs compared to the baselines but AOPC scores do not seem to show significant differences.\n\nOverall, I find the paper to be well written and authors provided analysis to support the research questions they set out for themselves. I also find the corpus to be useful for future explainability research on different parts of the input are interacting with each other to generate a prediction.\n\nWhile not necessary, one additional experiment that may strengthen the paper is to train a model with provided explanations to see if it returns superior accuracy compared to current models. This training may take a simple form of an additional loss supervising the attention matrix, etc.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "bXvPUjbZN3", "reviewer": "Reviewer_vXE4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for appreciating the novelty of the proposed dataset and its value for evaluating interaction explanations. Our responses to the comments are below.\n\n### Reasons to reject:\n\n_1. Details about the dataset construction are either not provided or difficult to understand. Specifically, descriptions regarding the criteria for segmenting spans, the implications of high-level and low-level spans, and the concept of relaxed span match are unclear._\n\nIn addition to the dataset construction description in Section 2, we have also provided the complete annotation guidelines in the appendix (as pointed out in L192) and plan to release the full implementations upon acceptance. We now provide additional details about the pointed aspects as follows. We will include these in the main manuscript as well.\n\n__Criteria for segmenting spans, the implications of high-level and low-level spans.__: A high-level span is the largest contiguous sequence of tokens that a) is _not_ the whole part, (i.e., not the entire premise or hypothesis) b) bears meaning in itself, and c) can be associated with a span from another part using one of the defined relations.\n\nConsider the premise “Two women are running” and the hypothesis “Two men are walking”. The label is \"contradiction’’.\n\nThe reasoning behind the \"contradiction\" decision needs to be established through the constituents in the sentences. We see that the subjects, made out of noun phrases are antonyms: \"two men\" (premise) and \"two women\" (hypothesis), and so are the predicates, made out of verb phrases: \"are running\" and \"are walking\". These are the largest meaningful constituents where such relationships can be established. Therefore, they are considered \"high-level\" spans.\n\nA low-level span is the smallest meaning-bearing constituent/text that still holds a relation. For the given example, these would be \"man\"/\"woman’\",  \"running’’/\"walking’’.\n\nFor annotating high-level span boundaries, the annotators were shown the constituency parse tree as a suggestion, but it was not enforced that the boundaries _must_ adhere to the constituents, as the semantic segmentation of a sentence does not always adhere to the syntactic one.\n\n__The concept of relaxed span match.__: In the relaxed span agreement, two annotated interactions \"match\" if there is at least one common token common between them, as pointed out in Table 3. The motivation is to _not_ penalize small boundary mismatches, as explained in L226-230. We will be sure to clarify this further should the paper get accepted.\n\n_2. The experiments mainly use feature (token*) selection/removal as means to evaluate the annotations quality. I disagree that this is a valid measure for span interaction. This harms implications of Section 3._\n\nWe want to point out that in our work a) an annotation constitutes an interaction explanation, and b) we adopt an evaluation protocol established in prior works (L266) on interaction explanations. While there are prospects for finding a more suitable evaluation protocol in future work, this is not the focus of this work.\n  \n_-   I think that if the synonym or hypernym span pairs are deleted, it is more probable that the classification decisions will be preserved. This would lead to a low AOPC-Comp score. Figure 2 also shows lower AOPC-Comp scores for synonym. This contradicts the paper’s argument in Line 313 which suggests that a higher AOPC-Comp score is desirable._\n\nAs pointed out in the manuscript, a higher AOPC-Comp score is indeed desirable, as it indicates a more faithful explanation. However, we note that Synonym/Hypernym-P2-P1 relations are important for instances with the Entailment/Supports labels. For other labels, e.g., contradiction, these relations are less relevant than other relations, say, Antonym. Consequently, the AOPC-comp scores for Synonym interactions should be lower as they are in Figure 2. Figure 2 and Table 4 further confirm that in most cases, the relevant relations indeed have higher AOPC-Comp scores than the non-relevant ones. We do agree that the Synonym relations are less important even for Entailment labels. __However, we believe that is a finding, not a limitation.__\n\n_-   However, even a small AOPC-Comp score is not effective measure for important interactions, because selecting two unimportant spans might also result in small score changes._\n\nThat is true, selecting two unimportant spans would not significantly change the AOPC-Comp score, which is desirable. This does not seem to be a limitation.\n\n_-   To simulate removing only the interaction features and not the span themselves, removing attention of Transformer architecture could be the better evaluation [1], or conditionally removing tokens [4].__\n    \nThank you for pointing out the resources [1] and [4]. However, we found that the evaluation methods proposed in [1] or [4] are not different from the evaluation methods employed in our work insofar they are all token deletion/substituition based. [1] evaluates explanations using \"Area Over Reservation Curve’’, which is very similar to AOPC-comp, and is a token selection/removal-based evaluation (see Eq. 13 in [1]). In [4], the evaluation metrics are again necessity (comprehensiveness) and sufficiency, based on token selection/removal: “_We use the two criteria sufficiency and necessity [1] in our metrics. Sufficiency measures whether a rationale is sufficient for a model prediction by comparing the model output for the full input to its output for the input built from the rationale. Necessity measures whether a rationale captures only the necessary information by comparing the model output when the rationale is removed._” Specifically, the alignment independent metrics (Sec. 4.1) in [4] are the same as ours if we constrain the perturbations to only one part (premise/hypothesis). The deletion-based metrics (Sec 4.2) are very similar to AOC-comp and AOC-suff. Finally, the substitution-based metrics (Sec 4.3) are not applicable to our case as that would require replacing the premise/hypothesis with pre-determnied spans. [4] does use an “attention mask’’ metric, which in our case would translate to zeroing out the attention weight between the explanation tokens. This is a) not a standard evaluation metric, and b) is proposed by [1] as a method to generate explanations. Also, [4] finds problems with this evaluation metric. In summary, both [1] and [4] _differ from us in how the explanations are generated_ but use _boradly the same protocol as ours in evaluating_ them. Nonetheless, we appreciate the pointer to [1] as that is very relevant to our work.\n\nAs for the reproducibility of the paper, the complete implementation with all hyper-parameters will be released along with the proposed dataset.\n\n### Typos Grammar Style And Presentation Improvements:\n\n _1. The notations in Section 3.1 may be confusing because they are not clearly describe what it means._\n\nWe understand the concern raised about the clarity of the notations and their descriptions. In the revised manuscript, we will focus on significantly improving the descriptions of the notations in question.\n\n_2. The definition of \"feature\" needs clearer elaboration. This clarity is essential, especially when interpreting descriptions such as \"removing the k most important features\" (as mentioned in line 287)._\n\nWe used the word feature to indicate both tokens/spans as we believed this would be clear from the context, but we would clarify this further.\n\n_3. The criteria distinguishing between high and low-level spans are ambiguous. I understand that if a span is longer and contains another span it can be considered \"high-level\" compared to another, but I am not convinced if clear binary classification as high and low-level is possible._\n\nPlease see above our elaboration on the low- and high-level spans. In addition, in our annotation protocol, an annotator chooses the high-level span first, and then the low-level span inside it. It is possible that looking just at a span one can not clearly say whether it is high or low level, but in our case, that problem is never raised by design.\n\n_4. My understanding is that a span is considered \"sufficient to bear meaning\" if it can be unambiguously aligned with a corresponding span in another text. However, this interpretation does not justify why 'has made' (as referenced in Figure 1) should be treated as a span, especially when 'made' by itself could also qualify as one.\nThere are papers on span-level modeling of NLI tasks , and they typically decide span boundary based on syntactic borders, such as splitting sentence by noun phrases [2], or extracting noun phrases and verb phrases [3]. If authors can provide how the guideline or annotators’ tendency aligns with existing practice of sentence segmentation for NLI or similar tasks it will be helpful for reader to understand. Authors might compare these spans with literature about token chunking to justify or hints the span annotation guidelines._\n\nWe have clarified the concept of high vs. low-level spans above. Importantly, we did provide the annotators with information about the constituent boundaries, but we did not enforce them. The suggestion for including the alignment between the annotated span boundaries and constituents is indeed valuable and could be addressed in future work.\n\n_5. Line 233: \"Relaxed Span Match agreement, where the case where interactions are annotated by three annotators at the high level and at least two at low level\" If the objective is to highlight the greater agreement among high-level spans, it might be more straightforward to report the agreement for high-level annotations and low-level annotations separately._\n\nThis is reported in Table 3.\n\n_6. Line 244: I believe it should reference Table 3, not Table 7._\n\nThis refers to Table 7 in the appendix, we will mention this explicitly in the revised manuscript.\n\nWe express our gratitude for your thorough review and valuable insights. We appreciate your favorable comments regarding the utility of the proposed dataset for further research in interaction explanations in NLP models. We have tried our best to address the issues raised in your review and hope that clears any misunderstandings. We understand your unhappiness with the lack of exposition in section 3, and we will address that if accepted. Finally, we hope that the clarifications provided here will reaffirm the __soundness__ of our study.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper introduces an interesting, novel dataset that contains annotations of span interactions. This dataset can serve as an invaluable resource for evaluating interaction explanations in complex LLM-based NLU systems.\n2. The dataset boasts high inter-annotator agreement.\n3. The example in Figure 1 provides a clear sense of what has been annotated.", "reasons_to_reject": "1. Details about the dataset construction are either not provided or difficult to understand. Specifically, descriptions regarding the criteria for segmenting spans, the implications of high-level and low-level spans, and the concept of relaxed span match are unclear.\n2. The experiments mainly use feature (token*) selection and removal to assess annotation quality. However, these evaluation techniques may not be ideal for comparing span interaction explanations. Consequently, the findings from Section 3 should be carefully examined.\n\n\n* I think that if the synonym or hypernym span pairs are deleted, it is more probable that the classification decisions will be preserved. This would lead to a low AOPC-Comp score. Figure 2 also shows lower AOPC-Comp scores for synonym. This contradicts the paper’s argument in Line 313 which suggests that a higher AOPC-Comp score is desirable. \n* However, even a small AOPC-Comp score is not effective measure for important interactions, because selecting two unimportant spans might also result in small score changes. \n* To simulate removing only the interaction features and not the span themselves, removing attention of Transformer architecture could be the better evaluation [1], or conditionally removing tokens [4].", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper introduces the SpanEx dataset, which contains human annotations that illustrate how text spans in NLU datasets interact with each other to reach a final classification decision.\nSpecifically, the natural language inference dataset SNLI and the fact-checking dataset FEVER have been annotated. Pairs of spans from two sentences are aligned and labeled as either synonym, hypernym, or antonym. Experiments demonstrate the agreement between the SpanEx dataset and the model, based on changes in model predictions after features are removed.\nMoreover, the paper proposes a novel community detection-based unsupervised method to extract interaction explanations.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "The following references are the works that are mentioned in this review. \n\n[1] Jiang, Zhongtao, et al. \"Alignment rationale for natural language inference.\" Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.\n\n[2] Stacey, Joe, et al. \"Logical Reasoning with Span-Level Predictions for Interpretable and Robust NLI Models.\" Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022.\n\n[3] Wu, Zijun, et al. \"Weakly supervised explainable phrasal reasoning with neural fuzzy logic.\" arXiv preprint arXiv:2109.08927 (2021).\n\n[4] Kim, Youngwoo, Razieh Rahimi, and James Allan. \"Alignment Rationale for Query-Document Relevance.\" Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2022.", "Typos_Grammar_Style_and_Presentation_Improvements": "1. The notations in Section 3.1 may be confusing because they are not clearly describe what it means. \n2. The definition of \"feature\" needs clearer elaboration. This clarity is essential, especially when interpreting descriptions such as \"removing the k most important features\" (as mentioned in line 287).\n3. The criteria distinguishing between high and low-level spans are ambiguous. I understand that if a span is longer and contains another span it can be considered \"high-level\" compared to another, but I am not convinced if clear binary classification as high and low-level is possible.  \n4. My understanding is that a span is considered \"sufficient to bear meaning\" if it can be unambiguously aligned with a corresponding span in another text. However, this interpretation does not justify why 'has made' (as referenced in Figure 1) should be treated as a span, especially when 'made' by itself could also qualify as one.\n* There are papers on span-level modeling of NLI tasks , and they typically decide span boundary based on syntactic borders, such as splitting sentence by noun phrases [2], or extracting noun phrases and verb phrases [3]. If authors can provide how the guideline or annotators’ tendency aligns with existing practice of sentence segmentation for NLI or similar tasks it will be helpful for reader to understand.\n* Authors might compare these spans with literatures about token chunking to justify or hints the span annotation guidelines. \n5. Line 233: \"Relaxed Span Match agreement, where the case where interactions are annotated by three annotators at the high level and at least two at low level\"\n* If the objective is to highlight the greater agreement among high-level spans, it might be more straightforward to report the agreement for high-level annotations and low-level annotations separately.\n6. Line 244:  I believe it should reference Table 3, not Table 7."}}, {"review_id": "RvQQacZ4MO", "reviewer": "Reviewer_yM6C", "summary": "", "questions": "Could you comment on the differences between the high-level and low-level spans?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for appreciating the novelty and soundness of our work. Our responses to the comments are given below.\n\n-  _Could you comment on the differences between the high-level and low-level spans?_\n\nA high-level span is the largest contiguous sequence of tokens that a) is _not_ the whole part, (i.e., not the entire premise or hypothesis) b) bears meaning in itself, and c) can be associated with a span from another part using one of the defined relations. \n\nConsider the premise “Two women are running” and the hypothesis “Two men are walking”. The label is ''contradiction’’.\n\nThe reasoning behind the \"contradiction\" decision needs to be established through the constituents in the sentences. We see that the subjects, made out of noun phrases are antonyms: \"two men\" (premise) and \"two women’’ (hypothesis), and so are the predicates, made out of verb phrases: \"are running\" and \"are walking\". These are the largest meaningful constituents where such relationships can be established. Therefore, they are considered \"high-level\" spans.\n\nA low-level span is the smallest meaning-bearing constituent/text that still holds a relation. For the given example, these would be \"man\"/\"woman\", \"running\"/\"walking\".\n\nFor annotating high-level span boundaries, the annotators were shown the constituency parse tree as a suggestion, but it was not enforced that the boundaries _must_ adhere to the constituents, as the semantic segmentation of a sentence does not always adhere to the syntactic one.\n\n - _The \"high-level vs low-level\" spans appear relative. For example, \"video games\" is considered as low-level when compared to \"54 video games\", yet it is high-level compared to \"games\". At a second thought, shouldn't \"video games\", which has an additional modifier that constrains the scope, be more low-level than \"games\"? But this point appears minor compared to the core contribution of this paper._\n\n We have clarified the concept of the high vs. low-level spans above. One final thing to note is that we consider a longer sequence as high-level and a shorter one, within the longer sequence, as low-level. These were not designed to represent hypernym/hyponym relations between the high-level and low-level span, i.e., they are not dependent on the constraining scope.\n\n-  _Another minor weakness: The models in 3.2 (BERT and RoBERTa with different sizes) are all bidirectional models. It is unsure whether the findings can generalize to uni-directional models like GPT._\n\nWe have indeed analyzed the inner workings of the most popular bidirectional Transformer models. With our dataset and the performed analysis, we have set the ground and only scratched the surface of the prospects to inspect the inner workings of a multitude of different architectures for span interactions, such as auto-regressive Transformer models. The implementation (released upon acceptance) can be easily adapted to perform the following studies in future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The span-level interaction is an important step in the decision-making procedure that deserves more attention. This paper provides an annotated dataset and tools that facilitate the research towards this direction.  \n- This paper provides interesting findings about how some LLMs respond to the span-level interactions.  ", "reasons_to_reject": "- The \"high-level vs low-level\" spans appear relative. For example, \"video games\" is considered as low-level when compared to \"54 video games\", yet it is high-level compared to \"games\". At a second thought, shouldn't \"video games\", which has an additional modifier that constrains the scope, be more low-level than \"games\"? But this point appears minor compared to the core contribution of this paper.  \n- Another minor weakness: The models in 3.2 (BERT and RoBERTa with different sizes) are all bidirectional models. It is unsure whether the findings can generalize to uni-directional models like GPT.  ", "questions_to_authors": "Could you comment on the differences between the high-level and low-level spans?", "paper_topic_and_main_contributions": "This paper considers a crucial step in the reasoning process over spans of tokens from different parts of the input: span interactions. There is a lack of explainability technique that unveils this step in the automatic solving of questions. There is also a lack of annotated ground truths.\n\nTo address these research gaps, this paper collects a dataset, SpanEx, containing 7000+ instances annotated for span interactions. This is the first dataset with human phrase-level interaction explanations with explicit labels for interaction types. The human annotators often agree on the interactions but provide complementary reasons for a prediction.\n\nFurther, this paper tests if the fine-tuned LLMs follow similar decision-making procedure (in terms of reasoning about the span-level interactions) as humans. With the sufficiency and comprehensiveness scores, this paper finds that the models rely on interactions that are consistent with the human annotations (and the consistency is higher when the humans agree more with each other).  \n\nTo add to the utility of the findings, this paper propose a novel approach for generating interaction explanations that connect textual spans from different parts of the input. This approach is based on community structure detection algorithms (the Louvain algorithm).", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "- Line 356: 4 out of 6 is not 70%. It is 66.7%, if you want to keep the number of precision points consistent throughout the paper (e.g., in lines 419-420 you took three precision points).  ", "Justification_for_Ethical_Concerns": "I do not see ethical concerns in this paper."}, "extra_text": {"Missing_References": "I found some prior works about span-level interaction. It might be nice to add them to the related works.  \n- Lu et al., (2021) Learning Span-Level Interactions for Aspect Sentiment Triplet Extraction \n- Dixit and Al-Onaizan (2019) Span-Level Model for Relation Extraction \n- Gong et al., (2017) Natural Language Inference over Interaction Space"}}]

Author rebuttal:

Meta-review: This work contributes a corpus of span interaction annotations for SNLI and FEVER instances. A primary goal of this work is to validate if NLP models use similar reasoning processes to humans. Reviewers appreciated the newly-created corpus, as well as the analyses in the paper.
Decision: Accept-Main