TextGenSHAP: Scalable Post-hoc Explanations in Text Generation with Long Documents (2024)

Abstract: Large language models (LLMs) have attracted great interest in many real-world applications; however, their "black-box" nature necessitates scalable and faithful explanations. Shapley values have matured as an explainability method for deep learning, but extending them to LLMs is difficult due to long input contexts and autoregressive output generation. We introduce TextGenSHAP, an efficient posthoc explanation method incorporating LLMspecific techniques, which leads to significant runtime improvements: token-level explanations in minutes not hours, and document-level explanations within seconds. We demonstrate how such explanations can improve end-toend performance of retrieval augmented generation by localizing important words within long documents and reranking passages collected by retrieval systems. On various opendomain question answering benchmarks, we show TextGenSHAP improves the retrieval recall and prediction accuracy significantly.

heading: 1 Introduction
text: Large language models (LLMs) continue to rapidly excel at different text-generation tasks alongside the continued growth of resources dedicated to training text-based models (Brown et al., 2020; Chowdhery et al., 2022; Touvron et al., 2023). LLM’s impressive capabilities have led to their widespread adoption throughout academic and commercial applications. Their capacity to reason cohesively on a wide range of natural language processing (NLP) tasks has motivated further efforts to enable a single model to automatically ingest increasingly large contexts. These long-context models have been shown to improve zero-shot, few-shot, and retrieval-augmented performance via in-context learning (Izacard et al., 2022b; Huang
∗enouen@usc.edu
et al., 2023a; Ram et al., 2023) and to reduce the need for training task-specific models, empowering non-experts to readily use LLMs.
Despite their remarkable text generation capabilities, LLMs which are trained to model statistical correlations in language can offer only limited insight into their internal mechanisms. Accordingly, LLMs are widely considered black-box models which are incredibly difficult to explain and understand. In the wake of widespread adoption amongst the general population, challenges beyond LLM prediction performance including safety, security, and truthfulness have gained increasing prominence. Growing reports of hallucinated material, harmful counseling, prejudiced content, and other real-world consequences continue to raise concerns. Often, explainability is hailed as a crucial avenue for addressing these concerns, enabling insights into the model’s decision-making process and allowing stakeholders to directly scrutinize the reasoning behind unsafe or untruthful responses.
Recent surveys in explainability for NLP juxtapose the two main criteria for model explanations: understandability and faithfulness (Lyu et al., 2023a; Zhao et al., 2023; Mosca et al., 2022). Understandability refers to how easily an explanation is understood by a human user, whereas faithfulness measures how accurately it reflects the model’s reasoning process. Effectively balancing these objectives for a given explanation technique remains an ongoing challenge (Rudin, 2019). Popular explanation approaches like attention scores, gradient saliency, and self-explained reasoning are generally considered to be understandable; however, ongoing debates question whether these approaches also provide high-fidelity explanations (Jain and Wallace, 2019; Adebayo et al., 2018; Ghorbani et al., 2019; Wang et al., 2020; Wei et al., 2022). For tabular and image data, the Shapley value (Lundberg and Lee, 2017) stands out due to its strong theoretical foundations, grounded in
13984
axioms guaranteeing: symmetry, efficiency, nullity, and linearity (Mitchell et al., 2022). In the NLP domain, however, faithful approaches like the Shapley value suffer greatly in their ability to scale to larger models and longer inputs, leading to impractically long wait times for explanations in LLM workflows.
To address the limitations of current explainability methods in the realm of NLP, we introduce TextGenSHAP, a novel approach to extend Shapley values for text generation while keeping a computational speed more suitable for real-world LLM use cases. We focus on explanations in the challenging scenario of open-ended text generation using long inputs as prompts, specifically focusing on the task of abstractive question answering from retrieval-augmented documents. We leverage wellfounded historical works in game theory to support our new Shapley score’s definition, and then develop an efficient algorithm designed specifically for transformer models. Accordingly, we demonstrate our method’s scalability to new applications across three key aspects shown in Fig. 1: (a) handling longer contexts with thousands of input tokens using the hierarchical structure of natural text; (b) accommodating larger models with billions of parameters using hardware-aware speedups; and (c) explaining free-form text generation, instead of only discriminative tasks like classification (which were the focus of previous work.)
Furthermore, we demonstrate how the explanations generated by our TextGenSHAP can enhance the performance of open-domain question answer-
ing on both MIRACL and NQ-Open, enhancing the recall of document retrieval systems by multiple points and closing the accuracy gap of opendomain question answering with a 5-10% point improvement.

heading: 2 Related Work
text: Post-hoc Model Explainability. There have been many works focusing on explanations which show how machine learning models utilize their input features to make predictions. Notable post-hoc explanation approaches include LIME (Ribeiro et al., 2016), SHAP (Lundberg and Lee, 2017), and Integrated Gradients (Sundararajan et al., 2017), although SHAP and Shapley have recently become dominant due to their strong foundations. For NLP, many related perturbation-based methods also exist, leveraging the hierarchical structure and sequential order of text (Chen et al., 2019; Jin et al., 2020; Chen et al., 2020). More recent methods extend beyond binary classification tasks with contrastive versions of the original techniques (Jacovi et al., 2021; Yin and Neubig, 2022). However, none of these existing works tackle non-binary hierarchies or generative text, which we identify as key challenges overcome by our approach, see Sec. 3.3 and 3.2 respectively. Although existing work has looked at accelerating Shapley value estimation (Jethani et al., 2022) for tabular and image data types, it remains challenging to extend such approaches to NLP because of generative text outputs. Specifically, all existing methods require prespecification of candidate outputs and cannot be applied
to the large output spaces of free-form text generation. Accordingly, current post-hoc methods for text generation are limited to the sequential application of tabular approaches (Sarti et al., 2023). Self-explanations and Rationales. For NLP explanations, another popular approach is training models generating ‘rationales’ to highlight important tokens for prediction, often by aligning with rationales collected either from human annotators (Arous et al., 2021; Joshi et al., 2022) or posthoc explanations (Stacey et al., 2022; Chan et al., 2022). Still, such approaches remain mostly limited to classification tasks instead of generative tasks, likely due to both the difficulties in collecting human rationales and the nonexistence of post-hoc explanations discussed above.
Natural language explanations, such as chain-ofthought (Wei et al., 2022), where LLMs emit explanations about themselves are hence some of the only available explanations for text generation. Unfortunately, such prompt-engineering and scratchpad approaches are only part of the mechanistic process of generation and provide no guarantees on faithfulness or explanation accuracy. Ongoing work aims to measure the degree of faithfulness which could be provided by such explanations (Jacovi and Goldberg, 2021; Zheng et al., 2022; Lyu et al., 2023b; Lanham et al., 2023). Information Retrieval from Long Documents. Question answering (QA) is a fundamental NLP task, evolving from reading comprehension into retrieval-augmented fusion with increasingly large knowledge bases. As early as the NQ dataset (Kwiatkowski et al., 2019), the bifurcation between the original long-document format (entire Wikipedia page) and the open-domain format (all of Wikipedia) had already emerged (Lee et al., 2019; Karpukhin et al., 2020). 1 Open-domain QA is dominated by pipelined approaches where fast retrievers rank relevant passages for slower, more thorough reader models. Recently, neural-based retrievers have emerged for this first stage, uprooting the long reign of term-frequency approaches (Izacard et al., 2022a; Karpukhin et al., 2020; Ma et al., 2021; Formal et al., 2021; Guu et al., 2020; Mao et al., 2021; Johnson et al., 2019). Simultaneously, improvements have been made on the reader model side of the pipelined approach with Fusionin-Decoder (FiD) (Izacard and Grave, 2021b,a) de-
1Unfortunately, this bifurcation leads to conflicting nomenclature for ‘document’. In the former, it is the entire long document. In the latter, it is the unit of retrieval.
signing an efficient QA architecture and ‘Lost in the Middle’ (LitM) (Liu et al., 2023) identifying the reader’s brittleness to passage order.
Architectures for long inputs. In pursuit of the impressive capabilities of large-scale, end-to-end training, there has also been a surge in architectures which can increase the context size of LMs. Maximum context windows have quickly expanded from thousands of tokens to many millions of tokens with the use of efficient sparsity methods (Wu et al., 2022; Bulatov et al., 2022; Ding et al., 2023). Two main approaches exist: methods utilizing sparsity which closely mimicks that of informationretrieval for relevant tokens or with external memory (Bertsch et al., 2023; Wu et al., 2022; Bulatov et al., 2022, 2023; Johnson et al., 2019), and methods instead using block sparse attention matrices to reduce the necessary computations of the attention mechanism (Beltagy et al., 2020; Zhang et al., 2022a; Ding et al., 2023; Dao et al., 2022).

heading: 3 Explainability Framework
text: Notation. Consider an LLM using a vocabulary of size V ∈ N for input sequences x ∈ X := [V ]d and output sequences y ∈ Y := [V ]m with input length d ∈ N and maximum output length m ∈ N, where [V ] := {1, . . . , V }. Broadly, a textgeneration model takes an input sequence of tokens and defines a probability vector over all possible outputs, F : X → [0, 1]Y . Hence, we have F (x)y denote y’s probability of being generated given x.
To enable explanation via feature attribution methods like the Shapley value, we need to be able to mask certain subsets of the input tokens. Let s ∈ M := {0, 1}d be a binary mask on the input tokens. We next define a masked text-generation model, f : X ×M → [0, 1]Y , which takes both an input sequence and an input mask. In practice, we replace all input tokens which are not in the mask s by the <pad> token before inputting it to the model. If we assume the <pad> or <mask> token is taken to be p ∈ [V ] and identify the d-vector composed of all p to be p, then we can write this as f(x, s) := F (x s+ p (1− s)).

heading: 3.1 Shapley Value
text: Shapley values, originally derived to allocate the worth of individual players in a cooperative game, have since become a dominant paradigm for explaining feature attributions of black-box models (Shapley, 1953; Lundberg and Lee, 2017). In Sec.
3.2, we describe the Shapley-Shubik and PenroseBanzhaf values which are extended to work for voting games (Shapley and Shubik, 1954; Banzhaf, 1965; Penrose, 1946). In Sec. 3.3, we describe the hierarchical extension, the Owen-Winter value (Owen, 1977; Winter, 2002). We use these two extensions to help overcome the challenges posed in Fig. 1c and 1a, respectively.
To define the ‘value functions’ required to define the Shapley score in a way that is consistent with the existing interpretability literature, we must first correspond our binary input masks with inputfeature subsets. In particular, for any element of the power set S ∈ P([d]) := {S ⊆ [d]}, there is a unique corresponding binary mask s ∈ {0, 1}d via the indicator function s = 1S . For any input token i ∈ [d], we will use the set notation (S+i) := S∪{i} and (S−i) := S\{i} to unmask or mask the token. For a fixed x, we will then write v`(S) := log(f(x, 1S)) and vp(S) := f(x, 1S) as our two candidate value functions (log-likelihood and likelihood).
The Shapley value is formulated as an expectation over uniformly distributed permutations:
ϕi = Eπ [ v`(Sπ,i + i)− v`(Sπ,i − i) ] , (1)
where π : [d]→ [d] denotes the sampled permutation, representing a random order of the features (tokens) and Sπ,i := {j ∈ [d] : π(j) < π(i)} is the set of elements which precede i in the order defined by π. Hence, Sπ,i + i = {j ∈ [d] : π(j) ≤ π(i)} and Sπ,i − i = Sπ,i = {j ∈ [d] : π(j) < π(i)}, where we unnecessarily subtract the element i in
preparation for Section 3.2. We follow the standard approach of permutation sampling to estimate the Shapley value as the empirical mean over a finite set of sampled permutations (Covert et al., 2021).
The key challenge of applying the conventional Shapley formulation is that we do not have access to the full probability vector F (x), which is of exponentially large size. For previous work in classification tasks, the log-probabilities may be computed exactly for every candidate output. In open-ended text generation, however, we utilize sequential decoding algorithms like greedy decoding and K-beam generation to recover only a sparse subset of the exponentially large probability vector F (x) ∈ [0, 1][V ]m . In the next section, we show how to adapt Shapley to handle generated text coming from distributions of a-priori unknown support.

heading: 3.2 Extension to Generative Outputs
text: Although the Shapley value has found wide success in tasks like classification and regression, it struggles to be applied to generative tasks using sequential decoding. Towards this end, we leverage the voting theory reformulation of the conventional Shapley value, called the Shapley-Shubik power index. We consider each input token as a ‘voter’ casting a vote for a generated answer, aiming to ‘elect’ their preferred answer under the LM’s blackbox voting system. While conventional Shapley employs a value function represented as the vector of log-probabilities, Shapley-Shubik formulation operates on the probability vector. Hereafter, we will refer to the ‘Shapley-Shubik power index’ as
still just ‘Shapley’ for brevity. We can equivalently reformulate Shapley as an expectation over a random subset instead of over a random permutation, highlighting its connection with the Banzhaf value:
ϕShi := ES∼PSh(S) [ [vp(S + i)− vp(S − i)]+ ]
ϕBzi := ES∼PBz(S) [ [vp(S + i)− vp(S − i)]+ ]
where PSh(S) is the Shapley distribution PSh(S) ∝ 1d+1 ( d |S| )−1
and the Banzhaf distribution is the same as the Bernoulli distribution PBz(S) ∝ p|S|(1 − p)d−|S|. In our experiments, we set both p = 50% as in the original Banzhaf value, but also p = 10% to consider smaller sets of documents. [·]+ is used to denote component-wise positive part (ReLU) which we use to take the positive part of the difference of the two probability vectors. By using these formulations on vp instead of v`, we can use standard decoding techniques like argmax decoding or K-beam search which
generate K-sparse approximations of the true vp. These zero-probability entries in vp will lead to −∞’s in the v` approximation, which cannot be easily handled by the usual Shapley value.

heading: 3.3 Extension to Hierarchical Inputs
text: Leveraging natural text’s intrinsic hierarchy, our method uses the structure of the retrieved passages to explain from the passage level to the sentence level to the word level. Unlike the original Shapley which treats each token as completely symmetric, no matter which document or sentence it came from, hierarchical Shapley ensures that the influence of a passage is distributed amongst its sentences and that the influence of a sentence is distributed amongst its words. While prior work (Jin et al., 2020; Chen et al., 2020) explored similar hierarchical extensions, they have only addressed binary hierarchies, lacking the support for more general structures. Instead, we support permutation sampling from a three-tiered hierarchy to calculate the Owen-Winter value. This replaces the Shapley distribution induced by sampling one ran-
dom permutation with the Owen-Winter distribution induced by sampling a hierarchy of random permutations (Owen, 1977; Winter, 2002).

heading: 4 TextGenSHAP: Faster Explanations
text: Input Hierarchy We leverage the hierarchical structure within natural text to also reduce the time complexity required for model explanation, following the theoretical foundations of the Owen-Winter value. We first break each long input document into its passages and measure the Shapley value of each passage, allowing us to select only those passages more important than some threshold to continue on to the sentence level and then word level.2 In our experiments, we consider thresholds of both 10% and 30% for the required importance for both paragraph and sentence level. This enables us to not waste computational effort on tokens which do not warrant further investigation. A full description is available in Algorithm 1.
Speculative Decoding Another major improvement in explanation speed is gained through utilizing speculative decoding similar to what was recently explored in Miao et al. (2023); Leviathan et al. (2023). Speculative decoding, similar to branch prediction in computer architectures, uses one or multiple guesses of the generated output sequence to reduce the time taken up by repeatedly applying the decoder model. Since mistakes are corrected by reapplying the decoder model, exact
2Besides the paragraph-sentence-word hierarchy we consider here, other hierarchies could be better suited to other applications such as structured documents, conversation agents, or code generation.
output probabilities are ultimately calculated, but speed is only enhanced when guessing correctly.
Fig. 3 depicts our speculative decoding approach which is tailored to the explainability application. Existing approaches can only speculate 2-10 tokens ahead and face relatively high error rates; however, our approach speculates the entire output sequence with lower error rates. While existing methods are designed for model inference which only gets ‘one guess’ at the full generated output, our application is perturbation-based resampling methods like the Shapley value. This means many of our generated outputs will be similar and allows us to gradually construct a bank of speculative outputs (using a tree structure) due to the redundancies of generating similar outputs. In our experiments, we verify that a large amount of total computation can be saved by speculatively decoding full outputs rather than sequentially running the decoder model.
Specifically, for each new sample (mask) coming from the Shapley distribution, we first verify (Fig. 3b) whether the argmax decoding exists or not within our speculative decoding calculation (if so we are already done with this sample). If not, then we need to generate the new candidate answer using autoregressive decoding. Afterwards, we graft the new answer to the existing causal decoding tree so we can generate this response in the future, making sure to update the causal attention matrix in order to respect the graph structure of the decoding tree (Fig. 3d). In all experiments, we use greedy decoding consistent with prior work on open-domain QA (Izacard and Grave, 2021b; Liu et al., 2023). However, we emphasize that the
speculative decoding tree can further support other popular sampling methods like beam search and nucleus generation (top-K and top-P) (Sina et al., 2021; Holtzman et al., 2020).

heading: 5 Architecture-Specific Accelerations
text: In this section, we include further details about the architectural implementations which we used to achieve the best wall-clock performance. The first section is about the widely adopted Flash Attention (Dao et al., 2022) which is used for all experiments within this paper and is what allowed for tens of thousands of tokens to be run on a single GPU. The second and third sections are additional tricks specific for the T5-FiD architecture, focusing on efficiency under the massive context sizes pursued by the long-document and open-domain communities.
Flash Attention To better address the challenges for long inputs, especially with limited compute resources, we follow recent adoptions of the Flash Attention mechanism (Dao et al., 2022) to improve both the memory efficiency and the runtime performance of LMs. Such approaches compute the attention matrix with the memory requirement scaling linearly with input size O(N) instead of quadratically O(N2) (Rabe and Staats, 2022; Dao, 2023). Block Sparse Attention We make a connection between Flash Attention and recent developments in long-document architectures (Izacard and Grave, 2021b; Beltagy et al., 2020; Ding et al., 2023) by using block sparse attention matrices for handling long inputs. Accordingly, we reformulate the original FiD to also incorporate a block sparse implementation of Flash Attention, still respecting the hardware-aware block sizes. These techniques demonstrate our method can be useful at immense context sizes, which is of heightened necessity as the context lengths of modern LLM architectures continue to grow.
In-Place Resampling We exploit the unique structure of chunking-based encoder-decoder models like FiD to get speedups significantly faster than previously attainable. In particular, we compute the encoder feature matrix just once while generating the entire explanation for a single example. Due to the independence of chunked input fragments, we only need to adjust the encoder-decoder crossattention mechanism to enable resampling with different document subsets. Reducing the memory
overhead not only reduces the computation time for re-encoding features, but allows for quicker memory accesses and larger throughput via a ‘decoding batch size’ which generates multiple outputs for a single input context. Increasing the decoding batch size enables much more hardware-efficient decoding (iterating through hundreds of permutation samples in only seconds on a single GPU).

heading: 6 Experimental Results
text: Datasets We focus on publicly-available datasets for the task of open-domain or long-document question answering: Natural Questions (NQ) (Kwiatkowski et al., 2019) and MIRACL (English subset) (Zhang et al., 2022b). We follow NQ as redesigned for open-domain question answering following (Lee et al., 2019; Karpukhin et al., 2020) called NQ-Open. In this setting, answers must be found from within all of Wikipedia, rather than a single Wikipedia page. The original NQ dataset provides short text answers and passages are rated as relevant so long as they contain the ground-truth answer. MIRACL is instead designed for information retrieval and for each query it provides binary relevance ground-truth for the ten most related passages in the corpus.3
Models For passage ranking of the corpus (retriever model) we use the recent Contriever (Izacard et al., 2022a) architecture following LitM. For question answering (reader models) we use different members of the T5 family (Raffel et al., 2020). We use the available flan tuned models at the large and XXL sizes (‘T5-large’ and ‘T5-XXL’) (Chung et al., 2022) and the fine-tuned T5 large model from FiD (‘T5-FiD’) (Izacard and Grave, 2021b).

heading: 6.1 TextGenSHAP Speed Benchmarking
text: We present benchmarks demonstrating the improved speed of TextGenSHAP. A single A100 40GB GPU is used for benchmarking all experiments. We note that our method would further benefit from parallelism across multiple GPUs.
First, we evaluate the standard Shapley value, which provides detailed token-level explanations using our Algorithm 1. In Fig. 4, we benchmark with 100 sampled permutations and 10 documents from the LitM setting for both T5-XXL and T5large. We observe that the standard Shapley value estimation requires a prohibitive 12-20 hours per
3In this work, ‘documents’ and ‘passages’ are used synonymously as the unit of retrieval for both NQ and MIRACL.
sample and show that our proposed hierarchical sampling algorithm significantly reduces this time. With the integration of speculative decoding, we can achieve an even more significant reduction in computation time, bringing computation time to nearly an hour or often faster. We note that additional speedups can be achieved in real-world settings by just sampling fewer permutations. In Appendix B.1, we show that much fewer than 100 permutation samples can suffice for accuracy gains. When using only 10 permutation samples, TextGenSHAP reduces the time for the T5-XXL model from about two hours to five minutes. We additionally benchmark the T5-FiD model accelerated with its architecture specific modifications as seen in Fig. 4c. We take document-level explanations from multiple minutes to less than ten seconds, enabling real-time improvements for document retrieval applications (see Sec. 6.4).

heading: 6.2 Visualizing Interpretations
text: We provide an example visualization in Fig. 2 to demonstrate the hierarchy enabled by TextGenSHAP. We observe the model consistently grounds its answer to the first document, which indeed contains the true answer to the example question. We also find that our hierarchical Shapley scores are effective for isolating important tokens from within contexts of thousands of tokens. We present further visualizations in Appendix D, and provide an interactive visualization hosted here.

heading: 6.3 Improved Question Answering
text: We study using TextGenSHAP to refine long information contexts. Following the recommendations in (Liu et al., 2023), we refine the model’s available documents before reaching a final answer. We evaluate top-K accuracy for small values ofK, narrowing the existing gap between the retriever’s re-
call and the reader’s accuracy, which highlights the importance of providing a diverse set of candidate answers. Fig. 5 illustrates the accuracy improvements achieved by the redistilled model compared to the majority voting baseline. TextGenSHAP significantly outperforms the baseline model, and further surpasses the majority voting baseline’s AUC scores in Table 1.

heading: 6.4 Improved Retrieval
text: model. Comparing to an existing method of reranking based on distilling attention scores (Izacard and Grave, 2021a), we see our approaches are able to achieve comparable performance.
Less pronounced improvements on the more challenging MIRACL dataset may primarily be due to its sparser label information, only providing labels for ten of the millions of available passages. We verify this claim by extending the label information using pseudo-labels. Specifically, we take all relevant passages according to the MIRACL labels and ask T5-XXL to give a short answer according to that passage alone. We then leverage this set of candidate answers to evaluate passage relevance similar to the NQ dataset. In the last column of Table 2, we see this not only improves the overall recall, but disproportionately boosts the success of TextGenSHAP, highlighting its ability to discover relevant passages missed by existing retrieval methods.

heading: 6.5 LLM Hallucinations
text: Addressing LLM hallucinations is of growing importance given modern LLM usage. Retrievalaugmented generation is seen as one effective solution (Shuster et al., 2021; Gao et al., 2023), making TextGenSHAP well posed to be able to identify and eliminate hallucinations from LLMs. By providing explainable results, our method enables human-in-the-loop approaches to further tackle the problem of hallucination. See Appendix E for further details.

heading: 6.6 Dataset Repair
text: As discussed in Sec. 6.4, our method can not only identify documents which are often underexplored by existing approaches, allowing for greater diversity in data collection, but also is able to localize critical information within extensive documents. Accordingly, we suggest that our method could enhance dataset construction pipelines by significantly reducing the burden of human annotation. Examples of this capability on the MIRACL dataset is provided in Appendix F .

heading: 7 Conclusion
text: In this paper, we introduce TextGenSHAP for enhancing the Shapley value, a trusted explainability method, to address the challenges in modern NLP applications featuring long inputs, large model sizes, and text generation. We introduce modifica-
tions to adapt the Shapley value for hierarchicallystructured input text and autoregressively-decoded output generations, drawing on insights from the game theory literature to support their theoretical motivations. Additionally, we incorporate multiple transformer-specific architecture modifications which significantly accelerate explanation generation. Our approach not only speeds up Shapley value computation for generated text but also demonstrates its effectiveness in improving performance at challenging question-answering tasks. We expect that such explanation methods will continue to find broad applicability in a variety of LLM use cases.

heading: 8 Limitations
text: The primary goal of this work is to introduce a variation of the Shapley value for generative LLMs. Although the definition is well-motivated and the experiments show significant improvement in wallclock time, many applications still have the potential to face the concern that the generated explanations could be too costly to incorporate. Models which are distributed over multiple GPUs or TPUs, are left unexplored in the current work, possibly requiring further verification to guarantee speedups of the hardware-specific modifications we make. Verification of speed and accuracy improvements over a larger range of tasks and architectures is left to future work, and evaluation against existing post-hoc techniques (IG and LIME) is limited due to the infeasibility of comparison.

heading: A LitM Reverification
text: We utilize many experiments to understand the degree of the claims from (Liu et al., 2023). In particular, we further verify how dependent it is on the semi-synthetic distribution introduced by the authors therein. There are a few major assumptions made in this semi-synthetic distribution (of planting a single document amongst a set of distractor documents) which may not always hold up in practical scenarios. First, the number of documents which are retrieved in real-world systems containing the true answer will not be exactly equal to one. Second, the order and relevancy of distractor documents may vary by retrieval system used and by documents within a corpus.
For all reader models we utilize, we verify the hypothesis from (Liu et al., 2023) on the effect of document position on model performance. In Fig. 6, we indeed see for the models trained in the typical way like T5-large and T5-XXL, we indeed reverify the hypothesis of LitM which shows a degradation in model performance whenever the true answer is placed towards the center of a very long context window. We additionally compare the performance of the permutation invariant T5-FiD model. Here, we consequently see that the model architecture trained to perform the long-document question answering task is able to increase the performance over the original T5-large model. In fact, we see that for some parts of the LitM curve, that the smaller T5-FiD model is able to outperform the much larger T5-XXL model.
To further prod the findings from (Liu et al., 2023), we investigate how changing the distractor documents in the context will alter the decision making with long context. Instead of taking the top 10 most relevant passages to serve as the distractor documents (as done (Liu et al., 2023)), we look at taking some less relevant retrieved passages by reversing the order of the top-K selected. Fig. 7 shows that making this change to the semi-synthetic setup indeed reduces the depth of the bowl-shaped curve.

heading: B Experiment Details
text: B.1 Models and Datasets Datasets Natural Questions (NQ) (Kwiatkowski et al., 2019) is a dataset originally designed for longdocument question answering, where both a relevant passage and a final answer must be selected from a single Wikipedia page. NQ is redesigned for open-domain question answering following (Lee et al., 2019; Karpukhin et al., 2020) which convert Wikipedia into a corpus of passages instead of pages, and only require giving a final answer which can be found amongst said passages. The original NQ dataset provides short text answers and passages are rated as relevant so long as they contain the ground-truth answer.
MIRACL (Zhang et al., 2022b). is a dataset designed for information retrieval over Wikipedia passages. Using an existing information retrieval score, the dataset selected the ten most relevant passages the corpus and labeled each as either relevant or irrelevant to the question at hand. Relevance judgements are made by a human annotator who decides whether the passage information is sufficient to answer the given question; however, they are not required to justify or describe the answer as part of the label. Accordingly, only a handful of passages have ground-truth single-judgement label information. This constitutes a much sparser signal than the NQ dataset which allows for any passage which contains the ground-truth text answer to be deemed as relevant. It is for this reason we generate psuedolabels based off of the relevant MIRACL passages to reevaluate MIRACL passages using the same criteria as NQ. In this work, we only focus on the subsest of MIRACL which uses English queries and English passages.
Models We follow the standard two-stage pipeline of ODQA, first using a retriever model to select a subset of relevant passages from a massive corpus and second using a reader model to extract the question’s answer from the subset of relevant passages.
For passage ranking of the corpus (retriever model), we use the recent Contriever (Izacard et al., 2022a) architecture following LitM , using FAISS to index the embeddings (Johnson et al., 2019). For question answering (reader model), we use different members of the T5 family (Raffel et al., 2020). We use the available flan-tuned models at the large and XXL sizes (‘T5-large’ and ‘T5-XXL’) (Chung et al., 2022) and the fine-tuned T5 large model from FiD (‘T5-FiD’) (Izacard and Grave, 2021b). Specifically, these correspond to flan-t5-large and flan-t5-xxl available from (Chung et al., 2022) which are originally trained on contexts of length 512. T5-FiD corresponds to nq_reader_large from (Izacard and Grave, 2021b) which is originally trained on context lengths of one hundred passages retrieved from their co-trained retriever. Despite the sizes of training context lengths, it is common to apply such models beyond their originally trained context lengths when applied to the task of long-document question answering (Liu et al., 2023) (which is feasible due to the relative position bias implemented within T5).
B.2 Additional Results
Here we provide the additional results for various values different values of the number of permutations used to generate explanations before evaluating. Because this is the main knob for sampling based algorithms to trade between estimation accuracy and time complexity, we calculate the AUC metrics of our target application across all levels of permutations to show the different effects. We see that even in as few as ten permutations we are getting multiple points of recall AUC in the end-to-end information retrieval system.
Table 3: AUC for 3 permutations.
Natural MIRACL MIRACL Questions (Original) (Pseudo)
Baseline 84.23 80.18 84.53 TextGenSHAP 86.01 69.58 84.71 TextGenBANZ 85.76 72.84 84.80 TextGenBANZ-10 87.53 79.08 85.40
Table 4: AUC for 10 permutations.
Natural MIRACL MIRACL Questions (Original) (Pseudo)
Baseline 84.23 80.18 84.53 TextGenSHAP 87.50 74.52 85.39 TextGenBANZ 87.86 75.65 85.71 TextGenBANZ-10 88.61 81.39 86.27
Table 5: AUC for 30 permutations.
Table 6: AUC for 100 permutations.

heading: C Further Details on the Shapley Value
text: As a reminder, we consider a language model F : [V ]d → [0, 1][V ]m and we take f(x, S) := F (x s+ p (1− s)) to define a masked language model f : [V ]d × {0, 1}d → [0, 1][V ]m where the inputs, input masks, and outputs are x ∈ [V ]d, s ∈ {0, 1}d, and y ∈ [V ]m, respectively. We consider a value function v : P([d]) → RM for M = V m, and consider the choices of value function as the log-probabilities or probabilities: v`(S) := log(f(x, 1S)) and vp(S) := f(x, 1S). Please refer back to the notation section in the main text for full details if necessary.
C.1 Shapley Value
The Shapley value is a long-existing solution concept from the game theory literature, originally designed to correctly attribute the value of each individual player within a cooperative game of forming a coalition (Shapley, 1953). In recent years, this solution concept has been repurposed towards the goal of explaining black-box machine learning models, treating each individual feature as a player and dividing up the prediction output correctly between the features (Lundberg and Lee, 2017). Between this time, however, many further advancements in the game theory literature building off of the seminal work by Shapley
have continued to progress. Herein, we focus on a few such extensions of the original Shapley value as we apply them to our particular structured data of text-to-text generation models.
The first such advancement occurred only shortly after the original Shapley value’s conception; the Shapley-Shubik power index is a reformulation of the original Shapley value instead designed for voting games (Shapley and Shubik, 1954). Here, the Shapley-Shubik value measures the amount of power or influence each voter has to influence the outcome of the vote. Also in the category of voting games, the Penrose-Banzhaf index (or more commonly Banzhaf power index) was first discovered by Penrose (Penrose, 1946) and was later independently discovered by Banzhaf (Banzhaf, 1965). Even now, both Banzhaf and Shapley-Shubik remain the two well-respected pillars for how to effectively evaluate the structure of a voting game.
Along the direction of further extensions to the Shapley value, Owen years later extended the Shapley value to additional deal with a two-level hierarchical structure (Owen, 1977). In particular, one can imagine that players form coalitions within an organization but moreover that organizations themselves form coalitions with one another. The value can further be defined for multi-level hierarchical structures and is sometimes called the Owen-Winter value (Winter, 2002). The corresponding extension to the Banzhaf value is instead usually considered more straightforward and is also referred to as the Banzhaf value. In this work, we use a combination of all listed approaches to be able to apply SHAP-style (Lundberg and Lee, 2017) explanations of machine learning algorithms in the case of sequence-to-sequence transformer models, adapting to the hierarchical structure of input text and the autoregressive structure of output text.
The Shapley value is commonly formulated as a uniform expectation over permutations, which lends itself to approximation via permutation sampling:
ϕi = Eπ [ v`(Sπ,i + i)− v`(Sπ,i − i) ] = 1 |Sd| ∑
π∈Sd
{ v`(Sπ,i + i)− v`(Sπ,i − i) } (2)
where π ∈ Sd := {π : [d]→ [d] : π is bijective} is the set of permutations of size d and the expectation is computed over the uniform distribution of permutations. In other words, π represents a random order of the features (tokens) and Sπ,i := {j ∈ [d] : π(j) < π(i)} is the set of elements which precede i in the order defined by π. Hence, Sπ,i + i = {j ∈ [d] : π(j) ≤ π(i)} and Sπ,i − i = Sπ,i = {j ∈ [d] : π(j) < π(i)}.
We can equally well write the Shapley value as the average over the induced distribution on the subsets S ∈ P([d]):
ϕi = ES∼PSh(S) [ v`(S + i)− v`(S − i) ] = ∑
S⊆[d]
1 (d+ 1) ( d |S| ) · { v`(S + i)− v`(S − i) } (3)
where PSh(S) is the Shapley distribution PSh(S) ∝ 1d+1 ( d |S| )−1
Because all such definitions of this solution concept involve at least an exponential amount of terms to compute exactly, the standard approach in the literature is to use permutation sampling (Covert et al., 2021; Mitchell et al., 2022). In this work, we additionally follow the approach of permutation sampling, making adjustments as necessary to apply to hierarchical structure as described in Algorithm 1.
C.2 Shapley-Shubik
Our first important departure from the existing Shapley literature is to be able to handle the case of autoregressively decoded output sequences. All existing post-hoc explanations including attention-based, gradient-based, and perturbation-based methods cannot be directly applied to text generations. Further details on these shortcomings of existing works are further described in Section 2. In such applications to text generation when they do exist, are done autoregressively, explaining each of the output tokens individually sometimes even without regard for the decoded outputs occurring prior to each autoregressive output. Not only does this pose a serious visualization challenge as decoded outputs get longer and longer in the era of LLMs, but also the correlations of explanations between adjacent output tokens are often left improperly handled.
Algorithm 1 Pseudo-code for efficient hierarchical Shapley computation
1: Input: data sample x ∈ [V ]d, masked text generation model f : [V ]d × {0, 1}d → [V ]m, number of passages p ∈ N, number of tokens d ∈ N, hierarchical partition of tokens P = (S1, . . . , Sp) 2: Parameters: hierarchy threshold τ , number of samples T 3: Output: computed Shapley values at document level {ϕk}k∈[p] and token level {ϕk,i}k∈I,i∈Sk 4: 5: function RANDPERM(N ) 6: return {random permutation of N} 7: function ONESHAPLEYPATH(f , P , I, ϕk, ϕk,i) 8: π ← RANDPERM(p), S ← ∅, textcurr ← “ ” . Initialize the loop 9: for k = 1 : p do 10: if k /∈ I then . Case 1: Add all of the unimportant document’s tokens to S 11: S ← S ∪ Sπ(k) . Add the entire document 12: if f(x; 1S) 6= textcurr then 13: Increment the count of text f(x; 1S) in ϕπ(k) by one 14: textcurr ← f(x; 1S) 15: else . Case 2: Add the important document’s tokens one by one 16: πk ← RANDPERM(Sk) . Random order of the tokens within the document 17: for i ∈ Sk do . Iterate through each token in the document 18: S ← S ∪ {πk(i)} . Add a single token 19: if f(x; 1S) 6= textcurr then 20: Increment the count of text f(x; 1S) in ϕπ(k),πk(i) by one 21: textcurr ← f(x; 1S) 22: 23: function HIERARCHICALSHAPLEY 24: Initialize ϕk ← ~0, for each k ∈ [p] 25: Initialize ϕk,i ← ~0 for each k ∈ [p], i ∈ Sk 26: for t = 1 : T do 27: ONESHAPLEYPATH(f, P, ∅, ϕk, ϕk,i) . First, only sample at the document level 28: I ← {k ∈ [p] : ϕk/S ≥ τ} . Select the set of important documents 29: for t = 1 : T do 30: ONESHAPLEYPATH(f, P, I, ϕk, ϕk,i) . Second, sample at the token level for certain
documents 31: return {ϕk}k∈[p], {ϕk,i}k∈[p],i∈Sk
This challenge stems from the fact that when using autoregressive sequence-to-sequence models, the full output probability vector is never calculated. We need to utilize decoding schemes like greedy decoding, K-beam generation, or nucleus decoding to approximate the most likely parts of the output generation space. In contrast to existing post-hoc approaches, our method is able to explain the full output sequence by reformulating Shapley into the Shapley-Shubik formulation on the probability vector and yielding an explanation on the entire prediction sequence.
We define the Shapley-Shubik and Banzhaf values as :
ϕShi := ES∼PSh(S) [ [vp(S + i)− vp(S − i)]+ ] ϕBzi := ES∼PBz(S) [ [vp(S + i)− vp(S − i)]+ ] (4)
where PSh(S) is the Shapley distribution PSh(S) ∝ d−1( d|S|)|S|(d−|S|) and the Banzhaf distribution is the same as the Bernoulli distribution PBz(S) ∝ p|S|(1− p)d−|S|. Accordingly, our Shapley explanation will be well-defined even on the sparse probability vectors vp which are induced by all natural decoding algorithms. It is for this reason we are able to generate
explanations on the entire prediction output unlike existing SHAP approaches, handling generated text coming from distributions of a-priori unknown support.
C.3 Existing Variations for NLP Applications In this section, we further detail existing work and the similarities and differences between the approaches taken therein.
C.3.1 Hierarchical Variants In the literature on Shapley for NLP or perturbation-based explanations for NLP, there have already been approaches leveraging the sequential and/or hierarchical structure of NLP data. In this section, we highlight the similarities and differences of existing approaches. One of the earliest approaches using structured versions of the Shapley value, (Chen et al., 2019) defines a Shapley value which can only consider coalitions with its neighbors (using linear structure for text data) meaning that word interactions will only span across adjacent phrases. This work does not explicitly leverage the further hierarchical structure of text data, but still utilizes input structure of text information. One of the earliest works using the hierarchical structure, (Jin et al., 2020), uses human-labeled grammatical hierarchies coming from the SST-2 sentiment classification dataset to assist in generating explanations. Their explanations give values to each node in the hierarchy and are done using their sampling and occlusion algorithm, similar to perturbation-based approaches from the interpretability literature. Finally, (Chen et al., 2020) automatically generates a hierarchy over the input text via a specially designed splitting algorithm. Phrases are split in binary pairs by choosing the weakest set of interacting phrases. Searching over phrase splits can be done in linear time by assuming phrases are sequential. Accordingly, all existing approaches will only apply to binary hierarchies and there are no existing approaches which can handle more complex hierarchies like the paragraph-sentence-word tiering which we consider in this work by utilizing permutation sampling on the Owen-Winter value.
C.3.2 Constrastive Variants Additionally, there have also been more recent advancements on the output structure side for Shapley-style attributions. In the context of language modeling (text to text) applications, there is a greater need to handle the growing complexity of an explanation with respect to the language model. While many works have tried the simple reformulation of language modeling as a classification task of the first produced token, fewer works have made further progress in providing sensible explanations beyond a vector over all possible output tokens (often amongst tens of thousands of tokens or more). In particular, the main approach leveraged is that of contrastive explanations, which specifically requires a comparison between two alternative output tokens, rather than a broad explanation across them all. (Jacovi et al., 2021) applies these techniques to still the simpler case of multiclass classification, highlighting the value of contrastive explanations for NLP applications. More recently, (Yin and Neubig, 2022) applies similar techniques to the case of language modeling on the first token, using grammatical information as useful candidates for contrastive explanations. Nevertheless, seemingly no existing work has yet developed post-hoc explanations which can adapt to the case of full-fledged output text generation.
D Visualization of Explanations
We can gain insights into how our hierarchically structured interpretations give values at different levels, attributing importance to passages from different documents and then further localizing these attributions to the sentence and word level. We also provide an interactive version of the following visualizations hosted here.

heading: E LLM Hallucinations
text: A problem of increasing importance is the issue of AI hallucinations created by LLMs (Dziri et al., 2022). With the wide dissemination of AI dialogue agents, there is a larger demand than ever before to resolve the longstanding problem of factual inaccuracy or hallucinations made by AI language models. In particular, the increased usage of LLMs as all-purpose information assistants has also added for the need to provide factual details to users requesting (and often expecting) accurate information.
Alongside larger service pipelines which will accurately branch to additional tools such as calculators, compilers, or external APIs, retrieval augmented generation is one of the leading candidate for ensuring the factuality of statements in text-generation provided by dialogue systems (Shuster et al., 2021; Huang et al., 2023b). Providing trusted source documents which are related to the current conversation topic or specifically answer a requested question is likely one of the only ways to continue to assert factual information across nearly the entire spectrum of human knowledge.
In accordance with the ultimate need for trusted sources of information to ensure the factuality of LLM-generated outputs, we envision a key application of TextGenSHAP will be towards eliminating hallucinations in LLMs. Existing methods have already demonstrated that the research-and-revise workflow is able to significantly reduce model hallucinations (Shuster et al., 2021; Gao et al., 2023). With the introduction of TextGenSHAP, we not only enable the improved retrieval step demonstrated in the main body of this work, but also further enable a cycle of improvement and human-in-the-loop feedback. Allowing human insight further enables previously successful methods like RLHF (Ouyang et al., 2022) to be incorporated to increase factuality and reduce hallucinations.

heading: F Further Analysis for Dataset Repair on the MIRACL Dataset
text: In this section we dive into specific example queries and passages found from within the MIRACL dataset to analyze how appropriately they are being judged. For each example, we provide the question being asked and a table of relevant passages. In particular, for each query we provide the top-three rated passages according to the Shapley value computed for the query. In addition, we provide some of the most relevant passages which were not significantly considered by the Shapley value or those which were specifically rated by the MIRACL dataset (are one of the ten total passages which have a positive/relevant or negative/irrelevant label.) We cover three main types of examples to try to give a good coverage of which differences exist across the interpretations and across the dataset labels.
F.1 Erroneous Labels These examples represent the relatively serious scenario where the original labels from the MIRACL dataset are found to be erroneous after exploration with our interpretabile explanations. We find that the selected passages from the explanation scores allow for us to quickly discover incorrect labels by finding the most important passages from a large corpus of potentially relevant information. In Table 7, we see that the original dataset mislabels paragraphs as irrelevant when they actually contain relevant information about grasshoppers’ diets. In Table 8, we see that the human annotator actually mistakes the ‘dialect test’ with the ‘dialectal method’, causing incorrect labeling of the passages.
F.2 Insufficient Labels These examples represent the relatively benign scenario where all labels are seemingly correct, but there is still an abundance of unlabeled passages which contain all of the necessary information. In particular, we highlight examples in Tables 9 and 10 where our method effectively locates passages which accurately answer the original query, but which are not in the top ten originally-retrieved passages from the information retrieval system. This paucity of label information in the MIRACL dataset restricts our method from its fullest potential when we consider the AUC metric only using the MIRACL’s top ten labels. It is for this reason we consider utilizing the psuedolabel evaluation in the main text as a better signal for the end-to-end ODQA task.
F.3 Explanations Insufficient In the final set of examples, we show the case where the explanations from the LLM identify incorrect passages. In Table 11, when looking for the origin of quantum field theory, the model focuses on the paper by Born, Heisenberg, and Jordan. Although extremely related, this work is generally considered a precursor to what is called quantum field theory rather than its first paper (Kuhlmann, 2023). In Table 12, we see the results finding the date of establishing the state flower of Texas. Although the highest rated explanation is a relevant passage, the next two highest have information both about Texan history and about the bluebonnet, but do not have the necessary dates to answer the question. We envision that even for such cases our method will still be useful for dataset construction and repair: since our method finds more relevant and more closely ambiguous paragraphs than existing retrieval-based systems, one will be able to more effectively utilize human annotators when using our method.
Shapley MIRACL True Label Ranking Rating Rating Agreement Title Text
1st Relevant Relevant Good Grasshopper Grasshoppers eat large quantities of foliage both as adults and during their development, and can be serious pests of arid land and prairies. Pasture, grain, forage, vegetable and other crops can be affected. Grasshoppers often bask in the sun, and thrive in warm sunny conditions, so drought stimulates an increase in grasshopper populations. A single season of drought is not normally sufficient to stimulate a major population increase, but several successive dry seasons can do so, especially if the intervening winters are mild so that large numbers of nymphs survive. Although sunny weather stimulates growth, there needs to be an adequate food supply for the increasing grasshopper population. This means that although precipitation is needed to stimulate plant growth, prolonged periods of cloudy weather will slow nymphal development. 2nd Irrelevant Relevant Erroneous Grasshopper Grasshoppers are plant-eaters, with a few species at times becoming serious pests of cereals, vegetables and pasture, especially when they swarm in their millions as locusts and destroy crops over wide areas. They protect themselves from predators by camouflage; when detected, many species attempt to startle the predator with a brilliantly-coloured wing-flash while jumping and (if adult) launching themselves into the air, usually flying for only a short distance. Other species such as the rainbow grasshopper have warning coloration which deters predators. Grasshoppers are affected by parasites and various diseases, and many predatory creatures feed on both nymphs and adults. The eggs are the subject of attack by parasitoids and predators. 3rd Irrelevant Relevant Erroneous Grasshopper Most grasshoppers are polyphagous, eating vegetation from multiple plant sources, but some are omnivorous and also eat animal tissue and animal faeces. In general their preference is for grasses, including many cereals grown as crops. The digestive system is typical of insects, with Malpighian tubules discharging into the midgut. Carbohydrates are digested mainly in the crop, while proteins are digested in the ceca of the midgut. Saliva is abundant but largely free of enzymes, helping to move food and Malpighian secretions along the gut. Some grasshoppers possess cellulase, which by softening plant cell walls makes plant cell contents accessible to other digestive enzymes.
– Irrelevant Irrelevant Good Kosher locust
In 1911, Abraham Isaac Kook, the chief rabbi of Ottoman Palestine, addressed a question to the rabbinic Court at Sanaá concerning their custom of eating grasshoppers, and whether this custom was observed by observing their outward features, or by simply relying upon an oral tradition. The reply given to him by the court was as follows: “The grasshoppers which are eaten by way of a tradition from our forefathers, which happen to be clean, are well-known unto us. But there are yet other species which have all the recognizable features of being clean, yet do we practice abstaining from them. [Appendage]: The clean grasshoppers () about which we have a tradition are actually three species having each one different coloration [from the other], and each of them are called by us in the Arabian tongue, “ğarād" (locusts). But there are yet other species, about which we have no tradition, and we will not eat them. One of which is a little larger in size than the grasshoppers, having the name of “’awsham". There is yet another variety, smaller in size than the grasshopper, and it is called “hanājir" (katydids).
– Irrelevant Irrelevant Good North American least shrew Its diet consists of mostly small invertebrates, such as caterpillars, beetle larvae, earthworms, centipedes, slugs, and sow bugs. It will also eat from the corpses of dead animals, and small amounts of seeds or fruits. This shrew will eat its prey whole, but when eating crickets and grasshoppers, the North American least shrew will bite off the head of its prey and eat only the internal organs. When fighting a larger creature, it will aim for the legs and try to cripple its adversary, and will bite lizards, which are often too large for it to kill, on the tail, which then falls off and provides it with a meal while the lizard escapes. The North American least shrew will also sometimes live inside beehives and eat all the larvae. It will often share its food with other shrews. It eats more than its body weight each day and is known to store food.
Table 7: Examples from the MIRACL Dataset for the query of “What do Grasshoppers eat?”
Shapley MIRACL True Label Ranking Rating Rating Agreement Title Text
1st Unrated Relevant Okay Interpersonal communication A dialectical approach to interpersonal communication was developed by scholars Leslie Baxter and Barbara Montgomery. Their dialectical approach revolves around the notions of contradiction, change, praxis, and totality. Influenced by Hegel, Marx, and Bakhtin, the dialectical approach is informed by an epistemology that refers to a method of reasoning by which one searches for understanding through the tension of opposing arguments. Utilizing the dialectical approach, Baxter and Montgomery developed two types of dialectics that function in interpersonal relationships: internal and external. These include autonomyconnection, novelty-predictability, openness-closedness. 2nd Unrated Relevant Okay Dialectical research
Dialectical research or dialectical inquiry or dialectical investigation is a form of qualitative research which utilizes the method of dialectic, aiming to discover truth through examining and interrogating competing ideas, perspectives or arguments. Dialectical research can be seen as a form of exploratory research, in that there is not so much a research hypothesis to be tested, but rather new understandings to be developed.
3rd Unrated Relevant Okay Dialectic Dialectic or dialectics (, “dialektike"; related to dialogue), also known as the dialectical method, is at base a discourse between two or more people holding different points of view about a subject but wishing to establish the truth through reasoned arguments. Dialectic resembles debate, but the concept excludes subjective elements such as emotional appeal and the modern pejorative sense of rhetoric. Dialectic may be contrasted with the didactic method, wherein one side of the conversation teaches the other. Dialectic is alternatively known as minor logic, as opposed to major logic or critique.
– Relevant Irrelevant Erroneous Dialect Test The Dialect Test was created by A.J. Ellis in February 1879, and was used in the fieldwork for his work “On Early English Pronunciation". It stands as one of the earliest methods of identifying vowel sounds and features of speech. The aim was to capture the main vowel sounds of an individual dialect by listening to the reading of a short passage. All the categories of West Saxon words and vowels were included in the test so that comparisons could be made with the historic West Saxon speech as well as with various other dialects. – Irrelevant Relevant Erroneous Frankfurt School
The Institute also attempted to reformulate dialectics as a concrete method. The use of such a dialectical method can be traced back to the philosophy of Hegel, who conceived dialectic as the tendency of a notion to pass over into its own negation as the result of conflict between its inherent contradictory aspects. In opposition to previous modes of thought, which viewed things in abstraction, each by itself and as though endowed with fixed properties, Hegelian dialectic has the ability to consider ideas according to their movement and change in time, as well as according to their interrelations and interactions.
Table 8: Examples from the MIRACL Dataset for the query of “When is the dialectical method used?”
Shapley MIRACL True Label Ranking Rating Rating Agreement Title Text
1st Relevant Relevant Good List of songs in Guitar Hero Live “Guitar Hero Live" is a 2015 music video game that’s developed by FreeStyleGames and published by Activision. It is the first title in the “Guitar Hero" series since it went on hiatus after 2011, and the first game in the series available for 8th generation video game consoles (PlayStation 4, Wii U, and Xbox One). The game was released worldwide on 20 October 2015 for these systems as well as the PlayStation 3, Xbox 360, and iOS devices including the Apple TV. 2nd Unrated Relevant Okay List of songs in Guitar Hero Live Two hundred songs were initially available on GHTV on the game’s release on 20 October 2015. 3rd Unrated Relevant Okay Guitar Hero
Following a five-year hiatus, as described below, Activision announced “Guitar Hero Live" for release in late 2015 on most seventh-generation and eighthgeneration consoles. “Live" was developed to rebuild the game from the ground up, and while the gameplay remains similar to the earlier titles, focusing primarily on the lead guitar, it uses a 3-button guitar controller with each button having “up" and “down" positions, making for more complex tabulators. The game using live footage of a rock concert, taken from the perspective of the lead guitarist, as to provide a more immersive experience.
– Relevant Relevant Good Guitar Hero
In 2015, Activision announced the first new title to the series in 5 years, “Guitar Hero Live", released in October 2015. The title is considered a reboot of the series, with development being performed by FreeStyleGames, who had developed the “DJ Hero" games previously. As of December 1, 2018, Activision disabled the GHTV servers for Guitar Hero Live, reducing playable content from approximately 500 songs to 42 on disc tracks.
– Irrelevant Irrelevant Good Guitar Hero Live
In an earnings report shortly following the gameś release, Activision stated that “Guitar Hero Live" was outselling their previous two “Guitar Hero" games, “" and “Guitar Hero 5", though did not report exact sales numbers. In their quarterly earnings results presented in February 2016, Activision reported that sales for “Guitar Hero Live" missed their expectations, and in March 2016, announced that they had to let go of about 50 of FreeStyleGamesémployees, though the studio still remains open to continue additional work for Activision. Prior to the Electronic Entertainment Expo 2016, Activision stated they will continue to produce content for “Guitar Hero Live" but have no present plans for another game.
Table 9: MIRACL Dataset Example for: “When was Guitar Hero Live first released?”
Shapley MIRACL True Label Ranking Rating Rating Agreement Title Text
1st Unrated Relevant Okay Origin of Hangul The Korean alphabet is the native script of Korea, created in the mid fifteenth century by King Sejong, as both a complement and an alternative to the logographic Sino-Korean “hanja". Initially denounced by the educated class as “eonmun" (vernacular writing), it only became the primary Korean script following independence from Japan in the mid-20th century. 2nd Unrated Relevant Okay Hangul The Korean alphabet, known as Hangul ( ; from Korean , ), has been used to write the Korean language since its creation in the 15th century by King Sejong the Great. It may also be written following the standard Romanization. 3rd Unrated Relevant Okay Jeong Inji
He is perhaps best known for having written the postscript of the “Hunmin Jeongeum Haerye", the commentary on and explanation of the native alphabet Hangeul invented by King Sejong in 1443. He also contributed to the “Goryeosa", the official history of Goryeo dynasty, and the “Yongbi Eocheon-ga".
– Relevant Relevant Good Korea The Korean alphabet hangul was also invented during this time by King Sejong the Great. – Relevant Relevant Good Origin of Hangul
Hangul was personally created and promulgated by the fourth king of the Joseon dynasty, Sejong the Great. Sejong’s scholarly institute, the Hall of Worthies, is often credited with the work, and at least one of its scholars was heavily involved in its creation, but it appears to have also been a personal project of Sejong.
Table 10: MIRACL Dataset Example for: “Who invented Hangul?”
Shapley MIRACL True Label Ranking Rating Rating Agreement Title Text
1st Irrelevant Irrelevant Good Quantum field theory Through the works of Born, Heisenberg, and Pascual Jordan in 1925-1926, a quantum theory of the free electromagnetic field (one with no interactions with matter) was developed via canonical quantization by treating the electromagnetic field as a set of quantum harmonic oscillators. With the exclusion of interactions, however, such a theory was yet incapable of making quantitative predictions about the real world. 2nd Unrated Irrelevant Okay History of quantum field theory
In 1925, Werner Heisenberg, Max Born, and Pascual Jordan constructed just such a theory by expressing the field’s internal degrees of freedom as an infinite set of harmonic oscillators, and by then utilizing the canonical quantization procedure to these oscillators; their paper was published in 1926. This theory assumed that no electric charges or currents were present and today would be called a free field theory.
3rd Unrated Irrelevant Okay Quantum field theory In 1913, Niels Bohr introduced the Bohr model of atomic structure, wherein electrons within atoms can only take on a series of discrete, rather than continuous, energies. This is another example of quantization. The Bohr model successfully explained the discrete nature of atomic spectral lines. In 1924, Louis de Broglie proposed the hypothesis of wave-particle duality, that microscopic particles exhibit both wave-like and particle-like properties under different circumstances. Uniting these scattered ideas, a coherent discipline, quantum mechanics, was formulated between 1925 and 1926, with important contributions from de Broglie, Werner Heisenberg, Max Born, Erwin Schrödinger, Paul Dirac, and Wolfgang Pauli.
– Unrated Relevant Okay History of quantum field theory The first reasonably complete theory of quantum electrodynamics, which included both the electromagnetic field and electrically charged matter as quantum mechanical objects, was created by Paul Dirac in 1927. This quantum field theory could be used to model important processes such as the emission of a photon by an electron dropping into a quantum state of lower energy, a process in which the “number of particles changes"—one atom in the initial state becomes an atom plus a photon in the final state. It is now understood that the ability to describe such processes is one of the most important features of quantum field theory. – Relevant Relevant Good History of quantum field theory
The third thread in the development of quantum field theory was the need to handle the statistics of many-particle systems consistently and with ease. In 1927, Pascual Jordan tried to extend the canonical quantization of fields to the many-body wave functions of identical particles using a formalism which is known as statistical transformation theory; this procedure is now sometimes called second quantization. In 1928, Jordan and Eugene Wigner found that the quantum field describing electrons, or other fermions, had to be expanded using anti-commuting creation and annihilation operators due to the Pauli exclusion principle (see Jordan–Wigner transformation). This thread of development was incorporated into many-body theory and strongly influenced condensed matter physics and nuclear physics.
Table 11: MIRACL Dataset Example for: “When was quantum field theory developed?”
Shapley MIRACL True Label Ranking Rating Rating Agreement Title Text
1st Relevant Relevant Good Bluebonnet (plant) Bluebonnet is a name given to any number of blue-flowered species of the genus “Lupinus" predominantly found in southwestern United States and is collectively the state flower of Texas. The shape of the petals on the flower resembles the bonnet worn by pioneer women to shield them from the sun. Species often called bluebonnets include:On March 7, 1901, “Lupinus subcarnosus" became the only species of bluebonnet recognized as the state flower of Texas; however, “Lupinus texensis" emerged as the favorite of most Texans. So, in 1971, the Texas Legislature made any similar species of “Lupinus" that could be found in Texas the state flower. 2nd Unrated Irrelevant Okay John Nance Garner
Garner was elected to the Texas House of Representatives in 1898, and reelected in 1900. During his service, the legislature selected a state flower for Texas. Garner fervently supported the prickly pear cactus for the honor, and thus earned the nickname “Cactus Jack". (The Bluebonnet was chosen.) In 1901 Garner voted for the poll tax, a measure passed by the Democratic-dominated legislature to make voter registration more difficult and reduce the number of black, minority, and poor white voters on the voting rolls. This disfranchised most minority voters until the 1960s, and ended challenges to Democratic power; Texas became in effect a one-party state.
3rd Irrelevant Irrelevant Good Alamo Fire
Maroon and white bluebonnets were developed as part of an effort to compose a Texas flag with red, white, and blue bluebonnets to celebrate Texas’ sesquicentennial in 1986. Pink bluebonnets were found in San Antonio, and reddish examples were selectively bred by Dr. Jerry Parsons of the Texas A&M AgriLife Extension Service to eventually give maroon bluebonnets in 2000. The color of these bluebonnets was fitting, as the color maroon is strongly associated with Texas A&M University.
– Irrelevant Irrelevant Good Bluebonnet Ordnance Plant The plant was operated by the National Gypsum Company but overseen by the military and was one of the four Ordnance plants in the United States during World War II. The army engineers were in charge of all plant construction while the Gypsum personnel and others worked out other strategies. Bluebonnet Ordnance Plant got its name from Major Paul Van Tuyl, who named the plant after the state flower of Texas (Bluebonnet). – Irrelevant Irrelevant Good Lupinus texensis
Lupinus texensis, the Texas bluebonnet or Texas lupine is a species of lupine endemic to Texas. With other related species of lupines also called bluebonnets, it is the state flower of Texas.
Table 12: MIRACL Dataset Example for: “When were bluebonnets named the state flower of Texas?”

heading: Future Work
text: Verification of speed and accuracy improvements over a larger range of tasks and architectures is left to future work, and evaluation against existing post-hoc techniques (IG and LIME) is limited due to the infeasibility of comparison