What-if I ask you to explain: Explaining the effects of perturbations in procedural text (2020)

Abstract: Our goal is to explain the effects of perturbations in procedural text, e.g., given a passage describing a rabbit’s life cycle, explain why illness (the perturbation) may reduce the rabbit population (the effect). Although modern systems are able to solve the original prediction task well (e.g., illness results in less rabbits), the explanation task - identifying the causal chain of events from perturbation to effect - remains largely unaddressed, and is the goal of this research. We present QUARTET, a system that constructs such explanations from paragraphs, by modeling the explanation task as a multitask learning problem. QUARTET constructs explanations from the sentences in the procedural text, achieving ~18 points better on explanation accuracy compared to several strong baselines on a recent process comprehension benchmark. On an end task on this benchmark, we show a surprising finding that good explanations do not have to come at the expense of end task performance, in fact leading to a 7% F1 improvement over SOTA.

## introduction procedural text is common in natural language (in recipes, how-to guides, etc.) and finds many applications such as automatic execution of biology experiments @xcite , cooking recipes @xcite and everyday activities @xcite . however, the goal of procedural text understanding in these settings remains a major challenge and requires two key abilities, (i) understanding the dynamics of the world inside a procedure by tracking entities and what events happen as the narrative unfolds. (ii) understanding the dynamics of the world outside the procedure that can influence the procedure. while recent systems for procedural text comprehension have focused on understanding the dynamics of the world inside the process, such as tracking entities and answering questions about what events happen, e.g., @xcite @xcite , the extent to which they understand the influences of outside events remains unclear. in particular, if a system fully understands a process, it should be able to predict what would happen if it was perturbed in some way due to an event from the outside world. such counterfactual reasoning is particularly challenging because, rather than asking what happened (described in text), it asks about what would happen in an alternative world where the change occurred. recently, @xcite introduced the wiqa dataset that contains such problems, requiring prediction of the effect of perturbations in a procedural text. they also presented several strong models on this task. however, it is unclear whether those high scores indicate that the mod-els fully understand the described procedures, i.e., that the models have knowledge of the causal chain from perturbation to effect. to test this, @xcite also proposed an explanation task. while the general problem of synthesizing explanations is hard, they proposed a simplified version in which explanations were instead assembled from sentences in the input paragraph and qualitative indicators (more/less/unchanged). although they introduced this explanation task and dataset, they did not present a model to address it. we fill this gap by proposing the first solution to this task. we present a model, quartet (qualitative reasoning with explanations) that takes as input a passage and a perturbation, and its qualitative effect. the output contains the qualitative effect and an explanation structure over the passage. see figure 1 for an example. the explanation structure includes up to two supporting sentences from the procedural text, together with the qualitative effect of the perturbation on the supporting sentences (more of or less of in figure 1 ). quartet models this qualitative reasoning task as a multitask learning problem to explain the effect of a perturbation. our main contributions are: ## related work procedural text understanding: machine reading has seen tremendous progress. with machines reaching human performance in standard qa benchmarks @xcite , more challenging datasets have been proposed @xcite that require background knowledge, commonsense reasoning @xcite and visual reasoning (antol et al., 2015; 1 all the code will be publicly shared upon acceptance @xcite . in the context of procedural text understanding which has gained considerable amount of attention recently, @xcite @xcite address the task of tracking entity states throughout the text. recently, @xcite introduced the wiqa task to predict the effect of perturbations. understanding the effects of perturbations, specifically, qualitative change, has been studied using formal frameworks in the qualitative reasoning community @xcite and counterfactual reasoning in the logic community @xcite . the wiqa dataset situates this task in terms of natural language rather than formal reasoning, by treating the task as a mixture of reading comprehension and commonsense reasoning. however, existing models do not explain the effects of perturbations. explanations: despite large-scale qa benchmarks, high scores do not necessarily reflect understanding @xcite . current models may not be robust or exploit annotation artifacts @xcite . this makes explanations desirable for interpretation @xcite . attention based explanation has been successfully used in vision tasks such as object detection @xcite because pixel information is explainable to humans. these and other token level attention models used in nlp tasks (wiegreffe and pinter, 2019) do not provide full-sentence explanations of a model's decisions. recently, several datasets with natural language explanations have been introduced, e.g., in natural language inference @xcite , visual question answering @xcite , and multihop reading comprehension (hotpotqa dataset) @xcite . in contrast to these datasets, we explain the effects of perturbations in procedural text. hotpotqa contains explanations based on two sentences from a wikipedia paragraph. models on the hotpotqa would not be directly applicable to our task and require substantial modification for the following reasons: (i) hotpotqa models are not trained to predict the qualitative structure (more or less of chosen explanation sentences in figure 1 ). (ii) hotpotqa involves reasoning over named entities, whereas the current task focuses on common nouns and actions (models that work well on named entities need to be adapted to common nouns and actions @xcite ). (iii) explanation paragraphs in hotpotqa are not ears less @xmath0: examples of our model's predictions on the dev. set in the format: "q @xmath1. supporting sentences x i , x j are compressed e.g., "the person has his ears less protected" → "ears less protected" procedural while the current input is procedural in nature with a specific chronological structure. another line of work provides more structure and organization to explanations, e.g., using scene graphs in computer vision @xcite . for elementary science questions, @xcite uses a science knowledge graph. these approaches rely on a knowledge structure or graph but knowledge graphs are incomplete and costly to construct for every domain @xcite . there are trade-offs between unstructured and structured explanations. unstructured explanations are available abundantly while structured explanations need to be constructed and hence are less scalable @xcite . generating free-form (unstructured) explanations is difficult to evaluate @xcite , and adding qualitative structure over them is nontrivial. taking a middle ground between free-form and knowledge graphs based explanations, we infer a qualitative structure over the sentences in the paragraph. this retains the rich interpretability and simpler evaluation of structured explanations as well as leverages the large-scale availability of sentences required for these explanation. it is an open research problem whether requiring explanation helps or hurts the original task being explained. on the natural language inference task (e-snli), @xcite observed that models generate correct explanations at the expense of good performance. on the cos-e task, recently @xcite showed that explanations help the end-task. our work extends along this line in a new task setting that involves perturbations and enriches natural language explanations with qualitative structure. ## problem definition we adopt the problem definition described in tandon et al. (2019) , and summarize it here. input: 1. procedural text with steps x 1 . . . x k . here, x k denotes step k (i.e., a sentence) in a procedural text comprising k steps. 2. a perturbation q p to the procedural text and its likely candidate effect q e . output: an explanation structure that explains the effect of the perturbation q p : see figure 1 for an example of the task, and an explanation consists of up to two (i.e., zero, one or two) supporting sentences i, j along with their qualitative directions d i , d j . if there is only one supporting sentence, then @xmath2. if d @xmath3, then @xmath4, @xmath5. while there can be potentially many correct explanation paths in a passage, the wiqa dataset consists of only one gold explanation considered best by human annotators. our task is to predict that particular gold explanation. assumptions: in a procedural text, steps x 1 . . . x k are chronologically ordered and have a forward flowing effect i.e., if @xmath6. prior work on procedural text makes a similar assumption @xcite . note that this assumption does not hold for cyclic processes, and cyclic processes have already been flattened in wiqa dataset. we make the following observations based on this forward-flow assumption. this assumption reduces the number of predictions, removing d j and answer label (see a2, a3). given x 1 . . . x k , q p , q e the model must predict four labels: i, j, d i , d e . ## quartet model we can solve the problem as a classification task, predicting four labels: i, j, d i , d e . if these predictions are performed independently, it requires several independent classifications and this can cause error propagation: prediction errors that are made in the initial stages cannot be fixed and can propagate into larger errors later on @xcite . to avoid this, quartet predicts and explains the effect of q p as a multitask learning problem, where the representation layer is shared across different tasks. we apply the widely used parameter sharing approach, where a single representation layer is followed by task specific output layers @xcite . this reduces the risk of overfitting to a single task and allows decisions on i, j, d i , d e to influence each other in the hidden layers of the network. we first describe our encoder and then the other layers on top, see figure 2 for the model architecture. encoder: to encode x 1 . . . x k and question q we use the bert architecture @xcite that has achieved state-of-the-art performance across several nlp tasks @xcite , where the question @xmath7. we start with a byte-pair tokenization @xcite of the concatenated passage and question (x 1 . . . x k ⊕ q) . let [x k ] denote the byte-pair tokens of sentence x k . the text is encoded as [cls] [x 1 ] [unused1] [sep] [x 2 ] [unused2] [sep] .. [q] [sep]. here, [cls] indicates a special classification token. [sep] and [unused1..k] are special next sentence prediction tokens. these byte-pair tokens are passed through a 12layered transformer network, resulting in a contextualized representation for every byte-pair token. in this contextualized representation, the vector @xmath8, ...u k , u q ] where u k denotes the encoding for [x k ], and u q denotes question encoding. let e l be the embedding size resulting from l th transformer layer. in that l th layer, the hidden representation of all transformer layers are initialized with weights from a self-supervised pre-training phase, in line with contemporary research that uses pre-trained language models @xcite . to compute the final logits, we add a linear layer over the different transformer layers in bert that are individual winners for individual tasks in our multitask problem. for instance, out of the total 12 transformer layers, lower layers (layer 2) are the best predictors for [i, j] while upper layers (layer 10 and 11) are the best performing predictors for @xcite found that the last layer is not necessarily the best performing layer. different layers seem to learn complementary information because their fusion helps. combining different layers by weighted averaging of the layers has been attempted with mixed success @xcite . we observed the same trend for simple weighted transformation. however, we found that learning a linear layer over concatenated features from winning layers improves performance. this is probably because there is very different information encoded in a particular dimension across different layers, and the concatenation preserves it better than simple weighted averaging. classification tasks: to predict the first supporting sentence x i , we obtain a softmax distribution from the forward-flow assumption made in the problem definition section earlier, we know that @xmath9, making it possible to model this as a span prediction x i:j . inline with standard span based prediction models (seo et al., 2017), we use an attended sentence representation here, denotes element-wise multiplication and ⊕ denotes concatenation. for classification of d i (and d j , since d @xmath10, we use the representation of the first token (i.e., cls @xmath11. classification of d e is performed in exactly the same manner. the network is trained end-to-end to minimize the sum of cross-entropy losses for the individual classification tasks i, j, d i , d e . at prediction time, we leverage assumptions (a4, a5, a6) to generate consistent predictions. metrics: we evaluate on both explainability and the downstream end task (qa). for explainability, we define explanation accuracy as the average accuracy of the four components of the explanation: acc @xmath12,j,d i ,de} acc(i) and acc @xmath13. the qa task is measured in terms of accuracy. hyperparameters: quartet fine-tunes bert, allowing us to re-use the same hyperparameters as bert with small adjustments in the recommended range @xcite . we use the bert-baseuncased version with a hidden size of 768. we use the standard adam optimizer with a learning rate 1e-05, weight decay 0.01, and dropout 0.2 across all the layers foot_1 . all the models are trained on an nvidia v-100 gpu. models: we measure the performance of the following baselines (two non-neural and three neural). ## downstream task in this section, we investigate whether a good explanation structure leads to better end-task performance. quartet advocates explanations as a first class citizen from which an answer can be derived. ## error analysis we analyze our model's errors (marked in red) over the dev set, and observe the following phenomena. gold: need fire for @xmath14: need fire for @xmath15 the model is generally good at predicting i, j and in many cases when the model errs, the explanation seems plausible. perhaps for the same underlying reason, human upper bound is not high on i (75.9%) and on j (66.1%). we show an example where i, j are incorrectly predicted (in red), but sound plausible. the following example shows an instance where '-' is misclassified as '+'. it implies that there is more scope for improvement here. 4 . the model is challenged by questions involving commonsense reasoning, especially to connect q p with x i in out-of-para questions. for example, in the following passage, the model incorrectly predicts • (no effect) because it fails to draw a connection between sleep and noise: pack up your camping gear, food. drive to your campsite. set up your tent. start a fire in the fire pit. cook your food in the fire. put the fire out when you are finished. go to sleep. wake up ... qp: less noise from outside qe: you will have more energy analogous to i and j, the model also makes more errors between labels '+' and '-' in out-of-para questions compared to in-para questions (39.4% vs 29.7%) -see gold: ears less @xmath16: ears less @xmath17 ## conclusion explaining the effects of a perturbation is critical, and we have presented the first system that can do this reliably. quartet not only predicts meaningful explanations, but also achieves a new state-of-the-art on the end-task itself, leading to an interesting finding that models can make better predictions when forced to explain. our work opens up new directions for future research: 1) can additional background context from the web improve explainable reasoning? 2) can such structured explanations be applied to other nlp tasks? we look forward to future progress in this area.