PI n KS : Preconditioned Commonsense Inference with Minimal Supervision (2022)

Abstract: Reasoning with preconditions such as “glass can be used for drinking water unless the glass is shattered” remains an open problem for language models. The main challenge lies in the scarcity of preconditions data and the model’s lack of support for such reasoning. We present PInKS , Preconditioned Commonsense Inference with WeaK Supervision, an improved model for reasoning with preconditions through minimum supervision. We show, empirically and theoretically, that PInKS improves the results on benchmarks focused on reasoning with the preconditions of commonsense knowledge (up to 40% Macro-F1 scores). We further investigate PInKS through PAC-Bayesian informativeness analysis, precision measures, and ablation study.

## introduction inferring the effect of a situation or precondition on a subsequent action or state (illustrated in fig. 1 ) is an open part of commonsense reasoning. it requires an agent to possess and understand different dimensions of commonsense knowledge @xcite , e.g. physical, causal, social, etc. this ability can improve many knowledge-driven tasks such as question answering @xcite , machine reading comprehension @xcite , and narrative prediction @xcite . it also seeks to benefit a wide range of real-world intelligent applications such as legal document processing @xcite , claim verification @xcite , and debate processing @xcite . multiple recent studies have taken the effort on reasoning with preconditions of commonsense knowledge @xcite @xcite . these studies show that preconditioned reasoning represents an unresolved challenge to state1 code and data on https://github.com/luka-group/pinks of-the-art (sota) language model (lm) based reasoners. generally speaking, the problem of reasoning with preconditions has been formulated as variations of the natural language inference (nli) task where, given a precondition/update, the model has to decide its effect on a common sense statement or chain of statements. for example, paco @xcite approaches the task from the causal (hard reasoning) perspective in term of enabling and disabling preconditions of commonsense knowledge, and evaluate reasoners with crowdsourced commonsense statements about the two polarities of preconditions of statements in conceptnet @xcite . similarly, δ-nli @xcite formulates the problem from soft assumptions' perspective, i.e., weakeners and strengtheners, and justifies whether the update sentence weakens or strengthens the textual entailment in sentence pairs from sources such as snli @xcite . obviously, both tasks capture the same phenomena of reasoning with preconditions and the slight difference in format does not hinder their usefulness @xcite . as both works conclude, sota models generally fall short of tackling these tasks. we identify two reasons for such shortcomings of lms on reasoning with preconditions: 1) relying on expensive direct supervision and 2) the need for improved lms to reason with such knowledge. first, current resources for preconditions of common sense are manually annotated. although this yields high-quality direct supervision, it is costly and not scalable. second, off-the-shelf lms are trained on free-text corpora with no direct guidance on specific tasks. although such models can be further fine-tuned to achieve impressive performance on a wide range of tasks, they are far from perfect in reasoning on preconditions due to their complexity of need for deep commonsense understanding and lack of large-scale training data. in this work, we present pinks (see fig. 2 ), a minimally supervised approach for reasoning with the precondition of commonsense knowledge in lms. the main contributions are 3 points. first, to enhance training of the reasoning model ( §3), we propose two strategies of retrieving rich amount of cheap supervision signals (fig. 1 ). in the first strategy ( §3.1), we use common linguistic patterns (e.g. " @xcite unless [precondition]") to gather sentences describing preconditions and actions associated with them from massive free-text corpora (e.g. omcs @xcite ). the second strategy ( §3.2) then uses generative data augmentation methods on top of the extracted sentences to induce even more training instances. as the second contribution ( §3.3), we improve lms with more targeted preconditioned commonsense inference. we modify the masked language model (mlm) learning objective to biased masking, which puts more emphasis on preconditions, hence improving the lms capability to reason with preconditions. finally, for third contribution, we go beyond empirical analysis of pinks and investigate the performance and robustness through theoretical guarantees of pac-bayesian analysis @xcite . through extensive evaluation on five representative datasets (atomic2020 @xcite , winoventi @xcite , an-ion @xcite , paco @xcite and dnli @xcite )), we show that pinks improves the performance of nli models, up to 5% macro-f1 without seeing any taskspecific training data and up to 40% macro-f1 after being incorporated into them ( §4.1). in addition to the empirical results, using theoretical guarantees of informativeness measure in pabi @xcite , we show that the minimally super-vised data of pinks is as informative as fully supervised datasets ( §4.2). finally, to investigate the robustness of pinks and effect of each component, we focus on the weak supervision part ( §5). we perform ablation study of pinks w.r.t. the linguistic patterns themselves, the recall value associated with linguistic patterns, and finally contribution of each section to overall quality and the final performance. ## problem definition common sense statements describe well-known information about concepts, and, as such, they are acceptable by people without need for debate @xcite . the preconditions of common sense knowledge are eventualities that affect happening of a common sense statement @xcite . these preconditions can either allow or prevent the common sense statement in different degrees @xcite . for example, @xcite model the preconditions as enabling and disabling (hard preconditions), whereas @xcite model them as strengthening and weakening(soft preconditions). beyond the definition of preconditions, the task of inference with preconditions is also defined differently among the literature. some task definitions have strict constraints on the format of statement, e.g. two sentence format @xcite or being human-related @xcite , whereas others do not @xcite . to unify the definitions in available literature, we define the preconditioned inference task as below: definition 1 preconditioned inference: given a common sense statement and an update sentence that serves as precondition, is the statement still allowed or prevented? this definition is consistent with definitions in the literature (for more details see appx. §g). first, similar to the definition by @xcite , the update can have different levels of effect on the statement, from causal connection (hard) to material implication (soft). second, similar to the one @xcite , the statement can have any format. ## preconditioned inference with minimal supervision in pinks, to overcome the challenges associated with inference with preconditions, we propose two sources of weak supervision to enhance the training of a reasoner: linguistic patterns to gather rich (but allowably noisy) preconditions ( §3.1), and generative augmentation of the preconditions data ( §3.2). the main hypothesis in using weaksupervision methods is that pretraining models on large amount of weak-supervised labeled data could improve model's performance on similar downstream tasks @xcite . in weak supervision terminology for heuristics, the experts design a set of heuristic labeling functions (lfs) that serves as the generators of the noisy label @xcite . these labeling functions can produce overlapping or conflicting labels for a single instance of data that will need to be resolved either with simple methods such as ensemble inference or more sophisticated probabilistic methods such as data programming @xcite , or generative @xcite . here, the expert still needs to design the heuristics to query the knowledge and convert the results to appropriate labels for the task. in addition, we propose the modified language modeling objective that uses biased masking to improve the precondition-reasoning capabilities of lms ( §3.3). ## experiments this section first showcases improvements of pinks on five representative tasks for preconditioned inference ( §4.1). we then theoretically justify the improvements by measuring the informativeness of weak supervision by pinks using pabi @xcite score and then experiment on the effect of precision (discussed in §3.1) on pinks using pabi score ( §4.2). additional analysis on various training strategies of pinks is also provided in appx. §c. ## analysis on weak supervision in this section, we shift focus from external evaluation of pinks on target tasks to analyze distinct technical component of pinks. here, through an ablation study, we try to answer four main questions to get more insight on the weak supervision provided by those components. first (q1), how each labeling function (lf; §3.1) is contributing to the extracted preconditions? second (q2), what is the quality of the weak supervision data obtained from different ways of data acquisition? third (q3), how does generative data augmentation ( §3.2) contribute to pinks? and finally (q4), how much does the precondition-aware masking ( §3.3) affect the overall performance of pinks? (q1) lf analysis: to address the first question, we use statistics of the 6 top performing lfs (see appx. §f for detailed results). these 6 top performing lfs generate more than 80% of data (coverage) with the highest one generating 59% of data and lowest one generating 1%. our results show that, in 0.14% of instances we have conflict among competing lfs with different labels and in 0.12% we have overlap among lfs with similar labels, which showcases the level of independence each lf has on individual samples. 4 (q2) quality control: to assess the quality of collected data, we used an expert annotator. the expert annotator is given a subset of the collected preconditions (preconditions-statement-label triplet) and asked to assign a binary label based on whether each the precondition is valid to its statement w.r.t the associated label. we then report the average quality score as a proxy for precision of data. we sampled 100 preconditions-statement-label triplets from three checkpoint in the pipeline: 1) extracted through linguistic patterns discussed in §3.1, 2) outcome of the generative augmentations discussed in §3.2, and 3) final data used in §3.3. 6 related work reasoning with preconditions collecting preconditions of common sense and reasoning with them has been studied in multiple works. @xcite uses the notion of "defeasible inference" @xcite in term of how an update sentence weakens or strengthens a common sense hypothesis-premise pair. for example, given the premise "two men and a dog are standing among rolling green hills.", the update "the men are studying a tour map" weakens the hypothesis that "they are farmers", whereas "the dog is a sheep dog" strengthens it. similarly, paco @xcite uses the notion of "causal complex" from @xcite , and defines preconditions as eventualities that either allow or prevent (allow negation @xcite of) a common sense statement to happen. for example, for the knowledge "the glass is shattered" prevents the statement "a glass is used for drinking water", whereas "there is gravity" allows it. in paco, based on @xcite and @xcite , authors distinguish between two type of preconditions, causal connections (hard), and material implication (tends to cause; soft). our definition covers these definitions and is consistent with both. hwang et al. (2020), sap et al. (2019), heindorf et al. (2020), and speer et al. (2017), provided representations for preconditions of statements in term of relation types, e.g. xneed in atomic2020 @xcite . however, the focus in none of these works is on evaluating sota models on such data. the closest study of preconditions to our work are @xcite , @xcite , @xcite and @xcite . in these works, direct human supervision (crowdsourcing) is used to gather preconditions of commonsense knowledge. they all show the shortcomings of sota models on dealing with such knowledge. our work differs as we rely on combination of distant-supervision and targeted fine-tuning instead of direct supervision to achieve on-par performance. similarly, @xcite , and @xcite also study the problem of reasoning with preconditions. however they do not explore preventing preconditions. weak supervision in weak-supervision, the objective is similar to supervised learning. however instead of using human/expert resource to directly annotate unlabeled data, one can use the experts to design user-defined patterns to infer "noisy" or "imperfect" labels @xcite @xcite , e.g. using heuristic rules. in addition, other methods such as re-purposing of external knowledge @xcite @xcite or other types of domain knowledge @xcite also lie in the same category. weak supervision has been used extensively in nlu. for instance, zhou et al. (2020) utilize weak-supervision to extract temporal commonsense data from raw text, brahman et al. (2020) use it to generate reasoning rationale, dehghani et al. (2017) use it for improved neural ranking models, and hedderich et al. (2020) use it to improve translation in african languages. similar to our work, aser @xcite and ascent @xcite use weak supervision to extract relations from unstructured text. however, do not explore preconditions and cannot express preventing preconditions. as they do focus on reasoning evaluation, the extent in which their contextual edges express allowing preconditions is unclear. generative data augmentation language models can be viewed as knowledge bases that implicitly store vast knowledge on the world. hence querying them as a source of weak-supervision is a viable approach. similar to our work, @xcite ## conclusion in this work we presented pinks , as an improved method for preconditioned commonsense reasoning which involves two techniques of weak supervision. to maximize the effect of the weak supervision data, we modified the masked language modeling loss function using biased masking method to put more emphasis on conjunctions as closest proxy to preconditions. through empirical and theoretical analysis of pinks, we show it significantly improves the results across the benchmarks on reasoning with the preconditions of commonsense knowledge. in addition, we show the results are robust in different precision values using the pabi informativeness measure and extensive ablation study. future work can consider improving the robustness of preconditioned inference models using methods such as virtual adversarial training @xcite . with advent of visual-language models such as @xcite , preconditioned inference should also expand beyond language and include different modalities (such as image or audio). to integrate in down-steam tasks, one direction is to include such models in aiding inference in the neuro-symbolic reasoners @xcite .