A (Mostly) Symbolic System for Monotonic Inference with Unscoped Episodic Logical Forms (2021)

Abstract: We implement the formalization of natural logic-like monotonic inference using Unscoped Episodic Logical Forms (ULFs) by Kim et al. (2020). We demonstrate this system’s capacity to handle a variety of challenging semantic phenomena using the FraCaS dataset (Cooper et al., 1996). These results give empirical evidence for prior claims that ULF is an appropriate representation to mediate natural logic-like inferences.

## introduction a monotone function between partially ordered sets either preserves or inverts the ordering of argument values. more precisely, a function f is said to be upward monotone if @xmath0. similarly, f is said to be downward monotone if @xmath1. if neither of these hold, f is said to be non-monotone. when used in the context of subset relations and entailment, monotonicity can be a tool for making natural language inferences. for instance, consider the second example in fig. 1 . never is downward monotone in entailment, since it flips the entailment ordering of (1) i had a girlfriend taller than me before entails (2) i had a girlfriend before to (2) i never had a girlfriend before entails (1) i never had a girlfriend taller than me before. natural logic is an approach to generating natural language inferences based on syntactic structure and knowledge of the semantic properties of the lexical items and local constructions @xcite . an important fragment of natural logic is monotonicity calculus which operates using syntactic structure and the knowledge of polarity inducing elements and monotonicity relationships. figure 1 shows the three basic cases of monotonicity inference, upward, downward, and non-monotone contexts leading to different entailment conditions. episodic logic (el) is an extended first-order logic designed to closely match the form and expressivity of natural language @xcite . unscoped logical form (ulf) is an underspecified form of el. ulf completely specifies the semantic type structure of el, but leaves scope, anaphora, and word sense unresolved @xcite . @xcite proposed that ulf is suitable for five classes of inferences, namely monotonic inferences, inferences based on clause-taking verbs, inferences based on counterfactuals, inferences from questions, and inferences from requests. @xcite experimentally demonstrated the capacity of ulf to generate all of those classes of inferences except monotonic inference. @xcite presented a proof-based formalism for natural logic-like monotonic inference for ulf. they established a correspondence between their formalism and the natural logic treatment @xcite , and showed that the formalism was capable of handling foundational natural logic inferences from the prior literature. we present an implementation of @xcite monotonic inference formalism and give empirical evidence for the feasibility of using ulfs as a basis for making natural logic-like inferences. our system achieves a high precision on monotonicity problems using a small number of sound inference rules on a variety of inference cases. we thereby complete the work of @xcite in experimentally demonstrating that ulf is in fact capable of handling the five kinds of inference outlined by @xcite . ## system description our inference system starts with a set of premise sentences and a hypothesis sentence in english which are automatically converted to ulf and then used to determine an entailment, contradiction, or unknown relationship between the premises and the hypothesis through a forward inference search from the premises. the inference process is modeled after the theoretical framework described by @xcite which uses slfs for identifying the polarity operator scopes and computing the global polarity context of each sub-expression. these polarities are then mapped back to the corresponding ulfs which are used as the basis for the inference rules. we simplify @xcite framework in two ways. first, we do not include the scoping possibilities in the proof process. that is, we compute a single slf for each ulf and assume that it is the correct scoping. second, we introduce variations of the monotonicity and conversion inference rules that correspond to ulf macros and specific syntax. these reduce two steps of inference (expanding the macro and then applying the inference rule) to a single step. both of these simplifications are introduced to reduce the search space and speed up the inference process. here we describe an example of the second simplification. we directly extract the monotonicity relation from the complex expression containing the nominal predicate with its premodifiers and postmodifiers (i.e., all but the determiner or kind-forming operator of the term derived from a noun phrase). in postnominal modification, the operands of the n+preds macro-combining a nominal predicate with postmodifying predicates-are used directly for inference, without expansion of the macro construct to a conjunctive lambda predicate. for example, for a postmodified noun, as in the phrase a dog in the park, we directly extract the entailments every dog in the park is a dog and every dog in the park is in the park, rather than first converting the phrase to something that is a dog and that is in the park and indirectly computing the rules via explicit predicate intersection. for an intersective prenominal modifier, as in the phrase a happy dog, we directly extract the entailments every happy dog is a dog and every happy dog is happy, rather than first converting the phrase to something that is happy and that is a dog first. the inference system has the following highlevel components: ## experimental setup in our experiments we allow a maximum of 50 inference steps and use a leaf label f1 heuristic (ll-f1) which alternates with breadth-first search (bfs) every 5 inference steps. ll-f1 computes the f1 score between the leaf labels of the new formula and the goal formula, ignoring order, but preserving repetitions. this is turned into a cost ranging 0-to-1 by subtracting it from 1. the fracas dataset is a set of entailment questions related to specific semantic phenomena that were curated by semanticists @xcite . it contains 346 problems, of which 12 do not have well-defined answers. we focus on the most relevant section of the fracas dataset, section 1: generalized quantifiers (gqs). this is also the largest section, making up almost a quarter of the dataset. due to the small size of the fracas dataset and the challenging phenomena it contains, prior research has trained and tested models on the same problems, focusing on the capacity of their systems to perform such inferences, rather than their competence in learning and generalizing to a larger scale. this aligns nicely with our goal to demonstrate the capacity to use ulfs as the basis for monotonic inferences, rather than present a system to compete with the state-of-the-art on entailment tasks. ## results our experiments show that our system is able to precisely cover a variety of semantic phenomena and constructions, but, as expected from a demonstration system, does not achieve the robustness of sota entailment systems. with a, as in, i saw the dog ⇒ i saw a dog. the imposes a flat entailment context on its restrictor whereas a imposes a positive entailment context which warrants a fresh computation of the global polarity markings. @xcite , lp @xcite , hu @xcite , and hr @xcite . @xcite and @xcite only report the overall accuracy of their systems. in the polarity propagation system used the fallback system (the polarity marking component of the natlog system) in 42 out of the 3,109 (1.3%) total polarity propagation calls made in the gqs section of the fracas evaluation. ## conclusion we have presented a simple implementation of forward monotonic inference starting with english sentences and using ulfs as the representational basis. our system shows a high degree of precision on a variety of monotonicity phenomena, successes id correct inference 18 every european has the right to live in europe; every european is a person; every (person who has the right to live in europe) ▼ can travel freely within europe ⇒ every european can travel freely within europe 59 (a few) ▲ (female committee members) ▲ are from scandinavia ⇒ at least a few committee members are from scandinavia 60 few (female committee members) ▼ are from southern europe ⇏ few committee members are from southern europe failures id correct inference 25 several delegates (got the results published in major national newspapers) ▲ ⇒ several delegates got the results published 48 at most ten commissioners spend (time) ▼ at home ⇒ at most ten commissioners spend a lot of time at home 76 few (committee members) ▼ are from southern europe ⇒ few female committee members are from southern europe empirically confirming the final class of inferences that @xcite proposed would be supported by ulf alongside a suite of pragmaticsoriented inference capabilities of ulf described in @xcite . the present effort is a feasibility demonstration and further engineering, expanding the coverage, is needed to create a system competitive with the state-of-the-art. the specifics of our demonstration system and the results point to many possible avenues of improvement. beyond direct improvements to the ulf parser, operator scoping, and inference rules to cover more constructions, the proof-search process can be expanded to explicitly include alternate parsing and scoping choices thereby enabling proper exploration of ambiguous constructions. for example, each particular english-to-ulf parse and each scoping choice leading to a distinct slf can be formulated as an inference rule that can be explored. the inference rules can also be made more flexible by implementing the ri-1 and ri-2 rules that @xcite describe as a generalization of umi. the direct access to syntactic structure from ulf leaves room for a much more sophisticated treatment of linguistic constraints (notably island constraints as discussed in section 3.4) and the logical type structure makes ulf theoretically capable of inferences from disjunctive conclusions; e.g., alice has a dog or a cat, given that alice has a furry pet and furry pets are either dogs or cats. finally, in the vein of merging ml/dl and symbolic approaches, ulf can be reliably translated back into english @xcite so that ml/dl approaches that work over raw english text can be accessed and used in conjunction with the symbolic rules. in fact, the polarity marking component of our system (section 3.3) is precisely an example of such a bridging of methods.