Becoming Different : A Language-Driven Formalism for Commonsense Knowledge

In constructing a system for learning commonsense knowledge by reading online resources for word definitions, a key challenge is to develop a formalism rich and expressive enough to capture commonsense concepts expressed in natural language. Derivations based on natural language impose strong requirements on the nature of the representation. Specifically, predicates should correspond to word senses and their argument structures in the language, and complex formulas should be constructed compositionally in a way that parallels the structure of language. To provide a suitable representation framework we need to extend interval temporal logic in several ways, including organizing time around objects rather than predicates, and developing a theory of scales. As a driving example, we analyze core meanings of the verbs change and become and the adjective different and show, after appropriate development of our formalism, how the desired meaning of change can be derived from one of its definitions in WordNet: become different. Introduction and Motivation Many applications of Artificial Intelligence, and natural language processing in particular, are hindered by a lack of extensive commonsense knowledge bases. Vast amount of knowledge is needed to understand language, as well as to plan and reason about the world. Much of it is quite mundane: if you fall asleep you become asleep; you use keys to unlock doors; people don’t like pain. While it is everyday ordinary stuff, such knowledge is critical if systems are to achieve human-levels of deep understanding of language. While there have been some efforts to encode large amounts of commonsense knowledge by hand, e.g., Cyc (Lenat, 1995), SUMO (Niles and Pease, 2001), such efforts barely make a dent in accumulating the knowledge that is needed. Further, such efforts generally are expressed in formal notations using predicates motivated by mathematics rather than attempt to create a close link to the elements of natural language (e.g, word meanings, semantic roles). Our goal is to create most of the commonsense knowledge base by reading. While recent efforts such as NELL (Carlson et al., 2010) and TextRunner (Yates et al., 2007) have been effective at collecting vast amounts of knowledge about instances (e.g., Chicago is a city) and semantic patterns (e.g., people kill people), the commonsense knowledge we need is definitional in nature to enable necessary entailments: e.g., kill means cause to die; murder means kill intentionally; fall asleep means change from awake to asleep. We are working on building knowledge bases automatically by reading definitions (Allen et al., 2013), starting with the definitions in WordNet (Fellbaum, 1998). The goal of this paper is to describe the formalism we have developed in order to facilitate the construction of effective axioms directly from natural language definitions. This requirement puts strong constraints on the nature of the representation that we note here and will develop further in the paper. In many ways, this paper has similar goals and motivations as those of Hobbs (2013). We both want to axiomatize core commonsense notions of events. Some of the differences are in the style of the formalism—we start from an explicit interval temporal logic and build from there, whereas Hobbs places eventualities as central and time plays a secondary role. But the most important difference is our emphasis on building a formalism that supports learning the knowledge by reading. Whereas Hobbs does a hand analysis of core verbs, such as cause and have, and identifies a few core meanings that he argues subsume all the WordNet senses, our goal is to axiomatize automatically most of the WordNet senses directly from their definitions. We would rather have a messy knowledge base that covers as much of the subtleties of language and word senses as possible, rather than developing a more minimal, but more abstract, theory. We base our formalism on the one developed in Allen & Ferguson (1994), henceforth AF, and Allen (1984), in which events are formalized in an interval temporal logic in a way that enables planning and reasoning. AF has reified events, with functional relations capturing semantic roles and arguments. For example, Jack lifted the ball (over interval t1) is represented as This work was supported in part by NSF grant IIS-1012205, ONR N000141210547 and James S. McDonnell Foundation 220020263. ∃e. (LIFT(e)∧(agent(e)=jack1)∧(affected(e)=ball1)∧(time(e)=t1)) While this is the underlying logic, as in AF, we usually abbreviate such expressions as LIFT(jack1, ball1, t1, e) when the specific roles are obvious. When using this abbreviation, predicates might appear to have a varying number of arguments, but this is just because of the abbreviation convention and not a formal part of the logic. The framework also builds from Allen’s interval logic of action and time (Allen 1983, 1984), where time periods can be related by Allen’s temporal relations. For this paper, we only need the meets relation, written t1:t2 and “during or equal”, written t1 ⊆ t2. A moment is an interval that has no true subintervals and captures minimally perceptible moments in time. Decomposable periods are often referred to as true intervals. The predicate Moment allows us to distinguish moments from true intervals. We also add the strong constraint on our temporal models by asserting that all intervals are constructed out of moments. This can be captured by the simple axiom that every interval contains a moment: Discrete Time Axiom: ∀t. ∃ t’ ⊆ t. Moment(t’) Representation and Linguistic Structure The key driving constraint of this work is that the representational framework should closely parallel linguistic elements and structure. We believe this is essential to enable learning conceptual knowledge by reading definitions. Specifically, we require an equivalence between predicates and functions in the knowledge base and word senses in the language. The word senses correspond 1-1 to the predicates and the arguments to these predicates correspond to the linguistic arguments that the word senses may take. This will allow us to introduce new predicates into the knowledge base in a systematic and straightforward way based on the words used. The representation of events in AF satisfies this constraint for verbs—the event predicates correspond to verb senses, and the reified events allow the argument functions that correspond directly to a verb’s semantic roles. Beyond events though, we need some extensions. First, if we are to maintain the close link between linguistic structure and the representation, we need to reify predicates (e.g., adjective meanings) so that they may serve as arguments to other predicates. To distinguish such predicates from the formal predicates in the logic we will call them property predicates (see Table 1). Intuitively, property predicates identify characteristics of the world that can be directly perceived in a moment of time (e.g., at the present moment). For instance, consider the sentence John’s mood changed from happy to sad. There are three arguments to the event predicate CHANGE: the object undergoing the change (John’s mood), the prior state (happy) and the resulting state (sad). By reifying property predicates, we can express this as: CHANGE(mood(john), Happy, Sad, t, e). While this is a natural mapping of the sentence meaning, such statements cannot be made in classical first order logic because predicates, such as Happy, cannot serve as arguments to other predicates. While there might be some technical tricks to try to avoid such a generalization of the formalism, we will soon see additional reasons for why the reified predicates are convenient for capturing commonsense knowledge, particularly when representing scales. We need one more significant change from AF to allow us to stay true to the structures of language. Consider one definition of change in WordNet: Become different. The meaning of this expression is that an object that changes becomes different from what it was before. We will spend some time defining exactly what this means, but for now just consider that the predicate Different needs to apply to the same object twice, but at different times. One cannot express such a relation if we can only associate times with predicates or properties, as in AF. Rather, we need a more general logic where terms, rather than the predicates, are temporally qualified. Specifically, we introduce a new function that takes the name of an object and a time and denotes that object over that time, e.g., x@t represents “object x over time t”. We refer to these as temporally situated objects. Thus, John is Happy today is written as TRUEOF(john@today, Happy) We need another predicate for binary relations. For example, we would express I am different today from yesterday as: TRUEOF2(me@yesterday, me@today, Different) Such a proposition cannot be directly expressed using a logic that only attaches time to the predicates. The notion that objects are temporally situated and properties are not is in stark contrast to standard temporal logics in which objects are atemporal and properties change over time. This view has been discussed in philosophy, going back to before Whitehead (1929). We introduce a predicate EXISTS that defines the temporal range of an object, i.e., when the temporally situated object o@t exists. For instance, if I was born in 1983, then EXISTS(me@1984) and ~EXISTS(me@1982) both hold. Properties only hold on temporally situated objects that exist: Construct Formal Status Linguistic correlate Notation and Example(s) Formal Predicates Predicates in the logic none Small caps TRUEOF Event predicates Predicates in the logic verb senses Small caps CHANGE Property predicates Terms that denote properties nouns and adjective senses Initial caps Happy, Dog, . Property functions Functions that apply to property predicates comparatives, nominalizations