Compositional Semantics and Inference System for Temporal Order based on J apanese CCG (2022)

Abstract: Natural Language Inference (NLI) is the task of determining whether a premise entails a hypothesis. NLI with temporal order is a challenging task because tense and aspect are complex linguistic phenomena involving interactions with temporal adverbs and temporal connectives. To tackle this, temporal and aspectual inference has been analyzed in various ways in the field of formal semantics. However, a Japanese NLI system for temporal order based on the analysis of formal semantics has not been sufficiently developed. We present a logic-based NLI system that considers temporal order in Japanese based on compositional semantics via Combinatory Categorial Grammar (CCG) syntactic analysis. Our system performs inference involving temporal order by using axioms for temporal relations and automated theorem provers. We evaluate our system by experimenting with Japanese NLI datasets that involve temporal order. We show that our system outperforms previous logic-based systems as well as current deep learning-based models.

## introduction natural language inference (nli) is the task of determining whether a premise entails a hypothesis. in particular, nli involving temporal expressions is crucial. (1) is an example of english nli involving temporal expressions. h: i arrived before may 2021. (entailment) the inference example with temporal expressions is challenging. this is because we need to represent the meaning of sentences that contain temporal adverbs like before and in, temporal expressions like april 2021, and verb tenses like arrived, and to compute temporal order of events written in the sentences. @xcite showed that deep learningbased models @xcite trained on a standard nli dataset such as multi-genre natural language inference (multinli; @xcite ) failed to perform simple temporal inference as in (1). furthermore, deep learning-based models have performed poorly on challenging nli datasets that involve various temporal inferences such as fracas @xcite for english and jsem @xcite for japanese. recently, logical inference systems based on compositional semantics @xcite @xcite @xcite @xcite (i.e., semantics in which the meaning of a phrase is determined compositionally from the syntax and the meaning of the lexicon contained in the phrase) achieved high accuracy in fracas and jsem. however, most previous systems did not cover temporal inference. in addition, because most previous research on nli has focused on english, research on other languages is desirable. in particular, research on nli in japanese is still in its infancy and is limited to deep learning-based systems using pre-trained language models and a few logical inference systems @xcite . @xcite attempted to implement a japanese logical inference system for temporal inference. however, the focus of this previous research was limited to a few temporal clauses in japanese, and temporal adverbs are out of scope. thus, there is still room for improvement in the accuracy of temporal inference in japanese. in this study, our aim is to realize the compositional semantics and a logical inference system for temporal inference in japanese based on combinatory categorial grammar (ccg) @xcite to derive a transparent syntaxsemantics interface and the analysis of tense and aspect studied in formal semantics @xcite @xcite @xcite . we focus on temporal order and develop a japanese logical inference system for temporal order. in our system, a ccg parser first parses the premise and hypothesis sentences and converts them into ccg trees. based on the analysis of the compositional semantics, we then modify the obtained ccg trees. next, using ccg2lambda @xcite , the meaning of the whole sentence is derived as a logical form. finally, we attempt to prove the entailment relations between the obtained logical forms by an automated theorem prover vampire @xcite . we experiment with two nli datasets involving temporal order in japanese: jsem and a japanese translation of the nli dataset focusing on temporal inference @xcite . we compare our system with the previous japanese logical inference system @xcite and the japanese bert model @xcite . our experiments show that our system outperforms previous logical inference systems as well as current deep learning-based models. our system will be available for research use at https: //github.com/ynklab/ccgtemp . ## background tense and aspect are important linguistic phenomena related to temporal expressions. this section provides standard background on the semantics of temporal expressions in japanese, which have been analyzed in previous studies @xcite @xcite @xcite . in japanese, verb tense is classified into past (ta) and non-past (-ru), and aspect is classified into stative (like iru) and non-stative (like kuru). the temporal interpretation of a matrix clause (i.e., a clause that contains a subordinate clause) is determined by the combination of tense and aspect, and is expressed by the constraints imposed on the relation between speech time and reference time. speech time represents the time that a sentence is uttered, and reference time is a concept proposed by @xcite and refers to the time used with location time (i.e., time when an event occurs) and speech time to represent the meaning of tense. past stative relation example + + @xmath0: constraints imposed on the relation between speech time s and reference time r by tense and aspect and example sentences to analyze the temporal interpretation of embedded clauses, the concepts of absolute tense and relative tense are necessary. absolute tense means that the temporal interpretation is determined by the relation between the speech time and the reference time, as in the matrix clause. however, relative tense means an interpretation in which the temporal interpretation does not depend on the relation between the speech time and the reference time. we explain the details with examples in section 3.2. this paper uses ccg to formalize the syntactic analysis of our method and analyzes the compositional semantics of temporal expressions based on the analysis by @xcite . ## system overview figure 7 shows the pipeline of our system. our system consists of three main steps. first, natural language sentences of premises and hypotheses are converted into modified ccg trees by ccg parsing and modifying trees. next, a meaning from the semantic templates is assigned to each lexical item. the semantics in lexical items are then composed by ccg2lambda to derive a logical formula that represents the meaning of the whole sentence. finally, an automated theorem prover determines whether the logical formula of the hypothesis is provable from the logical formula of the premises. in this section, we describe each of these steps. ## conclusion in this paper, we compositionally derived semantic representations of sentences with tense and aspect in japanese based on ccg. we developed a logicbased nli system that considers temporal order in japanese. we evaluated our system by experimenting with two japanese nli datasets involving temporal order. our system performed more robustly than previous logic-based systems as well as current deep learning-based models. the experimental results of our system suggest that a logical nli system based on an analysis of tense in formal semantics is effective for temporal inference. other previous studies of logic-based methods have shown the effectiveness of nli systems based on the analysis of various semantics such as degree semantics @xcite . by combining them, we will be able to construct one nli system capable of performing a variety of inferences. in the future, we plan to cover various temporal inferences involving comparative deletion and temporal anaphora. furthermore, we plan to construct inference test sets for these challenging inferences.