To Schedule or not to Schedule: Extracting Task Specific Temporal Entities and Associated Negation Constraints (2020)

Abstract: State of the art research for date-time entity extraction from text is task agnostic. Consequently, while the methods proposed in literature perform well for generic date-time extraction from texts, they don’t fare as well on task specific date-time entity extraction where only a subset of the date-time entities present in the text are pertinent to solving the task. Furthermore, some tasks require identifying negation constraints associated with the date-time entities to correctly reason over time. We showcase a novel model for extracting task-specific date-time entities along with their negation constraints. We show the efficacy of our method on the task of date-time understanding in the context of scheduling meetings for an email-based digital AI scheduling assistant. Our method achieves an absolute gain of 19% f-score points compared to baseline methods in detecting the date-time entities relevant to scheduling meetings and a 4% improvement over baseline methods for detecting negation constraints over date-time entities.

## introduction temporal entity extraction and normalization is an important aspect of natural language processing @xcite . there has been a substantial body of work on the task and there exist numerous well performing publicly available models for identifying and normalizing temporal entities @xcite @xcite . there exist however a growing number of nlp applications which require extraction of only a relevant subset of time entities that are useful for solving specific problems within a larger body of text. examples of such tasks include understanding search queries ("find me all emails sent by april between may 11 th and may 21 st "), goal oriented dialogue systems ("deliver george orwell's 1984 by next week.", "send the "fy 2020 budget" to watson monday morning.") etc. using the temporal entity extraction models for these tasks is insufficient, since they fail to disambiguate between general date-time entities and entities necessary to solve the task. in this paper, we address the task of recognizing date-time entities required by an ai scheduling assistant for correctly scheduling meetings. cortana from microsoft scheduler, clara from clara labs and amy from x.ai are examples of such email based digital assistants for scheduling meetings. for such systems, a user organizing the meeting adds the digital assistant as a recipient in an email with other attendees and delegates the task of scheduling to the digital assistant in natural language. for the assistant to correctly schedule the meeting, it must correctly extract the date-time entities expressed by the user in the email to indicate the times they want the meeting scheduled, as well as the times that do not work for them. the verbose nature of emails often exacerbates the difficulty of identifying relevant date-time entities; since the number of distractor (i.e valid date-time entities not pertinent to the task) tend to increase (eg: in fig. 1 "today" serves as a distractor entity). to this end, we present sherlock: scheduling entity recovery by looking at contextual knowledge, a novel model for detecting relevant date-time entities in the context of scheduling as well as identifying the entities associated with a negation constraint. sherlock comprises of 3 modules for identifying the relevant entities as well as negation constraints associated with them: the neural module then takes the email and the entities and generates scores for each entity. only the relevant entities are passed to the final negation module to detect times to schedule and times to avoid. ## contextual date-time extraction in order to correctly extract relevant temporal entities in the context of scheduling meetings from an email, we first extract potential entities using an off-the-shelf date-time entity extractor. both the email and the extracted entities are then encoded using neural modules. for each extracted entity, we then generate a context embedding using the encoded email and the encoded entity. both the contextual and encoded entity embedding are then used to predict if an entity is relevant or not. we describe each component in detail below: ## identifying negation constraints for a scheduling assistant to be able to correctly schedule meetings, understanding negations is crucial; otherwise it can lead to an unsatisfactory user experience (e.g.: in figure 1 , the meeting being scheduled on wednesday would be a frustrating experience for the organizer sherlock). only about 10% of scheduling requests in our dataset have negation constraints. building a model directly for the task did not show promising results from our preliminary experiments. we hypothesize this was due to the small volume of the data as well as the lack of good quality supervised data. consequently, to find negated time-entities, we adopt the approach of first finding the negation scope. if an entity occurs inside the negation scope, we mark it to be negated. in order to find the negation scope, we build on the approach proposed in @xcite . we first find the negation cue ("except" in figure 1 ). to find the negation cue, we first tokenize the email into sentences. for each sentence, we try fo find if cue from a set of negating cues (appendix a) occurs in the sentence. after finding the negation cue, we identify the pos tag of the negating cue (prep. for "except"). given the pos tag and the negation cue, we trigger a set of heuristics to identify the negation scope. most heuristics work by identifying the negation cue from the dependency parse of the sentence as well as the governor of the negating word. generating the narrow scope of negation (i.e. not containing the subject) then involves identifying the constituent from the constituency parse that contains both the negation cue and the governor word ("any day except wednesday", see figure 2 ). this constituent is considered to be the candidate narrow scope, and usually, the part following the cue is considered to be the narrow scope. for some cases, the narrow negation scope is not enough to identify the time entity being negated. consider the second example from figure 2: example: next week does not work watson. narrow: next week does not [work watson]. wide: [next week] [does] not [work watson]. for this case, the narrow scope is not enough to identify the entity being negated ("next week"). to find the wide scope, the heuristics leveraging the dependency path starting from the governor word are used. the main idea is to find the subject associated with the governor node, and extract that as the wide scope ("next week"). following the guidelines set by @xcite , we also include the aux dependency node in the wide scope ("does"). we also expand the heuristic set presented in @xcite , adding the following rules: due to space constraints, we include examples for the above in appendix b. after obtaining the narrow and wide scopes, we check if any entities are found in the narrow scope. if found, those entities are scored negated. if no entities are found in the narrow scope, we then check the wide scope to find negated entities. finally, for some cases, we also use domain specific cues that imply a non-availability (for example, in "dr. john out of office on monday.", "out of office" implies an unavailability to meet.) when such implied negation cues are encountered, we default to a custom heuristic which marks any entity occurring within the sentence containing the cue word as a negation. ## experimental setup we first show the effectiveness of our proposed entity scoring method of incorporating context for improving temporal entity extraction on the tempeval-3 dataset (uzzaman et al., 2013) ( §4.1). we then show the efficacy of sherlock for the task of extracting the correct temporal entities relevant for the context of scheduling, a task for which context becomes substantially more important ( §4.2). finally, we show that sherlock's negation module outperforms baseline methods on the task of identifying the entities with negation constraints ( §4.3). all our models have been implemented using the allennlp framework @xcite . the hyperparameters for all the experiments can be found in appendix c and d. ## conclusion in this paper, we presented a novel model that leverages conventional high recall rule-based models and neural models for utilizing contextual informa-tion for identifying task relevant temporal entities. our proposed model, when used in conjunction with 3 different rule-based models, achieves substantial precision gains for all of them without suffering from a huge recall drop. further, the model substantially outperforms baseline methods for the task of identifying relevant date-time entities for the task of scheduling a meeting. we also presented a novel approach for identifying the negation constraints of date-time entities. identifying the negation constraints associated with date-time entities correctly is necessary for the task of scheduling. we showed that the existing neural approaches for detecting negation scopes do not transfer well, and that our proposed model based on heuristics defined over constituency and dependency parses achieves strong performance gains, especially for the case of explicit negations.