Embedding Time Differences in Context-sensitive Neural Networks for Learning Time to Event (2021)

Abstract: We propose an effective context-sensitive neural model for time to event (TTE) prediction task, which aims to predict the amount of time to/from the occurrence of given events in streaming content. We investigate this problem in the context of a multi-task learning framework, which we enrich with time difference embeddings. In addition, we develop a multi-genre dataset of English events about soccer competitions and academy awards ceremonies, and their relevant tweets obtained from Twitter. Our model is 1.4 and 3.3 hours more accurate than the current state-of-the-art model in estimating TTE on English and Dutch tweets respectively. We examine different aspects of our model to illustrate its source of improvement.

## introduction the task of time to event (tte) prediction aims to determine the amount of time to/from the occurrence of a well-defined event. accurate prediction of this information is important for temporal tasks such as timeline generation @xcite , news summarization @xcite , and disease onset prediction in medical domain @xcite . current approaches mainly focus on news articles and expect at least one temporal expressions in each input data to predict tte @xcite @xcite @xcite . these approaches cannot be readily applied to streaming content (such as twitter data) because such data often do not carry any temporal expressions. figure 1 show * first and second authors equally contributed to this work. 1 our code and data are available at https://github . com/hajipoor/time2event looking forward to the clash between liverpool and arsenal ...must be fun to watch well done the gunners it's always nice to start with a win #livars liverpool vs arsenal +8 hours -2 hours time 2017-08-27 16:00 2017-08-27 8:00 2017-08-27 18:00 figure 1 : examples of tweets that don't carry any explicit time expression but indicate a future or past event due to the implicit temporal connotation in "looking forward to," "must be fun to watch," "well done" etc. the contributions of this paper are as follows: (a) an effective multi-task and context-sensitive framework that uses temporally-rich context and time difference embeddings to accurately predict tte in streaming content, (b) publicizing a time to event dataset that includes different genres of events (soccer competitions and academy awards ceremonies), their time of occurrence, their relevant tweets as well as tte information for each tweet. our framework is 1.4 and 3.3 hours more accurate than the current state-of-the-art model in estimating tte on large-scale english and dutch tweets respectively. in addition, our time difference model achieves an accuracy of 98.3% in terms of creating embeddings that encode temporal differences between given time pairs. ## context-sensitive model existing models often assume input data carry explicit temporal information about target events. although informative, these information may not be available in most textual content, especially in microblogs. we propose to utilize context information (in the form of neighboring tweets) and the relative temporal differences against neighbours to estimate time to event (tte) for given input texts. in particular, given a tweet about an event, we propose a multi-task learning framework to predict the absolute value of tte (in hours) for the tweet, as well as a binary sign which determines if the tweet has been posted before '(+),' or at the same time or after '(-)' the event. figure 2(a) shows our model for predicting tte for the target tweet t i , given its context tweet 3 , e.g. a previously posted tweet about the same event, t j , @xmath0, and their time difference embeddings, which encode the time differences between tweet creation times. our intuition for developing such embeddings is that if 3 neighboring or context tweets are randomly sampled from the set of previously posted tweets relevant to the target event. our model can be extended to greater context sizes. our model takes as input the concatenation of attention-weighted average embeddings of the target and context tweets (a and a � in figure 2 ) and their time difference embedding (d) (see section 2.1). the resulting concatenation are then used to predict tte sign and tte value for the target input. tte value is a regression task while tte sign is a binary classification task. to prevent the loss with larger gradient magnitudes dominate the training, we establish a common scale for the different loss magnitudes across the two tasks using the approach proposed in @xcite , which simultaneously learns classification and regression losses of varying quantities and combines them using homoscedastic uncertainty. ## conclusion and future work we developed a context-sensitive neural model that used rich-neighbouring tweets as well as time difference embeddings between target tweets and their neighbors for effective prediction of time to event. we evaluated our and current models on events and tweets of different genres (soccer competitions and academy award ceremonies) and languages (english and dutch). future works include expansion to temporal tasks that particularly focus on early prediction of time to events. in addition, it's worth investigating if user or social network information could be helpful for better time to event prediction.