Principles for Developing a Knowledge Graph of Interlinked Events from News Headlines on Twitter	The ever-growing datasets published on Linked Open Data mainly contain encyclopedic information. However, there is a lack of quality structured and semantically annotated datasets extracted from unstructured real-time sources. In this paper, we present principles for developing a knowledge graph of interlinked events using the case study of news headlines published on Twitter which is a real-time and eventful source of fresh information. We represent the essential pipeline containing the required tasks ranging from choosing background data model, event annotation (i.e., event recognition and classification), entity annotation and eventually interlinking events. The state-of-the-art is limited to domain-specific scenarios for recognizing and classifying events, whereas this paper plays the role of a domain-agnostic road-map for developing a knowledge graph of interlinked events.

Notation and Problem Statement	A tweet of a news headline contains a sequence of words INLINEFORM0 . tab:tweetsamples provides samples of news headlines on Twitter with provenance information such as publisher and publishing date. These were sampled for the type of embedded event discussed below. We aim to create an RDF knowledge base for such news headlines. An RDF knowledge base INLINEFORM1 consists of a set of triples INLINEFORM2 , where INLINEFORM3 is the union of all RDF resources ( INLINEFORM4 are respectively a set of classes, properties and instances), and INLINEFORM5 is a set of literals ( INLINEFORM6 ). We aim to extract rich set of triples INLINEFORM7 from each tweet INLINEFORM8 in the stream of news headline tweets (as discussed below), and populate an event knowledge graph INLINEFORM9 . Formally, the extraction task can be captured as INLINEFORM10 where INLINEFORM11 is the stream of news headline tweets and INLINEFORM12 is a knowledge graph of events (where a tweet INLINEFORM13 is mapped to a single event).

In contrast to NELL, it initially identifies Named Entities and their type on plain text, then it tries to infer mentions of relation expression to properties in DBpedia (e.g. taking the domain and range of properties into account). Open Information Extraction  is another extraction framework that is not limited to any predefined relation set. Furthermore, extracting triples from structured as well as semi-structured data sources has received adequate attention in the past, especially, DBpedia  and LinkedGeo Data  that leverage the loose structure of data for extraction. Another example is the work  which presents a holistic approach for extraction of RDF from templated websites.

Interlinking Events	The tasks described above have been considered independently before. The interlinking requirement, which has not hyet been adequately explored, comes from the two inherent facts of events as follows: A single event might be reported by various publisher sources using different expressions. Thus, it is necessary to identify same events across various publisher sources, and then interlink them using owl:sameAs or skos:related links. Events have an evolutionary nature in the sense that more information is added with time. Thus, it is essential to spot an event and its subsequent events reported to either complement the original event or reflect its causes or consequences. To interlink such events, skos:related can be utilized. The recognized events, entities and relations have to be published according to principles of LOD, RDF and the employed background data model. To maintain the knowledge graph's consistency and coherence, the generated triples must be de-duplicated, validated and assigned URIs disambiguated. The minted URI should be dereferenceable and interlinked to external RDF data sources.

For example,  initially recognizes named entities in a given sentence and then, by inferring over domains and ranges of properties in DBpedia, assigns an appropriate property between the recognized entities. Given an entity (e.g. Garry Marshall) with type director associated with a known movie (e.g. Pretty woman), it infers the property dbpedia:director from background ontology between the two recognized entities Garry Marshall and Pretty woman. So far, supervised and unsupervised learning approaches have been applied for these extractions, which rely on the use of a large number of specific lexical, syntactical and semantic features. We assume that each news headline maps to an event modeled by an n-ary relation that can be captured by generating multiple triples. An INLINEFORM9 -ary relation is a relation with n arguments INLINEFORM10 . For example, a binary relation triple INLINEFORM11 can be rewritten as INLINEFORM12 . Thus, the first challenge concerns the suitable background data model for representing various types of events and their associated entities by simulating n-ary relationships in terms of binary relationships.

Introduction	Several successful efforts have led to publishing huge RDF (Resource Description Framework) datasets on Linked Open Data (LOD) such as DBpedia  and LinkedGeoData  . However, these sources are limited to either structured or semi-structured data. So far, a significant portion of the Web content consists of textual data from social network feeds, blogs, news, logs, etc. Although the Natural Language Processing (NLP) community has developed approaches to extract essential information from plain text (e.g.,  ,  ,  ), there is convenient support for knowledge graph construction. Further, several lexical analysis based approaches extract only a limited form of metadata that is inadequate for supporting applications such as question answering systems. For example, the query “Give me the list of reported events by BBC and CNN about the number of killed people in Yemen in the last four days”, about a recent event (containing restrictions such as location and time) poses several challenges to the current state of Linked Data and relevant information extraction techniques.

Related Work	Overall, there is a lack of a holistic view on event extraction from free text and subsequently developing a knowledge graph from it. In this paper, we presented the full pipeline containing the required tasks such as (i) agreeing upon a data model, (ii) event annotation, (iii) entity annotation and (iv) interlinking events. The majority of previous research is either domain-specific or event-specific and do not undertake the full pipeline (e.g., limited to only event and entity extraction). We have provided a visionary review of the full pipeline which is merely applicable to any domain. In the following, we initially refer to research approaches for n-ary relation extraction on particular domains, then we refer the prominent approaches of binary relation extraction. We end by citing successful attempts at triple extraction from structured and semi-structured data sources. The work presented in  introduces complex relations as n-ary relations between n-typed entities. It proposes to factorize all complex relations into a set of binary relations. Then, a classifier is trained to recognize related entities of binary relations. After identifying all pairs of related entities for binary relations, it reconstructs the complex relation using a simple graph creation approach.

Conclusion and Future Work	In this paper, we presented the initial version of our framework for the real-time extraction of events. This framework is part of our project HeadEx for developing a knowledge graph of interlinked events. We presented the requirements for choosing a data model representing events and their arguments. We reviewed the existing data models which have been employed by the state-of-the-art applications. Furthermore, we outlined the required tasks for annotating events as well entities. Then, the interlinking strategies were discussed. As a proof-of-concept, we followed a case study of news headlines on Twitter. For our future agenda, we plan to develop the envisioned pipeline containing all the required tasks by either implementing new components or integrating the existing ones.

Considering our case study, news headlines are often one single sentence (potentially accompanied by subordinate clauses) along with a link directing to the body of the news report. In spite of its brevity, headline tweets provide dense and significant information. Various entities appear in the embedded core message (the latter commonly as verb phrase), including aspects that indicate temporal properties, location and agent. For example, consider the tweet no.2 in tab:tweetsamples that will serve as a running example: Instagram CEO meets with @Pontifex to discuss "the power of images to unite people" that contains several entities related to the verb phrase `meet' and are distinguished by separating boxes as [baseline=(X.base)] X) [draw, shape=rectangle, inner sep=0] Instagram CEO; [baseline=(X.base)] X) [draw, shape=rectangle, inner sep=0] meets with; [baseline=(X.base)] X) [draw, shape=rectangle, inner sep=0] @Pontifex; [baseline=(X.base)] X) [draw, shape=rectangle, inner sep=0] to discuss "the power of images to unite people";.

Using Singleton Property	We can adopting the concept of a singleton property introduced in  for modeling n-ary relations in the background data model. Singleton properties replace RDF reifications and enable efficient represention of statements about statements. Since news headlines contain both provenance information and multiple associated entities, SP is a suitable choice and furthermore, it enable systematic encoding of n-ary relations in terms of binary relations. Example 1 (Input/Output) Considering our running example which is about the occurrence of a meet event with two participant entities Instagram CEO and Pontifex and the topic INLINEFORM0 . The generated triples using singleton property are as follows: 1. :Meet#1 singletonPropertyOf :Meet. 2. :Instagram_CEO :Meet#1 :Pontifex. 3. :Meet#1 :about :t1. 4. :Meet#1 :hasSource :CNN. 5. :Meet#1 :extractedOn `26/2/2106'. 6. :t1 a :Topic. 7. :t1 :body `to discuss the power of images to unite people'.

The query seeks “fresh” information (e.g., last four days) whereas the current version of Linked Data is encyclopedic and historical, and does not contain appropriate information present in a temporally annotated data stream. Further, the query specifies provenance (e.g., published by BBC and CNN) that might not always be available on Linked Data. Crucially, the example query asks about a specific type of event (i.e., reports of war caused killing people) with multiple arguments (e.g., in this case, location argument occurred in Yemen). In spite of recent progress  ,  ,  , there is still no standardized mechanism for (i) selecting background data model, (ii) recognizing and classifying specific event types, (iii) identifying and labeling associated arguments (i.e., entities as well as relations), (iv) interlinking events, and (v) representing events. In fact, most of the state-of-the-art solutions are ad hoc and limited. In this paper, we provide a systematic pipeline for developing knowledge graph of interlinked events. As a proof-of-concept, we show a case study of headline news on Twitter.

The general intuition is that a core verb (i.e., relation) heads each headline tweet accompanied by multiple arguments (i.e., entities). The number of entities INLINEFORM0 depends on the type of relation but location and time are generic default arguments for any relation INLINEFORM1 . Thus, the core chunk (verb phrase) corresponds to the meet event and the remaining chunks of the given tweet likely function as dependent entities of this event. For instance, in the running example, the chunk [baseline=(X.base)] X) [draw, shape=rectangle, inner sep=0] meets; corresponds to the event INLINEFORM2 with the following recognized entities as associated arguments: DISPLAYFORM0  In this example, the temporal, as well as location arguments of INLINEFORM0 , are absent. Consistent with linguistic theory, not all arguments are always present for each occurrence of an event. The RDF and OWL (Web Ontology language) primarily allow binary relations, defined as a link between either two entities or an entity and its associated property value. However, in the domain of news, we often encounter events that involve more than two entities, and hence require n-ary relations.

Another domain for extracting n-ary relations is protein-protein interactions in the biomedical literature  ,  ,  . They first identify protein mentions in text and then recognize interaction relations before finally extracting interactions. The approaches employed for protein-protein interactions can be divided into three groups: (i) graph-based approaches (e.g. co-occurrence graph), (ii) rule-based approaches and (iii) learning approaches (e.g. maximum entropy). The other category of event extraction is based on binary relation extraction. NELL: Never-Ending Language Learning  is a learning agent that extracts new facts using existing binary relations in its knowledge base. It was initiated in 2010 using a couple of seed binary relations but after years of running has become self-learning. A notable feature of NELL is its dynamic approach for extracting facts, as it refreshes beliefs in its knowledge base and removes the incorrect or old ones. Linked Open Data as a valuable source of diverse ontologies also can be employed for extracting either new facts or new relations. The framework proposed in  ,  extracts facts using binary relations from DBpedia as background knowledge.

Outline of The Required Steps	Here, we outline the required steps for developing a knowledge graph of interlinked events. Figure FIGREF2 illustrates the high-level overview of the full pipeline. This pipeline contains the following main steps, to be discussed in detail later. (1) Collecting tweets from the stream of several news channels such as BBC and CNN on Twitter. (2) Agreeing upon background data model. (3) Event annotation potentially contains two subtasks (i) event recognition and (ii) event classification. (4) Entity/relation annotation possibly comprises a series of tasks as (i) entity recognition, (ii) entity linking, (iii) entity disambiguation, (iv) semantic role labeling of entities and (v) inferring implicit entities. (5) Interlinking events across time and media. (6) Publishing event knowledge graph based on the best practices of Linked Open Data.

All these can be imported from other datasets present on the Web as DBpedia links to other datasets in an easy manner. Similar to LODE ontology and SEM model, DBpedia fails to meet Requirement 2 as it does not include the publisher of events (provenance). Schema.org, a product of collaborative efforts by major companies (i.e., Google, Bing, Yahoo and Yandex) , presents similar generic concept of event. It considers temporal as well as location aspects and additionally provides a limited hierarchy. This hierarchy introduces types of events such as business events, sale events, and social events. The schemas in schema.org are set of these types which are associted with a set of properties. Furthermore, it considers multiple labels between the associated entity and the concept of the event (represented in fig:schema.org) such as actor and contributor, which distinguishes the role of the associated entity. Schema.org introduces hundreds of schemas for categories like movies, music, organizations, TV shows, products, places etc . For Schema.org, an event is an instance taking place at a certain time and at a certain location.

A factual aspect is given by 'What happened' (event), 'Where did it happen' (atPlace), 'When did it happen' (atTime), 'Who was involved' (involvedAgent)  . A visualization of LODE ontology is shown in Figure 2. We conclude that LODE meets (i) Requirement 1 as it defines a generic concept of the historic event, (ii) loosely Requirement 3 as it contains generic types for entities, e.g., Agent, SpatialThing, TemporalEntity, (iii) Requirement 4 as it includes necessary relations. But LODE ontology fails to meet Requirement 2 as it does not include the publisher of the event (provenance). Figure 3 depicts our running example in LODE. In 2011, SEM ontology was introduced from Vrije University and Delft. This ontology describes events as the central element in representing historical data, cultural heritage   and multimedia  . SEM is combined with a Prolog API to create event instances without the background knowledge. This API also helps in connecting the created event instances to Linked Open Data. SEM proposes a method to attain interoperability among datasets from different domains.

It organizes classes into semantically coherent event classes and event hierarchy, and notably, has an inventory of the corresponding lexical items. For example, tab:threeVerbClasses in the first column presents three event classes as (i) Communication event that corresponds to the event which causes transferring a message, (ii) Meet event which is an event related to group activities, and (iii) Murder event which is referring to an event that describing killing. The second column of tab:threeVerbClasses represents the lexical items (i.e., verbs) having shared meaning and are under the umbrella of a common event. In other words, an appearance of one of these verbs shows the occurrence of its associated event. For example, w.r.t. the running example, the appearance of the verb meet in the given tweet shows the occurrence of an event with the specific type `meet'. The CEVO ontology can be employed for recognizing events and more interesting classifying them w.r.t. their specific type. Specifically, it unifies apparently disparate lexical items under a single event class. More importantly, this can prove critical in reducing the number of apparent features for classifiers and in the support of inference necessary for query response.

Entity Annotation	Entity annotation is a significant task for creating a knowledge graph of events. It can be challenging when we have a fine-grained background data model, which makes the task of semantic role labeling of entities necessary. Overall, the required tasks for fulfilling entity annotation are as follows: Entity recognition: This task specifies a chunk of text as an individual entity which plays a role in the occurred event. An entity mention can be explicit or implicit. Regarding explicit entities, Named Entity Recognition (NER) tools can be used for open domain scenarios whereas alternatives such as knowledge graphs, gazetteers, and domain dictionaries are necessary for closed domain scenarios. E.g., for the tweet no.1 in tab:tweetsamples, the chunk `Michelle Obama' is recognized as a named entity with the type person. Entity linking: Entity linking can be attributed into two tasks, the first one  , which is required in our case, is about associating entity mentions in a given text to their appropriate corresponding entities in a given knowledge graph. Thus, it removes ambiguity. A textual mention of an entity might have a matching entity in the knowledge graph or not.

Using Existing Data Models	In this part, we review a number of state-of-the-art event ontologies. In 2009 UC Berkeley introduced the LODE ontology. In this ontology, an event is defined as an action which takes place at a certain time at a specific location. It can be a historical action as well as a scheduled action. There were previous models   for representing historic events and scheduled events. Some of them represent both types of events (i.e., historical and scheduled), e.g., EventsML-G2. The LODE ontology proposed to build an interlingua model, i.e., a model which encapsulates the overlap among different ontologies e.g., CIDOC CRM, ABC Ontology, Event Ontology, and EventsML-G2. This encapsulation is utilized to create a mapping among existing ontologies. LODE was introduced to publish historical events in a fine-grained manner as it assumes each event is a unique event even if it is a part of a series. Because the concept of sub-events does not exist in LODE, related events can be interlinked. This ontology helps us to link factual aspects of a historical event.

Developing a Data Model	The existing data models are basically coarse-grained. In case the domain or application requires a fine-grained data model, the existing data models can be extended. For example, here we extended event data model from CEVO ontology for three specific events. We take into account three subclasses (shown in Figure UID50 ) as (i) class communication INLINEFORM0 that refers to any event transferring a message, (ii) class meet INLINEFORM1 that ranges over all group activities, and finally, (iii) class murder INLINEFORM2 that includes any reports of killing. Furthermore, as Figure UID50 shows, the provenance information (e.g., publisher or date) is represented within the data model (default arguments for all events), to meet Requirement req:prov. Figure FIGREF49 (b-d) represents parts of data model for sub-event classes (i.e., INLINEFORM0 ) in detail. The type of all possible associated entities as well as their necessary relationships are represented within the data model. This meets the Requirements SECREF22 and SECREF23 . For example, the meet event is associated with entities with type of Participant and Topic (i.e., topic discussed in the meeting).

SEM strives to remove constraints to make it reusable by supporting weak semantics. Thus, in SEM, the concept of event is specified as everything that happens  . A schematic representation of SEM model is shown in fig:sem (summarized version). We conclude that SEM meets (i) Requirement 1 as it defines generic event, (ii) Requirement 3 as it specifies a type for entities, e.g., Actor, and (iii) Requirement 4 as it includes required properties. Similar to LODE ontology, SEM model fails to meet Requirement 2 as it does not include the publisher of events (provenance). Fig The DBpedia ontology defines the generic concept of event with a hierarchy which is broader, including lifecycle events (e.g. birth, death), natural events (e.g. earthquake, stormsurge), and societal events (e.g. concert, election). We conclude that DBpedia meets (i) Requirement 1 as it defines generic event, (ii) Requirement 3 as it specifies a type for entities, and (iii) Requirement 4 as it includes required properties.

Background Data Model	An initial key question is “What is the suitable background data model (serving as the pivot) for extracting triples associated to an event?” Contemporary approaches to extracting RDF triples capture entities and relations in terms of binary relations  ,  ,  . We divide the current triple-based extraction approaches into two categories: (i) those that (e.g.,  ) follow the pattern INLINEFORM0 to leverage existing relations (i.e., properties) INLINEFORM1 in the knowledge base to find the entities INLINEFORM2 and INLINEFORM3 for which the relation INLINEFORM4 holds. For example, for the relation plays holds between an athlete and his/her favorite sport, and NELL extracts the triple seve ballesteros plays golf for two entities seve ballesteros and golf, and (ii) others that (e.g.,  ,  ) utilize the pattern INLINEFORM5 to leverage the entities available in the knowledge graph (i.e., INLINEFORM6 ) to infer new relations (e.g., INLINEFORM7 ) that either did not exist in the knowledge base or did not hold between the entities INLINEFORM8 .

Like LODE, the repeated events are classified different events and thus keeping all the events unique even if it is a sub event. A schematic representation of Schema.org (summarized version) is shown in fig:schema.org. We conclude that Schema.org meets (i) Requirement 1 as it defines generic event, (ii) Requirement 3 as it specifies a type for entities e.g Actor (as type Person), Location (as type Place), Organizer (as type Person), StartDate (as type Date or DateTime) etc.. and (iii) Requirement 4 as it includes required properties for every entities defined above. Like LODE, SEM and DBPedia, Schema.org also fails in meeting Requirement 2 as it can define or import publisher of the event (provenance). The CEVO ontology relies on an abstract conceptualization of English verbs provided by Beth Levin  . Levin categorizes English verbs according to shared meaning and behavior. CEVO ontology, which is a machine-readable format (i.e., RDF format) of Levin 's categorization, presents more than 230 event classes for over 3,000 English verbs individuals.

The W3C Working group Note suggests two patterns for dealing with n-ary relations. We prefer the first pattern that creates INLINEFORM0 classes and INLINEFORM1 new properties to represent an n-ary relation. We formally define a generic event class representing all categories of events (n-ary relations) and then, use a template-based definition for any subclass of the generic event. This enables the representation of specific types of events (e.g. meet event). Definition 1 (Class of Generic Event) A generic event class refers to any event that can involve n multiple entities. In other words, the Generic Event Class denoted by INLINEFORM0 abstracts a relation among n entities. Definition 2 (Class of `X' Event) `X' Event denoted by INLINEFORM0 is a subclass (i.e. specific type) of the class INLINEFORM1 , i.e., INLINEFORM2 . Conceptually it refers to events sharing common behavior, semantics, and consequences. In the following, we provide requirements on the data model for developing a knowledge graph of interlinked events.

Requirement 1 (Inclusion of Generic Event) An event data model minimally includes the definition of the generic event while including the specific event as optional. Requirement 2 (Inclusion of Provenance) The provenance of each event must be represented within the data model. Requirement 3 (Inclusion of Entity Type) The type of each entity associated with a given event must be represented within the data model. This type can be fine-grained or coarse-grained. Requirement 4 (Inclusion of Properties) For any given entity INLINEFORM0 associated with a given event INLINEFORM1 , a property (i.e., binary relation) INLINEFORM2 between the entity INLINEFORM3 and the event INLINEFORM4 must be represented within the data model. Thus, for the given pair INLINEFORM5 , either the triple INLINEFORM6 or the triple INLINEFORM7 is entailed in the RDF graph of INLINEFORM8 .

We address three main challenges on the way: (1) agreeing upon a background data model (either by developing or reusing one), (2) annotating events, associated entities as well as relations, (3) interlinking events across time and media, and (4) publishing triples on the event knowledge graph according to the principles of Linked Open Data.

Considering the sample of tweets in Table TABREF9 , the tweets no.1, no.4, and no.7 are instances of the event Communication with the mentions tell, say, announce. The tweets no.2, no.5, no.8 are instances of the event Meet with the mentions meet, visit. The tweets no3, no6, no9 are instances of the event Murder with the mention kill. fig:exam demonstrates the running example within the developed data model. This event has two participants (i.e. instagram CEO and Pontifex) along with a specific topic.

Event Annotation	Events can be represented at different levels of granularity. The event annotation task potentially comprises of two subsequent tasks as follows: Event recognition: Typically, event recognition utilizes phrases and their parts of speech. Although, verbs are more common for distinguishing an event (e.g., `Obama met Merkel in Berlin'), the other POS might reveal an event (e.g., `G8 meeting in Berlin'). Furthermore, event recognition task ecan beither open domain or closed domain. In the former one, collecting a lexicon of event phrases is more challenging rather than for the latter one. In any case, a learning approach (either supervised or semi-supervised) can be applied for determining whether or not a piece of text contains an event phrase or not. Event classification: This task is necessary in case the employed background data model considers the specific type of events as part of event annotation. In this case, event phrases have to be labeled by specific types of events using multi-class classifier trained for distinguishing the specific type of a given event. For example, the tweets no.2, no.5, no.8 of tab:tweetsamples have the specific type “meet”.

In the former case, entity linking task is reduced to hook a suitable entity whereas in the latter case, it is required that a new IRI (i.e., International Resource Identifier) be minted and typed and then linked to the textual mention of the given entity. E.g., in the tweet no.1 of tab:tweetsamples, the named entity `Michelle Obama' should be linked to the entity dbr:Michelle_Obama, when DBpedia is employed as the background knowledge graph. The second type of entity linking is about linking entities across knowledge graphs using owl:sameAs links. While the first task is required in the pipeline of developing an event knowledge graph, the second one is optional but can enhance quality and visibility of the underlying knowledge graph. Semantic role labeling: Most of the existing event ontologies consider generic roles such as actor or agent for involved entities. For fine-grained background data model, the semantic role labeling can be done. E.g., w.r.t. the tweet no.1 in tab:tweetsamples, the entity `Michelle Obama' can be labelled by the generic role actor employing LODE ontology or the specific role giver applying the data model illustrated in fig:communicationpattern.

Entity disambiguation: An entity mention in a text might be polysemous, thus linking to the correct entity in the underlying knowledge graph requires a disambiguation phase. Furthermore, a single entity in multiple knowledge graphs might have various representations. Thus, interlinking them is challenging and requires a disambiguation phase as well  ,  ,  . E.g., w.r.t. the tweet no.7 in tab:tweetsamples, the named entity `Obama' is ambiguous as of whether it refers to `Michelle Obama' or `Barack Obama'. Regarding context (i.e., the remaining part of the tweet), it likely refers to `Barack Obama'. Implicit entity linking: As we mentioned before, not all of the mentions of entities are explicit. For example, w.r.t. the running example, the chunk `Instagram CEO' refers to the implicit entity `Kevin Systrom' who is the CEO of Instagram. The experiment performed in  shows that 21% entity mentions in movie domain and 40% of entity mentions in Book domain are implicit. Inferring implicit entities depends on capturing context as well as respecting time intervals.

The main contributions of this paper include: The remainder of this paper is organized as follows. Section SECREF2 is dedicated to notation and problem statement. Section SECREF3 outlines the required steps for developing a knowledge graph of interlinked events. Section SECREF4 frames our contribution in the context of related work. Section SECREF5 concludes the paper with suggestions for future work.

Question: Which news organisations are the headlines sourced from?
Answer: BBC and CNN