The DARPA W ikidata Overlay: W ikidata as an ontology for natural language processing (2023)

Abstract: With 102,530,067 items currently in its crowd-sourced knowledge base, Wikidata provides NLP practitioners a unique and powerful resource for inference and reasoning over real-world entities. However, because Wikidata is very entity focused, events and actions are often labeled with eventive nouns (e.g., the process of diagnosing a person’s illness is labeled “diagnosis”), and the typical participants in an event are not described or linked to that event concept (e.g., the medical professional or patient). Motivated by a need for an adaptable, comprehensive, domain-flexible ontology for information extraction, including identifying the roles entities are playing in an event, we present a curated subset of Wikidata in which events have been enriched with PropBank roles. To enable richer narrative understanding between events from Wikidata concepts, we have also provided a comprehensive mapping from temporal Qnodes and Pnodes to the Allen Interval Temporal Logic relations.

## introduction an ontology is a necessary framework for practical system representation of domain-specific world knowledge. however, since we each perceive the world somewhat differently, and different domains require access to varying levels of granularity, a unique, universal ontology is not a realistic goal. the biomedical portal foot_0 alone lists 1213 different biomedical ontologies. the largest structured repository of knowledge about world entities is wikidata @xcite , but, as should be expected from any collective knowledge repository, it has many inconsistencies, circular subclass links, and partially overlapping concepts and gaps. since every practical project designed for a specific task needs a consistent ontology, our challenge is to provide a common, mutually agreed upon vocabulary that speeds up the process of incorporating new domains or evolving existing ones by automatically leveraging existing knowledge bases, such as wikidata. we introduce the darpa wikidata overlay (dwd overlay), a curated subset of wikidata enriched with @xcite roles. we chose wikidata as our primary resource for ontological concepts because of its extensive coverage of concepts, its linking of those concepts to each other, and the ability to contribute additional concepts to the database as needed. importantly, wikidata concepts are linked to relevant wikipedia entries, a distinct advantage for nlp applications concerned with current events. we turned to propbank as our source of participant roles because of its wide coverage of verbs and eventive nouns that we could easily match to wikidata concepts and because its roles could be represented as both broad, general roles (e.g., arg0, arg1) and as more event-specific (e.g., attacker, victim). further, to enable richer narrative understanding between events, we have provided a comprehensive mapping from temporal qnodes and pnodes to the allen interval temporal logic relations @xcite . while wikidata is a multilingual project, and propbank is a languageindependent semantic description, both resources use a mainly english interface and semantic roles from resources like propbank can be, to some degree, language-specific @xcite . the overlay is currently hosted in a json format 2 , designed to give users the ability to browse concepts and easily ingest the ontology structure into their computing applications. we hope that by establishing a robust and accurate mapping be-tween propbank and wikidata, we enable the usage of well-established nlp methods for event extraction using propbank combined with the inference power and massive coverage of wikidata. ## background and related work wikidata 3 is a large, crowd-sourced knowledge base. each item in wikidata refers to a either a concept ("president") or a real-world instantiation of a concept ("joe biden"), and is called a qnode. qnodes are connected to one another via pnodes, which represent the relation @xmath0, relation, @xmath1. the original impetus for using wikidata to support darpa programs came from the darpa aida and kairos programs, closely followed by darpa maa. each of the aforementioned darpa programs is described in more detail below, as well the motivation for coming to a consensus on an approach to ontology development that could be quickly adapted to new domains. active interpretation of disparate alternatives (aida), now completed, aimed at the organization of natural language news and social network information into competing, alternative hypotheses (narratives) about events and situations. aida systems integrated multi-modal knowledge elements into a common semantic representation suitable for hypothesis generation. task area 1 (ta1) performers applied cutting-edge nlp techniques to extract and co-refer knowledge elements from streaming text, images and videos, producing entries in a knowledge base (kb) for each input document. the program goals originally included the challenging restriction of ta2's and ta3's performing their tasks without access to raw input data. without consulting the original ta1 inputs, ta2 performers link together entity and event entries from individual documents into a unified kb using cross-document co-reference techniques. the ta1's could provide ta2's the names of named entities but could not originally provide ta2's with the lexical items or images for other types of entities. instead, ta2's passed along type information for entity and event entries from an ontology. this made the development of a program-wide ontology that all performers could utilize a high priority. ta3 performers then mined the unified ta2 kb for competing hypotheses. at the outset of the program, aida worked 3 https://www.wikidata.org/ on developing an expressive, semantic program ontology that could be used by performers to encode and exchange kbs and hypotheses. midway through the program, the program ontology contained hundreds of entity, relation, and event types developed using a data-driven approach inspired by pre-existing knowledge resources. these included previous linguistic data consortium (ldc) annotation efforts, such as ace @xcite @xcite and ere @xcite and their extensions for other programs such as deft foot_2 , as well as publicly available resources such as yago @xcite @xcite , framenet @xcite , propbank (kingsbury and palmer, 2002), verbnet @xcite and the reference event ontology @xcite . an ongoing tension existed between performers' desires for expressive, expansive ontology models and ldc's need for a manageable ontology supporting cost-effective corpus construction and annotation that can assist in evaluation of resulting system output @xcite . aida's solution was to charge ldc with selecting from previous programs those elements of the program ontology that would support salient entities and events in current program evaluation scenarios. this approach resulted in a patchwork aida annotation ontology where the connections between different ontological elements were sometimes obscured. for instance, geographical areas and geographical points were suggested in the program ontology, but without clear definitions. ldc chose geographical points to define a location subtype that could be used for addresses. they chose geographical areas to define a subtype of facility that could be used for installations covering a significant area, larger than a point, such as borders and checkpoints. this choice of labels helped annotators to distinguish addresses and borders from other types of locations and facilities, but can be confusing to ontologists more familiar with geographical point and area as two subtypes of spatial region. the maa and kairos programs described below both relied heavily on the aida annotation ontology, reflecting ongoing program needs for an expressive ontological data model that can be easily extended to new domains and evaluation scenarios. modeling adversarial activity (maa), now completed, was directed towards mathematical and computational methods for graph alignment and merging as well as subgraph detection and subgraph matching. maa used the aida ontology, and maa graphs were direct projections of aida rdf graphs into a property graph format that supported efficient and scalable graph analytics developed by maa performers. maa predominantly used the ldc annotation ontology as encoded in the aida interchange format. maa also focused on the transactional aspects of interactions in addition to entity-and event-based knowledge graphs. the maa evaluation phase included data sources related to financial topics, e.g. scientific publications and social media, and required modeling temporal events and entities with both physical and abstract attributes. in addition to the modeling of such data sources, the aida annotation ontology was also used by maa performers to develop approximate entity alignment and subgraph matching algorithms. ## discussion use cases and limitations. the dwd overlay has mainly been used @xcite as the primary resource for general-purpose event extraction. the propbank mappings in the overlay enabled @xcite to create a large dataset starting from propbank annotations and ending with wikidata event qnodes chosen from our mappings by mechanical turk workers. although the overlay has facilitated these advances in event extraction, many limitations have been identified. the many-to-one mappings (many qnodes per propbank roleset) proved to be the biggest limiting factor, as well as inaccuracy of automatic mappings, and the inclusion of low frequency nodes. the ongoing annotation project described in section 3.1 should ameliorate these limitations. tension between resources. the same advantages we gain from combining wikidata with prop-bank-extensive coverage of real-world entities and concepts, plus a large, rich set of participant roles for events-create the largest problems. both resources are powerful as nlp tools in their own right, each created for a slightly different purpose under a slightly different ethos. as discussed in section 3, propbank is action and event oriented, while wikidata is oriented towards entities, more often containing nominalizations and nominal forms of events, if an event is represented at all. when determining how entity-denoting nodes in wikidata are to be mapped to propbank event predicates, it is useful to examine how events are lexicalized in language. for english, the most frequent lexical realization of an event is a predicative verb, e.g., eat, sink, write, sign. this is followed by event nominalizations (e.g., arrival, explosion, decay) and activity nominalizations (e.g., eating, sinking, writing), and finally event nominals, e.g., meal, war, accident. as mentioned above, since wikidata is largely organized around reified (entitycentric) conceptual nodes, it is not surprising that both nominalizations and nominal forms are more commonly represented as qnodes for event denotations. for example, event predicates denoting activities that have clearly unambiguous nominalizations can be found represented as qnodes in wikidata: in these cases, the propbank mapping is easy. many reifications of event nominalizations in wikidata, however, tend to denote the result of the event, rather than the activity or event itself: hence, mapping from wikidata concepts to prop-bank participant roles requires discernment between concepts that map to events themselves and concepts that should fill participant slots of events that are not represented in wikidata at all. deciding when to allow imperfect mappings for the sake of coverage, yet at the expense of semantic integrity, has been a constant tension in the annotation project. for this reason, our annotators have recommended adding thousands of qnodes to wikidata itself to match the sense of propbank rolesets exactly. ## future work the propbank-wikidata annotation project is still ongoing. the overlay is expected to become higher quality and less noisy as the project progresses. however, we hope to eventually retire the overlay by integrating it into wikidata itself. integrating our mappings into wikidata itself will allow maintenance to be handled by the crowd-sourcers that already maintain wikidata. in the meantime, we anticipate that our unique resource provides opportunities for further advancements in the field of semantic annotation and ontologies for natural language processing. we plan to incorporate our mapped propbank roles into wikidata itself. by moving these roles into wikidata, researchers will eventually be able to use wikidata directly and repeated updating of the dwd would not be necessary. in discussions with wikidata, it was suggested we hire a wikidata consultant-someone who is already a frequent contributor to wikidata-to assist in adding information to wikidata itself. it was also decided to first release this addition to wikidata as an appendix. this would allow users to try the enhancement before fully altering the main wikidata structure. ## conclusion we introduced the dwd overlay, a curated subset of wikidata enriched with propbank roles for use as an ontology for natural language processing. our mapping combines the extensive coverage of ontological concepts and the inference power of wikidata with participant roles in propbank, providing a comprehensive, open domain resource for information extraction especially geared toward natural language newstext. while the dwd overlay already includes 1,125 manually curated qnode-propbank mappings and 4,040 semi-automatically induced qnode-propbank mappings, event templates for every event, as well as mappings to allen interval temporal logic relations, the human annotation is still a work in progress and the overlay is expected to continue to increase in quality.