Extending W ordnet to Geological Times (2018)

Abstract: This paper describes work extending Princeton WordNet to the domain of geological texts, associated with the time periods of the geological eras of the Earth History. We intend this extension to be considered as an example for any other domain extension that we might want to pursue. To provide this extension, we first produce a textual version of Princeton WordNet. Then we map a fragment of the International Commission on Stratigraphy (ICS) ontologies to WordNet and create the appropriate new synsets. We check the extended ontology on a small corpus of sentences from Gas and Oil technical reports and realize that more work needs to be done, as we need new words, new senses and new compounds in our extended WordNet.

## introduction princeton wordnet @xcite works well as a dictionary and thesaurus for uses of english, as found, for instance, in newspapers and general knowledge texts, such as wikipedia. some attempts at extending it, for specific domains, such as bioinformatics, geography or law @xcite @xcite have been made, but it is not clear how these extensions should be done, as different stakeholders will want to extend the basic dataset into different directions and with different tools and objectives. the goal of our work is to describe a possible process of extension of the basic princeton word-net, for a restricted domain (geological time periods) and to discuss issues, challenges and opportunities for other generic extensions. one might wonder whether extensions of word-net are really necessary. to this we say that even for everyday language we are convinced that wordnet misses some necessary synsets. for example, there are several issues related to tokenization: words like ping-pong, kickboxing, water-ski and fistfight should appear with space, hyphens or not, in the respective synsets. they do not, which means that quite a bit of post-processing is necessary. it would be good to add many prefixes, suffixes and regular endings, which are perfectly understandable by humans, but not so much by machines, for instance shirtless and localizer, focalizer are not in wordnet. also many verbs ending in -ize, ise or ify are not present in pwn, while being in wiktionary, for instance adjectivise, africanize or incentify, girlify. we might also want to discuss why the kinds of extension of wordnet we describe in this work are useful. we offer two complementary explanations. first we want to use wordnet as a sort of "lightweight" ontology. as discussed in @xcite while full comprehensive ontologies like sumo @xcite or cyc @xcite would be best for reasoning formally with the information in texts, these tend to be very ragged. they only have detailed information in the specific domains that people felt the need to complete them for. for daily words and everyday, commonsense, events they miss many concepts. some shallow reasoning can be done on the basis of the information provided by lexical taxonomies and it seems best to cover all concepts, at the expense of being shallow than to have big gaping holes in the concepts covered. the second explanation has to do with bootstrapping specific domain ontologies for specific domains. even if we did have a fully comprehensive version of an open source ontology for commonsense, we would still need to complement it for specific domains like geology and paleontology. there are too many concepts specific to the field that english fluent speakers have never heard of and that should not be part of a basic lexical resource for english. but these specific, say, geological concepts, need to be fitted within the taxonomic framework of a lexical knowledge base like wordnet, so that we can take advantage of the aforementioned framework. some of us would like to use this aspect of wordnet expansion to construct gas and oil ontologies for supporting projects on information extraction on that industry. in the (small) experiments we report in this paper, we discuss a very specific extension to a hopefully not very controversial domain. we want to add to wordnet specific information concerning geological time periods. the geologic time scale (gts) is a system of chronological dating that relates geological strata of rocks (stratigraphy) to time as measured in years in earth's history. ## geological time periods the geologic time scale is used by geologists, paleontologists, and other earth scientists to describe the timing and relationships of events in earth's history. the table of geologic time spans set forth by the international commission on stratigraphy, which we take to be the official body for these scientists, is described in http://www . stratigraphy.org. both wikipedia and wiktionary have some information about geologic time periods that seem more complete than the information in wordnet. this is to be expected, as lexicographers tend to be conservative about the terms they add to their repository of the language. but to be useful, when analyzing scientific texts about geological descriptions, we need to take the newer and more specific information present in the wiktionary and wikpedia in consideration. this is a common pattern. for several specific domains wikipedia and wiktionary have more current and more specific information than wordnet. wordnet is concerned about not inflating the lexicon with terms that are clearly derived, when looked from a human perspective, (e.g. coaly is simply the adjective form of having to deal with coal) or easily compositional (like basinwardin the direction of a basin). also new expressions consisting of prefixes and suffixes are not considered good material for wordnet, so wordnet has aeon, but not superaeon. we would like to devise and describe a process to extend wordnet for a specific domain, when we do have specific information about the domain in the shape of a well curated ontology for the domain, as well as high quality texts in the same. we use geological time periods and a small collection of papers in petrology as a paradigmatic example of a domain specific extension. ## the isc ontology the isc ontology presents a view of the knowledge associated to the international stratigraphic chart. the isc ontology contains many subontologies, including the geologic timescale (gts foot_2 ) that would seem perfect for our uses. in this ontology, age, eon, epoch, era, period, sub-period, and super-eon are sub-classes of geochronologicera (abbreviated as ge), which seems simply a different name for what is called 'geological time' in wordnet. however, there is no formally defined hierarchy between these concepts. instead, greater emphasis is placed on the boundaries of the periods and only the approximate duration of the period is given in the chart. it is important to note that geologists qualify the units as "early", "mid", and "late" when referring to time, and "lower", "middle", and "upper" when referring to the corresponding rocks. for example, the lower jurassic series in chronostratigraphy corresponds to the early jurassic epoch in geochronology. the adjectives are capitalized when the subdivision is formally recognized, and lower case when not; thus "early miocene" but "early jurassic". while the commission was created exactly to unify and organize the classification of both strata and geochronological periods, it appears that the work is both not finished and bound to disagreement. the above mentioned paper also says [...] disagreement often arises, because type sections that are favoured for historical reasons may be abandoned, previously established boundary levels may be greatly changed, and in some instances historical units are replaced by different new ones. thus while the ontology might look very much a finished product, it seems that its contents are still subject to debate. the boundaries between periods seem to be annotated using another ontology, the temporal hierarchical ordinal reference system model (thors foot_3 ), which is used to formally define the hierarchy between instances of ge. fragments of the iso19108:2002 standard (geographic information -temporal schema) are also used to specify the temporal position of geochronologic boundaries. the time interval of a ge is given in terms of its boundaries to other ges via thors:begin and thors:end. each boundary is a geochronologicboundary and it is temporally located via iso19108:temporalposition which specifies a iso19108:coordinate with a value, frame (e.g., "ma"), and a positional uncertainty. for example, the maastrichtian period is defined by wiktionary in https://en . wiktionary.org/wiki/maastrichtian as "in the ics geologic timescale, the latest age or upper stage of the late cretaceous epoch or upper cretaceous series, the cretaceous period or system, and of the mesozoic era or erathem". in the isc ontology itself the definition is more complex. the maastrichtian period (66-72.1 million years) is defined using boundaries and frames (figure 1 ). maastrichtian a geochronologicera ; rank age ; begin basemaastrichtian ; end basecenozoic . basemaastrichtian a geochronologicboundary ; temporalposition basemaastrichtiantime . basecenozoic a geochronologicboundary ; temporalposition basecenozoictime . basemaastrichtiantime a coordinate ; frame ma ; value "72.1" . basecenozoictime a coordinate ; frame ma ; value "66" . the boundary modeling should be sufficient for representing the hierarchical relationship between ges, but isc further defines a explicit set inclusion relationship between ges via the thors:member property. also, skos @xcite is also used to represent inclusion via skos:narrower, skos:broader along with theirs transitive versions, skos:narrowertransitive and skos:broadertransitive. in any case a collection of 176 basic geologic period terms is easy to deal with, if the scientists are in agreement. however, we still have to deal with common nouns (e.g. play, basin, cleats) and compounds (e.g. golden spike), whose geological meanings are very different from their usual meanings. these need to be extracted from a geology corpus, similar to the one we describe in the next section. ## a corpus of geological reports the source documents for the our small experiment come from 155 randomly selected text passages relevant to petroleum systems extracted from a corpus of 1,298 publicly available english language geological reports, published by the united states geological survey (usgs), geological survey of canada (gsc), and british geological survey (bgs). the passages were segmented in 5,661 sentences (186,244 tokens) and parsed in the universal dependencies scheme by udpipe @xcite foot_4 . udpipe is a generic, off the shelf processing pipeline trained with the english corpus from the universal dependencies project @xcite . using the model available, trained on newswire data, it does not do well on named entity recognition in our corpus. our preliminary semantic pipeline looks up nouns, verbs, adjectives and adverbs in princeton wordnet. out of 8800 noun lemmas uncovered by udpipe, more than half were not recognized as present in word-net. because the reports are describing real world geological work, the corpus is full of named entities, e.g. names of places, people and organizations that cause named entity recognition to be such a hard task. some of these missing words are processing mistakes. for instance, the word 'reservoirs' was not correctly lematized to 'reservoir'. a large proportion are named entities, people, places and organizations that wordnet is not supposed to list in any case. but a small proportion are really common words that wordnet should have, in our opinion. finding these seems to be a positive side effect of trying to extend wordnet for specific domains. since our aim is not the processing of this corpus, but simply its use as a source of extra vocabulary for our extended wordnet, we decided to look at all tokens in the corpus with more than 10 occurrences, trying to decide whether they were named entities or not. and we assumed that the processing could be corrected, by hand, if need be. it is well-known that pwn lacks some important compounds and that the cut-off line for compounds to be lexicalized is a difficult one to decide on. moreover, in this specific field, we do not know exactly when compounds are compositional or not. but a shallow processing of the text provides us with some 20k proper nouns, so almost 4 proper nouns per sentence. this means that ner is a very hard job, even assuming near perfect geoname resources, which unfortunately we do not have. ## creating new synsets the language of isc and its various ontologies is complex, and for a reason. they want to be precise, while trying to merge different standards. as we want to map all their precision into an extended version of princeton wordnet we need a kind of a domain specific language (dsl) to describe new synsets. this language helps us not only to describe the new synsets we need, but also should helps us localize these new synsets within the original wordnet structure. the file format we decided to use is intended mainly for human consumption, even at the cost of a more complicated parsing routine. redundancies are eliminated, for example there is no need to specify both sides of reflexive relations, such as hyponymy and hyperonimy. artificial identities (ids) are avoided to make maintenance easy. actual ids are based on the lexical units, following the ideas of the original lexicographer files for princeton wordnet. instead of using symbols such as @, !, etc. for relations, we use mnemonics such as hyper (hypernym) and ant (antonym). the goal is to make a standalone domain specific language -one that is usable without any accompanying integrated development environment (ide) or other auxiliary program. synsets are defined by groups of lines, separated by a single empty line. words of the synsets should have their spaces converted to underscores and repeated words in the same file should have suffixes to distinguish them, also following the original lexicographer files of pwn. for example the synset for eon will be written as w: eon drf adj.pert:eonian w: aeon drf adj.pert:aeonian hyper: geological_time g: the longest division of geological time where drf stands for 'derived form', adj.pert is the usual wordnet description of the pertainym adjective file and g stands for the 'gloss'. each word entry is essentially a sense. links between senses are specified in the same line as the w: word, for example: w: uptime ant downtime means that 'uptime' and 'downtime' are antonyms. semantic relations (i.e., links between synsets) are specified on lines of their own, such as the hypernym hyper: geological time above. the first word of a synset is used as its identifier. the lexicographer file filename should also be included to further disambiguate words, if necessary. this is usually the case when there are semantic links across synsets defined in different files. for example, the file noun.location contains the following excerpt for the synset "brazil": w: brazil drf adj.pert:brazilian hp: noun.object:south_america to maintain compatibility with existing systems that already use pwn sense keys and synset ids we provide mappings between our sense ids and pwn. similarly, mappings that link synsets and existing ontologies can also be defined. the full set of pwn synsets extended with the nodes created for the geological time periods and the new concepts we deem necessary to understand our corpus could be called pwn gt s for wordnet extended for the geological time scale. in the next section we describe a toy application of the extension developed. in http://github . com/own-pt/wordnet-dsl we provide the pwn gt s and the mappings from the new synsets to the isc ontology. ## using pwn gt s the following discussion showcases an example where a number of geochronologic entities may be referenced implicitly by the text. consider the following sentence from our corpus: in this chapter, the kinematic interpretation of the west carbonate shear zone is placed in a regional context, with regard to intrusive and tectonic activity from 2740 to 2690 ma ago. assuming that a parser correctly identifies the numerical range above as being 2740-2690 and the unit 'ma' (for a million years), one can use our extended wordnet, creating a query to the isc ontology that searches for entities that encompass this period of time. the sparql query used is in the appendix, note that such a query does not take into consideration the variance of the boundaries of time periods (modeled by the ontology). we opted to omit this feature to keep the sparql code simple. this natural query is not enough to uniquely disambiguate the appropriate instance that is referenced above, since the query returns three isc entries: the neoarchean era (2500-2800 ma), the archean eon (2500-4000 ma), and also the precambrian super-eon (541-4567 ma). while this toy example shows one possible use we envisage for very restricted forms of extension of the basic english wordnet, the larger question of evaluating such extensions beckons. from our preliminary work we can see some possibilities, which we discuss next. ## evaluating extensions it is clear that different kinds of text and different content domains play a big role in the vocabulary that lexical resources are expected to cope with. this is clear for specific content domains, such as bioinformatics, where changes are recent and newer vocabulary is being created at impressive speeds. but even for domains, such as geology, where one might have expected the main vocabulary to have been established by the end of the 19th century, things are not as well settled as expected. certainly there is a need for more (open source, downloadable) online glossaries, apart from the (small) wikipedia one foot_5 , the openlearn project foot_6 and the one from usgs foot_7 that has not been updated since the mid 2000's. but it seems that the proprietary ones still have the upper hand. the american geosciences institute (agi) offers their fifth revised edition of the glossary of geology @xcite as a book and as paid subscribing content online. they say that their reference tool contains nearly 40,000 entries, including 3,600 new terms and nearly 13,000 entries with revised definitions from the previous edition. none of the open source glossaries we found has as many entries as that. one way of measuring how much we can do with the open resources online is to measure how much of the informational contents of technical reports can be gleaned by a impoverished nlp pipeline that builds bag-of-concept semantics from the sentences of the chosen corpus. in a previous experiment we have computed this kind of bag-of-concepts semantics for sentences of the corpus sick @xcite . the corpus sick is much easier to deal with, as it was engineered to not have any named entities at all. if we had no named entities in our geological reports, we could produce concepts from sumo @xcite using a bare bones pipeline that transforms sentences into universal dependencies (using udpipe), dependencies into wordnet concepts or synsets (using, say, freeling/ukb @xcite for disambiguation) and wordnet synsets into sumo con-cepts (using the sumo mappings). an example of a processed sentence is displayed in figure 2 . the idea here is not to produce knowledge representations of the meanings of the sentences, but simply to list the expressions for which we do not have a concept. for these 'empty concepts' we need either geographical information or new synsets, as they correspond to either new content words (that never appeared in wordnet before, like e.g. vitrinite or stratigraphic), or new compounds (e.g. pre-mississipian, antiform or subbasin, golden spike) or new senses of words already in wordnet (e.g. cleats, play, sequence, which have completely different meanings in geology from their usual ones). however we need to find a way of coping programmatically with named entities, for this baseline calculation to work. given the hardness of the ner problems in this particular kind of texts, we resorted to different open systems (with different training data and heuristics, e.g. opennlp foot_8 and freeling @xcite ) to try to extract most of the named entities. in this corpus apart from locations, people and entities we have many geological formations, which span counties and even states' lines. to help debug our processing, we are experimenting with interfaces that allow linguists, computer scientists and geologists to communicate more easily http://wnpt.brlcloud . com/demo. we hope to improve, using subject matter experts, the number of new synsets and new senses. the manual 'ensemble' effort to recognize named entities we produced for this small corpus, needs to be streamlined in the future, for the work in extending other domains. ## conclusions this preliminary work discusses extensions of princeton wordnet for specific content domains. the case we considered is the well delimited domain of geological time periods. we expected it to be less controversial and to have a more established vocabulary than it turned out to have. however, we stand by our initial suggestion that specific domains require specific extensions. that these specific extensions need to be built as much as possible from open source resources, in a collaborative fashion, using as much as possible associated ontologies produced by the subject mat-ter experts. however, a useful way to augment the specific knowledge required is to shallow process scientific texts on the specific subject (we used gas and oil technical reports) and try to extract more lexical information from them. our small experiment with geological reports indicate that a more robust mapping of named entities is required before we can evaluate the usefulness of our new geological time scale wordnet. we are working on a tool that would pre-annotate some of these geonamed entities and would facilitate the correction of the mistaken annotations.