Toward a Semantic Concordancer (2018)

Abstract: Concordancers are an accepted and valuable part of the tool set of linguists and lexicographers. They allow the user to see the context of use of a word or phrase in a corpus. A large enough corpus, such as the Corpus Of Contemporary American English, provides the data needed to enumerate all common uses or meanings. One challenge is that there may be too many results for short search phrases or common words when only a specific context is desired. However, finding meaningful groupings of usage may be impractical if it entails enumerating long lists of possible values, such as city names. If a tool existed that could create some semantic abstractions, it would free the lexicographer from the need to resort to customized development of analysis software. To address this need, we have developed a Semantic Concordancer that uses dependency parsing and the Suggested Upper Merged Ontology (SUMO) to support linguistic analysis at a level of semantic abstraction above the original textual elements. We show how this facility can be employed to analyze the use of English prepositions by non-native speakers. We briefly introduce condordancers and then describe the corpora on which we applied this work. Next we provide a detailed description of the NLP pipeline followed by how this captures detailed semantics. We show how the semantics can be used to analyze errors in the use of English prepositions by non-native speakers of English. Then we provide a description of a tool that allows users to build semantic search specifications from a set of English examples and how those results can be employed to build rules that translate sentences into logical forms. Finally, we summarize our conclusions and mention future work.

## introduction concordancers foot_0 enable the linguist to see the context of use of words or phrases. this is valuable in understanding how a word can have different senses, or in finding rules or exceptions for collocations. one issue for the linguist using such tools is that many linguistic constructions are patterns or types, rather than literal collections of words. we "take a pill" but "eat a muffin", we "play music" but "draw a picture", "fly a plane" but "drive a car" or "pilot a boat". for each of the nouns, a class or group determines the verb (such as "medicine", "2-d art" or "aircraft"), but enumerating those possibilities is cumbersome. a computational linguist could develop customized analysis software, but no general purpose tool fit for this task appears to exist. we have developed software that allows the linguist to specify dependency relations and semantic types, based on a formal ontology, that can alleviate the need to enumerate large numbers of alternative strings of search terms with a conventional concordancer. ## corpora to motivate development of this software we have two use cases. the first case is in analysis of corpora for classes of errors in usage that are common for non-native speakers of english. we chose to look at a small corpus of translated speech and analyze it for these classes of errors. in this way, we can provide specific feedback to translators on what problems to avoid in the future. to augment this work, we also examined a larger and broader corpus of non-native english usage, in order to help validate the utility of the tool on a corpus that has more, and more obvious, usage errors. we begin with a corpus of legal judgments translated from chinese into english. judgments translated from chinese into english are essential to the rule of law in hong kong. hong kong is the only common law jurisdiction where chinese and english languages are used alongside each other in the judicial system @xcite . judgments form an essential part of common law. because the majority of the population is chinese speaking, court cases are sometime heard in chinese. judgments in these chinese cases are written in chinese. judgments of cases with jurisprudence value are translated into english. these translated judgments may be used in the future by legal professionals who are not necessarily familiar with the chinese language. translated english judgments were downloaded from the hong kong judiciary website foot_1 to build the hong kong translated english judgments corpus. non-native speakers can find it challenging to use english prepositions properly. compared to english, chinese is a verb heavy language. the chinese language has significantly fewer prepositions than the english language does. unlike english, chinese sentences without prepositions are grammatically correct and comprehensible @xcite . chinese speakers, even with good english language abilities, may not be as sensitive to the use of prepositions when using the english language. therefore, one of the challenges facing chinese speakers when translating into english is the accurate use of prepositions. after removing titles, headings and other incomplete sentences in the legal corpus, we arrived at 8818 sentences in suitable for further processing by our semantic concordancer. to broaden our study, we also examined the cambridge learner corpus (clc) 3 (yannakoudakis et al., 2011), which has a greater number of english usage errors and is roughly twice the size of our legal corpus, at 16068 lines of text, also ignoring titles and headings. our second use case is in validating linguistic patterns and creating rules to translate language to logical forms, for which we employ two large corpora of native english writing. these are the corpus of contemporary american english (coca) @xcite and 2722 articles from wikipedia converted to plain text foot_3 . ## nlp pipeline our work relies upon the stanford corenlp @xcite pipeline, which is free and open source, and either the top performing system or at least state of art on each element of its pipeline. the system is structered as a series of annotations on tokens. each annotator builds up annotations on the textual input. to illustrate the pipeline, let's take a particular example. the stanford named entity recognizer @xcite identifies linguistic references to things like names and dates. it results in the following markings of our example (where "o" is a tag for "other", meaning not a named entity) we have added a multi-word phrase recognizer to the corenlp pipeline that uses wordnet and sumo as dictionaries. matching multi-word elements are reduced to a single token, so "yao ming" or "july 23rd" will become a single token with a class membership in sumo (human or day respectively here). dependency parsing (chen and manning, 2014) abstracts parse trees into a set of linguistic relations that are as independent of language as possible. we have the following dependency graph for example 1: yao ming drank tea in the morning root comp nsubj dobj case det nmod:in note that dependencies as a data structure can also be represented as just a list of triples. corenlp lacks a module for determining word senses so we have utilized our existing system from @xcite . this process normally addresses just nouns, verbs, adjectives and adverbs. determining named entities is done in the ner system described earlier. wsd annotations are shown as example 3, and definitions for some different senses of "tea" are shown in these ids are for wordnet 3.0 @xcite (with the part of speech number prepended) and they have been manually linked to the suggested upper merged ontology (sumo) 5 @xcite . since the original mapping effort in 2002, tens of thousands of synsets have been remapped to more specific sumo terms as they have been defined. in particular, several thousand have been remapped in 2017 alone. the current statistics for the mappings are shown in instance mappings are from a sumo term to a particular instance synset in wordnet, such as sumo's battle mapping to wordnet's "battle of britain". equivalence mappings are close but informal equivalences, such as the mapping between sumo's cloud and wordnet's synset 109247410 "a visible mass of water or ice particles suspended at a considerable altitude." subsuming mappings are between specific wordnet synsets and more general sumo terms, such as "meniere's disease" and sumo's diseaseorsyndrome. of note is that recently, with the growth of sumo in several domains, we increasingly have need for what we might term a "subsumed-by" relation, where a sumo term is more specific than any available word-net synset, as is the case with the new ontologies of law and weather. this relation is likely to appear in a future release of the mappings. we also augment the wordnet lexicon with lexical entries provided in the ontology for each new term, such as the string "mono crystalline" being associated with the recently-added sumo term monocrystalline. to perform word sense disambiguation, we rely on wordnet semcor @xcite , a corpus of manually-marked word senses, indexed to the wordnet semantic lexicon, and annotated on the brown corpus of english @xcite . for each word sense, we create a table counting the frequency of co-occurring words in the corpus. we use a frequency threshold so that low-frequency senses that have little cooccurrence data aren't influenced by random small amounts of data. one criticism of wordnet has been that it makes some overly fine distinctions among word senses @xcite . we use the sumo-wordnet mappings to collapse senses that map to the same term in the ontology. note however that this grouping is much more fine grained than the coarse-grained aggregation to categories done in semeval-17 on ontonotes @xcite , so that fewer (if any) meaningful distinctions in sense are lost. this approach has the added benefit of increasing the statistical significance of some of the merged cooccurrence relationships. this approach however does not perform as well as some recent effort in wsd that employ machine learning, such as @xcite . when tested on the ontonotes corpus @xcite we achieve roughly 66% accuracy, which approaches the score (stated at 72% in @xcite ) for inter-annotator agreement on fine grained senses. since we cannot assume a particular domain, accuracies are likely to be lower than the best results of other reported studies @xcite . however, it is likely that more training data from a wider set of corpora foot_5 will help improve performance. we augment stanford dependency parses with sumo terms. continuing the example above, we add the triples sumo(drinking,drank-3) sumo(morning,morning-7) sumo(tea,tea-4) while sumo does have a taxonomy, it also has definitions in a higher order logic that explain, in a computable way, the meaning of each term. so, sense key words definition 107575510 tea, teatime a light midafternoon meal 107933274 tea a beverage made by steeping tea leaves in water 107932841 tea, tea leaf dried leaves of the tea shrub that states that being thirsty implies a desire to drink something. axioms such as this are more specific and detailed than entailment links and can enable further logical reasoning. we have linked the stanford 7-class ner model to sumo types, which allows us to assert sumo(human,yao_ming-1) from the ner output shown in example 2. we also employ stanford's sutime @xcite to recognize temporal expressions. if we have the slightly modified example (4) yao ming drank tea in july. ## semantic concordance concordancers are very useful for checking intuitions with respect to language usage. searching on a word or phrase provides samples of usage in context. but not all language patterns are strict phrases. idioms can have insertions @xcite , such as "drop in the bucket" being modified to "drop in the proverbial bucket" or "drop in the fiscal bucket" but not "he put a drop of water in the bucket". being able to search a dependency parse for a grammatical pattern rather than a literal string or even a string with wildcards may be a useful tool. some patterns of usage are selected with respect to the types of participants in a phrase, rather than particular words. these can be quite specific. for example, if a linguist wants to examine usage of the preposition "in" in its physical, rather than temporal sense, an exhaustive number of searches would be required to enumerate physical words and phrases and temporal words or phrases. however, given that we have dependency parse forms and sumo terms we can search for patterns such as: nmod:in(?x,?y), sumo(?c,?y), issubclass(?c,timeposition) nmod:in(?x,?y), sumo(?c,?y), issubclass(?c,object) to carry on with example 1, note how the first pattern involving timeposition above matches with the clauses of the example, and the variables are bound to ?@xmath0, ?@xmath1. root(root-0,drank-3) det(morning-7, the-6) nmod:in(drank-3,morning-7) sumo(human,yao ming-1) sumo(drinking,drank-3) sumo(morning,morning-7) names(yao ming-1,"yao") dobj(drank-3,tea-4) case(morning-7,in-5) sumo(tea,tea-4) names(yao ming-1,"ming") nsubj(drank-3,yao ming-1) while wordnet noun synsets could be used to capture common classes of words, sumo provides extra utility when searching for groups of verbs. for example, one "looks for" or "searches for" something in order to find it and some language learners may omit the preposition. in each case there is a mapping to sumo's searching, but no common hypernym for those wn 3.0 senses (201315613 and 202153709, respectively). because wsd and dependency parsing are not always correct, it is necessary to review results rather than simply tabulating them. also, language is flexible, and what constitutes "correct" usage is more like correspondence to a preponderance of use than a strict rule in many cases. ## preposition errors we looked for common errors in preposition usage foot_6 in our corpora of non-native english. the first error type that was searched for was the use of prepositions with times of day (see example 5), where "night" is an exception. we can state the (ungrammatical) dependency pattern nmod:at(?x,?y), sumo(?c,?y), issubclass(?c,timeinterval) one sentence found in the corpus was example 6, (6) "we usually have lessons at the morning, till afternoon." this sentence has the augmented dependency parse of root(root-0, have-3) nsubj(have-3, we-1) advmod(have-3, usually-2) dobj(have-3, lessons-4) case(morning-7, at-5) det(morning-7, the-6) nmod:at(lessons-4, morning-7) case(afternoon-10, till-9) nmod:till(have-3, afternoon-10) sumo(subjectiveassessmentattribute, usually-2) sumo(educationalprocess,lessons-4) sumo(morning,morning-7) sumo @xcite other examples of linguistic errors in the corpus found by matching dependency patterns are (7) * i've been working here since five years. ## query composition one of the challenges in using this tool is that it requires some knowledge of dependency parsing and sumo. to address this, we have created a component that find the common structure of several sentences and returns a dependency parse for that common structure. that specification can then be used to search for other sentences that match the pattern. in this way, the linguist simply has to prepare several sentences that illustrate a common construction and let the system do the work to state the commonality in a formal language. take for example the following two sentences (8) john kicks the cart. note that the expression can be verified to unify with the original dependency parses, using the following substitutions for sentence 8 as an example. ## semantic rewriting the semantic concordancer is an intermediate result from efforts to translate language into logic. we are extending prior work on the controlled english to logic translation @xcite to use modern parsing techniques with stanford's corenlp instead of a restricted english grammar. when the semantics of sentences are fully captured it opens up opportunities for deductive reasoning that goes beyond simple retrieval of previous sentences. it also creates the possibility to vet utterances for contradictions with known facts about the world, thereby allowing a system to exclude faulty parses based on world knowledge. for example, the simple sentence 8 above becomes the following first-order logic sentence with sumo terms -(exists (?john-1 ?cart-4 ?kicks-2) (and (agent ?kicks-2 ?john-1) (attribute ?john-1 male) (names ?john-1 "john") (patient ?kicks-2 ?cart-4) (instance ?cart-4 wagon) (instance ?kicks-2 kicking) (instance ?john-1 human)) ) the process of accomplishing this is what we call semantic rewriting, and is based on previous efforts called transfer semantics or packed rewriting @xcite . it involves the iterative application of production rules to dependency parses. in the case of sentence 8 this involves execution of just two rules (along with a simple mechanical listing of the types of terms with instance and generation of the name of "john" as a male human from a common name database) - the first rule is a general default that if we have no more specific pattern, the direct object in a sentence becomes the "patient" in a sumo expression. the second rule is more interesting. it states that if the grammatical subject of a process is an agent (rather than some inanimate object) then we generate a sumo agent relationship between the entity and the process. while creating a few simple rules of this sort is easy, as the rule set grows and the remaining rules become more complex, authoring them through introspection become impractical. the query composition tool described above provides a principled way to create patterns by example, which form the left hand side of a semantic rewriting rule. the semantic concordancer then becomes useful as a way to validate the prevalence of a particular pattern of language use in a large corpus. ## conclusions and future work the software is available open source at https: //github.com/ontologyportal and has been used on a practical pilot project in analysis of non-native english. we expect to apply it further to more systematic studies in this area as well as others. the implementation is in java, using the h2 database 8 . all the words in each sentence and terms in dependency parses are indexed, so all semantic processing occurs at the time the database is built, rather than when a query is run. after sentences and dependencies matching a bag of terms are returned, a simple unification algorithm attempts to match the dependency parse literals with the dependency parse query, similar to a prolog-style unification algorithm @xcite . this enables the system to scale well to the requirements of modern large corpora. we are employing the semantic concordancer and its associated query composition tool to create and validate semantic rules that translate language into logical expressions. the system will be available by the time of gwc2018 on a server at https://nlp.ontologyportal.org : 8443/sigmanlp/semconcor.jsp.