COCO - EX : A Tool for Linking Concepts from Texts to C oncept N et (2021)

Abstract: In this paper we present COCO-EX, a tool for Extracting Concepts from texts and linking them to the ConceptNet knowledge graph. COCO-EX extracts meaningful concepts from natural language texts and maps them to conjunct concept nodes in ConceptNet, utilizing the maximum of relational information stored in the ConceptNet knowledge graph. COCOEX takes into account the challenging characteristics of ConceptNet, namely that – unlike conventional knowledge graphs – nodes are represented as non-canonicalized, free-form text. This means that i) concepts are not normalized; ii) they often consist of several different, nested phrase types; and iii) many of them are uninformative, over-specific, or misspelled. A commonly used shortcut to circumvent these problems is to apply string matching. We compare COCO-EX to this method and show that COCO-EX enables the extraction of meaningful, important rather than overspecific or uninformative concepts, and allows to assess more relational information stored in the knowledge graph.

## introduction conceptnet @xcite is a semantic network which contains general commonsense facts about the world, e.g. birds can fly or computers are used for sending e-mails @xcite . it originates from the crowdsourcing project open mind common sense @xcite that acquired commonsense knowledge from contributions over the web. the current version also includes expert-created resources such as word-net @xcite and jmdict @xcite , other crowdsourced resources such as wiktionary, knowledge obtained through games with a purpose such as verbosity, and automatically extracted knowledge (cf. @xcite ). knowledge facts in conceptnet are represented as triples, e.g. @xcite . the current version, conceptnet 5, comprises 37 relations, such as usedfor, isa, partof, or locatedat. conceptnet has been proven a useful resource of background knowledge for various nlp downstream tasks, and is thus widely used, e.g., for reading comprehension @xcite , machine comprehension @xcite , dialog modelling @xcite , argument classification @xcite , textual entailment @xcite , question answering @xcite or for explaining sentiment @xcite . as opposed to conventional knowledge bases such as nell @xcite , freebase @xcite , or yago @xcite , the nodes in conceptnet are represented as non-canonicalized, free-form text. this means that (i) concept nodes are not normalized: e.g. bake cake, bake cakes, baking cake, and baking cakes are represented as distinct nodes; likewise bin bag, binbag, bin bags, and bin-bag are separate nodes in conceptnet. (ii) concept nodes often consist of multi-word expressions, which can be very long and complex. often they consist of several nested phrase types, e.g., buying the ingredients of the recipe, or a friend was celebrating a birthday. (iii) since large parts of conceptnet have been crowdsourced, it contains noise (e.g., typos), uninformative concepts (e.g., there, it's), or very specific concepts (e.g., the second concept in the triple: [compute,hasprop,more complex than pencil]). these specific properties lead to a larger amount of nodes and a substantially sparser graph compared to conventional knowledge bases. this in turn is challenging for tasks such as knowledge base completion (cf. @xcite al. (2016); saito et al. (2018); bosselut et al. (2019); malaviya et al. (2020) ); the semantic representation of nodes and edges @xcite ; or the learning of new relations @xcite @xcite . moreover, non-canonicalized nodes become challenging when merging knowledge bases, as in @xcite , who introduce a graph database merging multiple hypernymy graphs extracted from conceptnet, dbpedia, webisagraph, wordnet, and wikipedia. they find that only 25% of the edges connect nodes from conceptnet to other databases, which can be traced back to the fact that conceptnet nodes are non-canonicalized, as opposed to common knowledge bases. finally, free-form concept nodes become problematic when we aim to project a conceptnet subgraph from natural language texts by mapping phrases from natural language text to nodes in con-ceptnet. in recent approaches, simple string matching has been applied to perform such a mapping (e.g. @xcite ; @xcite ). given the non-normalized nature of the concepts in con-ceptnet, this can, however, result in an incomplete and noisy mapping: e.g., if the word "brains" occurs in a text, it can be mapped to the concept-net node brains (which is connected by 131 edges within conceptnet), but not to brain (which is connected by 1799 edges). therefore, a lot of relational knowledge stored in conceptnet gets lost when mapping natural language text to concepts in conceptnet via string matching. moreover, since conceptnet contains many nodes that don't represent meaningful concepts (e.g. yes, there, it's, the), simple string matching can lead to the extraction of concepts that will most likely be useless for downstream applications. motivated by these observations, we built a concept extraction tool for conceptnet, coco-ex, which we present in this paper. coco-ex is a tool written in python 3.6 that selects meaningful concepts, possibly consisting of multiple tokens from natural language texts; it maps them to a collection of concept nodes in conceptnet, utilizing the maximum of relational information stored in the knowledge graph. it is thus perfectly suited for identifying and extracting concepts from natural language texts and mapping them to conceptnet, e.g., to project knowledge subgraphs from texts @xcite , or for detecting and classi-fying knowledge relations instantiated within texts @xcite . we describe our concept extraction tool coco-ex in section 2. in section 3 we evaluate the benefits of coco-ex in a practical application scenario, comparing it to simple string matching, by evaluating the retrieved concepts and their connectivity both automatically and manually. we conclude with a summary and results in section 4. ## coco-ex: extracting concepts from text and mapping them to conceptnet coco-ex is a pipeline implementation comprising several stages as shown in figure 1 . in step 1, we extract candidate phrases from a given text, which we preprocess in step 2. in step 3, we map the preprocessed phrases to conceptnet concepts, which we preprocess in the same manner: we first create a dictionary based on conceptnet, where we gather all concepts that are conceptually related (that is, referring to a similar or the same entity or event), but represented as distinct nodes. in this dictionary we then look up the preprocessed candidate phrases and get all conceptnet nodes which contain them. in order to avoid obtaining conceptually unrelated nodes, in step 4 we establish a method that allows us to filter out nodes that are not similar enough to the candidate phrase using similarity metrics and vector space representations. ## applications recent approaches that map natural language text to nodes in conceptnet apply simple string matching. @xcite for example use concept-net in order to retrieve multi-hop knowledge paths as background information for improving the task of question answering. they map concepts that appear in questions and answers from the two benchmark datasets, commonsenseqa @xcite and openbookqa @xcite , to conceptnet using plain string matching. irrespective of the question answering task, we want to evaluate the two methods of linking concepts from texts to conceptnet (plain string matching vs. coco-ex) by comparing the number of concepts that could be retrieved from conceptnet by both methods, respectively; and by evaluating the quality of the retrieved concepts, with regard to their coverage and informativity, as well as the amount of utilized relational knowledge from the conceptnet knowledge graph. we reimplement the string matching method and make it comparable to coco-ex by retrieving all noun phrases, verb phrases and adjective phrases and their nested phrases (as we do for coco-ex). additionally, as in coco-ex, we filter these phrases by removing articles, pronouns, adverbs, conjunctions, interjections and punctuation, and keep the original phrases and the chunked versions. the counts of concepts retrieved by simple string matching vs. using coco-ex are displayed in for evaluating concept quality, we set up a small annotation experiment where we provided our annotators with 50 questions randomly sampled from commonsenseqa and openbookqa. for each question, our annotators evaluated whether all meaningful concepts were extracted (coverage, in a binary setting (yes/no)); and if/how many informative (and thus, wanted) concepts are among the extracted concepts (which can be interpreted as reverse precision). 5 for each dataset, two annotators with linguistic background performed annotations. we measure annotator agreement in terms of cohen's kappa and achieve an agreement of 78%. remaining conflicts were resolved by an expert annotator (one of the authors). the number of concepts that could be accessed in conceptnet we evaluate automatically, in terms of the number of in-and outgoing edges connecting the node(s) which have been annotated as informative (wanted), identified by simple string matching vs. all nodes obtained by coco-ex through keys and values. the results of our manual evaluation experiment are displayed in next, we evaluate the informativeness of the extracted concepts. we find that the ratio between informative (wanted) and uninformative concepts (unwanted) is much better when using coco-ex opposed to simple string matching on both datasets (cf. ## conclusion in this paper we presented coco-ex, a tool for extracting concepts from texts and linking them to the conceptnet knowledge graph. as opposed to the common shortcut method of simply matching strings from texts to conceptnet nodes, coco-ex extracts meaningful concepts from texts and maps them to collections of concept nodes in con-ceptnet, which enables us to assess the maximum of relational information stored in the conceptnet knowledge graph. coco-ex takes into account that concepts in conceptnet are represented as noncanonicalized, free-form text and are often complex, noisy, uninformative, and/or over-specific. we evaluated coco-ex against the method of simple string matching, which confirmed our hypotheses that (i) coco-ex improves the precision of mapping by enabling the extraction of meaningful, important rather than overspecific or uninformative concepts, and (ii) allows to utilize the maximum of relational information stored in the knowledge graph, a step towards overcoming the well-known sparsity issue of commonsense knowledge graphs such as conceptnet.