A Survey on Ontology Enrichment from Text (2019)

Abstract: Increased internet bandwidth at low cost is leading to the creation of large volumes of unstructured data. This data explosion opens up opportunities for the creation of a variety of data-driven intelligent systems, such as the Semantic Web. Ontologies form one of the most crucial layers of semantic web, and the extraction and enrichment of ontologies given this data explosion becomes an inevitable research problem. In this paper, we survey the literature on semi-automatic and automatic ontology extraction and enrichment and classify them into four broad categories based on the approach. Then, we proceed to narrow down four algorithms from each of these categories, implement and analytically compare them based on parameters like context relevance, efficiency and precision. Lastly, we propose a Long Short Term Memory Networks (LSTM) based deep learning approach to try and overcome the gaps identified in these approaches.

## introduction there has been an explosion of data on the internet in the past few years, primarily caused by the drastic increase in the number of internet users over the years. about 90% of the data on internet has been created since 2016, mainly because of the massive increase in the user base and machine to machine communication. data is defined as unprocessed facts and figures that do not contain any added interpretation or analysis. information is interpretation of structured or unstructured data so that it holds meaning. knowledge is processed information, experience, and insight combined such that it is beneficial to the end user 1 . web pages, the primary source of knowledge on the world wide web (www) are primarily 1 https://tinyurl.com/datainfknowledge text documents annotated using hypertext markup language (html). lack of semantic markup of pages can result in irrelevant search results. the semantic web foot_0 provides a format or structure to machines to understand the meaning of the web page data rather relying on html markup, to make web intelligent and intuitive to user's queries. the semantic web includes data-centric publishing languages, including rdf (resource description framework -the data modeling language for the semantic web), sparql (sparql protocol and rdf query language for semantic web) and owl (web ontology language -schema language, or knowledge representation language, of the semantic web), which allows meaning and structure to be added to content in a machine-readable format. owl foot_1 allows definition of concepts composably, i.e. in such a way that it allows the reuse of concepts and relationships. given the amount of information being extracted from the data generated on a regular basis in various domains, it becomes essential for it to be stored in the form of knowledge in ontologies. however, the knowledge stored in ontologies is rarely static. like all other knowledge structures, its vital for ontologies to be enriched with time so as to improve the quality of search results. given recent advances in the fields of artificial intelligence and machine learning, as well as increased data processing capabilities with increase in compute power, newer, better and more accurate ways of extracting and enriching ontologies from text are now possible. ontology extraction from text has primarily been at lower layers in ontology "layer cake" @xcite . a pre-existing seed ontology created manually or through learning needs enrichment. ontology enrichment @xcite is population, updation, and adaptation (noy and klein, 2004) of concepts, relations and rules. in the context of this paper, we assume a pre-existing seed ontology that is enriched by learning (semi-automatic or automatic) from text. the survey in this paper attempts to address the following important research questions: we did not focus on the other knowledge representation methods such as knowledge graphs, frames, semantic nets and others in the survey considering the extensivity of ontology research and the generalizability of research trends to other knowledge representation methods. the further sections of the document contain our literature survey approach for identifying the state-of-the-art in section 2; we explain the broad genres identified in the ontology extraction in section 3; we proceed with a critical analysis of the major approaches through the years, by analyzing the algorithms on context relevance, efficiency and precision in section 4; we propose a deep learning based methodology (lstm -long short term memory) to possibly overcome the gaps in the ontology enrichment in section 5 and finally end with a conclusion summarizing our observations. ## approach for literature review we started the review on ontology learning from text before focusing on enrichment. research on ontology learning from text started in 1995 @xcite but still continues to be an area of interest. in the last two decades, there have been 20 survey papers on ontology tools, learning, evolution, construction, enrichment, change, generation, population, and matching with text. the large count of survey papers indicates the growing interest among researchers and changing research approaches in ontology learning. we classified these survey papers 4 on the basis of text format (structured or unstructured), evaluation methods, ontology layer cake, ai techniques, level of automation, etc. most survey papers recommended human intervention, continued automation, gold standards and graphical interfaces for improved quality, expressiveness and scalability. while the to ontology evolution, enrichment, updation, refinement, maintain, etc. the papers on "crud" operation dealt with creation, updation and deletion of redundant concepts and relations as well. for further analysis, the papers on "update" and "crud" on ontology were clustered into 4 categories based on the approach used for enrichment. 1. similarity based clustering algorithms 2. set theoretic based algorithms 3. web corpus based algorithms 4. deep learning based algorithms figure 2: ontology learning categories ## categories in ontology enrichment we proceeded with a review of the 23 shortlisted papers of the 4 categories in ontology enrichment. ## from clustering to learning algorithms: an in-depth analysis after categorizing our research set of 23 papers into 4 categories, the seminal algorithms from each category are listed below: ## discussion the guided agglomerative clustering algorithms used wordnet and hearst patterns on corpora to build its hypernym oracle. while wordnet is able to provide hypernyms for common nouns, it cannot handle proper nouns and phrases, which are often the primary focus while enriching domain specific ontologies. using hearst patterns is inefficient too and results in a lot of noise, due to pattern being matching being purely syntactic with no attention paid to context. though c-pankow is able to improve on precision, efficiency and also partially address the issue of context relevance (using a mixture of frequency-based mapping and document similarity scores), it uses naive syntactic pattern matching which results in selecting irrelevant terms and dropping relevant ones. the dynamo-mas algorithm, despite solving disambiguation and having better precision, has serious limitations like data sparsity and unscalability. fca, which uses pseudo-syntactic dependencies, was found to have better precision than both clustering and c-pankow. but construction of a concept hierarchy is time inefficient, which is where it loses out to c-pankow. the word2vec algorithm was able to improve the problems of efficiency, precision and data sparsity by using word embeddings and skip-grams, and was found to outperform previously mentioned algorithms. however, this algorithm also suffers from some shortcomings like the inability to handle previously unencountered words, selecting of too similar terms, scalability issues due to manual intervention etc. we used the 'information security' ontology @xcite based on iso 27001 for comparing the algorithms. figure 4 shows comparison of the metrics across these algorithms. all these algorithms have an area of improvement when the current concept and its pronouns are being extracted from text. in the previous approaches, the attributes and relations were mapped to the pronouns and not the concept itself. to address this gap, the algorithm needs to retain in memory the concept being extracted, instead of using naive pattern-matching approaches. also, the analysis of the shortlisted research papers in each category state the declining research on clustering and set-theoretic algorithms, and an increasing in research of learning algorithms. ## conclusion we started this survey paper by describing the need for enrichment of ontologies. we proceeded to survey the existing domain literature in the field of ontology learning from text and got a subset of 166 research papers and 20 survey papers. from shortlisted 101 papers, we narrowed down to the 23 most relevant research papers. these 23 papers were classified into four categories based on the approach used for ontology enrichment, namely clustering, set-theoretic, web corpus-based and learning-based algorithms. we selected a seminal paper from each category, based on criteria like the date of publication, the number of citations, relevance to our end goal etc. and then described the approach of the algorithms. next, we compared algorithms performance (context relevance, precision and efficiency) on the enrichment of information security ontology. we found that with each trend, some of the gaps were overcome but there still remained the problem of retaining concepts to improve relevance in a scalable manner. we proposed lstms as a possible solution for concept retention, since they use a memory state to partially remember/forget concepts over long periods of time as require. in future, we plan on implementing the proposed lstm model to improve precision and efficiency of the state-of-the-art. we also plan to validate further with complex ontologies, and extend our concept enrichment model to the addition of instances for building knowledge base.