Graph Based Automatic Domain Term Extraction (2020)

Abstract: We present a Graph Based Approach to automatically extract domain specific terms from technical domains like Biochemistry, Communication, Computer Science and Law. Our approach is similar to TextRank with an extra post-processing step to reduce the noise. We performed our experiments on the mentioned domains provided by ICON TermTraction - 2020 shared task. Presented precision, recall and f1-score for all experiments. Further, it is observed that our method gives promising results without much noise in domain terms.

## introduction domain term, is a word or group of words, carrying a special, possibly complex, conceptual meaning, within a specific domain or subject field or community. because of their low ambiguity and high specificity, these words are also particularly useful to conceptualize a knowledge subject. for each domain, there is an essential need to identify the domain-specific terms as they play a vital role in many natural language processing applications such as neural machine translation(nmt) @xcite , information retrieval @xcite , information extraction @xcite , text classification @xcite , etc. the task of automatically extracting domain specific terms from a given text of a certain academic or technical domain, is known as automatic technical domain term extraction. this is a predominant task in nlp. extracted terms can be useful in more complex tasks such as nmt @xcite , ontology construction @xcite , domain identification, semantic search, question-answering, word sense induction, etc. several research works have been carried out to extract domain-specific terms. most of them are either rule based @xcite or dictionary based @xcite . also, there are few term extraction techniques which uses machine learning algorithms @xcite , thereby demanding a huge labelled corpus. but, the existence of domain term annotated corpus is very rare in case of many domains. also, the availability of such huge labelled corpus is almost nil for low resource languages. therefore, our domain term extraction approach is motivated more by unsupervised than supervised strategies. hence, we used a graph based approach which extract not only unigrams but also collocations. collocations are expressions of multiple words which commonly co-occur in a given context than its individual word parts. these are the phrases that express stronger sentiment which can be easily captured with bigram, trigram and so on. hence our approach is not restricted to just unigram extraction, it also considers multi-word domain terms foot_0 . to demonstrate the performance of our approaches, we used data provided by icon termtraction -2020 shared task. the discussion and analysis on the performance of the approaches are mentioned in section 4. in this paper we performed our experiments on four domains in english. we are still in the process of exploring the possible unsupervised approaches to extract domain terms in a flexible and intuitive manner. further, it can be applicable to all domains irrespective of any language. ## background & motivation there have been a lot of studies regarding the automatic domain term extraction. but very less work carried on unsupervised approaches that too on technical domains like, computer science, chemistry, etc. automatic domain term extraction is a categorization or classification task where terms are categorized into a set of predefined domains @xcite . further this task is used in many nlp applications such as domain ontology construction and nmt with domain terminology by injecting custom terminology into neural machine translation at run time @xcite . in order to effectively make use of domain terms in various applications, an ultimate approach which is fast, flexible and reliable is highly required. in spite of many contributions on automatic domain term extraction, very limited study is done so far using unsupervised approaches. most of the explorations are done using supervised methods such as focusing on various features like contextual, domain concepts and topics to measure the semantic similarity of terms to assign domain concepts to domain-specific terms @xcite . similarly, another experimental evaluation is done by comparing the performance of two existing approaches for automatic domain term recognition: machine learning method and voting algorithm @xcite . but, the major well known drawback with these supervised algorithms is that, they demand huge labelled training data. therefore, an unsupervised algorithm is more preferable. most of such unsupervised approaches extract domain-specific terms using frequency count @xcite . the basic underlying idea is that, in a particular domain, domain-specific terms occur with markedly higher frequency than they do in other domains, similar to term frequency patterns captured by tf-idf. apart from these methods, another experimental approach for domain term extraction is executed using deep learning where possible term spans within a fixed length in the sentence, is considered to predict a domain term. deep learning technique is proven to yield high recall and a comparable precision on term extraction task @xcite . however, for training such deep learning models, an enormous training data is mandatory. conversely, availability of this sort of corpus for diverse multilingual domain is very scarce. our goal is to formulate a flexible and reliable approach which successfully extracts domain terms irrespective of the domain and language of a document. accordingly, we present experiments which extract domain terms in a given document disregarding of any domain without having a dependency on labelled corpus. our approach , textrank is an inspiration from pagerank algorithm @xcite . @xcite introduced tex-trank a graph-based ranking model for text processing, and showed how this model can be successfully used in natural language applications. in particular keyword and sentence extraction. we reimplemented textrank from @xcite for extracting domain-specific terms from technical domains like, computer science , chemistry, etc by handling noise generated in the outputs. textrank is merely a graph based approach where words are considered as nodes and the relation between them as edges. based on syntactic filters, such as parts of speech (pos) tags, words are selected as nodes and relation between the words is based on word co-occurrences . a window size (n) is assumed for word co-occurrences. for all words that fall in a particular window, an edge is allocated, resulting into a graph of nodes and edges. an undirected and unweighted graph is considered in our approach. this is further discussed in detail in section 3. ## approach a graph-based ranking algorithm is a way of deciding on the importance of a vertex within a graph by taking into account global information recursively computed from the entire graph, rather than relying only on local vertex-specific information. applying a similar line of thinking to lexical or semantic graphs extracted from natural language texts, results in a graph-based ranking model that can be applied to a variety of natural language processing applications, where knowledge drawn from an entire text is used in making local ranking/selection decisions. we implemented the textrank algorithm described in @xcite . @xcite described usage of textrank for keyword extraction and sentence extraction but we adopted that technique for automatic domain term extraction by doing few modifications in syntactic filters, and adding a post processing step for noise removal using top 1000 common words in english from wikipedia. we used noun, proper nouns, adjectives as syntactic filters, window size (@xmath0, recall and f1 score. textrank is completely unsupervised, and unlike other supervised systems, it relies completely on information drawn from the text itself, which makes it easily portable to other domains, and languages. intuitively, textrank works well because it does not only rely on the local context of a text unit (vertex), but rather it takes into account information recursively drawn from the entire text (graph). through the graphs it builds on texts, textrank identifies connections between various entities in a text, and implements the concept of recommendation. a text unit recommends other related text units, and the strength of the recommendation is recursively computed based on the importance of the units making the recommendation. the brief explanation of each step in text rank algorithms is given as follows, firstly the text is tokenized, and annotated with part of speech tags, for this task we used spacy @xcite . to evade the excessive growth of graph size by including all possible combinations of sequences consisting of more than one lexical unit(word), we consider only single words as nodes to build the graph, with multi-word domain terms being eventually reconstructed in the postprocessing step. following, all words that pass the syntactic filter are added to the graph, and an edge added between those words that co-occur within a window of n words. after the graph construction (undirected , unweighted graph), the value of each vertex is set to 1. next, the ranking algorithm will run on the graph for several iterations until it converges usually for 20-30 iterations, at a threshold of 0.0001 @xcite . once a final score is achieved for each vertex (for each word )in the graph, vertices are sorted in reversed order of their score then the top k words in the ranking are retained for post-processing. in our experiments we take top @xmath1. in post processing step along with constructing n-grams we reduce the noise using top 1000 english words from wikipedia. to construct n-grams from unigrams we get, first annotate the text with technical domain terms we get then retrieve the terms which occur side by side in the text. ## experiments & results we evaluate our approach on data provided by icon termtraction -2020 shared task for four domains, biochemistry, communication , computer science and law. each domain contains files with text related to that domain. in each domain we have minimum 10 files and maximum 16 files. we did experiments on individual files for the respective domain. as our approach comes under unsuper ## conclusion & future work in this paper we showed a graph based approach for automatic technical domain term extraction for four technical domains(biochemistry, computer science, communication,law). our approach showed high recall in all cases for all domains, from this we can conclude that our model has the power to extract domain-specific terms without much noise. our approach doesn't depend on any language dependant resources except pos tagger, hence we can adopt this method for any language. we plan to extend our approach to possible indian languages like telugu, hindi etc. and we would like to improve this approach with different word relationships(edge relations like we did using cooccurrence of words in given window). one approach for that is like using similarity of words using word2vec etc.