Unsupervised Technical Domain Terms Extraction using Term Extractor (2020)

Abstract: Terminology extraction, also known as term extraction, is a subtask of information extraction. The goal of terminology extraction is to extract relevant words or phrases from a given corpus automatically. This paper focuses on the unsupervised automated domain term extraction method that considers chunking, preprocessing, and ranking domain-specific terms using relevance and cohesion functions for ICON 2020 shared task 2: TermTraction.

## introduction the aim of automatic term extraction (ate) is to extract terms such as words, phrases, or multi-word expressions from the given corpus. ate is widely used in many nlp tasks, such as machine translation, summarization, clustering the documents, and information retrieval. unsupervised algorithms for domain term extraction are not labeled and trained on the corpus and do not have any pre-defined rules or dictionaries. they often use statistical information from the text. most of these algorithms use stop word lists and can be applied to any text datasets. the standard unsupervised automated term extraction pipeline consists of • simple rules: using chunking or pos tagging to extract noun phrases for multi-word extraction. currently, there are many methods for automatic term recognition. evans and lefferts (1995) used tf-idf measure for term extraction. @xcite used domain consensus which is designed to recognize the terms uniformly distributed over the whole corpus. the most popular method c-value @xcite is also a statistical measure that extracts a term based on the term's frequency, length of the term, and the set of the candidates that enclose the term such that the term is in their substring. @xcite proposed the method called basic, which is a modification of the c-value for recognizing terms of average specificity. the successor of c-value statistic called the nc value @xcite considered scored the term based on the condition if it exists in a group of common words or if it contains nouns, verbs, or adjectives that immediately precede or follow the term. the methods proposed by @xcite ; @xcite ; @xcite are based on extracting the terms of a text by considering the frequency of occurrence of terms in the general domain. a detailed survey of the existing automated term extraction algorithms and their evaluation are presented in papers by @xcite ; @xcite in this paper, we used the term extractor algorithm @xcite present in the pyate 1 library for domain term extraction. the term extractor algorithm is developed initially for ontology extraction from large corpora. it uses domain pertinence/relevance, domain consensus, and lexical cohesion for extracting terms. a detailed description of the modules is given in the next section. the paper is organized as follows. section 2 gives a detailed description of the term extraction algorithm used. section 3 gives information about the datasets used and results. section 4 concludes the paper. ## our approach in this section, we describe in detail the methods used in the term extractor algorithm. initially, termextractor performs chunking and proper name recognition and then extracts structures based on linguistic rules and patterns, including stop words, detection of misspellings, and acronyms. the extraction algorithm uses domain pertinence, domain cohesion, and lexical cohesion to decide if a term is considered a domain term. domain pertinence, or domain relevance (dr), requires a contrastive corpus and compares a candidate's occurrence in the documents belonging to the target domain to its occurrence in other domains, but the measure only depends on the contrastive domain where the candidate has the highest frequency. the domain pertinence is based on a simple formula, where tf i is the frequency of the candidate term in the input domain-specific document collection and max j (tf j ) is the general corpus domain, where the candidate has the highest frequency, and d i is the domain in consideration. domain consensus (dc) assumes that several documents represent a domain. it measures the extent to which the candidate is evenly distributed on these documents by considering normalized term frequencies (φ), here, we assume k distinct documents for the domain d i . lexical cohesion involves the choice of vocabulary. it is concerned with the relationship that exists between lexical items in a text, such as words and phrases. it compares the in-term distribution of words that make up a term with their out-of-term distribution. where n is the number of documents in which the term t occurs. the final weight of a term is computed as a weighted average of the three filters above, ## conclusion for domain term extraction from technical domains like bio-chemistry, law, computer-science, and communication, we used the term extractor method from pyate library for obtaining technical terms. the term extractor method uses keywords from the general corpora, and it considers domain pertinence, domain cohesion, and lexical cohesion methods for extracting domain terms in the given corpus. as mentioned above, it did not give preference to capitalized terms and did not consider some compound words. so we have to work towards better methods that consider capitalization, better formation of compound words for the more reliable performance of the automated domain term extractor.