Leveraging Large Language Models to Extract Terminology (2023)

Abstract: Large Language Models (LLMs) have brought us efficient tools for various natural language processing (NLP) tasks. This paper explores the application of LLMs for extracting domain-specific terms from textual data. We will present the advantages and limitations of using LLMs for this task and will highlight the significant improvements they offer over traditional terminology extraction methods such as rule-based and statistical approaches.

## introduction in the context of localisation projects, extraction of terminology is more often than not the first linguistic task we perform. this is especially true as the type of localisation projects handled by professional services tend to be higher in complexity and require more subject matter expertise (sme). the more general type of content is now handled with robust results directly with the use of neural network machine translation (mt) or llms. the importance of technical terminology extraction in various fields has long been established. having a solid glossary of terms is helpful to have a basis to agree on with the relevant stakeholders ahead of doing the translation. it is a helpful tool for the linguists and it allows to control the quality of the localised document via the use of term base and computer assisted tools (cat). it contributes to information retrieval and knowledge management. in the context of other natural language processing applications, term extraction is useful for text summarisation, semantic understanding and other usage. this paper explores the potential of llms for terminology extraction, providing an analysis of their effectiveness in comparison to traditional methods. ## context llms have emerged as powerful tools in part because of their ability to capture complex language patterns and context. the architecture and training process of llms such as gpt-4 by openai, bert, and others helped us identify their potential for extracting terms effectively. we considered different methods for terminology extraction using llms. contextual word embeddings -using pre-trained llms to generate contextualised word embeddings to capture word semantics based on surrounding text and enable better term identification. sequence labeling tasks ran with llms such as named entity recognition (ner) to identify and extract domain-specific terms. domain-specific fine-tuning to adapt pre-trained models to domain-specific corpora to improve term extraction accuracy within specialised texts. ## goal traditionally, in the context of a language service provider, the extraction of terms from textual data was done using a cat feature. we use phrase inbuilt term extractor in our regular workflow and for the purpose of this comparison. the list of terms is then cleaned-up manually by a linguist to eliminate words that are not useful and to then add some terminology present in the data but omitted by the automatic extraction. this step is lengthy and requires the skill of a specialised resource such as a terminologist or a linguist with the relevant 1 58 domain expertise. the main goal of this paper was to find out if the use of llms could lower the human effort needed in extracting terminology. ## hypothesis statistical tools are frequency-based. in this experiment, a word should appear either at least once or at least two times in the text in order to be extracted. with a minimal frequency of 1, this results in a big number of false candidates. their clean-up might take significant time. on the other hand, the tool with a minimal frequency of two generates less noise but it also omits numerous valid candidates. it is expected that llm will outperform the other two methods by generating the highest number of valid terms with a minimum of noise therefore reducing the human effort needed in the glossary creation workflow. ## methodology for this paper we are comparing a llm (gpt-4 by openai) with a statistical model (phrase term extractor). phrase was used in two modes: the terms were extracted with phrase 411, phrase 421 and gpt-4 from three text samples and covering three domains: legal, medical, technical. then, we also performed a manual extraction of the terms for each text sample and domain using a human specialist. this human extraction serves as a reference to identify the correct list of terms that should feature in the glossary. for each term extraction method, the number of correctly extracted terms (defined as a term also present in the list created by the human specialist) was divided by the total number of candidates proposed by the tool. the tools were compared based on the obtained accuracy score. prompt for the llm extraction: @xmath0: " + domain+") from the provided "+language+'' text. \ntext: "+text+"\nplease extract only terms that are directly related to "+domain + " and are present in the text (do not make up new terms, only extract existing ones in the text). keep the same grammatical form and capitalization as these terms have in the text. do not enumerate them in the output.`6