Termout: a tool for the semi-automatic creation of term databases (2023)

Abstract: We propose a tool for the semi-automatic production of terminological databases, divided in the steps of corpus processing, terminology extraction, database population and management. With this tool it is possible to obtain a draft macrostructure (a lemma-list) and data for the microstructural level, such as grammatical (morphosyntactic patterns, gender, formation process) and semantic information (hypernyms, equivalence in another language, definitions and synonyms). In this paper we offer an overall description of the software and an evaluation of its performance, for which we used a linguistics corpus in English and Spanish.

## introduction terminology-related software has been available for more than sixty years @xcite , first promoted by the vienna school @xcite , but later gravitating towards computational linguistics @xcite . currently, the field of computer assisted terminology consists of a large variety of tools and methods, not only for term management @xcite , but also for terminology extraction @xcite , bilingual terminology alignment @xcite and information extraction @xcite , among other related areas. despite all the efforts, there is still ample room for improvement not only in each of the individual areas but in the field as a whole. there is, in fact, no tool yet available that can offer an integral solution for all the different problems terminologists face up to when creating terminological databases. in this context, we present termout foot_0 , a tool for automatising, at least partially, many of those tasks. the tool we present allows the user to process a specialised corpus and extract a draft macrostructure (a lemma-list) as well as data for the microstructural level, such as grammatical and semantic information. the possibilities of this software are very diverse and there is potential to benefit different professionals, foremost terminologists and lexicographers. users are able to generate raw material which they can later improve manually by adding or correcting data. if the raw material is of some quality, it is undoubtedly better to build from it than starting from scratch. it is hoped that, with the help of this system, larger databases will be possible, saving time otherwise spent in tedious mechanical tasks. the current implementation of the software is a web-based prototype that can perform the tasks of corpus processing (file uploading, conversion to plain-text format, language detection, pos-tagging and indexing), terminology extraction (with optional human supervision), information extraction (hypernymy, definitions, equivalence in another language, term variation, etc.) and database management (editing, storage, retrieval and import/export options in html, csv and tbx). in this paper we focus on the evaluation of the results of the main functions of the software: terminology and information extraction. to this end, we experimented with a linguistics corpus in english and spanish. as the evaluation shows, in its current state the software can already be useful for terminology processing. the structure of the paper is as follows. section 2 offers a brief overview of computational terminology techniques with emphasis in terminology extraction. in section 3 we present a description of the proposed method. finally, in section 4 we discuss about the advantages and disadvantages of the method as well as the challenges ahead. ## related work as mentioned in the introduction, the first efforts in initiating the computational treatment of terminology were by members of the vienna school, but then the field took a turn towards empiricism and began to import methods from computational linguistics @xcite . this change was accompanied by the emergence of new schools and theories, since data analysis lead to the admission of previously unrecognised phenomena, such as polysemy and term variation, which are less evident when relying only on introspection @xcite . automatic terminology extraction (ate), i.e. the separation of terms from the general vocabulary of a corpus @xcite , was an early and strong force of change in practical terminology. the topic attracted the attention of many researchers and a wide variety of ideas were proposed. in the early years, some systems used statistical measures to detect multi-word terms @xcite . others incorporated syntactic knowledge @xcite . others used statistics to calculate keywordness or weirdness, which means exploiting reference corpora by comparing the frequency of a term in a specialised corpus versus a corpus of general language @xcite @xcite . the most recent tendency in the literature is the application of machine learning techniques, especially deep neural networks @xcite @xcite . a drawback is however that their complexity makes them difficult to use, to interpret their results and, as rigouts terryn et al. (2020) point out, their behaviour is often unpredictable. aside from terminology extraction, other relevant subfields must be commented upon. one of those is bilingual terminology alignment using parallel, comparable or unrelated corpora @xcite @xcite @xcite . another subfield consists of the application of text mining techniques to obtain information about the terms from the corpus, which can be definitions @xcite @xcite ; hypernymy relations @xcite @xcite and term variants (synonyms) @xcite . the work by @xcite is a recent example of a combination of term and term-relation extraction. the number of relevant and recent publications to the different subareas of computer assisted terminology is on the thousands and still rising. however, the tendency seems to be analytical, i.e., to specialise in the different individual problems. as a consequence, not may proposals exist for the comprehensive solutions needed in practical terminology. there are some terminology extraction services (e.g. oneclick terms foot_1 or multiterm foot_2 ), but no software exists, commercial or public, that can accompany the user in the different steps of a terminology project. the software terminus foot_3 (cabré and nazar, 2011) was a first attempt in that direction, but it was not further developed. the present year 2023 is, of course, one of unprecedented changes in the field of a.i., and it is likely that new proposals for terminology processing will come from that side. in fact, some lexicographers (de schryver and joffe, 2023) have already started using chatgpt (openai, 2023) to automate different lexicographic tasks. again, the results of neural network algorithms using large language models are promising although unpredictable, as they occasionally "hallucinate" 5 . in this juncture, we still think that there is room for experimentation with alternative methods, especially if they do not entail great complexity, require massive computing power and are not own by large private corporations. ## conclusions and future work in this paper we presented a software for terminology processing that integrates a variety of tools for the creation of a terminology database, and we reported on a series of tests to evaluate its performance. as a first take after our assessment, we believe that despite some limitations, it could be useful for professional lexicographers and terminologists. in addition, we see also a possible application of the tool in the teaching of terminology, as students may use it to learn from practical experience in term database creation. as pointed out in the introduction, there is today no single software product that can provide solutions for the different tasks involved in termdatabase creation. the software products now available for terminology and lexicography processing are too time-consuming. we believe, thus, that a tool such as the one we propose is useful not only for the convenience of automation but also because a technical glossary should be created using specialised corpora as input. another advantage of the proposal is that it is based on simple algorithms, compared to those using neural networks. dispersion and co-occurrence statistics can be performed in relatively cheap hardware, although it is still necessary to improve computational efficiency to reduce processing times. the current implementation of termout is freely available and has no restrictions of any kind. this might become a problem if the number of users increases significantly, since we lack the necessary infrastructure (manpower, servers, etc.). if confronted with such scenario, we would be forced to explore alternatives for sustainability. it is also worth pointing out that the system uses no information external to the user's own corpus. we are, however, considering the possibility of changing this in future versions, in order to include the optional use of wikipedia or other external knowledge sources. another point to mention is that, currently, the system only operates with english and spanish text. however, the method is fundamentally based on statistical and language-agnostic algorithms, apart from the pos-tagger and the lexical patterns used in the extraction of hypernyms and definitions. we are, indeed, already attempting to adapt the system to different european languages. we are also exploring new ways to let the users acquire corpora, and this function will soon be available. one alternative is to provide the program with a url that contains links to other documents, and let the program decide which links are relevant. the other possibility is to upload a single document that the program will use to automatically extract, using text-similarity measures, a subset of similar documents from a larger general corpus such as the tenten corpora collection @xcite . this will offer the user the possibility of having the most laborious tasks of a terminology project fully automated.