Deep Learning Methods for Identification of Multiword Flower and Plant Names (2023)

Abstract: Multiword Terms (MWTs) are domain-specific Multiword Expressions (MWE) where two or more lexemes converge to form a new unit of meaning. The task of processing MWTs is crucial in many Natural Language Processing (NLP) applications, including Machine Translation (MT) and terminology extraction. However, the automatic detection of those terms is a difficult task and more research is still required to give more insightful and useful results in this field. In this study, we seek to fill this gap using state-of-the-art transformer models. We evaluate both BERT like discriminative transformer models and generative pre-trained transformer (GPT) models on this task, and we show that discriminative models perform better than current GPT models in multi-word terms identification task in flower and plant names in English and Spanish languages. Best discriminate models perform 94.3127%, 82.1733% F1 scores in English and Spanish data, respectively while ChatGPT could only perform 63.3183% and 47.7925% respectively.

## introduction botany is a multidisciplinary field that encompasses different scientific disciplines such as genetics, ecology, physiology, biochemistry, architecture, gastronomy, commerce, art and design, etc. one of the key areas in botany is the study of flowers and plants. the market of flowers and plants is regarded as an economic engine of different economic and industrial activities. for this reason, its study and analysis are considered relevant in order to make this domain more accessible to all users, both at scientific and professional levels and also at layperson level, as flowers and plants have important national and international symbolisms and their roots are profoundly embedded in cultures and societies. the accurate identification and denomination of each plant is essential for the correct development and dissemination of science in all those multidisciplinary fields. it is also crucial for the correct communication of knowledge in different languages and also for the proper design of lexicographic resources and thesaurus. from the point of view of applied linguistics, the identification of names of flowers and plants is relevant to language professionals. from a terminological point of view, it helps in laying the basis of term coining processes and gives insights into the underlying mechanisms of term creation. translators also benefit from this information for the translation process. taking into consideration the quick development of nlp technologies and the importance of machine translation (mt) in the dissemination of knowledge and in building new resources, it is important to extend the studies and cover new areas of research, such as botany. the automatic identification of terms in this field helps in improving the quality of nlp applications, computer assisted translation tools and automatic translation tools @xcite as well as lexicon creation, acquisition of novel terms, text classification, text indexing, machine-assisted translation and other nlp tasks @xcite . for this reason, in this paper, we focus on the automatic extraction of flower and plant names, and we intend to address the shortcomings in this domain with the help of ai. specialised texts are rich with polylexical and monolexical terms @xcite . they are both essential for efficient scientific and technical communication. monolexical terms are formed of single lexical units, while polylexical terms are formed of more than one lexical unit. those last ones are also called multiword terms (mwt) and are defined as domain-specific multiword ex-pressions (mwe) @xcite where "two or more lexemes converge to form a new unit of meaning" @xcite . mwts are content-rich and are the most frequent type of lexical units in specialised discourse @xcite . in this context, a term is defined as the linguistic designation of specialised concepts @xcite . in terminographic and lexicographic studies, the detection and analysis of terms are considered key to comprehending and deciphering the semantic and conceptual relations that connect one lexical unit with the other to construct meaning @xcite properly. those semantic and conceptual relations also have an important role in the construction of specialised domains, ontologies and terminographic resources @xcite . moreover, they are also considered important for knowledge representation @xcite . however, the detection of terms in specialised domains is not an easy task. language users, such as professionals in specialised domains, terminologists and translators, need to acquire certain skills to be qualified to detect terms. the task is even more difficult in the cases of mwts, as language users find it more difficult to delineate where the mwt starts and where it ends in context. failure to detect terms leads to communicative problems, hinders the adequate construction of discourse, and provokes errors in translation processes. recently, automatic term recognition (atr) and automated term extraction (ate) have become more crucial to many nlp applications @xcite and @xcite . for example, those techniques are used for digital indexing, hypertext linking, text categorisation as well as in mt. moreover, the automatic detection of mwts at cross-linguistic level in specialised domains is also becoming more important and its study may help in different multidisciplinary research @xcite . for this reason, automatic translation of all types of texts is becoming an urgent priority in all fields, and more research is still required in order to obtain more insightful results. for this reason, and as a preliminary approach to the automatic extraction of mwts in specialised domains, in this study, we provide the results of a case-study for the atr and ate in the domain of botany in english and spanish. to the best of our knowledge, there are no programs that could automatically identify and retrieve those terms both as single-word terms and mwts in specialised domains, and no studies compare the already available resources in a comprehensive way. hence, this study seeks to fill in this gap and proposes a novel method based on transformer models @xcite for the automatic extraction of terms from the specialised domain of botany foot_0 . at the same time, it compares the results obtained by chatgpt to draw on conclusive results associated with their efficiency and whether they are promising to be used in further related research in different areas. the main contributions of this study are: the rest of the paper is structured as follows. section 2 outlines related work. section 3 describes the dataset used for our experiments, while section 4 presents the methodology. section 5 reports the evaluation results, and finally, section 6 summarises the conclusion of this study and suggests future research. ## related work in recent years, the computational treatment of mwes and mwts has received considerable attention, as it is essential for nlp applications, such as mt, indexing, terminology retrieval and translation technologies @xcite . they are considered relevant and highly important due to their ubiquity in both natural language and specialised language @xcite . @xcite highlight the importance of those terms in relation to nlp applications and propose including mwes and mwts in language technologies by means of type-based discovery, token-based identification, and mwe-aware language technology application models. studies such as @xcite show how the study of those terms may be relevant to detect synonym relations within distributional semantic models by using lexical substitution based and analogy based methods. others such as @xcite , show how the automatic detection of mwts is useful in tasks related to automatic formation of compound concepts within ontologies. within the field of language processing of specialised domains, previous research focused on the automatic detection of mwts in discourse. for example, @xcite used frequencies of occurrence of a text sequence in the corpus, combined with normalisation by lemmatising word by word in order to achieve the semi-automatic extraction of mwts in the domain of agricultural engineering. some authors such as @xcite used the approach of identifying candidate mwts in an automatically pos-tagged and lemmatised text, which is then weighted with the c-nc value in the domains of history of art and in legal domains. on the other hand, authors like @xcite centred their research on the automatic extraction of mwts from parallel corpora by using the compositional with word embedding projection (cmwep) approach in the domain of medicine. transformers based models have been used in previous research to detect mwts, such as @xcite . their study focuses on detecting mwt for filtering and indexing tasks. @xcite apply mwt extraction in irish, but they show that large pre-trained models struggle to perform better in a low-resource setting. @xcite employed transformers to evaluate mwt extraction in their own private dataset, and they could show that transformers were able to outperform the existed state-of-the-art results by greater margins. studies have been limited because of the lack of annotated datasets, but @xcite proposes an unsupervised way of annotations to combine with transformers to extract mwe. other studies, such as @xcite , also use transformer-based approaches to multilingual term extraction across domains. however, they believe more research based on neural models is still required to obtain more results. in this research, we combine the approaches to employ those methods on mwts in the specialised domain of botany, more specifically, on flower and plant names. in this case study, and since both mwts detection and ner tasks are about token classification, they can be modelled by using similar models. for this reason, we are using a set of models which are used in ner for the mwt detection task, too @xcite . we seek to fill the gap by empirically evaluating multiple transformers in the task of mwt identification and extraction in the domain of botany in english and spanish. ## data for the implementation of this case study, we extracted terms from different texts in english and spanish corpora. with respect to the english corpora used, firstly we compiled a corpus from the encyclopaedia of flowers and plants available in a digitalised editable format, published by the american horticultural society @xcite . this encyclopedia contains more than 8,000 plants and 4,000 photographs and is organised in different sections to serve all users. the first section provides information on how to use the book and explains the origin of the names of plants and their etymological origins. in the second section, it has a comprehensive plant catalogue which explains the type of plants, including information on their plant life cycle, their shape and size, and whether they are trees, shrubs, roses, bulbs, etc., or if they are water or rock plants, etc. finally, the encyclopedia offers a plant dictionary followed by an index of common names and a glossary of terms. the advantage of annotating this encyclopedia is that the scientific names will help as a common link in all languages written with the latin alphabet. it also has an important potential at cross-linguistic level in the field of botany. the data was preprocessed by annotating the proper names and their condition of being mwts or single-word terms. for example, the scientific name cynoglossum amabile is annotated as mwt, while the vernacular name of this flower, firmament, is annotated as a single word term. apart from the encyclopaedia of plants and flowers @xcite , we also compiled a corpus of other resources related to botany in en-glish. it consists of 437,663 words. some of the texts are monographs, others are journal articles, and some texts are retrieved from other online resources. those resources are @xcite , maghiar et al. (2021), @xcite , @xcite , @xcite . all those resources contained lists of names of plants and flowers, which were also annotated, and they all had relevant rich contexts on which we could rely to extract terms. with respect to the spanish dataset, we followed the same annotation criteria implemented in annotating the english dataset. the dataset in spanish consisted of a list of flower and plant names provided in selected monographs and glossaries. above all, we used books and articles in the domain of botany and botanical glossaries, such as the glossaries provided in los árboles en españa (de lorenzo cáceres, 1999), biología de la conservación de plantas en sierra nevada @xcite as well as the glossary of scientific names of plants and their vernacular names provided by the entomological museum in leon on the bio-nica webpage 3 . in order to obtain more context-rich corpora, we also used other texts in spanish, such as @xcite , guadalupe et al. (1985), blanca lópez and loépez onieva (2002), gonzáles et al. (2020), montserrat (1960), ar-mas, gómez garcía (2004) and the vademecum colombiano de plantas medicinales (de salud y protección social de colombia, 2008). for example, los árboles de españa includes a classification of trees in spain. above all, it describes their varieties, form and cultivation process and needs. it has glossaries with scientific names and family names. other scientific articles, such as biología de la conservación de plantas en sierra nevada contain tables with names of endemic plants and flowers in the national park of sierra nevada. the variety of resources allows for the list to be more inclusive. the same applies to the book vademecum colombiano de plantas medicinales (de salud y protección social de @xcite as it includes varieties of terms more specific to a concrete geographical area, in this case, in colombia. ## methodology with the emergence of transformers @xcite and large language models (llms), state-of-the-art results of many nlp tasks had pushed their existing boundaries with decent margins. attention mechanism @xcite played a major part in these language models, which could provide a contextual understanding of the left and right sides of a text sequence at once. bert @xcite was a prominent milestone in llms which is a variant from initial transformers architecture. similar llm architectures have emerged with the differences of having different learning objectives as well as using different datasets. having this motivation, we conduct our experiments on multiple popular transformer models to evaluate their performance on mwt extraction in flower and plant names. since this is a token-level classification task, we use macro averaged precision, recall and f1 score as our evaluation metrics. (3) the rest of this section discusses the models we used, with the categorisation of discriminative and generative models. discriminative models the original transformer @xcite consisted of two main parts; encoder and decoder. bert model can be described as a stack of encoders which has been pre-trained on masked language modelling primary objective function. generally, these discriminative models accept a sequence of tokens, and the output layer of the model can be configured such that the model is able to finetune on a downstream task such as classification. the general architecture of bert models on token-level classification tasks is shown in figure 1 . we used a mix of popular discriminative transformer models in our experiments with their variants as listed in for the experiments on the english corpus, we used all the models listed in since not all these models have multilingual capabilities, we used bert-base-multilingual-uncased, bert-base-multilingual-cased, xlm-roberta-base, xlm-roberta-large for spanish experiments. we used model training configurations shown in generative models these models took a different approach to bert-like models, by changing the objective function to predict only the next word. this variant of transformers leverages the decoder part of the initial transformer architecture, and a generative pretrained transformer (gpt) can be introduced as a stack of decoders in terms of the architecture. chatgpt foot_2 uses generative transformer architecture, and it has provided highly competitive results in conversational systems while it is capable of applying to non-conversational tasks like multi-word-terms identification. chatgpt is a human-like chatbot in which we can input a sequence of text and get an output accordingly. it is known that chatgpt produces different results for different inputs. therefore, finding the optimal prompt for better results is always encouraged. we tried multiple prompts to retrieve bio tags for mwts in the text directly, but this did not show good results since the model produced more tags than the number of tokens in the input text. after a couple of iterations, we settled for; -find whether there is a multi-word expression flower or plant name in the text delimited by "'if there is no multi-word expression found in the given text; just tell 'no' -if you find a multiword expression in the given text; say yes and then give the multiword flower or plant name for example; yes -'name' text : "'{sentence}"' as shown in the prompt, if there is no mwt in the given sentence, we retrieve 'no' as the output and if there is, we retrieve 'yes -{name}' as the output. in both cases, we post process the data using regular expressions to generate bio tags based on the chatgpt output. finally, we use the generated bio tags to evaluate the results. ## conclusions detection of terms is an important research area for many nlp applications and is considered a challenging task, above all when the task involves mwts besides single-word terms. the automatic identification of terms helps in improving the quality of nlp applications, such as computer assisted translation tools and automatic translation tools, as well as lexicon creation, knowledge representation, ontology building, text classification, text indexing, creation of terminographic resources and other nlp tasks. those nlp applications need to be developed in all fields of study in order to widen the scope of nlp applications and be more inclusive. botany is no exception. moreover, there is a need to fill this void as botany is one of the important interdisciplinary areas which is intertwined with many other activities and areas of research. we empirically show that general transformer models can produce very good results in multiword term identification of flower and plant names tasks for both english and spanish. further, we comparatively show that chatgpt is not performing as well as the other discriminative models. the results obtained from this experiment can be relevant for the comprehension of term formation processes and may be helpful for the design of new lexicographic resources related to new term formation in languages with low resources. in future research, we would like to explore more specialised domains and involve more languages and bigger datasets, and extend the study to multilingual parallel corpora.