Detecting Unassimilated Borrowings in S panish: A n Annotated Corpus and Approaches to Modeling (2022)

Abstract: This work presents a new resource for borrowing identification and analyzes the performance and errors of several models on this task. We introduce a new annotated corpus of Spanish newswire rich in unassimilated lexical borrowings—words from one language that are introduced into another without orthographic adaptation—and use it to evaluate how several sequence labeling models (CRF, BiLSTM-CRF, and Transformer-based models) perform. The corpus contains 370,000 tokens and is larger, more borrowing-dense, OOV-rich, and topic-varied than previous corpora available for this task. Our results show that a BiLSTM-CRF model fed with subword embeddings along with either Transformer-based embeddings pretrained on codeswitched data or a combination of contextualized word embeddings outperforms results obtained by a multilingual BERT-based model.

## introduction and related work lexical borrowing is the process of bringing words from one language into another @xcite . borrowings are a common source of out-ofvocabulary (oov) words, and the task of detecting borrowings has proven to be useful both for lexicographic purposes and for nlp downstream tasks such as parsing @xcite ), text-to-speech synthesis @xcite , and machine translation @xcite . recent work has approached the problem of extracting lexical borrowings in european languages such as german @xcite @xcite , italian @xcite , french @xcite , finnish @xcite , norwegian @xcite , and spanish @xcite , with a particular focus on english lexical borrowings (often called anglicisms). computational approaches to mixed-language data have traditionally framed the task of identifying the language of a word as a tagging problem, where every word in the sequence receives a language tag @xcite @xcite . as lexical borrowings can be single (e.g. app, online, smartphone) or multi-token (e.g. machine learning), they are a natural fit for chunking-style approaches. álvarez mellado (2020b) introduced chunking-based models for borrowing detection in spanish media which were later improved @xcite , producing an f1 score of 86.41. however, both the dataset and modeling approach used by álvarez mellado (2020a) had significant limitations. the dataset focused exclusively on a single source of news and consisted only of headlines. the number and variety of borrowings were limited, and there was a significant overlap in borrowings between the training set and the test set, which prevented assessment of whether the modeling approach was actually capable of generalizing to previously unseen borrowings. additionally, the best results were obtained by a crf model, and more sophisticated approaches were not explored. the contributions of this paper are a new corpus of spanish annotated with unassimilated lexical borrowings and a detailed analysis of the performance of several sequence-labeling models trained on this corpus. the models include a crf, transformer-based models, and a bilstm-crf with different word and subword embeddings (including contextualized embeddings, bpe embeddings, character embeddings, and embeddings pretrained on codeswitched data). the corpus contains 370,000 tokens and is larger and more topic-varied than previous resources. the test set was designed to be as difficult as possible; it covers sources and dates not seen in the training set, includes a high number of oov words (92% of the borrowings in the test set are oov) and is very borrowing-dense (20 borrowings per 1,000 tokens). the dataset we present is publicly available foot_0 and has been released under a cc by-nc-sa-4.0 license. this dataset was used for the adobo shared task on automatic detection of borrowings at iber-lef 2021 @xcite . ## error analysis we compared the different results produced by the best performing model of each type on the test set: ## discussion on the other hand, neural models (transformerbased and bilstm-crf) performed better. all of them performed better on the test set than on the development set, which shows good generalization ability. the bilstm-crf model fed with different combinations of transformer-based word embeddings and subword embeddings outperformed multilingual bert and spanish monolingual beto. the model fed with codeswitch, bpe, and character embeddings ranked first and was significantly better than the result obtained by the model fed with beto+bert, bpe, and character embeddings. our error analysis shows that recall was a weak point for all models we examined. among false negatives, upper-case borrowings (such as big data) and borrowings in sentence-initial position (in titlecase) were frequent, as they tend to be mistaken with named entities. this finding suggests that borrowings with capitalized initial should not be overlooked. similarly, words that exist both in english and spanish (like primer or red) are not rare and were also a common source of error. adding character embeddings produced a statistically significant improvement in recall, which opens the door to future work. concurrently with the work presented on this paper, de la rosa (2021) explored using supplementary training on intermediate labeled-data tasks (such as pos, ner, codeswitching and language identification) along with multilingual transformerbased models to the task of detecting borrowings. alternatively, @xcite used data augmentation to train a crf model for the same task. ## conclusion we have introduced a new corpus of spanish newswire annotated with unassimilated lexical borrowings. the test set has a high number of oov borrowings-92% of unique borrowings in the test set were not seen during training-and is more borrowing-dense and varied than resources previously available. we have used the dataset to explore several sequence labeling models (crf, bilstm-crf, and transformer-based models) for the task of extracting lexical borrowings in a high-oov setting. results show that a bilstm-crf model fed with transformer-based embeddings pretrained on codeswitched data along subword embeddings produced the best results (f1: 84.22, 84.06), followed by a combination of contextualized word embeddings and subword embeddings (f1: 83.63). these models outperformed prior models for this task (crf f1: 55.44) and multilingual transformerbased models (mbert f1: 82.02).