BERT ie Bott’s Every Flavor Labels: A Tasty Introduction to Semantic Role Labeling for G alician (2023)

Abstract: In this paper, we leverage existing corpora, WordNet, and dependency parsing to build the first Galician dataset for training semantic role labeling systems in an effort to expand available NLP resources. Additionally, we introduce verb indexing, a new pre-processing method, which helps increase the performance when semantically parsing highly-complex sentences. We use transfer-learning to test both the resource and the verb indexing method. Our results show that the effects of verb indexing were amplified in scenarios where the model was both pre-trained and fine-tuned on datasets utilizing the method, but improvements are also noticeable when only used during fine-tuning. The best-performing Galician SRL model achieved an f1 score of 0.74, introducing a baseline for future Galician SRL systems. We also tested our method on Spanish where we achieved an f1 score of 0.83, outperforming the baseline set by the 2009 CoNLL Shared Task by 0.025 showing the merits of our verb indexing method for pre-processing.

## introduction the goal of semantic role labeling (srl), also known as shallow semantic parsing, is to define the semantic roles of any given word in a sentence in regard to a specific noun or verb; the focus of this study specifically is srl in regard to verbal predicates @xcite . focus is placed on directly mapping a predicate to its role and recipients @xcite . computational systems are then able to use these roles to establish a shallow semantic representation of a string of words, allowing for inferences which otherwise would be lost to the system @xcite . as these roles seek to remove ambiguity as much as possible, they are specific to each verb's sense meaning, though tendencies can be identified. in general, arg0 represents that which causes an event or a change of state in another participant with respect to the verbal root, while arg1 represents that which is undergoing a change of state or causally affected by another participant @xcite . unfortunately, corpora including this shallow semantic information are extremely few and far between, with .1% of the languages of the world having the necessary language-specific information readily available and only .3% having access to the translated version of english propbank, the most extensive srl resource currently available @xcite @xcite . this lack of data makes it nearly impossible for srl systems to be developed for the vast majority of languages. the main purpose of this work was to produce a dataset and initial baseline for galician srl, a lowresource western ibero-romance language with approximately 4 million speakers worldwide @xcite . despite its large number of speakers, nlp tools and resources for galician are extremely limited and it remains classified as a low-resource language for most nlp tasks; it is our hope that the release of this dataset will help to improve and expand resources for galician (language archives services, university of british columbia, 2023; @xcite . several transfer-learning experiments were also performed to determine the efficacy of leveraging existing high-resource srl systems in the development of languages with lesser amounts of data available as it has seen success previously @xcite @xcite . ## related work recently, methods such as descriptive semantic role labeling (dsrl) and the inclusion of syntactic information have been proposed in an effort to increase interpretability, flexibility, and overall performance of srl systems @xcite . dsrl seeks to shift away from traditional argument labeling in favor of labeling through descriptions using natural language @xcite . both methods have seen great success, performing on par with or beating current state-ofthe-art results in high-resource contexts for english and chinese @xcite . despite their success, a choice to utilize standard methods of argument labeling was made due to the extremely low-resource status held by galician and the overall current lack of resources. ## method development and training scripts can be found on the project github foot_0 while the fully trained models and datasets can be accessed via huggingface foot_1 . ## results results are summarized across four tables, two for each language. overall f1 score and score by label are presented for galician models in ## discussion based on the performance of the monolingual models, xlm-r clearly outperforms mbert as a base architecture in the low-resource scenario. as both architectures are pre-trained on the same, unrelated task, have the same number of layers, hidden state, and attention heads, and include galician and spanish unlabeled text in their pre-training languages, this performance boost appears to be influenced by the difference in tokenization methods utilized by each architecture; mbert uses wordpiece while xlm-r uses sentencepiece. wordpiece is an algorithm that breaks down words into sub-words using a variation of byte-pair encoding (bpe) able to use a range of n-gram sizes, while sentencepiece is a re-implementation of sub-word units able to utilize both traditional bpe and unigram language modelling @xcite . generally, wordpiece is seen to perform better in text classi- an example galician sentence from the test dataset can be seen in only minor differences could be seen between the two architectures in the higher-resource spanish scenario, with both monolingual models achieving a score of 0.82. this indicates that the choice of base language model matters less the more target language training data exists. pre-training on the srl task also seemed to have little to no effect on the final f1 score, further supporting that the most important variable toward accurate performance is the availability of data. as all scores were still able to outperform the baseline and maintained no differences in the training set other than our proposed method, this also suggests that verbal indexing is advantageous.