Add Noise, Tasks, or Layers? M ai NLP at the V ar D ial 2025 Shared Task on N orwegian Dialectal Slot and Intent Detection (2025)

Abstract: Slot and intent detection (SID) is a classic natural language understanding task. Despite this, research has only more recently begun focusing on SID for dialectal and colloquial varieties. Many approaches for low-resource scenarios have not yet been applied to dialectal SID data, or compared to each other on the same datasets. We participate in the VarDial 2025 shared task on slot and intent detection in Norwegian varieties, and compare multiple set-ups: varying the training data (English, Norwegian, or dialectal Norwegian), injecting character-level noise, training on auxiliary tasks, and applying Layer Swapping, a technique in which layers of models fine-tuned on different datasets are assembled into a model. We find noise injection to be beneficial while the effects of auxiliary tasks are mixed. Though some experimentation was required to successfully assemble a model from layers, it worked surprisingly well; a combination of models trained on English and small amounts of dialectal data produced the most robust slot predictions. Our best models achieve 97.6% intent accuracy and 85.6% slot F1 in the shared task.

## introduction slot and intent detection (sid) is a classic natural language understanding (nlu) task. research today has mainly focused on standard languages with many speakers (e.g., @xcite @xcite . however, even when performance on a related standard language is high, sid for non-standard varieties can be challenging. this can be due to spelling variation @xcite and syntactic differences that complicate cross-lingual slot filling @xcite . furthermore, the lack of task data in the relevant language varieties *equal contribution. complicates the adaptation of sid models to underresourced varieties. in this paper, we report on the results of our participation in the vardial 2025 shared task on slot and intent detection in norwegian standard and dialect varieties (norsid; @xcite . we compare multiple strategies for improving the performance of sid systems (figure 1 ): we share our code at https://github.com/ mainlp/norsid . ## related work in the past few years, research on sid for dialects and non-standard languages has gained popularity. the multilingual sid dataset xsid @xcite @xcite ) contains evaluation data in over a dozen languages, including non-standard varieties like neapolitan, and german dialects spoken in switzerland, south tyrol, and bavaria. it has recently been extended with translations into norwegian dialects @xcite , which are the focus of this shared task. we provide more details in §3. using xsid, van der goot et al. (2021a) investigate multi-task learning with auxiliary tasks in the target language (or a closely related standard language). similarly, @xcite include auxiliary tasks in multi-task learning and intermediatetask training set-ups for dialectal sid. both studies find that the effects depend on both the auxiliary task(s) and the target task. we include auxiliary tasks in one of our experiments ( §4.3). two previous shared tasks have focused on dialectal sid @xcite . useful approaches by the participants were to train on sid data in multiple languages @xcite , injecting character-level noise into the training data @xcite ; we use a similar method in §4.2), and ensembling models trained on dialectal translations of the training set @xcite @xcite . outside the context of a shared task, abboud and oz ( 2024) numerous other methods for improving nlp performance in low-resource settings exist @xcite , many of which have not yet been applied to dialectal or cross-lingual sid. one recently proposed approach is assembling layers of models trained on different tasks or languages into a new model @xcite , which we explore in §4.4. ## data we use the xsid 0.6 dataset (van der goot et al., 2021a) and its norwegian extension nomusic @xcite . xsid combines re-annotated versions of two sid datasets @xcite . it includes 43k english training sentences, as well as smaller development and test datasets that have been translated into other languages. the shared task also includes an automatic translation of the training set into bokmål @xcite . for these sentence-level translations, the intent labels remained unchanged, while the slot annotations were automatically projected during the translation. nomusic provides translations of xsid's development and test utterances into the norwegian bokmål orthography and norwegian dialects from three of the four major dialect groups ( slots are annotated in the bio scheme. intent classification is measured with accuracy, and slot filling with strict span f 1 . one of our approaches uses datasets for auxiliary tasks; these are described in §4.3. we also use additional datasets for layer swapping experiments, described in §4.4. ## methodology we construct several baselines that differ in their training data and pretrained language model (plm) choices ( §4.1). we subsequently build on (some of) these baselines to examine the effects of different recent approaches to improving performance on low-resource language data. we submitted three systems each for intent classification and for slot filling; here we also discuss models we tested but did not submit to the shared task. when selecting systems for submission, we considered their performance on the development set while also aiming for a diverse set of systems. we use machamp (van der goot et al., 2021b; v.0.4.2, commit 052044a) with default hyperparameters to fine-tune the plms to simultaneously predict slot and intent labels. the slot predictions are decoded via a conditional random field (crf; @xcite . each model is fine-tuned for 20 epochs, and the best epoch is chosen based on performance on the development set. for layer swapping, we build on an implementation of joint-bert ( §4.4). ## results and analysis in this section, we mainly focus on the test scores. for the shared task, we submitted models considering their development set performance. these are denoted by asterisks in the results tables, and further discussed in §5.5. all models were trained (and evaluated on the development set) before the test set was released. ## discussion and conclusion the strength of our baselines suggest that the nor-sid task is, relatively speaking, less challenging than other dialectical variants of xsid (cf. @xcite @xcite @xcite @xcite . we suspect that there is less deviation from standard norwegian, and less variation between the dialects. this limits the gains we could expect from additional methods, particularly on the intent classification task, where the accuracy of our baselines ranges from 92.4% to 96.7% on the test set. we observe somewhat of a trade-off between performance on intent classification (strongest for models trained on norwegian data) and slot filling (strongest for models trained on the gold-standard english training or norwegian development data; §5.1). we hypothesize that the latter is due to the poor quality of the slot labels in the machinetranslated norwegian training data. we see noise injection as a simple way to improve transfer between a standard language and related varieties ( §5.2), although it requires access to appropriate training data. where a language has enough resources for additional annotated datasets, we see mixed effects from the inclusion of auxiliary nlp tasks ( §5.3). which auxiliary tasks help sid performance depends on the target-task training data and sid subtask (intent classification vs. slot filling) and remains hard to predict, requiring further research. improving performance on the slot-filling task proved to be quite difficult; our most successful method by a small margin is the assembled model made up of layers from a model trained on the nomusic development set (norsid expert), and another on the english xsid data ( §5.4). using layers from both of these models seems to have a regularizing effect and produces a model that is able to perform well on both languages and suffers less from overfitting than the norsid expert. we successfully adapted layer swapping -originally applied to a 32-layer decoder -to a 12-layer encoder, demonstrating its potential for resourceefficient cross-lingual transfer. layer swapping could prove useful for modular solutions, as layers for different languages could dynamically replace those of a "base" sid expert to adapt the model. we again note that the subset of the development set of nomusic we used, at 2.9k examples, is much smaller than the set used to train our ensid expert, at 43.6k examples; this modular approach would allow adaptation to different languages in a fairly lightweight manner post-hoc. we encourage further research comparing (and combining) different methods for low-resource nlp with the same training and/or evaluation data.