D an T ok: Domain Beats Language for D anish Social Media POS Tagging (2023)

Abstract: Language from social media remains challenging to process automatically, especially for non-English languages. In this work, we introduce the first NLP dataset for TikTok comments and the first Danish social media dataset with part-of-speech annotation. We further supply annotations for normalization, code-switching, and annotator uncertainty. As transferring models to such a highly specialized domain is non-trivial, we conduct an extensive study into which source data and modeling decisions most impact the performance. Surprisingly, transferring from in-domain data, even from a different language, outperforms in-language, out-of-domain training. These benefits nonetheless rely on the underlying language models having been at least partially pre-trained on data from the target language. Using our additional annotation layers, we further analyze how normalization, code-switching, and human uncertainty affect the tagging accuracy.

## introduction language data from social media offer unique insights into how communities use language to communicate in a natural, spontaneous setting, using a highly domain-specific vocabulary. this domain is, however, also subject to frequent changes, high noise, and variability, making it difficult to process @xcite both for high @xcite and especially for lower-resourced languages @xcite @xcite . to better understand and improve how to target such code available at: github.com/kkirsteinhansen/dantok. data is available upon request (contact robv@itu.dk). a highly specialized domain, we present the first linguistically annotated dataset of contemporary danish social media from tiktok, a relatively new platform focused on short videos. the danish dependency treebank @xcite from the parole corpus @xcite , and its universal dependencies (ud) conversion @xcite is the primary part-ofspeech (pos) resource for danish @xcite . however, it contains well-edited text only. we present not only the first danish social media dataset with pos annotation, but, to the best of our knowledge, also the first linguistic exploration of tiktok language in any language. the dataset contains annotation for pos, normalization, codeswitching and annotator uncertainty (figure 1 ). it further prioritizes quality over quantity, covering over 8k tokens, which have been manually verified to ensure relevance, correctness, and privacy. it has been shown that training language models on the target domain can be used for cross-domain learning @xcite . in this work, we aim to tease apart the effect of the pre-training data of the language model and the source of the annotated training data for the target task. for this purpose, we contribute: foot_0 dantok, the first linguistically annotated dataset of tiktok comments and the first danish posannotated social media dataset; foot_1 an extensive study of transfer learning targeting this highly specialized domain; 3 an in-depth analysis investigating the unique features of this new domain, and how specific data/model properties help boost transfer performance. ## conclusion we presented dantok, the first linguistically annotated tiktok dataset and the first danish social media dataset with pos annotation. we conducted an extensive analysis of how to best transfer to a highly specialized domain in a mid-resource language, and we demonstrated that llms benefit from common approaches such as normalization, while struggling with the same cases as the human annotators. simultaneously, our results show that although in-language data and models form the basis for high performance, in-domain data, even from another language, should not be neglected in order to achieve state-of-the-art results.