Pre-annotation Matters: A Comparative Study on POS and Dependency Annotation for an A lsatian Dialect (2025)

Abstract: The annotation of corpora for lower-resource languages can benefit from automatic pre-annotation to increase the throughput of the annotation process in a a context where human resources are scarce. However, this can be hindered by the lack of available pre-annotation tools. In this work, we compare three pre-annotation methods in zero-shot or near-zero-shot contexts for part-of-speech (POS) and dependency annotation of an Alsatian Alemannic dialect. Our study shows that good levels of annotation quality can be achieved, with human annotators adapting their correction effort to the perceived quality of the pre-annotation. The pre-annotation tools also vary in efficiency depending on the task, with better global results for a system trained on closely related languages and dialects.

## introduction automatic pre-annotation is often considered a cost-effective way of producing high-quality corpora, as it streamlines the process for human annotators. in the context of low-resource languages, pre-annotation can be a particularly beneficial practice, given that annotation tasks are often undertaken with limited human and financial resources. however, low-resource languages frequently lack training data or existing tools to obtain good quality pre-annotations. in this article, we address the impact of preannotation on pos and dependency annotation in the universal dependencies (ud) framework @xcite for the alsatian alemannic dialects. alsatian is a hypernym which refers to both alemannic and franconian dialectal varieties spoken in the alsace region, in northeastern france. the different upper german dialects referred to by the term "alemannic alsatian dialects" are northern low alemannic, spoken in the northern and central parts of alsace, southern low alemannic, spoken in the southern part of alsace (south of colmar), and high alemannic, in the very south of the region. the alemannic alsatian dialects are closely related to other alemannic german dialects, as for example swiss german and swabian, and to other dialectal varieties in the oberdeutsch dialect family, as for example bavarian. 1 rhine franconian is also spoken in the northwest of alsace, but it is not included in our study, which focuses on low alemannic alsatian. it is also worth mentioning that there is no consistent spelling standard for alsatian dialects, which leads to high levels of variation in writing. in this work, we compare three different preannotation methods, focusing on out-of-the-box tools that are easy to use without requiring extensive computational resources, advanced information technology skills or financial resources to pay for apis. these methods rely either on tools trained for the closest standard language, german, or on a mix of german and related dialects, as well as an instruction-tuned generative large language model (llm). instruction-tuned llms have sparked the interest of researchers in recent years for annotation tasks with both positive and negative-or at least more cautious-conclusions. one of our goals was therefore to gain a better understanding of their advantages and pitfalls. we address the following research questions (rq): rq1 is it possible to obtain good annotation quality with zero-shot pre-annotation only, when no existing tools are available for the target language? rq2 which pre-annotation method is the most useful? rq3 can pre-annotation bias be mitigated by using a mix of pre-annotation tools or, on the contrary, does it have a detrimental effect on quality? rq4 what are the advantages and pitfalls of instruction-tuned llms for our target tasks? 2 previous work ## conclusion and perspectives in this work, we have compared three preannotation methods for pos and dependency annotation for low alemannic alsatian. since there is no pre-existing annotated corpus for the language, we used mostly zero-shot methods, relying on closely-related languages or an instructiontuned llm. we were able to obtain good annotation quality and showed that the human annotators adapted their correction effort to the perceived quality of the pre-annotation. moreover, the best method for pre-annotation is task-dependent, with the arboratorgrew model trained on a mixture of closely-related languages and dialects achieving the best overall performance for both tasks. the corpus described in this paper is currently being reviewed for its release on the ud repository and will complement the resources already available for high german languages. we also used this corpus to train a parser specifically for alsatian and pre-annotate a second corpus of texts natively written in alsatian.