Annotating Second Language in U niversal D ependencies: a Review of Current Practices and Directions for Harmonized Guidelines (2025)

Abstract: Universal Dependencies (UD) is gaining popularity as an annotation standard for second language (L2) material. Grammatical errors and other interlanguage phenomena, however, pose significant challenges that official guidelines only address in part. In this paper, we give an overview of current annotation practices and provide some suggestions for harmonizing guidelines for learner corpora.

## introduction ever since @xcite proposed to represent learner corpora as parallel dependency treebanks, universal dependencies (ud) @xcite has been gaining popularity as an annotation standard for both written and spoken second language (l2) data. at the time of writing, treebanks have already been released for written chinese @xcite , english @xcite , now retired 1 ), italian @xcite and korean @xcite , as well as spoken english @xcite . two more are in progress, one for written russian @xcite and one for written swedish. when it comes to the annotation of l2 productions, the main advantage of using ud is that it provides a cross-lingually consistent morphosyntactic annotation layer that enables both quantitative and qualitative comparisons between a learner's l1 and l2, between different l2s and, most importantly, between the target language (tl) in its standard form and as an l2. in the latter scenario, an especially helpful format is that of parallel, socalled "l1-l2" treebanks, proposed by @xcite and adopted by the vast majority of the seven aforementioned annotation efforts. in an l1-l2 treebank, each learner sentence is paired with a 1 "retired" ud treebanks are available for download but not up-to-date with current annotation guidelines and therefore not part of the latest ud release. ud annotation of l2 productions, however, comes with its challenges. learner texts often deviate from standard language in ways that make them difficult to analyze in a framework designed with standard use of the tl in mind. relevant phenomena are not limited to grammatical errors themselves, but also include code switching, calques and non-idiomatic expressions. as a result, treebank developers have produced extensive project-specific guidelines, not always consistent with each other. this study has been carried out as part of an ongoing annotation effort whose aim is to produce a treebank based on the swell-gold (swedish learner language) corpus @xcite , expanding the resources available through the språkbanken research infrastructure. to ensure that the ud annotation of swell is consistent with that of other similar datasets, convenient for treebank users and theoretically motivated, we have reviewed the relevant ud guidelines and compared practices across five l2 treebanks for as many tls. in this paper, we present our findings and propose directions for harmonizing l2 annotation guidelines. we demonstrate the application of our suggested principles on selected examples from our yet-to-be-released treebank. ## challenges of treebanking l2 as mentioned in the introduction, a number of phenomena typical of -but not exclusive to -learner language pose significant challenges for treebank developers. most prominently, l2 productions are often characterized by the presence of grammatical errors, here understood in a broad sense, i.e. also including issues of orthography and lexical choice. some of these issues, such as misspellings, are to some extent addressed in the universal ud guidelines (cf. section 3.1) and do not typically alter the morphosyntactic analysis of the sentences in which they occur. however, it is not uncommon for grammatical errors to create a conflict between observed language use and intended meaning. as a consequence, much of the discussion that has taken place in the context of l2 treebank development has revolved around trying to strike a balance between two seemingly contradictory approaches: following the principle of literal reading (as formulated by @xcite , according to which morphosyntactic analysis should be guided solely by the surface word forms and observed language use, and applying distributional criteria, in an attempt to reflect the writer's intentions. while annotating literally is often key to an informative analysis of nonstandard language, some amount of distributional insight is not only indispensable to deal with nonexisting word forms and otherwise unanalyzable constructions, but also intrinsic to the ud annotation scheme at large, independent of what material is analyzed. we reframe the problem, guided by a similar but not identical question, i.e. to what extent the analysis of a learner sentence should be informed by its assumed intended meaning. while the principle of literal reading encourages not to rely on any particular interpretation of learner productions, it can be difficult to analyze ungrammatical constructions without some degree of speculation as to what the writer is trying to convey. in addition, when a correction hypothesis is available -such as in l1-l2 treebanks -it seems reasonable to try to ensure that the two analyses are consistent with each other, at least for the sake of comparability. intuitively, the two trees constituting a sentence-correction pair should be "as different as necessary," so as to not hide any information that may help describe any discrepancies more accurately, and "as similar as possible" to facilitate fine-grained comparisons and error retrieval. an example showing the difference between literal and correction-driven annotation is given in figure 1 . another challenging situation, also illustrated in figure 1 , occurs when what the most informative annotation results in a category combination invalid in the underlying framework, which is descriptive of the standard use of the tl. this happens not only because of grammatical errors, but also in conjunction with other interlanguage phenomena such as code switching and syntactic calques. relevant ud guidelines exist for some of these cases and will be discussed in section 3.2. ## l2 phenomena in the universal guidelines in this section, we summarize the general ud guidelines that relate to different aspects of the annotation of learner language. we restrict this discussion to the official guidelines available on the ud website, foot_0 although several proposals for unified guidelines for spoken data @xcite and user-generated content @xcite also cover a variety of relevant phenomena. ## annotation practices across l2 treebanks in this section, we provide a comparative analysis of the annotation practices found in different l2 treebanks. its objective is twofold: on the one to achieve these goals, we systematically review the papers describing the various treebanks, focusing on the sections dedicated to annotation criteria. when available, we integrate this with information with instructions from the treebanks' annotation manuals and complement textual sources with tree queries performed with the stund tool for parallel treebanks @xcite . we consider five of the seven annotation efforts mentioned in the introduction and listed in it must be noted that, for some annotation layers, cfl also provides a separate conll-x file with double (literal and distributional) tags. in this paper, we focus on the annotation strategies that were chosen as the "default", i.e. on the values present in the connl-u file distributed as part of the official ud releases. as for esl, an important caveat is that the version described by @xcite follows ud version 1 guidelines, which are now largely outdated. furthermore, it should be kept in mind that such treebank is not annotated for feats or misc. similarly, csl and ksl do not use the feats fields. finally, the papers describing the russian and korean treebanks only discuss dependency annotation. ## directions for harmonized guidelines as shown in in the following, we distinguish token-level annotation, i.e. the assignment of lemmas, morphological features and pos tags to individual words, from syntactic annotation, consisting in establishing and labeling head-dependent relations between pairs of tokens. we argue that the two can be guided by different, complementary principles, and that this results in more informative analyses. ## concluding remarks we discussed the challenges l2 poses for ud annotators and gave an overview of the universal guidelines relevant for learner corpora, as well as of the current annotation practices across five of the existing treebanks. we concluded the paper with some suggestions for harmonization, which we hope can be the starting point for a productive discussion, primarily with other members of the ud community working directly with l2 material, but also with annotators of spoken and/or code-switched data, user-generated content and nonstandard language varieties such as pidgins and dialects, all of which share some features with learner language.