A Dependency Treebank of Spoken Second Language E nglish (2022)

Abstract: In this paper, we introduce a dependency treebank of spoken second language (L2) English that is annotated with part of speech (Penn POS) tags and syntactic dependencies (Universal Dependencies). We then evaluate the degree to which the use of this treebank as training data affects POS and UD annotation accuracy for L1 web texts, L2 written texts, and L2 spoken texts as compared to models trained on L1 texts only.

## introduction in the field of applied linguistics, natural language processing tools such as part of speech (pos) taggers and syntactic parsers have been and continue to be used to investigate characteristics of second language (l2) use at scale (e.g., @xcite @xcite . although taggers and parsers are increasingly accurate (achieving f1 scores of around .98 for pos taggers and .95 for dependency annotation) when evaluated on in-domain texts (i.e., texts with similar linguistic characteristics as the training data), accuracy can drop precipitously for out of domain texts (e.g., @xcite . a pressing issue, then, is the availability of appropriate annotated corpora to test and train tagging and parsing models on the types of data applied linguists often use @xcite . although a treebank of written second language (l2) english is available @xcite , to our knowledge no treebanks of spoken l2 speech are publicly and freely available. in this paper, we report on the development of an annotated corpus of spoken l2 english and evaluate the accuracy of a pos tagger and dependency parser when trained on l1 texts and a combination of l1 and l2 texts. ## applied linguistics research and nlp the use of nlp tools such as taggers and parsers to examine characteristics of language use has a long history in the field of applied linguistics. early studies (e.g., @xcite focused on the analysis of lexical and lexicogrammatical variation across registers (e.g., different spoken and written language use domains). as the subfield of learner corpus research has grown, taggers and parsers have also been used to investigate how second language learners' linguistic patterns change over time (e.g., @xcite and/or differ across proficiency levels (e.g., @xcite @xcite .