An Annotated Social Media Corpus for G erman (2020)

Abstract: This paper presents the German Twitter section of a large (2 billion word) bilingual Social Media corpus for Hate Speech research, discussing the compilation, pseudonymization and grammatical annotation of the corpus, as well as special linguistic features and peculiarities encountered in the data. Among other things, compounding, accidental and intentional orthographic variation, gendering and the use of emoticons/emojis are addressed in a genre-specific fashion. We present the different layers of linguistic annotation (morphosyntactic, dependencies and semantic types) and explain how a general parser (GerGram) can be made to work on Social Media data, pointing out necessary adaptations and extensions. In an evaluation run on a random cross-section of tweets, the modified parser achieved F-scores of 97% for morphology (fine-grained POS) and 92% for syntax (labeled attachment score). Predictably, performance was twice as good in tweets with standard orthography than in tweets with spelling/casing irregularities or lack of sentence separation, the effect being more marked for morphology than for syntax.

## stuttgart-tübingen tagset for spoken german obviously, this is a softer evaluation method than the ones used in conll x and empiricist, that had access to separate, multi-annotator gold corpora. on the other hand, when using training data for ml, a gold corpus is simply a reserved sub-section of the corpus, and compatibility of the tag set is automatically ensured when using an ml method. however, when a parser is trained on one treebank and measured on another, evaluation can be biased due to different encoding schemes @xcite . a rule-based system, with its own tagset, is a worst-case scenario in this regard, because it cannot be re-trained. therefore, in our case, evaluation with an external gold corpus was impractical, and creating one from scratch manually was deemed excessive, since the resource would be needed only for evaluation, not for training, and because parser evaluation is an -unfundedside issue for the hs project as a hole 13 . still, though not directly comparable, evaluation results ( as can be seen from a break-down of error types ( at the syntactic level, due to the larger set of function categories, there were 128 different confusion pairs, even without taking into account attachment errors. however, only 13 pairs occurred more than 3 times, the most frequent being the classical ambiguity between postnominal and adverbial pp-attachment and the semantically important confusion between subject and accusative or dative objects. judging from false positives and negatives, the gergram parser has a bias towards adverbial over postnominal attachment, but is "errorbalanced" with regard to subjects and objects. because german has case constraints both for clause-level constituents and for arguments of prepositions, there is also some confusion between e.g. direct objects and accusative preposition arguments.