The Denglisch Corpus of G erman- E nglish Code-Switching (2023)

Abstract: When multilingual speakers involve in a conversation they inevitably introduce code-switching (CS), i.e., mixing of more than one language between and within utterances. CS is still an understudied phenomenon, especially in the written medium, and relatively few computational resources for studying it are available. We describe a corpus of German-English code-switching in social media interactions. We focus on some challenges in annotating CS, especially due to words whose language ID cannot be easily determined. We introduce a novel schema for such word-level annotation, with which we manually annotated a subset of the corpus. We then trained classifiers to predict and identify switches, and applied them to the remainder of the corpus. Thereby, we created a large scale corpus of German-English mixed utterances with precise indications of CS points.

## introduction multilinguality is becoming more and more ubiquitous, to the extent that psycholinguists increasingly acknowledge that bilingualism is the rule and not the exception @xcite . @xcite stated that "bilingualism is a worldwide phenomenon, found on all continents and in the majority of the countries of the world" and @xcite assessed that more than half the world's population is multilingual. multilingual speakers have two or more language systems active in their minds, and they tend to use them interchangeably, especially when communicating with other multilinguals. this process of mixing two or more languages within a discourse or even within a single utterance is called codeswitching @xcite . in order to understand and produce natural language, nlp systems need to cope with this phenomenon, but today's language technology still cannot efficiently process cs, partly due to lacunae in our understanding of the factors driving cs, and partly due to lack of resources. we introduce a corpus of german-english cs in spontaneous written communication. 1 we discuss challenges in determining the language id of tokens in multilingual texts in section 4, and present our novel annotation scheme in section 5. we describe the corpus in section 6, and then describe classifiers (section 7) that accurately identify the language id of tokens in the corpus, thereby allowing us to effectively identify switch points in unseen texts. we conclude with suggestions for future research. the phenomenon of cs code-switching is the process of mixing two or more languages within a discourse or even within a single utterance, where the mixed words or fragments do not suffer any syntactic or phonological alternation. cs can happen on various linguistic levels (phonological, morphological, lexical, syntactic), and can be intrasentential (the switch occurs within the boundaries of a sentence or utterance), or inter-sentential (the switch occurs between two sentences or utterances). there are two competing theories on how this process works: as a symmetric relation or as an asymmetric relation. in the symmetric approach both languages are equally dominant, and any lexical items from either language can be replaced by the corresponding items of the other language, as long as the switch happens at syntactic boundaries that are shared by both languages. the monolingual fragments conform to the grammar of the corresponding language they are taken from @xcite . in the asymmetric approach one of the languages is more dominant than the other, and only content morphemes can be taken from both languages, whereas late system morphemes indicating grammatical relations can only be taken from the subordinate language. the dominant language from which the grammatical framework is taken is called the matrix language, and the subordinate language that is mixed into it is called the embedded language @xcite . oral cs cs in oral communication has been studied extensively. it interacts with speakers' proficiency as well as style and content of the utterances, serving several, partly contradicting, purposes, such as compensating for words the speaker does not know in one language or expressing nuanced meanings that cannot be expressed precisely with the other language @xcite . but cs can also serve sociolinguistic purposes such as conveying identity, interpersonal relations and formality. conclusions from past research have differed greatly in whether cs is a strategy used by highly adaptive speakers to convey very subtle meaning differences between words of different languages @xcite , or a strategy used by speakers less familiar with one of the languages to overcome lexical deficiencies @xcite . written cs cs in written communication has not drawn much attention in research so far. written communication differs significantly from spoken interaction, especially in formality and spontaneity: e.g., literary texts undergo an inherent process of conscious reflection, correction, editing and review. findings thus far have differed on whether oral and written cs behave in the same manner and serve the same purposes. written cs in literary texts does partially serve the same purposes as in spoken cs @xcite , but there are additional functions and purposes that are not found in spontaneous oral speech, such as serving as a poetic device @xcite . online forums with the increasing ubiquity of online discussion platforms, there are large amounts of written communication reflecting more spontaneous speech productions than classical written texts, thereby constituting a hybrid between speech and formal writing. research on cs in online forums has so far mainly focused on computational challenges for nlp algorithms @xcite . sociolinguistic aspects of the communicative purpose of cs in these settings are severely understudied. most sociolinguistic works mainly focused on very limited data of a small number of language-pairs or authors @xcite . @xcite developed a large-scale corpus of written cs data from reddit posts con-taining various languages switched with english, but not including the german-english pair that we focus on here. they compared monolingual and code-switched posts, finding that there are topical and stylistic distinctions, as well as a difference in the proficiency of speakers. @xcite compiled an arabizi corpus from twitter and reddit posts which contains cs between arabic, english and french, and trained classifiers to identify switches. annotating cs language annotation of bilingual data is not always trivial @xcite , especially when borrowings and named entities are involved. borrowing is a continuous process, with different stages, where a word is first introduced as a completely foreign sounding word and is then phonologically and morphologically adapted to the borrowing language, until it becomes a common word of the language's lexicon. clear cuts on when a word is still to be considered a foreign word or already a common word of the language are hard to make. due to the geographical and phylogenetic closeness of german and english and their common cultural and religious roots, it is often hard to determine whether a word is borrowed, adapted, foreign or native to the language. alvarez-mellado and lignos (2022) added a "language" tag, bor, to indicate recent borrowings, in addition to a tag for named entities. @xcite proposed the use of a shared category for words that can be used in both languages. we further refine their annotation scheme and the definition of the shared category. for a different approach to language id annotation of multilingual texts, see @xcite . predicting cs points cs is influenced by various sociolinguistic characteristics, such as topic and setting or the speakers involved in the conversation and their level of familiarity. it can serve several sociopragmatic functions such as direct quotation, emphasis, clarification, parenthetical comments, etc. several linguistic features can be exploited for predicting cs points. @xcite showed that pos-tags, cognates, and entrainment of a word can trigger switches on the succeeding word, but not on the preceding word. this suggests that predicting cs points from the previous words alone is possible. @xcite predicted cs points using lexical and syntactic features, such as tokens, part-of-speech (pos) tags, and tree tags. recent works show that the strong relationship between cs and cognate words, as proposed by @xcite @xcite in the triggering hypothesis, can be used to improve language models @xcite . it is important to note that predicting cs is a difficult task because cs is a subjectively motivated process, subject to the speaker's preferences and background. clearly, bilingual speakers do not have to code-switch, as by definition they can converse in any of their two languages. understanding when and where they do code-switch is an ultimate goal of our research program, but undoubtedly some degree of arbitrariness is inherent to the phenomenon. @xcite therefore proposed the use of human judgments additionally to standard statistical evaluation measures. ## experimental setup data reddit is a large-scale social news and discussion platform, with several hundreds of thousands of sub-categories (sub-reddits) on different topics, and over 100 million new posts a year. there are many region-based sub-reddits, which attract large bilingual communities. the posts and comments are length-unlimited, and unlike in labsettings the interlocutors produce language spontaneously, which allows us to analyze natural conversation flow. german is one of the most widely-used languages in the world. with approximately 100 million native speakers, it is the most prevalent mother-tongue and, after english, the most widely understood language in europe. 2 since english is the world's main lingua franca, that non-native speakers across europe use on a regular basis, german speakers are constantly exposed to english (through movies, music, the internet, etc.) and cs exists in their daily life. it is thus worthwhile to investigate cs in german-english. however, to the best of our knowledge, no corpus or any work on written german-english cs is available, although a german-turkish corpus of twitter posts does exist @xcite . most existing cs corpora and studies on cs use language pairs in communities where both languages are either co-official or co-native to the community (e.g., hindi-english, where english is an official language and a lingua franca throughout india @xcite ; maltese-english, where malta as a former british colony maintained english as a lingua franca @xcite ; turkish-german in the german-turkish community @xcite ; etc.) here, we address cs in a country that is officially monolingual (german) and neither has a major community of englishnatives nor uses english as a lingua franca. we investigate german-english cs using country-specific sub-reddits for german-speaking countries/regions, like r/germany or r/de. since these sub-reddits contain discussions about regionbased topics, we expect authors in these communities to be speakers of both german and english. ## shared lexicon the key to identifying cs points is precise annotation of the language id of each token in the text. in multilingual texts, this problem is non-trivial (alvarez-mellado and lignos, 2022). we now discuss some of its challenges. we provide examples from german-english, but most of the observations are valid for any language pair. many words are shared across the german and english lexicons. we differentiate between inher-ited words, or cognates, which developed from words in an earlier stage of the language, and borrowed words, which are taken from or developed from words of another language. borrowing is a continuous process with different stages: words are first introduced into the language as a completely foreign sounding word; they are gradually adapted to the phonological and morphological rules of the borrowing language until eventually they are considered to be common words of the language @xcite @xcite . loan words are fully integrated borrowings, i.e., fully adapted to the borrowing language in flexion, phonology and orthography. borrowings without (or with minimal) adaptions are called foreign words. a pseudo-borrowing is a word created from elements of a borrowing language, but which does not exist in the donor language (e.g., handycellphone) @xcite . the reasons that words are borrowed or shared across languages include geographical language contact, phylogenetic closeness, and common cultural background @xcite @xcite . it is not always easy to tell whether a word is borrowed, native, or a switch. german and english are both germanic languages, which share a common ancestral lexicon and many similar-looking words. both languages were religiously and culturally influenced by greek and latin. nevertheless, words can be marked by native morphology or orthography and some of these adaptions may intuitively look more german than others (e.g., -ieren, which is usually used on long integrated latin words instead of -en). further, not all of these words are actually shared. many latinate words entered english through old french and by today either displaced their germanic equivalents or shifted their meaning. many cultural terms are borrowed into other languages as full new concepts without translations. this is the case for modern inventions, but many everyday words entered the german lexicon centuries ago, and native speakers are often unaware of their foreign roots (e,g, meschugge, schal, cotton, assassin). named entities are usually borrowed without translation, but they may take different forms: they can be shared completely or adapted orthographically, phonologically, and morphologically, sometimes with very distant looking forms, or even be taken from different etymological roots. addition-ally, they can take derivational or inflectional morphemes of the borrowing language or even be used in compounds with native words. additional challenges are due to the fact that some very high-frequency words share spelling with a word of the other language (e.g., was, die) although they are totally unrelated. furthermore, words can be composed of components of two different languages (e.g., pushnachrichten-push notifications). using english entities like fifth avenue, or untranslatable terms like hamburger, in a german sentence cannot be considered a regular switch, since there is no actual german equivalent for such terms. nevertheless, the use of these terms might activate the english lexicon and trigger a future switch. the extent of such triggering may be reduced for entities or terms that are adapted to german in orthography and morphology. these considerations are the motivating principles for our annotation scheme, which we now present. ## annotation scheme we introduce a novel, highly-detailed annotation scheme that reflects the observations of section 4 above. we present the scheme in section 5.1, and then propose a flattened version of it in section 5.2. crucially, while we define the schemes and exemplify annotated instances in terms of english and german, the schemes are applicable to any language pair. ## corpus creation we used a modified version of the method used by @xcite to collect and extract our data. we downloaded approximately 17 million comments from the german-language sub-reddits r/de, r/deutschland, r/germany, r/berlin using the pushshift reddit api. we extracted 10,000 comments that potentially contained cs using the polyglot language detector. we foot_2 annotated 950 of the extracted comments manually following the detailed scheme of section 5. these contained over 75,000 tokens in 4,200 sentences, of which 1,250 contained intra-sentential switches. we then generated a version with the collapsed annotation scheme. we then downloaded another set of 25.5 million comments from german-language sub-reddits, including also sub-reddits dedicated to cities and regions in austria and switzerland, as well as a few general topics. of those, 21,500 comments were extracted as potentially including switches. these comments, together with the remainder of the initially downloaded comments, were used to create a larger automatically annotated corpus. the data for the automatic annotation thus consists of 31,500 comments containing 230,000 sentences with over 5 million tokens. to identify code-switches in the automaticallytagged corpus we use two different criteria. the strict definition requires a sentence to contain at least one word annotated "pure english" (1), and at least one tagged as "pure german" (2). the relaxed definition only requires a token tagged as english-origin, excluding named entities (1,4b-e,4d-e,3-e) and a token similarly annotated as german-origin, excluding nes (2,4b-d,4d-d,3-d), or a token tagged as merge-word, excluding ne-merger (3c-m,3c-c). we now provide some observations on the manually-annotated portion of the corpus. ## identifying switches in order to identify switches in an unseen utterance, we need to identify the language id tag of the words in the sentence. we now describe a classifier that establishes this task. ## conclusion we presented a corpus of german-english codeswitched utterances from user generated social media content, which contains precise language annotation indicating code switches. our corpus is partly hand-annotated and partly automatically annotated. we addressed some challenges in annotation of multilingual data by introducing various types of shared and mixed categories. we trained classifiers to predict our word-level annotation and switch-points. first experimental results from the prediction of switch-points indicate that properties of shared and mixed words are relevant factors for cs. this encourages us to use our corpus as a basis for further sociolinguistic research on spontaneous written cs, specifically for studying the use and effects of shared and mixed words on switches in german-english and how these compare to other language pairs. such work is currently underway.