S y MC o M - Syntactic Measure of Code Mixing A Study Of E nglish- H indi Code-Mixing (2022)

Abstract: Code mixing is the linguistic phenomenon where bilingual speakers tend to switch between two or more languages in conversations. Recent work on code-mixing in computational settings has leveraged social media code mixed texts to train NLP models. For capturing the variety of code mixing in, and across corpus, Language ID (LID) tags based measures (CMI) have been proposed. Syntactical variety/patterns of code-mixing and their relationship vis-a-vis computational model’s performance is under explored. In this work, we investigate a collection of English(en)-Hindi(hi) code-mixed datasets from a syntactic lens to propose, SyMCoM , an indicator of syntactic variety in code-mixed text, with intuitive theoretical bounds. We train SoTA en-hi PoS tagger, accuracy of 93.4%, to reliably compute PoS tags on a corpus, and demonstrate the utility of SyMCoM by applying it on various syntactical categories on a collection of datasets, and compare datasets using the measure.

## introduction code-mixing refers to mixing of linguistic units and structures from multiple languages in a single utterance and/or conversation @xcite . the complexity of code-mixing can be intuitively understood as the degree of structural interleaving between the languages at the level of the lexicon and morpho-syntax @xcite , and also at the level of pragmatic and sociolinguistic functions of code-mixing in a linguistic community @xcite @xcite . it is an important notion that is linguistically well-studied and provides insights into cognitive and cultural aspects of human language. additionally, quantification of this complexity has recently attracted attention of computational linguists because studies have shown that the performance of the same model can widely vary on different code-mixed corpora. as a result, different metrics of complexity of code-mixing have been proposed such as cmi, ratio-based measures time-course measures and memory-based measures @xcite . but as srivastava and singh (2021) points out, these metrics are limited, partly because they are primarily based on language switch patterns at the token level, being completely agnostic to structural features. for instance, figure 1 shows en-hi code-mixed sentences with the same language tag distribution, but in example (1), verb is switched, while in (2) nouns are switched. the former seems much more complex and difficult to process cognitively and computationally, than the latter, where switching seems to be an extension of noun-borrowing @xcite . motivated by such cases, in this paper, we ask the following research question: can syntactic category information be deduced and used as a measure of structural complexity of a codemixed sentence and corpus? we attempt to tackle this question by formulating: syntactic measure of code mixing sym com , a simple metric that encodes the distributional difference of various syntactic categories across languages in a sentence. sym com can be computed for any corpora, as long as there is a reasonably accurate pos tagger for the code-mixed language pair. through empirical studies of several existing en-hi code-mixed corpora we provide, for the first time, a strong quantitative evidence in support of a widely held theoretical notion that open class categories (e.g., noun, adjectives) are more likely to be switched than the closed class categories (e.g., pronouns, verbs) within a sentence. further, we show that different corpora have significantly different distribution of sym com values. the polarity of sym com su indicates the language, among l1 and l2, that is contributing higher number of tokens for a particular su , and its absolute value captures the degree of skew towards a particular language. if sym com su is closer to zero, it indicates that the contribution of l1 and l2 for su is balanced. we define the sym com sent score for a sentence, as the weighted average of absolute sym com su scores for all su , where the weights are the fraction of tokens in the sentence belonging to an su . values closer to zero indicate that l 1 and l 2 contribute nearly equally for most types of su s in the sentence, whereas values close to 1 indicate that in the sentence each su is majorly contributed by a single language. note that while a low sym com sent implies that the tokens in a sentence are nearly equally contributed by the two languages, a high sym com sent does not say anything about the language distribution of the tokens. sym com sent can be averaged over the corpus to capture the syntactic variation at a corpus level: in figure 1 , sym com su scores are computed for two en-hi code-mixed sentences, whose cmi scores are equal. for each utterance, we calculate the number of nouns and verbs belonging to l 1 = en and l 2 = hi. in example 1, sym com n ou @xmath0, indicating that l 2 is contributing all the nouns in this sentence. the opposite polarity of sym com v erb indicates that all the verbs are contributed by l 1 . ## experiments & discussion to demonstrate the utility of the proposed sym com measure, we analyse a) en-hi code-mixed corpus; b) compare sym com sent sym com corpus score across different datasets. to compute sym com scores we need token wise lid and pos tags. we use pre-trained character level bilstm language id tagger released by @xcite for obtaining token wise lids. we train our pos tagger using the en-hi universal dependency dataset released by @xcite , which used universal dependency tagset @xcite . ## conclusion & limitations in this work, we have proposed sym com , a syntax-aware measure of code-mixing, to analyze code-mixed corpora from a syntactic perspective. our analysis confirms a few important tenets of the matrix language theory, including the fact that closed class categories and (finite) verbs are less likely to be switched. additionally, we have trained a english-hindi (hinglish) pos tagger using xlm-r which is able to achieve state-of-the-art-results. sym com relies on the strength of pos tagger and lid tagger . the errors made by the tagger would propagate into the subsequent analysis thus adding noise to the sym com scores as well. extending sym com to code-mixing between 3 or more languages and to deeper syntactic structures (nested phrases) are left as part of future work. detection, humor detection and conversational systems.