Revealing the impact of synthetic native samples and multi-tasking strategies in H indi- E nglish code-mixed humour and sarcasm detection (2025)

Abstract: In this paper, we reported our experiments with various strategies to improve code-mixed humour and sarcasm detection. Particularly, we tried three approaches: (i) native sample mixing, (ii) multi-task learning (MTL), and (iii) prompting and instruction finetuning very large multilingual language models (VMLMs). In native sample mixing, we added monolingual task samples to code-mixed training sets. In MTL learning, we relied on native and code-mixed samples of a semantically related task (hate detection in our case). Finally, in our third approach, we evaluated the efficacy of VMLMs via few-shot context prompting and instruction finetuning. Some interesting findings we got are (i) adding native samples improved humor (raising the F1-score up to 6.76%) and sarcasm (raising the F1-score up to 8.64%) detection, (ii) training MLMs in an MTL framework boosted performance for both humour (raising the F1-score up to 10.67%) and sarcasm (increment up to 12.35% in F1-score) detection, and (iii) prompting and instruction finetuning VMLMs couldn’t outperform the other approaches. Finally, our ablation studies and error analysis discovered the cases where our model is yet to improve. We provided our code for reproducibility.

## introduction: humour and sarcasm are complex and subjective emotions that impact the nature of human communication. they can appear in different forms such as exaggeration, dark humour, gross humour, adult or slang expression, insult, offence, etc. @xcite . past study (bleakley and sailofsky, 2023) highlighted how they can affect politics amid tragedy. detecting humour and sarcasm becomes more challenging in a code-mixed setting. this is because models now need to understand humour and sarcasm in an utterance expressed through altering multiple languages. more details on the phenomenon of code-mixing are presented in the appendix. an example of humorous and sarcastic expression in hindi-english codemixed language is given in the following. more examples are presented in figure 1 of appendix b. in the following example, the english parts are marked in red, and the hindi parts are marked in blue. we have provided their translations for readability. the nlp community has shown significant interest in monolingual humour and sarcasm detection @xcite @xcite . unfortunately, there is relatively less focus on the code-mixed settings @xcite @xcite . therefore, we have few publicly annotated code-mixed corpora available, further acting as a bottleneck in developing new models @xcite @xcite . the evolution of multilingual large language models (mlm hereafter) has shown a new path to address this issue. they can learn task-specific knowledge from samples in one language and make predictions for samples in different languages. this phenomenon is known as cross-lingual learning. it is very effective if training and testing samples share similar linguistic and cultural contexts @xcite . mlms learn their embeddings from a corpora spanning multiple languages, thus they are aware of vocabulary of multiple languages. in the context of codemixed languages, it means we can fine-tune the mlms using native monolingual task samples (e.g. for hindi and english task samples for a hindi-english code-mixed task) and do prediction for code-mixed samples. the hypothesis is that since code-mixed corpora (in hindi-english code-mixed language) and the native monolingual corpora (in hindi and english languages) are likely to share similar linguistic and cultural contexts, the training with native task samples or adding them in codemixed training sets can improve code-mixed task performance. in fact, @xcite , in a set of empirical experiments, have shown that adding hindi and english hate samples in codemixed hate training corpora improves code-mixed hate detection @xcite . however, nobody has tested it for code-mixed humour and sarcasm detection. a detailed discussion of prior works in this direction is presented in the appendix a. further, we observed that hindi humour and sarcasm datasets are not publicly available. we experimented with synthetic hindi samples and multi-tasking strategies to fill this gap. overall, we asked for three research questions, r1: does mixing native samples (english and synthetic hindi samples in our case) in code-mixed training sets improve code-mixed humour and sarcasm detection? r2: do jointly training with a semantically related third task (hate detection) along with native sample mixing improve code-mixed humour and sarcasm detection? r3: do adding native samples in the prompting context or in instruction finetuning of very large mlms (vmlms hereafter) improve the performance? in summary, our contributions are the following, in this section, we reported the details of codemixed and native datasets considered in our study. please note that we ignored the datasets containing dialogues and multi-modal samples to simplify our task formulation. the list of the datasets and their basic statistical details were reported in the dataset examples were illustrated in figure 1 in the appendix. a brief description of the individual datasets were reported in appendix b. apart from the class distribution, we also provided the kullback-leibler (kl) divergence @xcite values for the individual datasets in we also reported the fraction of total samples containing english hurtful (offensive, aggressive and hateful) keywords in each class. we used the lexicon given by @xcite to calculate it. finally, we also reported the inter-annotator agreement (iaa) scores published with individual datasets. in this section, we reported the details of our experiments conducted as a part of this study. apart from reproducing the baselines (section 3.1), we designed three experiments each based on a unique research philosophy. the three philosophies are i) native sample mixing (section 3.2), ii) multitask learning (section 3.3) and iii) prompting and instruction finetuning very large multilingual language models (vmlms) (section 3.4). while the native sample mixing strategy intends to improve the code-mixed tasks by adding monolingual native samples to the code-mixed training sets, the multi-task learning strategy tries to do the same by learning linguistic knowledge from the samples of a third task (here, it is hate detection). our last strategy, i.e. prompting and instruction finetuning vmlms, evaluates the performance of very large multilingual language models for the considered code-mixed tasks in a few-shot context prompting and instruction finetuning scenarios. the detailed experimental set up is given in appendix e.1. ## conclusion: from our findings, we drew the following conclusions: sl no sample translated english cm nsm mtl m1 m2 m3 m1 m2 m3 m1 m2 m3 1 .@shashitharoor sir kejriwal power cut nahi karenge to bill kam kaise hoga? (humor) .@shashitharoor sir kejriwal power cut nahi karenge to bill kam kaise hoga? × × × × × × × ✓ ✓ 2 musalmaano ka intolerance kuch zyada hi badh raha hai.. par media gaalia sirf hindu ko deti hai (nonhumor) the intolerance among muslims seems to be increasing excessively... but the media abuses only hindus. ✓ ✓ ✓ × ✓ × × × × 3 kehte hain agar kisi cheez ko dil se chaaho to puri kayanat usey tumse milane ki koshish mein lag jaati hai. #dada is back #ipl4 #srk #irony (non-sarcasm) they say if you truly desire something from the heart, the whole universe conspires to make it happen. #dada is back #ipl4 #srk #irony ✓ ✓ ✓ × × ✓ × × × 4 culturally rich honge..par gavaaro ki basti bhi mera desh. #susaid #irony #india (sarcasm) they might be culturally rich, but my country is also a haven for the uneducated. #susaid #irony #india in this section, we reported some of the limitations of our work. a related works: in this section, we discussed the past works of this domain. we listed task-wise description of the past literature in the following subsections.