Curse of bilinguality: Evaluating monolingual and bilingual language models on C hinese linguistic benchmarks (2025)

Abstract: We investigate cross-lingual transfer effects in large language models (LLMs) trained on two high-resource languages, English and Chinese. Four monolingual Chinese and four bilingual English–Chinese models are evaluated on two Chinese linguistic benchmarks. The monolingual models consistently outperform the bilingual ones on 12 out of 55 tasks, while the reverse is true for only 4 tasks, highlighting the prevalence of negative (rather than positive) transfer from English to Chinese. Additionally, we carry out a feature attribution analysis in a monolingual and a bilingual model, showing that the differences in their performance may be explained by more predictable attribution patterns in the monolingual model. Our findings have implications for the ongoing effort of training bilingual LLMs.

## introduction in multilingual nlp, cross-lingual transfer is traditionally described in positive terms. for example, a model's performance in low-resource languages can be improved by leveraging transfer from highresource languages. at the same time, adding lowresource languages to the training data may cause a model to perform worse in high-resource languages due to the negative cross-lingual transfer, a phenomenon known as the curse of multilinguality @xcite . despite the abundance of studies that address this problem @xcite @xcite , they primarily focus on multilingual llms trained on a variety of languages with very unbalanced amounts of data per language. what happens, however, when a model is trained on exactly two high-resource languages? english and (mandarin) chinese are the two languages with the largest amounts of data available for training, and the recent years have seen a surge in the development of llms for both languages. while a few chinese models are monolingual (e.g., @xcite @xcite , most others are either bilingual (i.e., trained on a mix of english and chinese data: @xcite @xcite or multilingual (see a survey by @xcite . while bilingual and multilingual models perform well on some english benchmarks (e.g., @xcite , it is unclear whether they always outperform their monolingual counterparts in chinese linguistic tasks. in this paper, we study cross-lingual transfer effects in bilingual chinese-english llms. we evaluate four monolingual chinese models and four bilingual chinese-english models on two commonly used chinese linguistic benchmarks. for a number of paradigms in these benchmarks, the monolingual models (including the relatively small monolingual chinese bert) consistently outperform the bilingual ones, indicating negative transfer from english to chinese. we then present an interpretability analysis using feature attribution methods on two selected models, showing that the bilingual model may be worse at capturing the relations between words in the target sentences than the monolingual one. foot_0 2 method ## conclusion we have evaluated four monolingual chinese and four bilingual chinese-english models on two chinese linguistic benchmarks. across 55 test tasks, we observe consistent performance differences between monolingual and bilingual models on 16 tasks -despite their smaller sizes, monolingual models perform better on 12 and worse only on 4 tasks. this result suggests that bilingual chinese-english models may suffer from negative crosslingual transfer. it extends prior findings on negative transfer in multilingual models @xcite to a bilingual setting where both languages are high-resource and well-represented in training data. our feature attribution analysis suggests that monolingual models' higher scores may stem from the fact that they better capture the key relations between words in sentences, compared to bilingual models. our findings have implications for the ongoing effort of training bilingual llms on highresource languages (e.g., @xcite @xcite .