Learning to Write Rationally: How Information Is Distributed in Non-native Speakers’ Essays (2024)

Abstract: People tend to distribute information evenly in language production for better and clearer communication. In this study, we compared essays written by second language (L2) learners with various native language (L1) backgrounds to investigate how they distribute information in their non-native L2 production. Analyses of surprisal and constancy of entropy rate indicated that writers with higher L2 proficiency can reduce the expected uncertainty of language production while still conveying informative content. However, the uniformity of information distribution showed less variability among different groups of L2 speakers, suggesting that this feature may be universal in L2 essay writing and less affected by L2 writers’ variability in L1 background and L2 proficiency.

## introduction with the progress of globalization, more people have started acquiring new languages. for instance, the proportion of individuals who speak multiple languages daily in the united states has doubled over the past four decades, rising from about one in ten speakers to about one in five @xcite . these rapid changes in linguistic diversity offer unique opportunities but also present challenges for the multilingual population: not all speakers achieve perfect or proficient levels in their non-native languages (l2s) due to various factors, including the quantity and quality of exposure to l2s @xcite , the length and styles of their acquisition process @xcite , and their native language (l1) backgrounds and experiences @xcite . the cognitive mechanisms underlying language use in multilingual speakers may differ from those of native speakers, not only due to variations in proficiency but also because of diverse language backgrounds and experiences @xcite . many previous studies have explored whether and how speakers with different language backgrounds comprehend and produce languages differently. for example, spanish-english speakers may produce "spanish-like" sentences in their english production, where such types of grammar are rarely used or even prohibited in english. most of these studies have reached a similar conclusion: for multilingual speakers, representations are integrated across languages, forming a unified system for human language processing @xcite . consequently, for individuals who know more than one language, the language(s) that are not seemly involved in the target language production task, can also contribute to and influence comprehension and production processes in the target language, leading to unique patterns in human language processing that can reveal information and knowledge from other languages. despite variations in language production among multilingual speakers, the overarching goal of speaking and writing remains the same: to deliver information effectively. to achieve this goal, people distribute information evenly across language production, maintaining relatively equal predictability for each upcoming word @xcite @xcite . furthermore, the information carried by a unit of production (e.g., a word) can be quantified in several ways, including surprisal @xcite , entropy @xcite , and the uniformity of information distribution (uid) @xcite . these metrics help characterize the underlying rules of human language production, which can be summarized as follows: these rules have been substantiated by a wealth of empirical studies. for instance, people need longer time to process unexpected words during comprehension @xcite ; during production, people maintain uniformity of information and constancy of predictability by selecting shorter words @xcite , repetitive/familiar syntactic structures @xcite , or faster speech rate @xcite . using information-based metrics, prior studies also explored how the complexity of language production changes across language acquisition, and whether we can predict learners' proficiency based on those changes @xcite @xcite . what remains unknown, despite numerous studies exploring how individuals use these rules to enhance language production, is how l2 speakers apply these rules to distribute information in their l2 production-a topic that remains under-researched. given that l2 speakers often exhibit different preferences in lexical selection and syntactic structures compared to native speakers @xcite -variations influenced by their language backgrounds-it is reasonable to assume that these differences may result in distinct patterns in their l2 output. in this paper, we use several well-established metrics from psycholinguistics and information science to investigate how speakers with diverse l1 backgrounds and varying levels of l2 proficiency distribute information in their written production. ## related work the cognitive mechanisms underlying multilingual language processing represent a significant research topic spanning multiple fields, including psychology @xcite , linguistics (bhatia and ritchie, 2014), and cognitive neuroscience @xcite . as an integrated mechanism covering multiple languages, multilingual speakers demonstrate several typical cognitive and language patterns, such as cross-lingual priming effect @xcite , cross-lingual cognate effects @xcite , and code-switching effect @xcite . other studies explored how multilingualism impacts general cognitive capabilities and neural structures @xcite . recently, studies also started involving artificial intelligence to explore multilingualism and potential applications for multilingual populations @xcite , which provides new opportunities to explore and simulate multilingual processes and potential new methods for language education and proficiency assessment. while some prior studies take an informationbased approach to investigate human language production, few of them specify the nature of their multilingual sample. some studies offer intriguing evidence regarding cross-lingual production, such as the observation that multilingual speakers switch languages to avoid using uncommon words, demonstrating the surprisal effect @xcite . specifically, bilingual speakers are more likely to switch languages when the coming words are difficult to predict, leading to a reduction of information density @xcite . even though some previous works proposed that different mechanisms may exist to help l2 speakers better deliver information in communication @xcite , details regarding these mechanisms remain under-researched. ## discussion this study explored how speakers with different l1 backgrounds distribute information in their l2 written production. our results revealed more "nativelike" trends in metrics such as surprisal and entropy as the speakers' l2 proficiency increased in contrast, metrics such as the uid score indicated that l2 writers tend to adhere to the fundamental principles of information distribution, even when they are less proficient in l2. these results provide additional insights regarding the learning progress among l2 speakers in language production and communication. language surprisal and entropy emphasize language production from different aspects: surprisal measures the exact information carried by the in-coming word, while entropy estimates the expected certainty about upcoming words. as demonstrated by native speakers in figure 1 , speakers want to maximize the information in each word while minimizing the overall expected uncertainty for effective and clearest communication. as learners' proficiency in l2 increases, they develop more nativelike language production. with increased l2 proficiency, they have more l2 resources, which further lead to more advanced, sophisticated, and coherent lexical selection, longer production units, and more complex syntactic structures in their production outcomes @xcite @xcite . our results provide additional insights through the information distribution among l2 speakers, showing that higher l2 proficiency enables learners to produce language more effectively and efficiently by carrying more information and reducing expected uncertainty in their production. even though we observed significant group differences in mean surprisal and entropy scores among l2 speakers with different l2 proficiency levels and l1 backgrounds, the uid scores showed a different pattern with fewer variations and a more native-like distribution across all proficiency groups (see figure 2c and ## conclusion and future work this paper studies how information is distributed in written essays from native and non-native english speakers using information-based metrics. the increasing surprisal and decreasing entropy values showed that proficient l2 speakers distribute information in a more native-like style by maximizing the usage of information channels while reducing the uncertainty of upcoming words. in contrast, the uid score showed fewer differences among proficiency groups, indicating that maintaining smooth communication channels is a more general skill among human language users. future studies can investigate the relationship between linguistic features and information-based metrics regarding speakers' language production, as well as how prior language experiences impact the information distribution patterns.