Addressing Linguistic Bias through a Contrastive Analysis of Academic Writing in the NLP Domain (2023)

Abstract: It has been well documented that a reviewer’s opinion of the nativeness of expression in an academic paper affects the likelihood of it being accepted for publication. Previous works have also shone a light on the stress and anxiety authors who are non-native English speakers experience when attempting to publish in international venues. We explore how this might be a concern in the field of Natural Language Processing (NLP) through conducting a comprehensive statistical analysis of NLP paper abstracts, identifying how authors of different linguistic backgrounds differ in the lexical, morphological, syntactic and cohesive aspects of their writing. Through our analysis, we identify that there are a number of characteristics that are highly variable across the different corpora examined in this paper. This indicates potential for the presence of linguistic bias. Therefore, we outline a set of recommendations to publishers of academic journals and conferences regarding their guidelines and resources for prospective authors in order to help enhance inclusivity and fairness.

## introduction artificial intelligence (ai) research has experienced a boom in recent years, seeing a significant increase in published works from across the globe. with the international language of academia being english, a significant number of publications are written by non-native speakers of english. this phenomenon brings about the issue of linguistic bias @xcite @xcite @xcite . described by @xcite , linguistic bias, also known as linguistic injustice, is the case of academic writing being judged more harshly by reviewers and editors if it doesn't meet the standards of international academic english, * *corresponding author even if the quality of the content is sufficient and communicability isn't impacted. this is supported by @xcite ; @xcite , who conclude that reviewers are more likely to accept papers whose language is more nativelike. to make steps towards addressing the issue of linguistic bias, we conduct a comprehensive contrastive statistical analysis of paper abstracts from the nlp domain, where we investigate how writing differs between authors from different linguistic backgrounds. we choose this domain due to its value in the nlp community, where computational methods are used to study language and linguistic expression is seen as important. in our analyses, we explore aspects of organizational competence, as outlined by @xcite . more specifically, we focus on the sub-components of grammatical competence and textual competence, within which we analyse how lexical, morphological, syntactic and cohesion usage varies for authors from different geographical locations. through our analyses, we identify a number of characteristics that vary widely between writers from different linguistic backgrounds. for instance, we identify that in the china, japan and india corpora from our dataset, there are varying preferences for specific lexical bundles, and all corpora in our dataset have vast differences in the discourse connectors that are preferred. therefore, to support writers whose usage patterns may impact the expression of the ideas presented in their work and enhance their chances of being evaluated fairly in international academic publications, we also outline a set of recommendations for conferences and journals regarding the author guidelines, resources and tools they make available. in summary, the contributions of our work are as follows: ## related work some of the earliest work related to differences in language output based on one's native language comes from @xcite , who posited that features of the target language that differed greatly from a learner's native language would be more difficult for the learner to learn. conversely, features of the target language that are similar to a learner's native language would be easier for the learner to learn. more recently, with the support of learner corpora, such as @xcite , researchers have sought to identify how writers of different native languages differ in their writing when compared with native english speakers. we summarize findings of works focusing on lexis, morphology, syntax and cohesion below. lexis existing works on lexis have identified that measures of lexical diversity, sophistication and density are higher for non-native english-speaking students with higher proficiency @xcite . non-native english speakers also tend to use more lexical bundles and use them more frequently than native english speakers @xcite @xcite in order to produce text that appears more native-like. morphology it has been found that it is possible for morphological complexity to reach native-like levels for second-language speakers @xcite . however, it is common for complexity to be lower for those whose native language is less morphologically complex than the target language, as they may have yet to acquire the target structure. syntax comparisons of syntax are typically made through the lens of complexity at different levels, such as phrasal, clausal and sentence. previous works have identified developmental stages for syntactic complexity in second-language learners of english. for example, @xcite proposed that complement clauses are developed first and then phrasal modifiers are acquired later. other research, however, has found that these developmental stages are not always observed. for instance, @xcite recognized that chinese, japanese and russian writers use far less subordination than both native english speakers and other non-native english speakers. they also recognized that chinese writers use fewer coordinating clauses than writers from other backgrounds. the authors suggested that these observations are a result of native-language influence. cohesion it is common for researchers to compare the usage of discourse connectors of different groups of writers. for instance, milton and tsang (1993) compared native english speakers and cantonese speakers in their use of 25 connectors in the categories of additive, adversative, causal and sequential. they found that the cantonese speakers had a considerably higher usage for a number of connectors such as firstly, secondly, lastly, besides, moreover and therefore; and lower usage for connectors such as likewise and previously. existing works on contrastive analysis in a second-language setting generally focus on analysing writing by language learners or undergraduate students. moreover, they are limited in their scope, typically focusing on one area of language -e.g., syntax @xcite or cohesion @xcite . this study differs from previous research in that it is the first to perform a contrastive analysis of professional academic writing in the nlp domain. thus, we are able to analyze characteristics specific to academic writing in our target domain and can give more relevant recommendations. a further point of difference is that we carry out a more comprehensive analysis than previous research, focusing on a wider range of language features. ## data collection our data foot_0 are acquired from the following academic nlp venues: acl 2018-2021, emnlp in order to ensure we have a sufficient quantity of text on which to perform our analyses, we limit our analyses to the five most common geographic locations in the dataset. an overview of our data is shown in ## analyses as outlined in section 1, we perform contrastive analyses across the lexical, morphological, syntactic, and cohesion dimensions of the different geographic locations in our dataset. in summary, we find the following: lexis in a number of corpora in our dataset, synonyms, hypernyms and hyponyms are used at a lower rate, resulting in lower lexical complexity. in addition, in these corpora there is an increase in usage of fixed lexical expressions. this is evident in the china, india and japan corpora, for example, where the usage of fixed lexical expressions is roughly double that of the united states corpus. ## recommendations in order to make progress on the issue of linguistic bias and based on the findings of our analyses, we outline a set of recommendations to academic journals and conferences regarding how their author guidelines could support prospective authors from around the globe. our recommendations are focused on the four areas of analysis performed in this study and are summarized as follows: lexis we found variation in lexical diversity across each of the investigated corpora due to differences in the use of synonyms, hypernyms and hyponyms. we recommend that author guidelines include lists of commonly used terms in nlp research and include examples of suitable synonyms. we also found that there was a higher use of lexical bundles in a number of the corpora studied. to help avoid over-reliance on lexical bundles, we propose that author guidelines include suggestions for how different pragmatic functions of the paper can be expressed. for instance, we found that different corpora in our dataset had different preferences for bundle usage as a means for introducing a paper's main idea. the author guidelines should provide multiple examples for how authors can introduce the main idea of their work, in addition to examples for other common pragmatic functions. morphology as we have identified, there are different preferences for grammatical tense across the different corpora in our dataset. in academic nlp papers, there are a number of common functions expressed, such as introducing one's contributions, citing related work, describing methods, introducing experiments, discussing results, etc. each function carries its own set of expectations regarding the grammatical tense to be used (for example, which tense to use when citing literature in a related work section). to aid writers regarding usage of grammatical tense, we suggest that clear guidelines accompanied by sufficient examples be provided regarding tense usage for specific functions. syntax from our analysis, we can see that in some of the corpora investigated, there is a preference for expressing complex points through complex phrase structures. increased complexity with regards to phrasing can impact understanding and communicability. we suggest that author guidelines include clear examples of how to increase readability through splitting phrases containing multiple modifiers. journals and conferences could also provide free access to automated writing tools capable of paraphrasing, thus providing authors with ideas on how to rewrite complex content. cohesion as we have shown in this study, the preference for different discourse connectors across many of the corpora investigated are highly variable. we recommend providing a list of common connectors and examples of how they can be used. this can also be accompanied by a list of frequently overused or misused connectors along with alternative options for connectors. ## conclusions and future work this paper seeks to address the issue of linguistic bias in academic publishing. our comprehensive contrastive analysis of academic writing in the nlp domain identifies a number of characteristics that are highly variable across writers from different nationalities. these findings highlight the potential risk of linguistic bias. to mitigate this risk, we outline a set of recommendations to academic publishers about how their resources could better support writers in presenting their ideas and contributions and thus ensuring that their work is fairly evaluated. regarding possible directions for future work, we propose that more work be done to analyze the writing of a wider range of author backgrounds. moreover, our analyses could provide a basis for integration into writing correction and suggestion tools. with current solutions typically focusing on providing corrections for grammatical or convention-based errors, there is scope to extend this to suggestions based on writing characteristics where expression may impact an author's chance of being assessed fairly, but not necessarily be incorrect grammatically or according to conventions.