Fairness in Automated Essay Scoring: A Comparative Analysis of Algorithms on G erman Learner Essays from Secondary Education (2024)

Abstract: Pursuing educational equity, particularly in writing instruction, requires that all students receive fair (i.e., accurate and unbiased) assessment and feedback on their texts. Automated Essay Scoring (AES) algorithms have so far focused on optimizing the mean accuracy of their scores and paid less attention to fair scores for all subgroups, although research shows that students receive unfair scores on their essays in relation to demographic variables, which in turn are related to their writing competence. We add to the literature arguing that AES should also optimize for fairness by presenting insights on the fairness of scoring algorithms on a corpus of learner texts in the German language and introduce the novelty of examining fairness on psychological and demographic differences in addition to demographic differences. We compare shallow learning, deep learning, and large language models with full and skewed subsets of training data to investigate what is needed for fair scoring. The results show that training on a skewed subset of higher and lower cognitive ability students shows no bias but very low accuracy for students outside the training set. Our results highlight the need for specific training data on all relevant user groups, not only for demographic background variables but also for cognitive abilities as psychological student characteristics.

## introduction educational equity is seen as a foundation for learning with technology @xcite , because all students need effective instruction. one of the most effective instructional practices is feedback @xcite , which can support students in acquiring complex skills like writing @xcite . automated essay scoring (aes) can be used to provide students with feedback on their writing at scale @xcite . the foundation of equity in automated feedback systems is the fairness of the algorithm ( @xcite , @xcite ), i.e., the absence of any prejudice or favoritism toward groups of students based on their inherent or acquired characteristics, including their background and their psychological variables( @xcite , @xcite ). algorithmic fairness is widely discussed in various educational contexts from normative @xcite ; european commission, directorate-general for education, youth, sport and culture, 2022), societal @xcite , or methodological @xcite perspectives, but literature reviews have shown that it is rarely investigated empirically @xcite . specifically in the aes context, only six empirical studies have examined algorithmic fairness, examining differences in algorithmic accuracy and biases for students with different gender, race, and language backgrounds in english-language corpora @xcite @xcite @xcite . this means that while aes is widely used in education in many countries @xcite including non-english speaking countries, it is unclear whether the algorithms used are fair to all groups of students confronted with the results or whether they might disfavor some student gropus. compounding the problem, the few existing studies have shown that, depending on the algorithms used, students' essays were not scored fairly and disfavored groups related to race/ethnicity, economic status, and english language learner status (e.g., @xcite ; @xcite ; @xcite ). so far, previous studies only analyzed fairness in relation to students' demographic variables in corpora with students' essays in english: extending this research to a corpus on argumentation essays in the german language, we address three main re-search questions: (1) how fair are aes algorithms for students with different levels of cognitive abilities as psychological characteristics strongly related to writing competence? @xcite . addressing this question is linked to the wider equity issue of whether aes systems are likely to widen or narrow the gap between high and low-performing students. (2) how fair are aes algorithms in languages other than english? the question is especially important when automated scoring is based on large language models, mostly trained on english text data. (3) how is the distribution of student characteristics in the training data impacting the mean accuracy and fairness of the prediction? by answering these questions, our paper makes the following contributions: first, we provide a set of baseline models, including shallow learning, deep learning, and generative large language models (llm), for the newly released darius corpus, thus enriching the automatic scoring landscape with models for a large german argumentative writing corpus. second, we conduct fairness evaluations on our results indicating that none of the models trained on the entirety of training data shows unfair behavior towards specific subgroups. finally, to assess the role of the distribution of the training data on algorithmic fairness, we train shallow and deep models with subsets of data from students of low and high cognitive ability, as well as a mixed subset based on low, medium and high cognitive ability, and show that the models are unfair to the groups not included in the training set. we make all of our code publicly available. 1 ## related work: fairness in aes algorithm according to a literature review by @xcite , there have been 49 peer-reviewed empirical studies focused on fairness and predictive bias in education since 2010, highlighting the growing academic interest in these issues. the studies included multiple fairness measures, including the accuracy for the included groups and the mean differences between predicted and annotated scores for each score (e.g., @xcite ). most of these studies were conducted in contexts other than aes, such as predicting students' course performance or their likelihood of 1 https://github.com/darius-ipn/fairness_aes dropping out of a course. to our knowledge, there are only two papers that diagnosed the predictive bias displayed by aes models @xcite , even though the importance of this task has been pointed out as early as in 2012 @xcite . @xcite evaluated the fairness of shallow and deep learning aes algorithms for essays from the upper elementary level in the english language using three measures: overall score accuracy (osa), overall score difference (osd), and conditional score difference (csd). they found that shallow and deep aes algorithms showed systematically overly positive and negative scoring depending on students' gender, race, and socioeconomic status. arthurs and alvero (2020) showed that a shallow learning aes system for college admissions essays based on word vectors favored high-income students over low-income students (see also @xcite for similar results for essays from the test of english as a foreign language). additionally, the authors trained models on only essays from the highest quartile of students in terms of performance, showing that these models are not suitable for students from the other quartiles. @xcite further emphasized that the fairness of aes systems is compromised if such models are used on students or tasks for which they have not been trained. in addition to the studies included in the literature review, recent studies added an investigation of fairness in large language models scoring essays from a high school context @xcite in the persuade 2.0 corpus @xcite . the authors compared the winning entries of the kaggle feedback prize competition. foot_0 they show differences in the model's accuracy based on demographic factors such as student race/ethnicity, and economic disadvantage. similar fairness issues based on students' demographic variables were shown for large language models in essays in the english language written by first @xcite and second language students @xcite . in summary, previous studies on fairness in aes have used shallow learning models, deep learning models, and llms and compared whether the accuracy of judgments and systematic over/underrating can be explained by students' demographic vari-ables. the results showed some fairness problems, which were exacerbated in the studies where the aes was additionally trained only on a homogenous group of students. ## data the darius corpus is a collection of 4,589 annotated argumentative texts written by 1,839 students from german high schools, spread across 114 classes in 33 different schools @xcite . essays that were off-topic, shorter than two sentences, empty, or contained names or other data relevant to data protection were removed beforehand. the final dataset consists of essays from two writing assignments focused on socio-scientific issues on the topics energy and automotive, containing 2,307 and 2,282 essays respectively. students wrote a draft and revision on one task, followed by an essay on the other task, resulting in up to 3 essays per student. an example text is listed in the appendix 7. students also provided demographic data voluntarily, a selection of which is listed in the dataset has been extensively annotated with information about argumentative structure on different levels of granularity. in the present study, we focus specifically on a subset of these annotations, namely content zone, major claim, position and warrant. out of the nine original annotation categories, we selected those as they reflect different parts of an argumentative text, e.g. structure and content, and are annotated on different granularity levels (token level to whole texts). we used the demographic data to measure fairness with respect to gender, profile, school, cognitive ability (kft), and languages, which are further explained after providing more details on the annotations in section 3.1. a more extensive description can be found in the original paper @xcite . ## method in the following section, we describe the experimental setup for our evaluation study. ## experimental study in the following, we discuss the results of our experimental studies. we compare the three classification model types (shallow, deep, and llm) with respect to both fairness and kappa. in the first experiment, we trained on the complete dataset and evaluated the fairness for certain subgroups. in a second experiment, we trained models on subsets of the training data that represent only a specific part of the whole population (in our case, the upper and lower quartiles of the cognitive ability values) and examined the fairness of such models. ## conclusion and future work in our work, we provide three basic models (shallow learning models, deep learning models, and llm) trained on the annotations of the darius corpus of learner texts in german. these models are ready to use in schools, for example, to create a feedback tool for training argumentative skills. evaluation of model fairness showed that all models produced fair scores for all students, considering demographic and psychological differences among students. in a second experiment, we trained our models on subgroups of students, based on either low, high, or mixed cognitive abilities, to investigate the extent to which skewed training data leads to unfair aes system scores. our results showed lower performance for students who were not in the training data, emphasizing the importance of including samples of the full range of users in the training data for aes, not only for demographic background variables but also for psychological aspects such as cognitive abilites. fail-ure to do so risks reducing the predictive accuracy of the algorithm for those who are not adequately represented. to mitigate the risk of students receiving unfair scores based on their demographic and psychological variables, we advocate that future aes systems incorporate the goal of fairness in addition to accuracy into their training data collection and algorithm optimization function, going beyond the current state of retrospective analysis of model fairness.