Improving Domain Generalization for Prompt-Aware Essay Scoring via Disentangled Representation Learning (2023)

Abstract: Automated Essay Scoring (AES) aims to score essays written in response to specific prompts. Many AES models have been proposed, but most of them are either prompt-specific or prompt-adaptive and cannot generalize well on “unseen” prompts. This work focuses on improving the generalization ability of AES models from the perspective of domain generalization, where the data of target prompts cannot be accessed during training. Specifically, we propose a prompt-aware neural AES model to extract comprehensive representation for essay scoring, including both prompt-invariant and prompt-specific features. To improve the generalization of representation, we further propose a novel disentangled representation learning framework. In this framework, a contrastive norm-angular alignment strategy and a counterfactual self-training strategy are designed to disentangle the prompt-invariant information and prompt-specific information in representation. Extensive experimental results on datasets of both ASAP and TOEFL11 demonstrate the effectiveness of our method under the domain generalization setting.

## introduction automated essay scoring (aes), which aims to score essays written for specific prompts, is helpful in reducing the burden of scoring staff in various writing tests @xcite . over the past few years, supervised deep learning has achieved remarkable success on the prompt-specific aes task @xcite @xcite , which assumes that the training and test data are from the same prompt. however, in many real-world scenarios, the training and test data often come from different prompts, which leads to a performance degradation of promptspecific aes model on the out-of-distribution tar- ## conclusion in this paper, we focus on the prompt-generalized aes task. we propose the prompt-aware neural network model pann to comprehensively evaluate the essays in terms of both prompt adherence and writing quality. to improve its generalization, we further propose a disentangled representation learning framework, including two representation disentanglement strategies. experimental results demonstrate the effectiveness of the proposed method for prompt-generalized essay scoring.