Enhancing Automated Essay Scoring Performance via Fine-tuning Pre-trained Language Models with Combination of Regression and Ranking (2020)

Abstract: Automated Essay Scoring (AES) is a critical text regression task that automatically assigns scores to essays based on their writing quality. Recently, the performance of sentence prediction tasks has been largely improved by using Pre-trained Language Models via fusing representations from different layers, constructing an auxiliary sentence, using multi-task learning, etc. However, to solve the AES task, previous works utilize shallow neural networks to learn essay representations and constrain calculated scores with regression loss or ranking loss, respectively. Since shallow neural networks trained on limited samples show poor performance to capture deep semantic of texts. And without an accurate scoring function, ranking loss and regression loss measures two different aspects of the calculated scores. To improve AES’s performance, we find a new way to fine-tune pre-trained language models with multiple losses of the same task. In this paper, we propose to utilize a pre-trained language model to learn text representations first. With scores calculated from the representations, mean square error loss and the batch-wise ListNet loss with dynamic weights constrain the scores simultaneously. We utilize Quadratic Weighted Kappa to evaluate our model on the Automated Student Assessment Prize dataset. Our model outperforms not only state-of-the-art neural models near 3 percent but also the latest statistic model. Especially on the two narrative prompts, our model performs much better than all other state-of-the-art models.

## introduction automated essay scoring (aes) automatically evaluates the writing quality of essays. essay assignments evaluation costs lots of time. besides, the same instructor scoring the same essay at different times may assign different scores (intra-rater variation), different raters scoring the same essay may assign different scores (inter-rater variation) @xcite . to alleviate teachers' burden and avoid intra-rater variation, as well as interrater variation, aes is necessary and essential. an early aes system, e-rater @xcite , has been used to score toefl writings. recently, large pre-trained language models, such as gpt @xcite , bert @xcite , xlnet @xcite , etc. have shown the extraordinary ability of representation and generalization. these models have gained better performance in lots of downstream tasks such as text classification and regression. there are many new approaches to fine-tune pre-trained language models. @xcite proposed to construct an auxiliary sentence to solve aspect-based sentiment classification tasks. @xcite added extra separate tokens to obtain representations of each sentence to solve sequential sentence classification tasks. @xcite summarized several fine-tuning methods, including fusing text representations from different layers, utilizing multi-task learning, etc. to our knowledge, there are no existing works to improve aes tasks with pre-trained language models. before introducing our new way to use pre-trained language models, we briefly review existing works in aes firstly. existing works utilize different methods to learn text representations and constrain scores, which are the two key steps in aes models. for text representation learning, various neural networks are used to learn essay representations, such as recurrent neural network (rnn) @xcite , convolutional neural network (cnn) @xcite , recurrent convolutional neural network (rcnn) @xcite , etc. however, simple neural networks like rnn and cnn focus on word-level information, which is difficult to capture word connections in long-distance dependency. besides, shallow neural networks trained on a small volume of labeled data are hard to learn deep semantics. as for score constraints, prediction and ranking are two popular solutions. from the prediction perspective, the task is a regression or classification problem @xcite @xcite . besides, from the recommendation perspective, learning-to-rank methods @xcite aim to rank all essays in the same order as that ranked by gold scores. however, without precise score mapping functions, only regression constraints could not ensure the right ranking order. and only ranking based models could not guarantee accurate scores. in general, there are two key challenges for the aes task. one is how to learn better essay representations to evaluate the writing quality, the other one is how to learn a more accurate score mapping function. motivated by the great success of pre-trained language models such as bert in learning text representations with deep semantics, it is reasonable to utilize bert to learn essay representations. since self-attention is a key component of the bert model, it can capture the interactions between any two words in the whole essays (long texts). previous work @xcite shows that fusing text representations from different layers does not improve the performance effectively. for the aes task, the length of essays approximates the length limit of the bert model, so it is hard to construct an auxiliary sentence. meanwhile, only score labels are available; it is also difficult to utilize multitask learning. summarized existing works in aes, they utilize regression loss or ranking loss, respectively. regression loss requires to obtain accurate score value, and ranking loss aims to get precise score order. unlike multi-task learning requires different fully-connected networks for different tasks, we propose to constrain the same task with multiple losses to fine-tune the bert model. in addition, it is impossible to rank all essays in one batch so that the model is required to learn more accurate scores. during training, the weight of the regression loss is increasing while that of ranking loss is decreasing. in this paper, we propose r 2 bert (bert model with regression and ranking). in our model, bert is used to learn text representations to capture deep semantics. then a fully connected neural network is used to map the representations to scores. finally, regression loss and batch-wise ranking loss constrain the scores together, which are jointly optimized with dynamic combination weights. to evaluate our model, an open dataset, automated student assessment prize (asap), is used. with the measurement of quadratic weighted kappa (qwk), our model outperforms state-of-the-art neural models on average qwk score of all eight prompts near 3 percent and also performs better than the latest statistical model. especially on the two narrative prompts (7 and 8), only the regression based model performs comparably even better compared with other models. and our model with combined loss gains much better performance. to explain the model's effectiveness, we also illustrate the attention weights on two example essays (an argumentative essay and a narrative essay). the self-attention can capture most conjunction words that reveal the logical structure, and most key concepts that show the topic shifting of the narratives. in summary, our contributions are: according to different score constraints, existing works fall into three categories, namely prediction, recommendation, and reinforcement learning based models. prediction is the most general approach, including classification and regression. for classification, the models directly predict labels that point to different scores. in comparison, regression models constrain calculated scores to be the same as gold ones. generally, hand-crafted features and neural network based features are two popular methods to learn text representations. early works mainly focus on the construction of hand-crafted features such as statistical features and linguistic features. there are several early aes systems including e-rater @xcite , peg (project essay grade) @xcite , and intellimetric @xcite . e-rater utilized ten linguistic features, including eight representing aspects of writing quality and two representing content. peg used a larger feature set with more than 30 elements of writing quality. intelli-metric aggregated all the features into five types, namely focus/coherence, organization, elaboration/development, sentence structure, and mechanics/conventions. @xcite combined string kernel and word embeddings to extract features. with the success of deep learning, researchers start to utilize various neural networks to learn text representations. @xcite explored several neural networks, such as long short-term memory (lstm) and cnn. finally, they found that the ensemble model combining lstm and cnn performs best. @xcite proposed a hierarchical text model that utilized cnn to learn sentence representations, and lstm was used to learn text representations. @xcite introduced a model called skipflow, which aimed to capture neural coherence features of the text via considering the adjacent hidden states in the lstm model. in the recommendation view, learning to rank approaches is another popular method to solve this task. @xcite firstly addressed this problem as a rank preference problem. based on statistical features, ranksvm, a pairwise learning to rank model, was used as score constraint. @xcite utilized listwise learning to rank model to learn a ranking model based on several linguistic features. reinforcement learning based models are also possible solutions. @xcite utilized dilated lstm to learn text representations. then scores calculation was guided by quadratic weighted kappa based reward function. for text representation, previous works only consider the relations among sentences. in this paper, we focus on all the interactions between any two words. besides, existing works only utilize regression or ranking loss, respectively. we combine two losses dynamically in our model. ## r 2 bert in this section, we first introduce the framework of our model, briefly review the bert model, as well as self-attention. in addition, we will illustrate the regression model as well as some useful tricks. finally, we will show batch-wise learning to rank model and the combination metric. our model, as shown in figure 1 , takes a batch of essays as input. with preprocessing (adding a special token, [cls], at the beginning of each essay), each token is transformed into its embedding and sent into the bert model. the representations of all essays are the output vectors mapping to @xcite . essay scores could be obtained by passing the representations into the score mapping function. they are constrained by regression loss and ranking loss, which are optimized jointly with the dynamic combination. as shown in the color bar, the weight of regression loss is gradually increasing, while that of ranking loss is decreasing. ## experiment in this section, the asap dataset is introduced firstly. then we illustrate experiment settings and evaluation metrics. in addition, baseline models, experiment results, and analyses are shown. furthermore, we also visualize the attention weights of different words in two examples. ## conclusion and future works from experimental results, we can obtain several conclusions: 1) bert is a significantly effective model to improve the performance of downstream natural language processing tasks. 2) regression loss and ranking loss are two complementary losses. 3) simply fine-tuning on bert is not enough. multi-loss objective is an effective approach to finetune the bert model. 4) self-attention is useful to capture conjunction words and key concepts in essays. in the future, we will investigate how to uti-