Autoregressive Score Generation for Multi-trait Essay Scoring (2024)

Abstract: Recently, encoder-only pre-trained models such as BERT have been successfully applied in automated essay scoring (AES) to predict a single overall score. However, studies have yet to explore these models in multi-trait AES, possibly due to the inefficiency of replicating BERT-based models for each trait. Breaking away from the existing sole use of *encoder*, we propose an autoregressive prediction of multi-trait scores (ArTS), incorporating a *decoding* process by leveraging the pre-trained T5. Unlike prior regression or classification methods, we redefine AES as a score-generation task, allowing a single model to predict multiple scores. During decoding, the subsequent trait prediction can benefit by conditioning on the preceding trait scores. Experimental results proved the efficacy of ArTS, showing over 5% average improvements in both prompts and traits.

## introduction automated essay scoring (aes) is a prominent task to efficiently assess large volumes of essays. currently, there is a growing trend in holistic aes to use pre-trained bert-based models, showing promising results @xcite @xcite @xcite . however, these models have yet to be explored in multi-trait aes, which evaluates essays on diverse rubrics, possibly due to the inefficiency of duplicating encoders for different traits. existing multi-trait scoring approaches @xcite @xcite typically adopted holistic scoring models @xcite , adding multiple linear layers or separate trait-specific layers for different traits. however, achieving multi-trait aes as a holistic method overlooks the trait dependencies, and constructing separate trait-specific modules is resource-inefficient, leading to inferior qualities in data-scarce traits. these limitations highlight the need for optimized multi-trait strategies. in this paper, we propose autoregressive multitrait scoring of essays (arts), which incorporates the decoding process by leveraging a pre-trained language model, t5 @xcite . moving beyond the conventional sole reliance on the encoder, we introduce a novel text-to-text aes framework. unlike existing regression or classification approaches to output a separate numeric value, we aim at precise sequence generation by considering multi-trait scores as an entire sequence; thus, a single model can yield multi-score predictions. arts employs causal self-attention to capture the intrinsic relations of the traits by sequentially predicting text-transformed trait scores. the autoregressive generation allows the subsequent trait prediction to benefit from referencing preceding trait scores. arts remarkably outperformed the baseline model on the asap and asap++ @xcite datasets. ablation studies and additional discussions of trait order further verify our method. furthermore, arts achieved training efficiency by using a single model to generate multiple predictions across all prompts, avoiding the duplication of the same modules. codes and datasets are available on github foot_0 . ## related work early studies of aes mainly focused on holistic essay scoring that only predicts the overall score and already achieved high assessment performance @xcite @xcite @xcite . in contrast, multi-trait scoring has been studied for detailed assessments lately, yet showing far-lagged quality. holistic scoring structures are typically employed either for a trait-shared model followed by multiple linear layers @xcite , [input] ## autoregressive essay multi-trait scoring (arts) to predict multiple trait scores in an auto-regressive manner, we fine-tune the pre-trained encoderdecoder language model, t5. specifically, we treat aes as a generation task to predict a single sequential text rather than multiple numeric values for traits. subsequently, we extract each trait score from the generated text comprising the predicted trait scores along with trait names (figure 1 ). ## results our model is primarily compared with the baseline mtl-bilstm model @xcite , multitask learning where auxiliary multi-trait scoring tasks aid holistic scoring ( ## conclusion in this paper, we introduce an autoregressive multitrait scoring of essays that leverages the capacity of the pre-trained language model, t5. our model exhibits remarkably improved results, demonstrating its ability to overcome far-lagging multi-traitscoring performances. furthermore, our approach allows a single model to make multi-trait score predictions across all prompts, avoiding the use of redundant modules and promoting simplicity and training efficiency. this indicates that a new paradigm of generating score sequences holds profound implications for future aes, opening new avenues for advanced multi-trait scoring.