# Large Language Model-Powered Automated Assessment: A Systematic Review
## 1 Introduction  
Assessment in education is a crucial element used to determine whether learning has been achieved. Various types of questions are utilized in assessment processes, which can be categorized into two main groups: objective questions (multiple choice, true-false, fill-in-the-blank) and subjective questions (open-ended questions, essays) [1]. Today, objective questions are often preferred because they are easier to grade than subjective questions. However, subjective questions are considered to contribute to more long-term and effective learning than objective questions, as they encourage students to think critically, explain their thought processes, and apply their knowledge [2]. However, subjective questions present several challenges, including no single correct answer, the requirement for evaluators to analyze student responses, the time-consuming nature of assessment, increased workload for educators, and potential evaluator bias [3]. Considering these challenges, there has been growing interest in automated scoring systems, which are often assumed to improve consistency and reduce human bias. This assumption has been questioned. Recent research indicates that automated assessment tools can also perpetuate or amplify inherent biases [4]. As a result, there is a need for new tools to facilitate the evaluation of subjective questions. In this context, automated essay scoring (AES) and automated short answer grading (ASAG) have emerged as key components of automated assessment [5].  
AES is a software-based system developed to automatically score essays [6]. Similarly, ASAG is a system that builds a model for automated short-answer scoring [7].  
Although both systems use natural language processing (NLP) and machine learning (ML) to evaluate text, AES focuses on long-form essay scoring, whereas ASAG is designed for short-answer responses.  
Recently, with the emergence of LLMs, their potential applications in education have become a focal point for researchers. The ability of LLMs to be adapted to various assessment tasks (e.g., grading, scoring) with minimal training data and their lack of need for technical expertise make them promising tools for assessing student learning more quickly and effectively while providing personalized feedback [8; 9]. Thus, there has been a noticeable increase in research on LLM-powered AES and ASAG in the recent literature. While these capabilities are promising, they raise significant ethical and practical concerns. Emerging studies highlight that LLMs can exhibit inherent biases that may affect grading fairness [10]. [PERSON] et al. found that algorithmic bias in LLMs can lead to inequitable scoring for minority and non-standard language students. Additionally, data privacy and security have been identified as pressing issues, since student information processed by LLMs requires robust protection. [PERSON] emphasizes the need for provable ethical safeguards and transparency to build trust in AI educational tools [11]. Moreover, human oversight remains crucial: educators should review and contextualize automated scores to ensure reliability and address nuances that AI might miss [4]. However, it remains unclear how LLM-powered models influence assessment processes, their agreement with human scoring, the overall reliability and validity of these tools, and whether they can effectively conduct automated assessments.  
In summary, this study systematically analyzes research on Large Language Model-Powered Automated Assessment (LLMPAA). Throughout this document, AES and ASAG studies incorporating LLMs are collectively referred to as LLMPAA. Within this scope, the following research questions are explored:  
(RQ1) What are the usage domains of studies with LLMPAA?  
(RQ2) What are the details of using LLMPAA?  
(RQ3) What are the challenges in LLMPAA?