# Evaluating LLMs for Automated Scoring in Formative Assessments  
[PERSON]  
1 Faculty of Exact Sciences and Engineering, Penteada University Campus, University of Madeira,  
9000-082 Funchal, Portugal; [EMAIL_ADDRESS] (F.Q. ); [EMAIL_ADDRESS] (F.M.) [PERSON]  
1 Faculty of Exact Sciences and Engineering, Penteada University Campus, University of Madeira,  
9000-082 Funchal, Portugal; [EMAIL_ADDRESS] (F.Q. ); [EMAIL_ADDRESS] (F.M.) [PERSON]  
1 Faculty of Exact Sciences and Engineering, Penteada University Campus, University of Madeira,  
9000-082 Funchal, Portugal; [EMAIL_ADDRESS] (F.Q. ); [EMAIL_ADDRESS] (F.M.) Footnote 1: email: {fim  
tasks, such as essay evaluations, by analyzing content and structure. These systems streamline assessments, reduce instructor workloads, and ensure consistent and unbiased evaluations [1; 4]. Compared to manual scoring methods, the scalability and efficiency of LLM-based systems are especially evident in larger classrooms and online environments [1; 4]. [PERSON] et al. [4] emphasize that AI-driven systems provide relevant time savings while maintaining quality and reliability in scoring processes. Similarly, [PERSON] et al. [2] underline how these systems deliver precise assessments tailored to individual student needs, addressing disparities often present in traditional scoring. Such innovations allow for greater scalability, making them particularly effective in larger classrooms or online environments [2; 4]. These complementary findings collectively highlight LLMs' role in reducing variability across assessments, thus increasing equity in student evaluations [2; 4]. A recent study by [PERSON] et al. [5] focused on Automatic Short Answer Grading (ASAG) tasks, finding that GPT-4, with minimal prompt engineering, performed at a level comparable to expert human raters. This result suggests that generative LLMs like GPT-4 could be integrated into real-world educational systems to grade formative assessment tasks reliably and efficiently [5]. Additionally, GPT-4's flexibility in adapting to various scoring contexts with few-shot prompting and minimal adjustments underscores its advantage over fine-tuned systems [6]. This highlights a key benefit of generative models in educational applications, where their ability to align with the gold standard of human ratings demonstrates potential for scalable, automated assessment systems [5; 7]. Despite these advancements, several studies stress the importance of human oversight [6; 7; 8]. LLMs can effectively automate grading, but educators play a crucial role in addressing non-standard questions, ensuring fairness, and upholding ethical practices. [PERSON] and [PERSON] [7] emphasize the need for AI systems to complement rather than replace instructors, maintaining a balanced approach that leverages the strengths of both human judgment and machine efficiency [6; 7; 8]. Despite their potential, LLMs face critical challenges regarding reliability, fairness, and ethical considerations [1; 4]. An important concern is their tendency for hallucinations, where models generate inaccurate or non-existent information [1; 4; 9; 10]. Additionally, data privacy and security remain pressing issues, especially in educational settings, where sensitive information requires robust protection to maintain trust [1; 4; 11]. Algorithmic bias in LLMs is another concern, as it can result in inequitable grading for minority groups or non-standard language users, undermining fairness [4]. Misuse by students, such as employing LLMs to generate answers, further raises ethical questions and challenges the integrity of the learning process [4]. Moreover, the rapid development of new models complicates research comparability and generalizability, highlighting the need for continuous evaluation across diverse contexts [6]. To address the limitations of LLMs, particularly hallucinations and outdated information, Retrieval-Augmented Generation (RAG) offers a practical solution by integrating external knowledge sources.