LLM Agents at the Roundtable: A Multi-Perspective and Dialectical Reasoning Framework for Essay Scoring (2025)

Abstract: The emergence of large language models (LLMs) has brought a new paradigm to automated essay scoring (AES), a long-standing and practical application of natural language processing in education. However, achieving human-level multi-perspective understanding and judgment remains a challenge. In this work, we propose Roundtable Essay Scoring (RES), a multi-agent evaluation framework designed to perform precise and human-aligned scoring under a zero-shot setting. RES constructs evaluator agents based on LLMs, each tailored to a specific prompt and topic context. Each agent independently generates a trait-based rubric and conducts a multi-perspective evaluation. Then, by simulating a roundtable-style discussion, RES consolidates individual evaluations through a dialectical reasoning process to produce a final holistic score that more closely aligns with human evaluation. By enabling collaboration and consensus among agents with diverse evaluation perspectives, RES outperforms prior zero-shot AES approaches. Experiments on the ASAP dataset using ChatGPT and Claude show that RES achieves up to a 34.86% improvement in average QWK over straightforward prompting (Vanilla) methods.

## introduction essay scoring has long been used in educational settings as a tool for evaluating students' reasoning, expression, and thinking skills in a comprehensive manner @xcite . however, human evaluation is often limited by the high cost and time required @xcite . to address this issue, automated essay scoring (aes) systems have been actively studied for decades, with efforts focusing on improving efficiency and reliability through statistical features and machine learning techniques @xcite . ## roundtable essay scoring (res) res is a multi-agent framework that evaluates essays from multiple perspectives through collaborative dialectical reasoning. as shown in figure 1 , res operates in two stages: multi-perspective evaluation and dialectical reasoning. ## conclusion we propose res (roundtable essay scoring), a multi-agent framework for zero-shot aes consisting of two stages: multi-perspective evaluation, where agents independently generate trait-based rubrics and evaluate essays, and dialectical reasoning, where agents engage in dialogue to reach a consensus score. this structure improves evaluation precision and promotes collaborative judgment, achieving stronger alignment with human evaluators than single-llm approaches. experiments on the asap dataset demonstrate res's superior performance and the practical utility of llm-based evaluation methods in educational assessment, a domain where accurate and scalable evaluation remains a long-standing challenge.