Title: SketchMind:  A Multi-Agent Cognitive Framework for Assessing Student-Drawn Scientific Sketches
Venue: NeurIPS 2025
Accepted: yes
Keywords: Educational AI

Multimodal Learning

Scientific Reasoning

Cognitive Modeling

Sketch-Based Assessment
Abstract: Scientific sketches (e.g., models) offer a powerful lens into students' conceptual understanding, yet AI-powered automated assessment of such free-form, visually diverse artifacts remains a critical challenge. Existing solutions often treat sketch evaluation as either an image classification task or monolithic vision-language models, which lack interpretability, pedagogical alignment, and adaptability across cognitive levels. To address these limitations, we present SketchMind, a cognitively grounded, multi-agent framework for evaluating and improving student-drawn scientific sketches. SketchMind introduces Sketch Reasoning Graphs (SRGs), semantic graph representations that embed domain concepts and Bloom's taxonomy-based cognitive labels. The system comprises modular agents responsible for rubric parsing, sketch perception, cognitive alignment, and iterative feedback with sketch modification, enabling personalized and transparent evaluation. We evaluate SketchMind on a curated dataset of 3,575 student-generated sketches across six science assessment items with different highest order of Bloom's level that require students to draw models to explain phenomena. Compared to baseline GPT-4o performance without SRG (average accuracy: 55.6%), the model with SRG integration achieves 77.1% average accuracy (+21.4% average absolute gain). We also demonstrate that multi-agent orchestration with SRG enhances SketchMind performance, for example, SketchMind with GPT-4.1 gains an average 8.9% increase in sketch prediction accuracy, outperforming single-agent pipelines across all items. Human evaluators rated the feedback and co-created sketches generated by SketchMind with GPT-4.1, which achieved an average of 4.1 out of 5, significantly higher than those of baseline models (e.g., 2.3 for GPT-4o). Experts noted the system’s potential to meaningfully support conceptual growth through guided revision. Our code and (pending approval) dataset will be released to support reproducibility and future research in AI-driven education.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a multi-agent framework for evaluating student-drawn scientific sketches and providing automated feedback. The framework comprises four specialized agents:\n\nAgent 1 generates gold-standard sketch reasoning graphs (SRGs) for given scientific questions using multimodal large language models (MLLMs). \nAgent 2 constructs corresponding SRGs based on student-drawn sketches, also utilizing MLLMs. \nAgent 3 evaluates student performance by computing similarity scores between the gold-standard and student-generated SRGs, and additionally provides learning level assessments and feedback signals through rule-based logic. \nAgent 4 generates visual sketch hints to guide student learning using MLLMs.\n\nThe authors evaluate their proposed framework on the dataset from Zhai et al. (2022) [28], which consists of six scientific modeling assessment tasks. The evaluation employs both closed-source models (e.g., GPT-4) and open-source LLaMA models. Experimental results demonstrate that the proposed framework effectively automates the assessment of scientific sketches and provides feedback aligned with human assessment.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n- The paper proposes an effective multi-agent framework for evaluating students' responses to scientific questions, which could be valuable for developing AI-assisted tools in STEM education.\n- The paper is well-written.\n- The experiment is well-designed to demonstrates the effectiveness of the proposed framework. The authors utilizes both closed-source and open-source models and also includes the human evaluation studies.\n\nWeaknesses\n\n- Some details are missing about the human evaluation process. Specifically, the authors do not provide sufficient information about the number of evaluators and assessment criteria used by human experts.\n\nQUESTIONS:\nThe details regarding the human evaluation are insufficient and require clarification for reproducibility and validity assessment.\n- How many human evaluators participated in the study? Did they evaluate results from all baselines (e.g., GPT-4.1, O3…) across the 3,575 student-generated sketches, or was a subset used?\n- Are there established evaluation guidelines for human experts? What specific criteria do human experts use to assess sketch quality and feedback usefulness?\n- Could the authors provide cases where the framework's assessment aligns with human expert judgment, as well as cases where they diverge? This would help readers understand the quality of the alignment.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a multi-agent framework for evaluating and enhancing student-drawn scientific sketches. The system consists of modular agents dedicated to rubric parsing, sketch perception, cognitive alignment, and iterative feedback with sketch modification, enabling personalized and transparent evaluation. Additionally, the framework integrates Bloom’s Taxonomy—a standard in cognitive theory—to construct and analyze Sketch Reasoning Graphs (SRGs), facilitating structured evaluation of visual student work and delivering pedagogically sound feedback alongside real-time sketch refinement. Compared to the baseline GPT-4.0 performance, SKETCHMIND demonstrates a significant improvement in quality.\n\nSTRENGTHS AND WEAKNESSES:\n+ Overall, the idea is novel and compelling.\n+ The proposed methodology and experimental section are clearly presented.\n\n– A major weakness, however, is that several parts of the paper are difficult to follow unless the reader is already deeply familiar with prior work in this niche domain. A significant improvement would be to make the writing more self-contained, which would broaden the paper’s accessibility and appeal to a wider audience beyond those specifically focused on sketch-based systems.\n\nQUESTIONS:\n- The concept of “Bloom level annotation” would benefit from a more self-contained explanation. Including a brief description along with a schematic diagram could significantly improve comprehension for a broader readership.\n- Similarly, the term “rubric concept” requires a clearer, standalone explanation. Expanding on this would enhance the paper’s accessibility to readers from diverse backgrounds.\n- The representation of the visual suggestion H(t) is unclear. Further elaboration on how this is formulated and how step 5 (line 175) is operationalised would be helpful.\n- A more detailed and standalone discussion of Bloom’s cognitive hierarchy would strengthen the theoretical clarity and support the framework’s pedagogical grounding.\n- The caption for Table 1 could be more comprehensive, offering sufficient context and explanation to ensure that it is interpretable without referring extensively to the main text.\n\nLIMITATIONS:\n- There exist some inconsistencies in the citations. Please proofread carefully. For example, SketchFusion citation is wrong. \n- I wonder if the codebase will be made public for the ease of reproducibility.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents SketchMind, a multi-agent cognitive framework for the automated assessment and improvement of student-drawn scientific sketches. The core innovation lies in the introduction of Sketch Reasoning Graphs (SRGs), which are semantic graph representations embedding both domain-specific concepts and cognitive labels based on Bloom’s Taxonomy. The authors evaluate SketchMind on a curated dataset, showing significant accuracy improvements over baseline MLLMs.\n\nSTRENGTHS AND WEAKNESSES:\n*Strengths*:\n\n• The integration of Bloom’s Taxonomy introduces a cognitive dimension beyond traditional visual sketch understanding, substantially enhancing the system’s interpretability and depth of analysis.\n\n• The use of a multi-agent architecture further improves the interpretability and modularity of the proposed framework.\n\n*Weaknesses*:\n\n• **Scope and Fit**: I am not sure that NeurIPS is the most suitable venue for this submission. The work lies primarily within the domain of AI4Education, and its application scenario is quite limited. In my view, this paper might be a better fit for a workshop focused on AI for Education.\n\n• **Limited Methodological Novelty**: The paper’s novelty appears somewhat limited. While the use of Bloom’s Taxonomy provides a new perspective for understanding sketches, there is not much methodological innovation beyond this integration.\n\n• **SRGs as a Prompting Strategy**: While SRGs are effective in enhancing LLM performance, their primary role in the system is as a sophisticated prompting strategy that leverages structured data. As such, this contribution is more closely related to prompt engineering than to the development of a new algorithm.\n\n• **Lack of Analysis**: The paper reports overall accuracy and human evaluation scores, but lacks detailed analysis on failure cases, which would be valuable for understanding the system’s limitations.\n\nQUESTIONS:\n• What precisely constitutes a “scientific sketch”? What are the differences between *student*-drawn and *teacher*-drawn scientific sketches? Providing examples would help clarify this distinction.\n\n• Could the authors provide more examples and analysis, detailing the types of errors SketchMind makes (e.g., errors related to specific Bloom levels, complex relationships, or specific visual representations)?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper  tackles the problem of automatically evaluating student-generated scientific sketches and propose a cognitively-grounded approach using four specialized agents: \n- rubric parsing, \n- sketch perception, \n- cognitive alignment evaluation,\n- feedback generation with sketch modification. \n\nThe system represents sketches as semantic graphs with Bloom's taxonomy annotations and demonstrates great improvements over baseline approaches.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- The paper introduces Sketch Reasoning Graphs with novel Bloom's taxonomy annotations. It provides a cognitively-grounded approach to sketch assessment that bridges educational theory with the model evaluation\n\n- The evaluation is comprehensive, testing many state-of-the-art models (GPT-4o, GPT-4.1, o3, o4-mini, LLaMA variants) across six assessment items spanning different Bloom cognitive levels. It also includes human evaluation study showing that expert educators rate the system's feedback highly (4.1/5 for GPT-4.1)\n\n- Consistnet performance gains across all models from SRG integration (e.g., GPT-4o: +21.4% average improvement), with strong results on cognitively demanding tasks\n\n**Weakness**\n- The paper lacks quantitative analysis of system efficiency. Four sequential agents create great computational overhead that could be hard for real-world classroom deployment. For real-time educational guidance, response times must be fast enough to maintain student engagement. \n\n- The paper would be improve if the it includes validation of core components. Agent 1 relies entirely on MLLM interpretation of rubrics to generate gold-standard SRGs, but there's lack of validation method to quantify how well these generated SRGs match expert-constructed graphs. And Agent 2's sketch perception capabilities lack quantitative assessment. It would be interesting to measure how accurately MLLMs can detect and interpret visual elements in student sketches, especially given that hand-drawn student sketches may not be commonly found in MLLM training data.\n\n- Not a weakness but as the author mentioned in the limitation section, it would be interesting to see how student behavioral data could be incorporated into the SRG pipeline for improving alignment with cognitive engagement signals beyond final sketch appearance.\n\nQUESTIONS:\nN/A\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-04"}