SketchMind:  A Multi-Agent Cognitive Framework for Assessing Student-Drawn Scientific Sketches
Authors: Ehsan Latif
Zirak Khan
Xiaoming Zhai
Venue: NeurIPS 2025 poster ()
Keywords: Educational AI
Multimodal Learning
Scientific Reasoning
Cognitive Modeling
Sketch-Based Assessment
Primary area: applications
TL;DR: Students' drawn scientific sketch evaluation with cognitive theory based multi-agent framework

Abstract: Scientific sketches (e.g., models) offer a powerful lens into students' conceptual understanding, yet AI-powered automated assessment of such free-form, visually diverse artifacts remains a critical challenge. Existing solutions often treat sketch evaluation as either an image classification task or monolithic vision-language models, which lack interpretability, pedagogical alignment, and adaptability across cognitive levels. To address these limitations, we present SketchMind, a cognitively grounded, multi-agent framework for evaluating and improving student-drawn scientific sketches. SketchMind introduces Sketch Reasoning Graphs (SRGs), semantic graph representations that embed domain concepts and Bloom's taxonomy-based cognitive labels. The system comprises modular agents responsible for rubric parsing, sketch perception, cognitive alignment, and iterative feedback with sketch modification, enabling personalized and transparent evaluation. We evaluate SketchMind on a curated dataset of 3,575 student-generated sketches across six science assessment items with different highest order of Bloom's level that require students to draw models to explain phenomena. Compared to baseline GPT-4o performance without SRG (average accuracy: 55.6%), the model with SRG integration achieves 77.1% average accuracy (+21.4% average absolute gain). We also demonstrate that multi-agent orchestration with SRG enhances SketchMind performance, for example, SketchMind with GPT-4.1 gains an average 8.9% increase in sketch prediction accuracy, outperforming single-agent pipelines across all items. Human evaluators rated the feedback and co-created sketches generated by SketchMind with GPT-4.1, which achieved an average of 4.1 out of 5, significantly higher than those of baseline models (e.g., 2.3 for GPT-4o). Experts noted the system’s potential to meaningfully support conceptual growth through guided revision. Our code and (pending approval) dataset will be released to support reproducibility and future research in AI-driven education.

SKETCHMIND: A Multi-Agent Cognitive Framework for Assessing Student-Drawn Scientific Sketches
Ehsan Latif
Zirak Khan School of Computing University of Georgia Athens, GA 30605
Xiaoming Zhai A14STEM Education Center University of Georgia Athens, GA 30605
AI4STEM Education Center University of Georgia Athens, GA 30605
Abstract
Scientific sketches (e.g., models) offer powerful lens into students' conceptual understanding, yet Al-powered automated assessment of such free-form, visually diverse artifacts remains a critical challenge. Existing solutions often treat sketch evaluation as either an image classification task or monolithic vision-language models, which lack interpretability, pedagogical alignment, and adaptability across cogmitive levels. To address these limitations, we present SKETCHMIND, a cog- nitively grounded, multi-agen framework for evaluating and improving student- drawn scientific sketches. SKETCHMIND introduces Sketch Reasoning Graphs (SRGs), semantic graph representations that embed domain concepts and Bloom's taxonomy-base cognitive labels. The system comprises modular agents responsi- ble for rubric parsing, sketch perception, cognitive alignment, and iterative feedback with sketch modification, enabling personalized and transparent evaluation. We evaluate SKETCUMIND
curated dataset of 3,575 student-generated sketches across six science assessment items with different highest order of Bloom's 's level that require students to draw models explain phenomena. Compared baseline GPT-4o performance without SRG (average accuracy: 55.6%), and with bSRG in- tegration achieves 77.1% average accuracy (+21.4% average absolute gain). We also demonstrate that multi-agent orchestration with SRG enhances SKETCHMIND performance, for example, a SketchMind with GPT-4.1 gains an average 8.9% increase in sketch prediction accuracy, outperforming single-agent pipelines across all items. Human evaluators rated the feedback and co-created sketches generated by SKETCHMIND with GPT-4.1, which achieved an average of 4.1 out of 5, signifi- cantly higher than those baseline models (e.g.. g. 2.3 for GPT-4o). Experts noted the system's 's potential to meaningfully support conceptual growth through guided revision. Our code and (pending approval) dataset will be released support reproducibility and future research in Al-driven education.
Introduction
Sketching such as drawn models fundamental tool science education, allowing students to externalize their thinking, represent causal mechanisms, and engage in higher-order reasoning [30]. However, assessing the quality and cognitive depth of suudent-generater sketches remains a long- standing challenge due their open-ended nature and semantic variability. Automated systems often struggle with interpreting free-form, domain-rich visual input, which makes effective feedback and evaluation particularly difficult. Recent advances in multimodal large language models (MLLMs), such as GPT-4V, have enabled breakthroughs in languue reasoning, leading to promising applications in educational assessment [16]. Specifically, systems like NeRiF have shown that
"Correpoondng author email: xiaomig www.zaaaaaaae..m edu
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

MLLMs can approximate expert grading of student-drawn models by extracting latent structure from images and comparing them against rubrics [15]. However, such monolithic models still face limitations regarding their reasoning processes, feedback personalization, and inconsistent across conceptually diverse tasks [28, 12].
To address these limitations, we propose SKETCHMIND, cognitively grounded, multi-agent frame- work for evaluating and enhancing scientific sketches. SKETCIMIND models sketches as SRGs, which embed both structural semantics and Bloom's 's Taxonomy-basee cognitive annotations [13, 8, 6]. Each SRG encodes domain concepts, their relationships, and Bloom's levels, enabling meaningful alignment with rubrics and scaffolding -aatgngggtggeeeddt feedback. Inspired by the pedagogical principles behind systems like Betty's 's Brain [3, 18. 4], SKETCHMIND framework decomposes the assessment task across four specialized agents. These agents perform (1) rubric parsing and reference SRG generation, (2) sketch perception and SRG inference, (3) cognitive alignment and scoring, and (4) iterative feedback and sketch modification. This modular design is grounded in cognitive science and agentic learning principles, enabling transparent reasoning and pedagogically informed intervention [14, 9].
Through extensive evaluation on curated NGSS-aligned dataset of student-drawn science sketches [30], we show that SKETCHMINN not only improves the baseline monolithic MLLM approaches but also increases the capabilities of reasoning models in both quantitative metrics (accuracy and alignment) and qualitative human feedback (clarity, relevance, pedagogical value). Human experts highlighted that SKETCHMIND with models like GPT-4.1 can iteratively improve students' conceptual understanding and sketch quality via visual hints and structured revision cycles. Here are key contributions of this paper:
We introduce SKETCHMIND, multi-agent framework that integrates cognitive theories with AI to assess student -eneeratee sketches effectively using our proposed SRGs.
We integrate Bloom's Taxonomy as cognition theory standard to construct and analyze SRGs for structured evaluation of visual student work and provides pedagogically sound feedback along with real-time sketch modification.
With empirical studies, we found that SKETCHMIND with State-Of-The-Ar MLLM such as GPT-4.1, ,, is able to achieve an average 90.2% sketch prediction accuracy and generate pedagologically sound feedback with sketch modifications highly rated by human experts (4.1 out of 5), highlighting its potential for advanced Al-supported learning of scientific concepts.
To promote transparency and facilitate further research, we have open-sourced our codebase at our repository and plan to make the dataset publicly available upon receiving the necessary approvals. This work represents step forward in Al for Education, demonstrating how cognitively-awan agentic systems can advance the quality, transparency, and effectiveness of automated reasoning over student-generated visual content.
2
Related Work
Sketch Understanding and Visual Reasoning. Recent years have seen significant progress in sketch understanding, particularly within the computer vision community. Approaches such as SketchFusion [2]. Sketch2Saliency [2], and SketchXAI [21] have explored the utility of human-drawn sketches for learning visual concepts and providing interpretable representations. These works primarily focus on object detection, image retrieval [5], or 3D modeling [19], rather than assessing the conceptual depth embedded in scientific sketches. Educationally focused sketch models such as SEVA [20] and DrawEduMath [1] analyze human abstraction or math reasoning but lack cognitive scaffolding like Bloom's 's taxonomy. SKETCHMIND departs from these efforts by representing student-drawn sketches as cognitively annotated semantic graphs and grounding visual elements in educational rubrics, allowing for pedagogical interpretation and targeted feedback.
Multimodal and Agentic Reasoning in Education. MLLMs such as GPT-4V have opened new opportunities in visual question answering and diagrammatic reasoning [16]. While tools like NeRiF [15] demonstrate GPT-4V's ability to grade drawn models, these systems often operate as monolithic
hnttps://githu http:/twww..../taaaaeeee/eaaaaaa

---

black boxes, limiting transparency and pedagogical udaptability. Similarly, recent work in multimodal chain-of- of-thought reasoning [30, 27, 29] and multi-agent systems [7, 9, 25, 14] show promise for decomposing complex tasks. However, most frameworks either lack cognitive modeling or fail to integrate sketch modifications explicitly. In contrast, SKETCHMIND brings together modular reasoning agents with fine-grained cognitive alignment, enabling both transparent evaluation and actionable feedback.
Educational Al for Scientific Sketch Assessment. Several studies have explored automated grading and classification of hand-drawn sketches in educational settings. Rakhmanov [23] proposed a quality- classification framework for freehand sketches, while Rahaman et al. [22] applied CNN-based models for accuracy prediction. Lee et al. [17] developed rubric-driven grading system focused on particulate matter diagrams. These systems, however, primarily emphasize surface-level visual features and often lack semantic or cognitive interpretation. More recent work has tried to integrate learning objectives, such as [28, 12]. yet they remain constrained to textual responses or fixed rubrics. SKETCHMIND bridges this gap by introducing SRGs embedded with Bloom-level annotations and enabling multi-agent-driven sketch modification.
3 Proposed Approach
We present SKETCHMIND, cognitively grounded, multi-agent framework for evaluating and improving scientific sketches through iterative, feedback-driven modification. SKETCHMIND is anchored in Bloom's 's Taxonomy [13], hierarchical model of cognitive processes ranging from recall to creative synthesis. By modeling sketches as semantic structures called SRGs, SKETCHMIND align suudent-generati content with domain rubrics and provide interpretable, formative feedback across cognitive levels.
3.1 Cognitive Framework: Bloom's Taxonomy in Sketch Understanding
Bloom's Taxonomy structures learning objectives into six ascending levels of cognitive complexity [6]:
B (Remember, Understand, Apply, Analyze, Evaluate, Create].
(I)
These range from basic recall of knowledge (REMEMBER) to the synthesis of novel ideas (CREATE). This cognitive hierarchy has long served a foundation in science education for designing assess- ments and scaffolding learning [13]. Prior work further shows that aligning instructional technologies with Bloom's 's levels supports measurable gains in higher-order thinking [8]. In SKETCHMIND, scientific sketches are not merely visual representations but are conceptualized as cognitive artifacts that externalize learners' mental models. To operationalize this, annotate each node (concept) and edge (relation) in the SRG with a Bloom level, a process we call Bloom-Level Annotation. This provides a fine-grained measure of the depth of conceptual engagement demonstrated by the student.
To implement this systematically, rubric statements are parsed to extract key verbs and criteria, which are then mapped to Bloom levels using a curated lexicon [24]. The resulting numeric levels (I for REMEMBER through 6 for CREAtE) are encoded as attributes on SRG nodes and edges. These attributes directly support semantic similarity scoring, nuanced evaluation, and feedback generation.
Pedagogical Integration. Each gold gold-standard SRG is labeled at the level of highest Bloom demand. Student sketches are then evaluated by comparing the Bloom-level annotations of their SRGs against this reference. This supports both diagnostics (identifying the highest level achieved) and adaptive feedback. For example. if student's 's sketch demonstrates UNDERSTAND, the system can generate targeted textual feedback or visual hints nudging them toward APPLY. Such progression- oriented scaffolding is consistent with educational research showing that adaptive support aligned with Bloom's 's hierarchy fosters deeper learning in STEM domains [13, 8].
A schematic illustration (see Fig. 1) depicts this process: rubric statement (e.g., "Develop a model to explain the transfer of thermal energy") maps to an SRG nodes of (FASTER_ (MNTE___OOOOO and WWATER_PARTCCLEF (WATER_ PARTICLF_ HOT), which are then assigned Bloom level (UNDERSTAND). This visual link between task, rubric, SRG structure, and Bloom's hierarchy enhances transparency for both learners and instructors.
3

---

3.2 Sketch Reasoning Graphs (SRGs)
We define an SRG as a cognitively annotated semantic graph extracted from a sketch:
fSRG( - G E,,,,
(2)
where E I is sketch image, V - C are concepts from ontology O (Concept, Relation), E S V. V are directed relations such as causality, and é V + maps each node to a Bloom level. The annotation function À V E & captures visual and textual evidence (E) supporting the cognitive label.
We treat both the representative gold standard reference sketch for given task and the student drawn sketch as inputs to this mapping, producing reference graph G. and student graph G, respectively. Importantly, the Bloom level annotations are central to how these graphs are constructed and interpreted throughout the system.
3.3 Agent Roles
SKETCHMIND comprises four agents, each of which contributes to the construction, interpretation, refinement of SRGs for Bloom-aligned assessment prediction:
Agent 1: Rubric Parser. Agent 1 in SKETCHMIND performs static analysis of the representative gold standard reference sketch and rubric for given task r to construct G., the gold-standard SRG. This process includes explicit mapping of each rubric concept For example, components that recall facts are labeled REMEMBER, while those requiring functional understanding multi-step reasoning becomes the benchmark for evaluating conceptual depth. Figure I delineates the SRG creation from the given question, textual rubric and gold standard reference sketch. Agent gold-standard Go but also provides reverse mappings ø to create visuals from cognitive concepts for the target question. These reverse mappings will be used by subsequent agents to provide visual support and sketch modification for improved learning.
Bloom level using expert annotations.
labeled APPLY or ANALYZE. This Bloom-informed rubric
only generates
Red dye dfficioon (DF) 942 022.0--021
srroudding hep vblii den com umponetre seviers net nemmmaann enery ner the tini healing 6.T Thee Valco
make
Boe Vegeeta
Temporature, Inojpase (Applr)
Op bobe meci cla Aue
Water, Particie frot (Understand)
Partilet (Understond)
Water, Wiste_Particke Partide, Room (Undensannd)
(Azokyye)
(a) Question Snippet
Dye Particie Roam (Undensand)
Key (Rementen)
sutdl whach
the
and
Dyp Farscle Gole (Understand)
Taleess offee
Bur sor dibiriii
Ietura ONE
Faesomer youles
mual PPalicn
(Analyza)
Water: Partideeeeeec Card (Undenstan))
Temperature,
(b) Raw Textual Rubric
(c) Agent1's Generated SRG Gcold-Standard
Figure 1: Overview of Reference SRG Generation. Given the textual description of question, an expert-designed textual rubric for student sketch performance evaluation, and golden standard reference sketches; Agent processes the information and extracts SRG components and builds Level 4 Bloom's 's taxonomy ordered SRG (Bloom level set the Gold standard for further evaluation and sketch modification.

---

Agent 2: Perception. Agent 2 in SKETCHMIND applies MLLM to infer the student SRG G. from the sketch image æ. Beyond identifying visual elements, it infers semantic roles and Bloom levels using MILM directly. For instance, correctly labeling diagram element might reflect UNDERSTAND, whereas indicating a dynamic interaction (e.g., force, flow) might reflect APPLY or higher. Thus, Agent 2 directly constructs the cognitive structure of the student's s mental model. Figure 2 delineates the sample student drawn sketch and Agent 2's perceived SRG, which is then further used for cognitive alignment by Agent 3.
Cod Nater
Room Tempenstan Watrr
Pariide (Understand)
red dye particle
Water, Particie, Cold (Understand)
Hot Water
Temperature Incraaso (Apply)
Modion (Anatyzo)
(Remmember)
Partice Hot (Undenstand)
Dyn ParticeeeeCoo Gold (Undersand)
(a) Input to Agent (Student's drawn sketch)
(b) Agent 2's Perceived SRG
Figure 2: Sample sketch drawn by student and Agent 2 to extract perceived SRG. (a) student's drawn
sketch, (b) Agent 2's perceived SRG based on the given sketch.
Cognitive Alignment Evaluator. Agent 3 in SKETCHMIND compares G. to Gos com- puting structural and semantic similarity while analyzing Bloom-level mismatches. To get the similarity score, it first computes the ontology-base node alignment in such a way that for each pair (Ug, Uo) E V, x Vo, it calculates weight wVVs, vo)
Agent 3:
w(U,, vo) = a simo vo) + (1 a1) I[((v.) = {(v.)),
(3)
where simo is semantic similarity from ontology O. The summation these weights then normalized by the number of total nodes in both graphs to get the overall semantic similarity:
foA Vo) = Ve + Vol
Σ
w(v,,
(4)
Eaalgn)
The similarity score S E [0, 1] is defined as:
S(G,, Go) =1 (m -
fGED(G,,
+ (1- (ossoo(V..V))) vo)).
(5)
where JGED is function to calculate graph-edit distance (node/edge insertions, deletions, substitu- tions) noramlized by (total number edges and nodes in both graphs). foA measures semantic alignment via ontology-baat Notably, both the edit and alignment components consider Bloom-level mismatches as part of the error cost, penalizing regressions in cognitive complexity. It then computes the dominant Bloom level expressed in the sketch as:
node similarity, and weights 71, 72 are calibrated on training set.
= mode(E(a) (ue) = S(G,, Go) > >T)
(6)
where the minimum similarity threshold pre-defined to extract overlapping features. This highlights any regression in complexity (e.g., if concept expected at ANALYZE is represented at only REMEMBER). These mismatches guide the diagnosis of underdeveloped concepts, forming the basis for targeted, Bloom- aolonee feedback.
5

---

Similarity Score from Agent 3 and Textual Feedback from Agent 4
Similarity score: 0.592
Feedback:
Your Proficiency Level: Developing
What You Did Well:
The student demonstrated sound structure despite missing some components.
What Needs Attention:
Missing Concepts: Dye_ Particle_Roome Temperature Decreaae, and Slower_Motion Motion
modification Guidance (Next Sketch Revisions by Visual Hint):
Water Particle Room (understand): Add markup to highlight water particles/molecules on the first block.
Reasoning Gaps Detected
D0yePPari Room, Temperature Decrease, and Nower_Mooion.
Cognitive alignment score and feedback for the perceived SRG (See Figure 2) generated by Agent 3 after similarity score calculations.
Figure 3:
Agent 4: Feedback Generator and Sketch Modification. Agent 4 in SKETCHMIND initiates feedback loop when the similarity score S(G,, Go) falls below threshold (pre-defined to determine skecth proficiency). In our case, the curriculum expert defined three levels of proficiencies (Beginning, Developing, and Proficient) as can be seen in Figure 1b. value is carefully calculated for each level and used by the agent. The agent identifies missing or misaligned nodes and edges, and traces each to its expected Bloom level. Using trained reverse mapping Ø (v, e) VisualHint, Agent generates cognitively aligned suggestions. For instance, if a student omits a causal interaction labeled ANALYZE, the system may overlay arrow with textual prompt like "What causes this effect?". Below is the step-wise procedure of the sketch revision loop: Given G. and (0);
Compute _ fSRG(x(()))
2. If S(G!), G.) 2 r, exit loop.
3. Identify deficient elements:
A (+)
Va v & aligned nodes} u {e E E, e EJO
4. Use reverse mapping Φ (u. + visual hint generated by Agent to provide visual suggestions (t)
5. Render (1)) sketch canvas and generate Python code modify the canvas and run locally.
6. Modify the canvas with updated overlay and prompting student revise (1) ((++1)
7. Repeat until 2 T or maximum iterations Tmax reached.
This sketch revision loop directly scaffolds the student toward higher-order cognitive tasks, iterating until the revised sketch meets conceptual fidelity (Modified SRG with additional node and updated sketch using the Python toolkit for the sample image given in Fig. 4). SKETCHMIND is designed to adaptively scaffold students across diverse ability levels. If a sketch yields an incoherent or low-complexity SRG (e.g., all nodes REMEMBER), Agent 4 shifts focus guided reconstruction. layering in increasingly complex prompts. The evaluator distinguishes perceptual errors from conceptual ones by analyzing visual evidence X. ensuring the feedback remains diagnostic rather than punitive. By explicitly modeling Bloom's 's cognitive hierarchy at each stage of analysis analysis-from rubric parsing to feedback generation SKETCHMIND transforms sketch-based assessment into learning-orienten interpretable, and cognitively aligned process. SKETCHMIND positions sketching not just as representational task but as an active, assessable pathway for scientific reasoning and growth.
from

---

Coll Mater
Baon Trrperitu Parter
Water Particie__ Retiiiee_dmmmmmee (Understand)
water molecule
red dye particle
Satte_Partiie_Cooo Cold (Understand)
Watte_Pariclee Pariice_eeee (Urderstand)
Het Water
Faster (Analyza)
(Rememberr)
(Apply)
_ye_Part Partidee_ Hot (Understand)
Dyo, Particee.ecoo Cold (Understand)
(b) Agent 4's updated sketch for per- ceived feedback
(a) Modified SRG with Feedback from Agent 4
Figure 4: Sample modified SRG based on the given score from Agent 3 and feedback from Agent and updated sketch with embedded python toolkit.
4 Experimental Setup
We evaluate SKETCHMIND using two distinct model configurations to comprehensively examine its performance and versatility across closed-source and open-source MILMM frameworks. In Configu- ration 1, we utilize GPT-4o, GPT-4.1, GPT-4. --nao,, o3, and 04-mini (all latest GPT models with different reasoning capabilities for comprehensive evaluation) accessed via the OpenAl API. This setup leverages black-box prompting tailored explicitly to encourage structured JSON output and support multimodal reasoning. Configuration 2 deploys open-source MLLMs, specifically the INT4- quantized Llama-4 Maverick and INT8-quantizA
Scout (400B and 109B parameters respectively, with 17B active parameters), running locally on four NVIDIA H100 GPUs, facilitated by Hugging Face Transformers (version 4.39+) and Py Torch (version 2.1+). We keep the same model for each agent maintain the model performance consistency, for example, if targeted GPT-4o, then each agent uses GPT-4o and mainatin local chat session.
For closed-source configurations, inference requests are systematically routed through the OpenAl API, meticulously logging timestamps, model versions, and full prompt-response interactions to ensure transparency and reproducibility. Open-source models executed on-premises, utilizing four H100 GPUs with CUDA 12.1 and cuDNN 8.9, loading quantized model weights from the Hugging Face Model Hub, and multimodal data inputs are managed by the Llama4Processor.
Both configurations operate with a multi-agent pipeline detailed in Section 3, where all the agents in sequence. Agents 1, 2, and 4 incorporate specifically designed prompt templates optimized for effective cnatruction-followiii structured output enforcement, and multimodal reasoning integration. Agent 3 functions deterministic and agnostig component, focusing exclusively graph comparison tasks. Comprehensive documentation of prompt templates is available in supplementary materials to enable precise experimental replication.
Dataset and Evaluation. Despite growing interest in applying machine learning to educational domains, there remains a significant lack of publicly available, high-quality datasets that capture student-generated visual reasoning, particularly scientific sketches. To our knowledge, no large-scale, general-purpos. benchmark exists that includes both raw student-drawn diagrams and structured expert annotations for meaningful semantic evaluation.
Given this gap, we base our study on a rigorously developed dataset originally introduced by Zhai et al. [30], which has since become one of the most widely recognized resources for evaluating automated reasoning over student- generrted scientific models. This dataset, adapted from the NGSA (Next Generation Science Assessment) initiative [10], aligns closely with the NGSS framework (26) and has been used by AIED researchers to assess students' conceptual understanding through multimodal evidence [16, 15). Comprehensive details about dataset selection rationale and statistics are provided Appendix To ensure pedagogical validity, we conducted structured human evaluation of model-generated feedback. Four domain-expert educators, each with graduate-level
7

---

training in science education, independently assessed the pedagogical quality of system responses. Each rater evaluated stratified random sample of 890 suudett-generate sketches (25% of the dataset), ensuring balanced representation across models (GPT-40, GPT-4.1, O3), grade levels, and science task types.
All evaluators participated in calibration phase: they jointly annotated and discussed 10 represen- tative examples to align rubric interpretation, and then independently scored the same 10 sketches. Inter-rater reliability, measured using Quadratic Weighted Kappa, reached K = 0.83, which is con- sistent with benchmarks for expert judgment in science assessment [11]. J. For the main study, each response was rated by at least two experts in a cuuble-blind manner.
Evaluation guidelines were based on extended rubric adapted from Zhai et al. [30] and tailored for formative science assessment. Experts rated each response along three dimensions: 1) Clarity: Whether the textual and visual feedback was understandable and actionable. 2) Conceptual Accuracy:
Whether suggested modifications and explanations correctly reflected the target scientific concepts. And In Value: The feedback's is potential to promote student learning and progression along Bloom's 's taxonomy. Each dimension was scored on 5-point ordinal scale (1-5). Consistency across raters was supported through the initial calibration process, and ratings were conducted under double-blind conditions to mitigate bias. Framework-human agreement rates, along with detailed rubric codebooks and annotated examples, are reported in Section 4 and the supplementary material.
Implementation Details. The SRG construction pipeline in SKETCHMIND utilizes a shared SRGBuilder class, which efficiently constructs, validates, and caches graphs, significantly reduc- ing computational overhead and cost during repeated evaluations. Sketch adequacy is determined by a similarity threshold (r = 0.75). with dynamic generation of visual hints guided by re- verse mapping (Φ) embedded within Agent I's implementation. We calculate the sketch predic- tion accuracy for each assessment item by comparing with human-expert annotated proficiencies as (Sum of correctly predicted samples across each proficiency level/Toote samples) and average for all items as (Sum of all item's accuracies/ Total number of items). We have evaluated the perfor- mance of SKECTIMIND by decomposing into combination of target model with proposed SRG. This decomposition can help understand the impact of target model and proposed SRG to deter- mination best possible combination for SKETCUMIND, Detailed evaluation scripts and additional implementation specifics provided comprehensively in Appendix A.
Results
Table presents item- wise and macro-average accuracy across a range of MLLMs, both with and without SRG integration. The results consistently demonstrate that incorporating SRG supervision significantly improves performance across all models and items. For instance, GPT-4o, shows a substantial accuracy increase from 47.7% to 76.5% Item H4-1, relative gain of nearly 30 percentage points. Averaged across all items, GPT-4o benefits from an improvement of approximately 21.4%. Even tfate-of-the-art models such GPT-4.1 show meaningful accuracy gains ranging from 11.0% to 15.4% when SRG guidance is applied, with performance increasing from 74.2% to 89.6% on Item R1-1 and from 73.1% 1% to 87.2% Item H4-1, Hence, SKETCHMIND works best with GPT-4.1 integrated with SRG.
Models with lower bascline performance, such LLaMA 4 Maverick and Scout, experience even greater relative improvements. LLaMA 4 Maverick, for example, improves by 29.7% on Item J2-1 and achieves up to 26.0% gains on Item H4-1, suggesting that structured supervision via SRGs can dramatically enhance reasoning capabilities in non- non-reasoning models for open-source SKETCHMIND.
Why Multi-Agent Framework? All the above-mentioned results are performed with the multi- agent framework proposed for SKETCHMIND, but here comes the question: why not single agent? To answer that and assess the impact of modularization inherent in the multi-agent framework for reasoning tasks, we conducted ablation study comparing unified single-agen framework with our proposed multi-agent pipeline. Each setting was evaluated with and without SRG supervision using two strong backbone models: GPT-4o and G.T-4.11
The results in Table demonstrate that modularizing the reasoning process via a multi-agent frame- work consistently improves SKETCHMIND's performance over the single-agent baseline across
8

---

Table (tem-wish accuracy (%) across models with and without SRG integration for SKETCHMIND.
Model
R1-1
J2-1
M3-1
H4-1
H5-1
J6-1
Average
GPT-4o
63.2
58.4
53.5
47.7
52.3
58.6
55.6
+ SRG
78.5
77.4
75,8
76.5
74.7
79.1
77.1
Gain
+15.3
+19.0
+22.3
+28.8
+22.4
+20.5
+21.4
GPT-4.1
74.2
78.5
77.4
73.1
79.6
81.5
77.4
+ SRG
89.6
91.6
88.4
87.2
91.7
92.6
90.2
Gain
+15.4
+13.1
+11.0
+14.1
+12.1
+ +11.1
+12.8
GPT-4.I-nano
62.5
61.3
59.6
57.3
63.8
67.5
62.0
SRG
73.7
72.6
70.4
68.7
78.5
79.3
73,9
Gain
+11.2
+11.3
+ +10.8
+11.4
+14.7
+ +11.8
+11.9
O3
75.2
79.5
76.4
75.3
77.5
79.4
77.2
+ SRG
89.5
91.1
87.3
86.4
89.6
90.3
89.0
Gain
+14.3
+11.6
+10.9
+11.1
+12.1
+ +10.9
+11.8
O4-mini
71.4
75.3
73.3
69.2
74.6
76.1
73.3
+ SRG
79.5
81.5
79.4
77.6
82.8
83.9
80.8
Gain
+8./
+6.2
+6.1
+8.4
+8.2
+7.8
+7.5
LLaMA 4 Scout
48.6
43.2
39.5
38.4
45.6
47.5
43.8
+ SRG
63.8
69.3
59.6
61.7
67.5
66.8
64.8
Gain
+15.2
+26.1
+20.1
+23.3
+21.9
+ 19.3
+21.0
LLaMA 4 Maverick
53.4
49.7
44.5
42.7
46.8
49.6
47.8
SRG
77.3
79.4
63.6
68.7
71.8
73.5
72.4
Gain
+23.9
+29.7
+1.11
+26.0
+25.0
+23.9
+24.9
all six items. Without SRG integration, GPT-4o's accuracy increases from 50.1% (single-agent) to 55.6% (multi-agent). while GPT-4.1 improves from 62.9% to 77.4%, indicating that decomposing tasks into specialized agents enables more structured, context-aware reasoning even without explicit graph guidance. This performance gap widens with SRG supervision: GPT-4o's accuracy rises from 69.5% to 77.1% and GPT-4.1 from 82.8% to 90.2%, with item-wise gains ranging from 5.2% to 13.4%. Notably, Item 14-1 (Hot Shower Effect), which demands the highest Bloom's taxonomy level (Create), sees accuracy climb from 63.2% to 76.5% for GPT-4o and from 79.3% to 87.2% for GPT-4.1 when switching multi-agent reasoning with SRG, confirming that modular agents better equipped to leverage structured semantic guidance. These results highlight that a multi-agent architecture, where reasoning responsibilities explicitly segmented and coordinated, facilitates more robust and interpretable scientific reasoning.
Table 2: Ablation study comparing single-agent and multi-agent agent frameworks for SKETCHMIND (accuracy in %)
Model configurations
R1-1
J2-1
M3-1
H4-1
H5-1
J6-1
Average
GPT-4o (Single Agent w/o SRG)
56.3
52.4
49.3
43.1
48.3
51.2
50.1
GPT-4o (Multi-Agent w/o SRG)
63.2
58.4
53.5
47.7
52.3
58.6
55,6
GPT-4o (Single Agent w/ SRG)
73.5
69.6
68.4
63.2
68.8
74.3
69.5
GPT-4o (Multi-Agent w/ SRG)
78.5
77.4
75.8
76.5
74.7
79.1
77.1
GPT-4.1 (Single Agent w/o SRG)
69.6
61.3
59.2
57.1
58.8
71.2
62.9
GPT-4.1 (Multi-Agent w/o SRG)
74.2
78.5
77.4
73.1
79.6
81.5
77.4
GPT-4.1 (Single Agent w/ SRG)
84.4
81.2
83.6
79.3
82.7
85.3
82.8
GPT-4.1 (Multi-Agent w/ SRG)
89.6
91.6
88.4
87.2
91.7
92.6
90.2
Feedback and Sketch Modification Evaluation. To assess the pedagogical quality and usefulness of SKETCHMIND'S generated sketches and feedback, we conducted human evaluation study focusing Agent 4, the component responsible for modification of visual representations and

---

providing formative feedback. Expert evaluators, comprising experienced science educators, rated the outputs across six science items based on their clarity, conceptual accuracy, and instructional value.
As shown in Table 3, GPT-4.1 achieved the highest average rating (4.1), followed closely by o3 (4.0) and LLaMA 4 Maverick (3.5). Evaluators consistently noted that sketches generated using GPT-4.1 were pedagogically sound, well-aligned with scientific principles, and accompanied by feedback that could directly support improved student learning. Lower-performing models, such as GPT-4o and LLaMA 4 Scout, received average ratings of 2.3 and 2.5, respectively, often due to missing or vague concepts and less actionable feedback. The evaluators emphasized that when integrated with tegh-performing language models like GPT-4.1, SKETCHMIND has the potential to significantly improve the quality of student- -eneentte scientific sketches through guided modification and tailored feedback.
Table 3: Human Evaluation Ratings Feedback and Sketch Modification (1 = Poor, 5 Excellent).
Model
R1-1
J2-1
M3-1
H4-1
H5-1
J6-1
Average
GPT-4o
2.5
2.0
2.5
2.5
2.5
2.0
2.3
GPT-4.1
4.5
4.0
3.5
3.5
4.5
4.5
4.1
GPT-4.1-namo
3.0
3.0
3.5
2.5
3.0
3.5
3.1
03
4.5
4.0
4.0
4.0
3.5
4.0
4.0
O4-mini
4.0
3.5
3.0
3.0
3.5
3.0
3.3
LLaMA 4 Scout
3.0
2.5
2.0
2.5
2.0
3.0
2.5
LLaMA 4 Maverick
3.5
4.0
3.5
3.5
3.0
3.5
3.5
6 Conclusion
In this work, introduced SKETCHMIND, cognitively grounded multi-agent system for assessing and improving student-generated scientific sketches. By leveraging SRGs annotated with Bloom's taxonomy, SKETCHMIND enables interpretable evaluation, pedagogically aligned feedback. and iterative modification of higher-quality visual explanations. Empirical results across six NGSS- aligned items show that SKETCHMIND (model SRG) substantially outperforms both MLLM baselines without SRG and single-agent pipelines, achieving up to 90.2% average accuracy with GPT-4.1 and receiving high expert ratings of (4.1) for feedback quality. Overall, each MLLM shows significant improvement for sketch reasoning and prediction with the proposed SRG integration which highlights the significance of structural reasoning for scientific sketch evaluations. SKETCHMIND bridges the gap between visual AI reasoning and education by embedding cognitive theories directly into the assessment pipeline.
Limitations and future directions. Despite promising results, several limitations warrant attention. First, while the multi-agent system is modular, inter-agent coordination is static and predefined, future work could explore dynamic planning strategies using large language model (LLM) controllers or reinforcement learning, demonstrated in recent advances in task decomposition for multi-agent collaboration [7, 9]. Second, our findings relies on overal sketch prediction for given proficieicny level; however, SRG-level evaluation may provide in-depth analysis of predictions. Future work could involve experts to create manual SRGs for in-depth analysis. Lastly, Incorporating student behavioral data (e.g., stroke sequence eye tracking) into the SRG modeling pipeline may also further enhance alignment with cognitive engagement signals [14].
Acknowledgments and Disclosure of Funding
The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305C240010 (PI Zhai). The opinions expressed are those of the authors and do not represent views of the Institute the U.S. Department of Education.
10

---

References
[1] Sami Baral, Li Lucy, Ryan Knight, Alice Ng. Luca Soldaini, Neil T Heffernan, and Kyle Lo. Drawedumath: Evaluating vision language models with expert-annotated students' hand-drawn math images. arXiv preprint arXiv:2501. 14877, 2025.
[2] Ayan Kumar Bhunia, Subhadeep Koley, Amandeep Kumar, Aneeshan Sain, Pinaki Nath Chowd- hury, Tao Xiang, and Yi-Zhe Song. Sketch2saliency: Learning to detect salient objects from human drawings. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages
2023.
[3] Gautam Biswas, Krittaya Leelawong, Daniel Schwartz, and Nancy Vye. Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence, 19(-4)):33-392.
2005.
[4] Gautam Biswas, James R Segedy, and Krithika Bunchongchit. From design to implementation practice: A learning by teaching system: Betty's brain. International Journal of Artificial Intelligence in Education, 26(1 ):350-364, 2016.
[5] Abhra Chaudhuri, Ayan Kumar Bhunia, Yi-Zhe Song, and Anjan Dutta. Data-free aketch-based image retrieval. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pages 12084-18 22084-1203,, 2023.
[6] Emily Hui Sein Yue Elim. Promoting cognitive skills in ai-supported learning environments: the integration of bloom's 's taxonomy. Education 3-13, pages 1-11, 2024.
[7] Longling Geng and Edward Y Chang. Realm-bench: A real-world planning benchmark for Ilms and multi-agent systems, arXiv preprint arXiv:2502. 2025.
[8] Chahna Gonsalves. Generative ai's impact on critical thinking: Revisiting bloom's taxonomy. Journal of Marketing Education, page 02734773244130999) 2024.
[9] Shuchen Guo, Ehsan Latif, Yifan Zhou, Xuan Huang, and Xiaoming Zhai Using generative ai and multi- multi-agents to provide automatic feedback. arXiv preprint arXiv:2411.07407, 2024.
[10] Christopher J Harris, Joseph S Krajeik, and James W Pellegrino. Creating and using instruc- fionally supportive assessments in NGSS classrooms. NSTA Press, National Science Teaching Association, 2024.
[11] Edouard M Hickling and Jane E Bowie. Applicability of human reliability assessment methods to human-computer interfaces. Cognition, technology & work, 15()):19-27, 2013.
[12] Shengxin Hong, Chang Cai, Sixuan Du, Haiyue Feng, Siyuan Liu, and Xiuyi Fan. "my grade is wrong!": A contestable ai framework for interactive feedback in evaluating student essays. arXiv preprint arXiv:2409.07453 2024.
[13] Emily Sein Yue Elim Hui. Incorporating bloom's 's taxonomy into promoting cognitive thinking mechanism in artificial inteligence-supporeed learning environments. Interactive Learning Environments, 33(2): -18811000 2025.
[14] Firuz Kamalov, David Santandreu Calonge, Linda Smail, Dilshod Azizov, Dimple R Thadani, Theresa Kwong, and Amara Atif. Evolution of in education: Agentic workflows. arXiv preprint arXiv:2504.20088, 2025.
[15] Gyeong-Geon Lee and Xiaoming Zhai. Nerif: Gpt-4v for automatic scoring of drawn models. arXiv preprint arXiv:2311. 129900, 2023.
[16] Gyeong-Geon Lee, Ehsan Latif, Lehong Shi, and Xiaoming Zhai. Gemini pro defeated by gpt-4v: Evidence from education. arXiv preprint arXiv:2401.08660 2023.
[17] Jaeyong Lee, Gyeong-Geon Lee, and Hun-Gi Hong. Automated assessment of student hand drawings in free-response items on the particulate nature of matter. Journal of Sci- ence Education and Technology, 32:549 566, 2023. URI. https:/ :/dd:i/////// htt///i..ogg/..../ org/10.1007/
s10956-023 2200000022-110--33311113103 10042-3.

---

[18] Krittaya Leelawong and Gautam Biswas. Designing learning by teaching agents: The betty's 's brain system. International Journal of Artificial Intelligence in Education, 18(3):181-208 2008.
[19] Aryan Mikaeili, Or Perel, Mehdi Safaee, Daniel Cohen-Or, and Ali Mahdavi-Amiri. Sked: Sketch-guided text-based 3d editing. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pages 14607-14619, 2023.
[20] Kushin Mukherjee, Holly Huey, Xuanchen Lu, Yael Vinker, Rio Aguina- Ariel Shamir, and Judith Fan. Seva: Leveraging sketches to evaluate alignment between human and machine visual abstraction. Advances Neural Information Processing Systems. 36:67138-67155, 2023.
[21] Zhiyu Qu, Yulia Gryaditskaya, Ke Li, Kaiyue Pang. Tao Xiang, and Yi-Zhe Song. Sketchxai: first look at explainability for human sketches. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13327-23333 2023.
[22] Md. Afzalur Rahaman, Tahmid Rahman, and Md. Monowar Hossain. Automated grading and classification of hand-drawn sketches using deep learning. 2024 International Conference on Innovations Science, Engineering and Technology (ICISET), pages 1-6, 2024. URL http://ieeexxplore. ieee. org/ stamp/ stamp/stampppppa stamp. jaffpp=kkarmummbbr010999999
[23] Ochilbek Rakhmanov. A novel algorithm classify hand drawn sketches with respect content quality. Computaaioonal Science and Its Applications - ICCSA 2020, 12252:179 193, 2020. URL httpss//app..eenatttcccooooooll www.eeicscccmmnaeeeee.ee...
[24] Malini Reddy and Heidi Andrade. A review of rubric use higher education. Assessment & evaluation in higher education, 35(4):435-448, 2010.
[25] Rimsha Shahzad, Muhammad Aslam, Shaha T Al-Otaibi, Muhammad Saqib Javed, Am- jad Rehman Khan, Saeed Ali Bahaj, and Tanzila Saba. Multi-agent system for students cognitive assessment in e-learning environment. IEEE Access, 12:15458-15467 2024.
[26] NGSS Lead States. Next generation science standards: For states, by states. National Academies Press, 2013.
[27] Jiacong Xu, Shao- Yuan Lo, Bardia Safaei, Vishal M Patel, and Isht Dwivedi. Towards zero-shot anomaly detection and reasoning with multimodal large language models. arXiv preprint arXiv:2502.07601, 2025.
[28] Antoun Yaacoub, Jérôme Da-Rugna, and Zainab Assaghir. Assessing ai-generated ques- tions' alignment with cognitive frameworks educational assessment. arXiv preprint arXiv:2504.1422 14232, 2025.
[29] Hao Yin, Guangzong Si, and Zilei Wang. Clearsight: Visual signal enhancement for object hallucination mitigation in multimodal large language models. arXiv preprint arXiv:2503. 13107, 2025.
[30] Xiaoming Zhai, Peng Hc, and Joseph Krajeik. Applying machine learning to automatically assess scientific models. Journal of Research in Science Teaching, 175-144175 2022.
12

---

Appendices
Prompts and Code
14
B Dataset Statisties
18
Costs
20
D Impact Statement
20
13

---

A Prompts and Code
Prompts for Cognitive Sketch Assessment Agents This section details the prompt structures used to interact with the MLLMS for agents 1, 2, and 4 in our cognitive sketch assessment pipeline. The prompts are designed to elicit structured JSON outputs and guide the MLLMs through their respective tasks, including SRG generation, sketch analysis, and visual feedback generation. For clarity and visualization in this document, prompts containing Python class definitions multi-line ISON examples are presented within code-like blocks; in our actual implementation, these are passed as standard text strings to the MLLM. The placeholders like [QUESTION. [EXT], etc., are dynamically filled at runtime.
This agent is responsible for interpreting the educational question and rubric, along with golden standard sketches, to generate a reference SRG and a mapping of concepts to visual drawing hints.
Agent-1:
System Prompt
You are Agent I: Rubric Parser cognitive sketch evaluation system.
Your job is to construct a Bloom-aligned Sketch Reasoning Graph (SRG) based on the rubric, question, and example sketches.
Use the SRGBuilder class [SRCBuilde] Dllass_StringPPeP Placeholder] to:
Extract semantic concepts relevant to the given question and rubric.
2. Label them with Bloom levels (Remember, Understand, Apply, Analyze, Evaluate, Create), Define directed edges that capture causal logical relationships.
4. Use example sketches to identify additional visual concepts and reinforce what is expected. Validate the graph for connectivity (There should be path from each node to other nodes)
and Bloom level ordering.
Provide a reverse mapping from concepts to drawing hints using Bloom's taxonomy. Return JSON in this format:
"srg": ("nodes": [("label":"... [("from":"... treverse_mapping": d"concept_named "visual "risual_hint"]
"}], "edges";
Be consistent with node naming across examples. Be thorough but not redundant.
Note: [SRGBuilder, Stapindddeerrlasss_eeeee
Classcla__ss____eee CwwssslStainngaa_aeeeae BaarinngPlacchhodde is replaced by the full string definition of the SRGBuilder class as shown below:
class SRGBuilder:
BLOOM_ORDER ["Remember", "Understand", "Apply", "Analyze", "Evaluate", "Create"]
def diint___seef,, question: str, rubric: str):
self question question
self rubric rubric self.nodes. List (Taple str]] [ (concept, Bloom level)
self, edges: List (Tuple str]] = [ (from, to)
def add add_node (self, concept: str, bloom_ level: str): assert bloom_level in self BLOOM_ORDER, f"Invalid Bloom level:
+ (bloom_level)*
self. nodes (concepp,, bloom_level))
def add_ [id_edde (self, source: str, target: str): self edges. append (source, target))
def build_graph -> [str,str, List [Tuple[str, str]]]: return ("nodes"; self nodes, "edges"; self edges)
14

---

aalidate_graphhe bool: node_names
names [n[0] for n in self. nodes)
valid_edges all (u in mode_names and v in node_names for u, V in self edges)
Check for connectivity
G nx. DiGraph()
a.ddd _.add_edges, edges_from www.frrrwl.....ierroooorree edges)
connected ixx.ss_weall wew.s_ewallycoooeecttooe False
if Gf else
Ensure Bloom level ordering is respected bloom_leveis levels ordered bloom_leveis
CODDRO index (level) for -, level in self nodes]
return _dgis and connected and ordered
Initial User Prompt Sequence
The initial interaction with Agent-l 1 involves sequence of user messages:
1. Text Input:
Question: [QUESTION_TEN] [EXT]
Rubric: [RUBRIC_TEXT]
Please analyze the following golden standard sketches guidance.
2. For each of the 3 golden standard images provided:
Image Input: [COLDEN_STANDAR] IMAGE_N_BALE [___E66_]
Associated Text:
"This is golden standard sketch."
This agent analyzes the student's sketch, using reference SRG (from Agent-1) template, to identify concepts and relationships present the sketch.
Agent-2:
System Prompt
You are Agent 2: Sketch Parser in cognitive sketch evaluation system.
You receive student's sketch and analyze its contents construct Sketch Reasoning Graph (SRG).
Your SRG output must use the same node labels and edge labels as the reference SRG possible.
Only include node or edge if is visibly present or clearly inferable.
Instructions:
1. Use the reference SRG node and edge names as template. Detect which concepts and relationships from the reference actually present in the sketch.
3. Label each node with the Bloom level shown by the sketch.
4. Return only valid components and their Bloom levels.
Return ISON in the format:
{
"srg": {
"nodes": [["label": "...", "bloom level": "bloom_level"; "...")),
"edges": [["rrom": "...", "to": "... "}]
15

---

Be strict and do not assume missing content. Match names exactly where applicable.
Initial User Prompt Sequence
The initial interaction with Agent involves:
1. Text Input (Reference SRG Information):
Reference SRG Node Labels: [Labeli]
[Label2]
Reference SRG Edges:
[Source1] -> [Targeti] [Source2] [Target2]
2. Image Input (Student's Sketch):
Prompt Variations for LLaMA-4 (Maverick & Scout) While the core structure of all prompts remained identical, for the LLaMA-4 variants we applied only these minimal changes get the same structured JSON replies as our GPT-based agents:
Agent-1 (LLaMA-4 Variant): based prompts above:
Only the bold content in the following lines differ from the GPT-
System Prompt
Use the SRGBuilder class
to:
Follow these steps:
1. Extract concise semantic concepts relevant to the question and rubric.
Label each concept with its corresponding Bloom's taxonomy level: Remember, Under- stand, Apply, Analyze, Evaluate, or Create.
Define directed edges that capture causal or logical relationships between these concepts.
4. Incorporate insights from example sketches to identify additional visual concepts and reinforce expectations.
5.
Your output must strictly adhere to the following JSON format:
"srg": ("nodes": [["label":"., "bloom_ "iboo__evee""".... "edges": [["from": "...., "to":"..." "H]}, nneverse_mapping" ("concept_name"" "visual_hint"}
Use double quotes for all keys and string values. Do not include any explanations additional text outside the JSON structure.
Initial User Prompt Sequence
Please analyze the above information and generate the Sketch Reasoning Graph (SRG) and reverse mapping as per the system instructions and Please analyze the following golden standard sketches for guidance
16

---

Agent-2 (LLaMA-4 Variant): based prompts above:
Only the bold content in the following lines differ from the GPT-
System Prompt
Return JSON in the format (double quotes only):
Use double quotes for all keys and string values. Do not include any explanations or additional text outside the JSON structure.
While Agent performs several functions, its primary MLLM interaction for generating new content involves creating Python code for visual overlays the student's sketch. Agent-3 is deterministic and does not use an MLLM.
Agent-4;
System Prompt
You are Agent-4; That returns Python code to overlay visual sketch hints on images. Before generating the code, understand the given image on which this overlay will apply, and carefully position objects appropriately. Do not just randomly place the overlay on the image in the code.
Initial User Prompt Sequence
This prompt is sent along with the student's sketch image.
1. Text Input:
Generate a Python function using PIL to visually overlay a hint onto a student sketch.
The original image is JIMAGE_WIDTHE px wide and [IMAGE_ HEIGHT] px tall.
Concept: [CONCEPT_MMG BAAETT__AME_FFFFE [O___IOTTT
Hint: LWAAATTEEDIISSAALIIYY
Return only the function named
overlay hint hint (image: Image Image) -> Image Image"
2. Image Input (Student's Sketch): SETUDENT_SKETCHE ]
Agent-3 Core Evaluation Logic. The following code implements the similarity computation,
classification, and feedback signal extraction used by Agent 3.
class Agent3:
def run(self, reference srg, student_arg) ref SRGBuilder ("", "")
stu SRGBuilder ("", "")
ref nodes, ref edges reference_ srg ('oods')ss') reference srg[ ['edges ']
stu nodes, stu edges student_srg ('nodes student_srge "'deedgss '
score stu, compute, wwwpwtteeemmleeeeeeeeeeeeeeeeeee wwwilariityreeeerreeee
missing nodes In for in reference_ste srgg'nodes') 'nodes'] if not in student_ _aagtt_sg___g nodes ']]
17

---

daasing_edges for in reference_ste "rggeeeeeee"""" eedgee') not + Student_argl (edges'' ']]
irrelevant_nodee _ndee In for n in student_srg( nnodes '] if n not in eetereene_ssrlnnooee ']]
irrelevant. edges [e e student_arg( edges '] if e not in reference_arge segnnceess
Compare Bloom level eapectations bloom_ discrepancies
ref ref_node_dict dict (raet for concept, level in student_srg ''ooddes'' 'I:
if concept in ref_node_dic and level != ref_node__ee nictedidddicdddddeeeeeed [concept]] bloom_ wwwcrepancees.appeeeeee
"concept"; concept,
"expected": ref_ denctddddictedeeoeeeeeee [concept],
"observed": level
}
Classify sketch based on similarity score if score >= SCORE_THRESHON
label "Proficient"
elif score >= 0.5:
label "Developing"
else:
label "Beginning"
Rank missing nodes by Bloom level priority_ fix sorted(miesing nodes, key=lambda x:
SRGBuilder. BLOOM_ORDER index(x
Detect gaps in reasoning flow sopected_sources farc for sre, reference _arnceerrrgaae edges actual_sources (src for sre, student_srg student_ssg("edges: edges ']}
conceptual_gaps list (isppeeccted__sourc actual_sources)
return {
"similarity_soom round 3),
"elassification" label,
"missing.nodes" nodes": missing_nodes,
"missinggedges": missing_edges,
"irrelevant_nodes"" irrelevant_nodee
"irrelevant_edges"" irrelevant_edges.
dilloooddiscrrepccciess" bloom_discrepanccee,
periority_fix" priority_fix,
gaps": conceptual_gaps
B Dataset Statisties
We selected this dataset for three primary reasons: Relevance and Authenticity: contains real, open-ended student responses in the form of scientific sketches, reflecting a range of cognitive and conceptual models that go beyond synthetic or constrained data sources. This authenticity is essential for evaluating models that aim to interpret provide feedback on student thinking. 2) Expert Annotation and Interpretability: Each assessment item supported by a rubric and three gold- standard (SRGs), representing Beginning, Developing, and Proficient levels. These gold standards were constructed by domain experts, enabling supervised learning and interpretability in model predictions. 3) Domain Breadth and Volume: comprises six items across physical sciences spaning from physical chemistry, heat transfer, and thermodynamics, dataset contains over 3,500 curated student sketches. Each assessment item is associated with specific highest order of Bloom's 's level as
18

---

each assessment is designed for comprehensive sketch evaluation. Each response has been cleaned to remove noise and non non-sensical content, making it suitable for reliable evaluation. Further, each item is supported by an expert-authored rubric and three corresponding gold-standard (SRGs), representing Beginning, Developing, and Proficient levels of performance.
The dataset consists of student drawn scientific sketches and golden-standard sketches drawn by teacher. A "scientific sketch" visual representation intended to illustrate scientific concept, process, or phenomenon. For example, for thermodynamies concept, the scientific sketch shows heat source, its transition phases, arrows may represent the flow of heat, and object/medium transferring the heat. Teacher-drawn sketches serve as gold-standard reference sketches, explicitly containing essential visual components necessary to fully represent the target scientific concept (e.g.. arrows clearly indicating heat transfer direction in thermodyyamiic).. In contrast, student-drawn sketches often reflect incomplete, varied, or erroneous understanding, exhibiting substantial diversity and ambiguity. We explicitly distinguish these categories in our data annotation by using teacher sketches as gold- standard "reference SRGs" and grounding student sketches pedagogically Bloom's cognitive levels (See Fig. 1. We also have included the representative examples, illustrating these differences in Fig. 5,
Cold Water
Room Temperature Water
30 seconds
0 seconds
30 seconds
seconds
TD
Waze
Hot Water
30 seconds
seconds
(a) Teacher Drawn golden-standard sketch for the repsentative example shown in Fig. reaching Level
(b) student drawn scientific sketch (Barely reaching the Understanding Level of Bloom's 's Teaxonomy)
Figure 5:
Sample sketches drawn by teacher and student(a) Teacher's drawn sketch (Level 6, (b)
Student drawn sketch at Level
Table 4: Summary of Assessment Items and Student Response Distribution based on the Proficiency Level.
Item#
Name
Bloom's 's Level
Total Samples
Beginning
Developing
Proficient
RI-1
Red dye diffusion
4 (Analyze)
476
194
205
77
12-1
Jane's inflated ball
5 (Evaluate)
538
177
288
73
M3-1
Melting Butter
5 (Evaluate)
520
155
266
99
H4-1
Hot Shower Effect
6 (Create)
772
494
107
171
HS-1
Heated Cup of Water
5 (Evaluate)
453
61
262
130
J6-1
Jennifer's Teapot
4 (Analyze)
816
390
271
155
The Table. summarizes each item, domain, the distribution of student responses across the three proficiency levels, and highest order of Bloom's level. Evaluation criteria for model quality include: structural validity and semantic alignment via accurate node and edge labeling consistent with Bloom's 's taxonomy in term of accuracy for final predictions based on SRGs, and feedback quality
19

---

as rated by domain experts on a 5-point Likert scale (1 poor, = excellent), assessing the clarity and instructional value of generated visual hints and textual explanations.
C
Costs
Table Cost and Token Usage Per Sample across Agents for each Model.
Item#
Model
Input Tokens/sample
Output Tokens/sample
Cost per sample
E2E Latency
RI-I
GPT-4o
55,105
~1491
S0.0479
9.48s
RI-1
GPT-4.1
55,105
~1441
50.0221
20.35s
RI-1
GPT-4.1 nano
~5, 105
1441
$0.0011
16s
R1-1
O3
5,,10
1449
S0.1107
51.62s
R1-1
O4-mini
55,105
1441
S0.0122
169.79%
R1-1
Llama-4 Scout
5,,17
~335
Local Inference
13.30s
R1-1
Llama-4 Maverick
55,170
1135
Local Inference
13.82s
D Additional Evaluation
E
Impact Statement
SKETCHMIND advances AI in Education by introducing a cognitively grounded, multi-agent system for evaluating student-drawn scientific sketches. By combining semantic reasoning with Bloom's 's Taxonomy, enables interpretable, standards-aligned feedback that supports formative assessment. This approach benefits science educators by automating consistent, high-quality evaluation and guiding students through conceptually meaningful revisions. For researchers, SKETCHMIND provides a reproducible framework and dataset for studying multimodal reasoning, cognitive alignment, and agentic learning savveng-paving the way for more transparent and pedagogically sound AI tools in STEM education.
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
fine include aspirational goals as motivation, long as clear that these goals are not attainable by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The paper discusses limitations in Section
Guidelines:
The answer NA means that the paper has no limitations, while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not to penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
21

---

The paper does not present new theorems or formal proofs or make any theoretical claims.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in the appendix or supplemental material. Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
We provide implementation details in Section 4 and A.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in the supplemental material?
22

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they Answer: [No]
Justification: Dataset access is pending approval. Code will be open sourced.
open-sourced.
Guidelines:
The answer NA means that the paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
oewpemisssssollliiiooioooooooooo ilyyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: cc/ www.//pwlcgggpuiieeeeeeeeeeeeeee / /twpps.ccppub..l////
cesssoooooocyyyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, generated data, etc.
proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Test Details are specified in Sections 4, 5 and B.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in the appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined, or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Since our work does not involve any pre- pre-training or fine-tuning, and only involves MLLM inference, Therefore, the results shows the evaluation scores on test set. Which is expained in detail in 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. normally distributed errors).
23

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report 2-sigma error bar rather than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g., negative еπхо rates).
If error bars are reported in tables or plots, the authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Detailed computational resources, including hardware specifications for locally executed open-source models and costs for closed-source models are proivded in and C.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers, CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more computing than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https
Answer: [Yes]
Justification: Our research conforms fully with ethical guidelines and best practices as outlined by NeurIPS Code of Ethies.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Yes, we discuss broader impact in Appendix E.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for the responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: No high-risk assets being released, as the paper mainly involves MILM inference and students' scientific sketch dataset (pending approval); therefore, the paper does not describe any safeguards.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example, requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best-faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Yes, we have properly credited and cited the dataset and models in 4
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URI
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, phpore thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
25

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented, and is the documentation provided alongside the assets?
Answer: [No]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the datasettcoodemmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This work does not involve crowdsourcing human-subject studies.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: We have used fully anonymized dataset from Zhai et al.[30] study, hence doen't require IRBs conduct our research.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
26

---

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigor, or originality of the research, the declaration is not required.
Answer: [Yes]
Justification: Section details the of MLLMs in our multi-agent pipeline.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non non-standard components.
Please refer to our LLM policy (https / /tt///nerrpp/////e wwweurips.cc/ooneeeeeeeeeee cctp.//oeeeeenncc//225////////// for what should or should not be described.
27


Reviews:
[{"review_id": "Hq14XsA9C0", "reviewer": "Reviewer_bfZH", "summary": "The paper  tackles the problem of automatically evaluating student-generated scientific sketches and propose a cognitively-grounded approach using four specialized agents: \n- rubric parsing, \n- sketch perception, \n- cognitive alignment evaluation,\n- feedback generation with sketch modification. \n\nThe system represents sketches as semantic graphs with Bloom's taxonomy annotations and demonstrates great improvements over baseline approaches.", "questions": "N/A", "limitations": "N/A", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n- The paper introduces Sketch Reasoning Graphs with novel Bloom's taxonomy annotations. It provides a cognitively-grounded approach to sketch assessment that bridges educational theory with the model evaluation\n\n- The evaluation is comprehensive, testing many state-of-the-art models (GPT-4o, GPT-4.1, o3, o4-mini, LLaMA variants) across six assessment items spanning different Bloom cognitive levels. It also includes human evaluation study showing that expert educators rate the system's feedback highly (4.1/5 for GPT-4.1)\n\n- Consistnet performance gains across all models from SRG integration (e.g., GPT-4o: +21.4% average improvement), with strong results on cognitively demanding tasks\n\n**Weakness**\n- The paper lacks quantitative analysis of system efficiency. Four sequential agents create great computational overhead that could be hard for real-world classroom deployment. For real-time educational guidance, response times must be fast enough to maintain student engagement. \n\n- The paper would be improve if the it includes validation of core components. Agent 1 relies entirely on MLLM interpretation of rubrics to generate gold-standard SRGs, but there's lack of validation method to quantify how well these generated SRGs match expert-constructed graphs. And Agent 2's sketch perception capabilities lack quantitative assessment. It would be interesting to measure how accurately MLLMs can detect and interpret visual elements in student sketches, especially given that hand-drawn student sketches may not be commonly found in MLLM training data.\n\n- Not a weakness but as the author mentioned in the limitation section, it would be interesting to see how student behavioral data could be incorporated into the SRG pipeline for improving alignment with cognitive engagement signals beyond final sketch appearance.", "was_revised": true, "final_justification": "It is technically solid paper and I keep my positive score unchanged.", "rebuttal": "Thanks for your valuable feedback! We are honored that you recognize our\nnovel approach, comprehensive evaluation, and consistent performance\ngains. We address your concerns as follows:\n\n## **Weaknesses**\n\n**Weakness 1:** The paper lacks quantitative analysis of system\nefficiency. Four sequential agents create great computational overhead\nthat could be hard for real-world classroom deployment. For real-time\neducational guidance, response times must be fast enough to maintain\nstudent engagement.\n\n**Authors' Reply.** We acknowledge this important practical\nconsideration. We clarify that the current multi-agent structure\nprimarily aims to establish interpretability and specialized modular\nsemantic reasoning that is cognitively grounded for evaluating\nstudent-drawn scientific abstract sketches. Despite this focus, we have\nincorporated following efficient design measures:\n\n-   Agent 1 generates the gold-standard \\\"reference SRG\\\" only once per\n    task, with caching of Agent 1's outputs per rubric reducing repeated\n    computation overhead during inference (mentioned in Section 4\n    Implementation details).\n\n-   Agent 3 computes similarity between cached gold-standard \\\"reference\n    SRG\\\" and Agent 2 output using pre-coded evaluation logic (Appendix\n    A), requiring no MLLM calls.\n\n-   Agent 4 is only utilized when there are deficiencies in student\n    sketches, making it a conditional component in the sequence.\n\nHence, the system mainly depends on Agent 2's computational cost, as it\nis executed for every sample in the task.\n\nNow, for response times, our system relies on API latency for\nclosed-source models (GPT-4o, GPT-4.1, etc.). We will add the total\nend-to-end latency for each MLLM in Table 5, Appendix C, in the revised\nversion. In contrast, local deployment with Llama variants eliminates\nAPI latency. While not in the current study's scope, it would be\ninteresting to study the application of state-of-the-art model\ncompression techniques like low-bit weight/activation quantization \\[a\\]\nand pruning \\[b\\] to open-source local Llama variants, especially for\nAgent 2, which could further improve inference latency while maintaining\naccuracy. Finally, our cost analysis (Appendix C) shows per-sample\ninference costs of $0.0221$ for GPT-4.1 (best performing model), making\nclassroom deployment economically feasible even with closed-source\nmodels.\n\n**Weakness 2a:** The paper would be improve if the it includes\nvalidation of core components. Agent 1 relies entirely on MLLM\ninterpretation of rubrics to generate gold-standard SRGs, but there's\nlack of validation method to quantify how well these generated SRGs\nmatch expert-constructed graphs.\n\n**Authors' Reply.**\n\nWhile we relied on MLLMs for initial rubric parsing and generating\ngold-standard \\\"reference SRGs\\\" due to scalability and automation\ngoals, we acknowledge that full modular validation would increase system\nreliability but it requires detailed component-wise expert annotation\n(planned for future work). However, we implement several safeguards for\nrobust reference SRG generation:\n\n-   For each of the 6 tasks in the dataset, Agent 1 generates a single\n    gold-standard \\\"reference SRG\\\". To make the results and process\n    more reliable, the Agent 1 prompt is carefully crafted with\n    suggestions from domain experts. For each task, it takes into\n    account the expert-designed textual rubric, textual description of\n    the question, three expert-designed golden standard example\n    sketches, and a detailed modular example of components it expects in\n    the final reference SRG.\n\n-   This implements inherent cross-validation against multiple\n    gold-standard sketches per task generated by domain experts.\n\n-   The MLLM-generated final gold-standard \\\"reference SRGs\\\" for each\n    task are validated manually by four seasoned domain experts in this\n    field, ensuring semantic consistency checks using domain ontologies.\n\nOur ablation study (Table 2) showing consistent gains across all models\nsuggests our reference SRG quality is reliable to guide the system\ntowards high performance gains. However, we still acknowledge that\nquantitative expert-SRG alignment metrics would further strengthen this\nclaim.\n\n**Weakness 2b:** Agent 2's sketch perception capabilities lack\nquantitative assessment. It would be interesting to measure how\naccurately MLLMs can detect and interpret visual elements in student\nsketches, especially given that hand-drawn student sketches may not be\ncommonly found in MLLM training data.\n\n**Authors' Reply.** This is an excellent point. While assessing MLLMs'\ncapabilities to interpret visual elements in abstract sketches would\nrequire detailed experiments and benchmarking across different SOTA\nMLLMs, constituting a research question in itself, we provide indirect\nassessment of sketch perception capabilities across different MLLMs for\ndetecting and interpreting visual elements in student hand-drawn\nsketches, despite such sketches being uncommon in MLLM training data:\n\n-   Table 1 results demonstrate that MLLMs' capability to detect and\n    interpret visual elements in hand-drawn student sketches (with or\n    without SRG pipeline) correlates positively with model size. Larger\n    MLLMs achieve both higher initial accuracy and higher performance\n    with SRG integration (e.g., GPT-4.1 demonstrates the highest average\n    accuracy across all tasks both with and without SRG integration).\n\n-   Human expert ratings of final outputs also correlate with model\n    capability (4.1/5 highest for GPT-4.1), suggesting that larger\n    models better perceive and interpret sketch elements for generating\n    pedagogically sound feedback.\n\nWe agree that direct perceptual accuracy metrics (e.g., concept\ndetection precision/recall and spatial relationship identification\naccuracy among concepts) would be valuable and plan to include these in\nfuture work with expert-annotated sketch components.\n\n**Weakness 3:** Not a weakness but as the author mentioned in the\nlimitation section, it would be interesting to see how student\nbehavioral data could be incorporated into the SRG pipeline for\nimproving alignment with cognitive engagement signals beyond final\nsketch appearance.\n\n**Authors' Reply.** We appreciate this forward-looking suggestion. While\nour current work focuses on static sketch assessment, incorporating\nstroke sequences, timing, and revision patterns into SRG construction\ncould indeed enhance cognitive alignment beyond final sketch appearance.\nThis represents an exciting direction for extending our framework beyond\nstatic sketch analysis to dynamic learning process modeling.\n\n**References:**\n\n\\[a\\] Xiao, G., Lin, J., Seznec, M., Wu, H., Demouth, J., & Han, S.\n(2023, July). Smoothquant: Accurate and efficient post-training\nquantization for large language models. In International conference on\nmachine learning (pp. 38087-38099). PMLR.\n\n\\[b\\] Sun, M., Liu, Z., Bair, A., & Kolter, J. Z. (2023). A simple and\neffective pruning approach for large language models. arXiv preprint\narXiv:2306.11695."}, {"review_id": "1jkXHlXyW9", "reviewer": "Reviewer_PZAq", "summary": "This paper presents SketchMind, a multi-agent cognitive framework for the automated assessment and improvement of student-drawn scientific sketches. The core innovation lies in the introduction of Sketch Reasoning Graphs (SRGs), which are semantic graph representations embedding both domain-specific concepts and cognitive labels based on Bloom’s Taxonomy. The authors evaluate SketchMind on a curated dataset, showing significant accuracy improvements over baseline MLLMs.", "questions": "• What precisely constitutes a “scientific sketch”? What are the differences between *student*-drawn and *teacher*-drawn scientific sketches? Providing examples would help clarify this distinction.\n\n• Could the authors provide more examples and analysis, detailing the types of errors SketchMind makes (e.g., errors related to specific Bloom levels, complex relationships, or specific visual representations)?", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "*Strengths*:\n\n• The integration of Bloom’s Taxonomy introduces a cognitive dimension beyond traditional visual sketch understanding, substantially enhancing the system’s interpretability and depth of analysis.\n\n• The use of a multi-agent architecture further improves the interpretability and modularity of the proposed framework.\n\n*Weaknesses*:\n\n• **Scope and Fit**: I am not sure that NeurIPS is the most suitable venue for this submission. The work lies primarily within the domain of AI4Education, and its application scenario is quite limited. In my view, this paper might be a better fit for a workshop focused on AI for Education.\n\n• **Limited Methodological Novelty**: The paper’s novelty appears somewhat limited. While the use of Bloom’s Taxonomy provides a new perspective for understanding sketches, there is not much methodological innovation beyond this integration.\n\n• **SRGs as a Prompting Strategy**: While SRGs are effective in enhancing LLM performance, their primary role in the system is as a sophisticated prompting strategy that leverages structured data. As such, this contribution is more closely related to prompt engineering than to the development of a new algorithm.\n\n• **Lack of Analysis**: The paper reports overall accuracy and human evaluation scores, but lacks detailed analysis on failure cases, which would be valuable for understanding the system’s limitations.", "was_revised": true, "final_justification": "The authors' response has addressed my concerns well. Therefore, I raise my score.", "rebuttal": "Thanks for your valuable feedback! We are honored that you recognize how\nour Bloom's Taxonomy integration introduces cognitive dimensions that\nenhance interpretability and analysis depth, and that our multi-agent\narchitecture improves framework interpretability and modularity. We\naddress your questions below and would be grateful if you could consider\nimproving the rating after seeing our responses.\n\n## **Weaknesses**\n\n**Weakness 1:** Scope and Fit: I am not sure that NeurIPS is the most\nsuitable venue for this submission. The work lies primarily within the\ndomain of AI4Education, and its application scenario is quite limited.\nIn my view, this paper might be a better fit for a workshop focused on\nAI for Education.\n\n**Authors' Reply.** We respectfully clarify that the submission aligns\nclosely with NeurIPS's \\\"Applications of AI\\\" track, as the core\ncontributions extend beyond education-specific scenarios to domains\nrequiring structured interpretation of visual artifacts. Although our\nprimary demonstration focuses on education, our method, introducing SRGs\ngrounded in pedagogical frameworks \\[a\\], addresses the common challenge\nof interpreting abstract sketches, a task currently not well handled by\nexisting MLLMs as highlighted in Table 1. Our framework not only\nenhances educational assessments but also provides foundational\nmethodological innovations applicable to broader domains where abstract\ngraphical interpretation is required, such as scientific communication,\ndiagram understanding, grounding abstract visual artifacts and visual\nreasoning. Recent advances such as SketchPad \\[b\\] and Sketch\nUnderstanding \\[c\\] further illustrate the growing interest in robust\nmultimodal reasoning frameworks, reinforcing the broader relevance of\nour methodological contributions to the NeurIPS community.\n\n**Weakness 2:** Limited Methodological Novelty: The paper's novelty\nappears somewhat limited. While the use of Bloom's Taxonomy provides a\nnew perspective for understanding sketches, there is not much\nmethodological innovation beyond this integration.\n\n**Authors' Reply.** We respectfully clarify that our contributions\nsignificantly exceed mere integration of Bloom's taxonomy. Specifically,\nour proposed SRG framework algorithmically transforms abstract\nstudent-generated sketches and rubric texts into structured, cognitively\nannotated graph representations through a novel multi-agent pipeline.\nAgent 1 parses rubric text into SRGs, providing structured cognitive\ngrounding; Agent 2 algorithmically maps diverse, free-form student\nsketches into consistent SRGs annotated with Bloom's cognitive levels.\nThis dual-agent algorithmic pipeline is fundamentally novel: existing\napproaches (e.g. VisualSketchPad \\[b\\]) rely on multimodal LLM reasoning\nwithout structured cognitive annotations, and SketchFusion \\[d\\] focuses\npurely on sketch embedding without cognitive grounding. We also\nintroduce: reverse mapping $\\phi$ for visual hint generation from\ncognitive concepts; graph-based similarity metrics to incorporate both\nstructural and cognitive alignment (Eq. 3-4); multi-agent orchestration\nfor modularity and interpretability, improving MLLMs' performance by\n7.5-24.9% across models. Thus, our methodological novelty not only lies\nin the mere usage of Bloom's taxonomy, rather it's a set of\naforementioned contributions.\n\n**Weakness 3:** SRGs as a Prompting Strategy: While SRGs are effective\nin enhancing LLM performance, their primary role in the system is as a\nsophisticated prompting strategy that leverages structured data. As\nsuch, this contribution is more closely related to prompt engineering\nthan to the development of a new algorithm.\n\n**Authors' Reply.** We respectfully disagree with the reviewer's\nperspective on SRGs as a prompting strategy. We would like to clarify\nthat SRGs are not merely prompts but structured intermediate\nrepresentations that fundamentally change how MLLMs process visual\ninformation. Unlike prompting strategies that modify input text, SRGs:\nenforce structured reasoning through graph constraints; enable\nquantitative similarity computation between scientific sketches (i.e.,\nquantitatively comparing abstract visual artifacts one by one in those\nsketches with gold-standard \\\"reference SRGs\\\"); and provide\ninterpretable decision paths. SRGs constitute an original algorithmic\ninnovation rather than merely advanced prompt engineering. Unlike\nstandard prompting, SRG generation involves systematic analysis and\ncognitive annotation of abstract visual inputs using Bloom's taxonomy\nthrough multiple distinct algorithmic stages. Specifically, Agent 1\nalgorithmically generates SRGs from rubric texts by extracting cognitive\ncomponents, and Agent 2 transforms diverse, free-form sketches into\ncognitively annotated structured graphs. The resulting structured SRGs\nare embedded into prompts only subsequently (Agent 4), serving as\nstructured cognitive representations rather than simple prompt\naugmentations. The structured cognitive annotations within SRGs not only\nguide inference but enable sophisticated reasoning about abstract visual\nrelationships, thus constituting a substantial algorithmic advance\nbeyond standard prompt engineering methods \\[e\\]. Results in Table 1\nshow that incorporation of SRGs with MLLMs significantly improves the\nperformance by 7.5-24.9% across all models.\n\n**Weakness 4:** Lack of Analysis: The paper reports overall accuracy and\nhuman evaluation scores, but lacks detailed analysis on failure cases,\nwhich would be valuable for understanding the system's limitations.\n\n**Authors' Reply.** We thank the reviewer for highlighting the need for\ndeeper analysis. In response, we will provide detailed failure case\nanalysis in Appendix E of the revised manuscript, categorizing errors\nexplicitly based on Bloom's cognitive levels and the types of visual\ncomplexities encountered. Specifically, we found that in 89% of the\ncases, our framework's classifications aligned closely with human\nexperts. However, discrepancies arose predominantly in 7% of cases,\ntypically involving complex visual relationships or ambiguous sketches,\ncausing the system to overestimate cognitive complexity (e.g.,\nclassifying \"Understand\" as \"Analyze\"). An additional 4% of sketches\nwere inadequately processed due to poor sketch quality or excessive\nambiguity. Each failure type will be explicitly illustrated with\nrepresentative examples and discussed to provide transparency and to\nidentify future directions for improvement in Appendix E of the revised\nmanuscript.\n\n## **Questions**\n\n**Question 1:** What precisely constitutes a \"scientific sketch\"? What\nare the differences between student-drawn and teacher-drawn scientific\nsketches? Providing examples would help clarify this distinction.\n\n**Authors' Reply.** A \"scientific sketch\" is a visual representation\nintended to illustrate a scientific concept, process, or phenomenon. For\nexample, for thermodynamics concept, the scientific sketch shows heat\nsource, its transition phases, arrows may represent the flow of heat,\nand object/medium transferring the heat. Teacher-drawn sketches serve as\ngold-standard reference sketches, explicitly containing essential visual\ncomponents necessary to fully represent the target scientific concept\n(e.g., arrows clearly indicating heat transfer direction in\nthermodynamics). In contrast, student-drawn sketches often reflect\nincomplete, varied, or erroneous understanding, exhibiting substantial\ndiversity and ambiguity. We explicitly distinguish these categories in\nour data annotation by using teacher sketches as gold-standard\n\\\"reference SRGs\\\" and grounding student sketches pedagogically to\nBloom's cognitive levels. We will include representative examples,\nillustrating these differences in Appendix B of the revised manuscript.\n\n**Question 2:** Could the authors provide more examples and analysis,\ndetailing the types of errors SketchMind makes (e.g., errors related to\nspecific Bloom levels, complex relationships, or specific visual\nrepresentations)?\n\n**Authors' Reply.** As detailed in response to Weakness 4, we will\nprovide the detailed documentation of failure cases in Appendix E of the\nrevised manuscript. For instance, one frequent error category involves\nsketches with complex visual relationships, leading\n[SketchMind]{.smallcaps} to erroneously assign higher cognitive levels\n(e.g., labeling \"Understand\" tasks as \"Analyze\"). Another distinct\ncategory (4% of cases) involves sketches so ambiguous or incomplete that\nSRGs could not be meaningfully constructed.\n\n**References:**\n\n\\[a\\] Hall, K. (2012). Grounding assessment in authentic pedagogy: A\ncase study of general education assessment (Doctoral dissertation,\nUniversity of St. Thomas, Minnesota).\n\n\\[b\\] Hu, Y., Shi, W., Fu, X., Roth, D., Ostendorf, M., Zettlemoyer, L.,\n\\... & Krishna, R. (2024). Visual sketchpad: Sketching as a visual chain\nof thought for multimodal language models. Advances in Neural\nInformation Processing Systems, 37, 139348-139379.\n\n\\[c\\] Shrivastava, M., Isik, B., Li, Q., Koyejo, S., & Banerjee, A.\n(2024). Sketching for Distributed Deep Learning: A Sharper Analysis.\nAdvances in Neural Information Processing Systems, 37, 6417-6447.\n\n\\[d\\] Koley, S., Dutta, T. K., Sain, A., Chowdhury, P. N., Bhunia, A.\nK., and Song, Y. Z. (2025). SketchFusion: Learning Universal Sketch\nFeatures through Fusing Foundation Models. In Proceedings of the\nComputer Vision and Pattern Recognition Conference (pp. 2556-2567).\n\n\\[e\\] Wang, C., Yang, Y., Gao, C., Peng, Y., Zhang, H., & Lyu, M. R.\n(2022, November). No more fine-tuning? an experimental evaluation of\nprompt tuning in code intelligence. In Proceedings of the 30th ACM joint\nEuropean software engineering conference and symposium on the\nfoundations of software engineering (pp. 382-394)."}, {"review_id": "XKzkU00Gg8", "reviewer": "Reviewer_Y5Kd", "summary": "The authors propose a multi-agent framework for evaluating and enhancing student-drawn scientific sketches. The system consists of modular agents dedicated to rubric parsing, sketch perception, cognitive alignment, and iterative feedback with sketch modification, enabling personalized and transparent evaluation. Additionally, the framework integrates Bloom’s Taxonomy—a standard in cognitive theory—to construct and analyze Sketch Reasoning Graphs (SRGs), facilitating structured evaluation of visual student work and delivering pedagogically sound feedback alongside real-time sketch refinement. Compared to the baseline GPT-4.0 performance, SKETCHMIND demonstrates a significant improvement in quality.", "questions": "- The concept of “Bloom level annotation” would benefit from a more self-contained explanation. Including a brief description along with a schematic diagram could significantly improve comprehension for a broader readership.\n- Similarly, the term “rubric concept” requires a clearer, standalone explanation. Expanding on this would enhance the paper’s accessibility to readers from diverse backgrounds.\n- The representation of the visual suggestion H(t) is unclear. Further elaboration on how this is formulated and how step 5 (line 175) is operationalised would be helpful.\n- A more detailed and standalone discussion of Bloom’s cognitive hierarchy would strengthen the theoretical clarity and support the framework’s pedagogical grounding.\n- The caption for Table 1 could be more comprehensive, offering sufficient context and explanation to ensure that it is interpretable without referring extensively to the main text.", "limitations": "- There exist some inconsistencies in the citations. Please proofread carefully. For example, SketchFusion citation is wrong. \n- I wonder if the codebase will be made public for the ease of reproducibility.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "+ Overall, the idea is novel and compelling.\n+ The proposed methodology and experimental section are clearly presented.\n\n– A major weakness, however, is that several parts of the paper are difficult to follow unless the reader is already deeply familiar with prior work in this niche domain. A significant improvement would be to make the writing more self-contained, which would broaden the paper’s accessibility and appeal to a wider audience beyond those specifically focused on sketch-based systems.", "was_revised": true, "final_justification": "Most of my concerns have been addressed, and overall, this is a reasonably good paper. Improving the paper’s self-containment could help it appeal to a broader readership.", "rebuttal": "We thank the reviewer for the insightful comments and encouraging\nfeedback! We are honored that you acknowledge the novelity of the\napproach and admire the clarity of proposed methodology and experiments.\nWe address your raised conerns as follows:\n\n## **Weaknesses**\n\n**Weakness 1:** A major weakness, however, is that several parts of the\npaper are difficult to follow unless the reader is already deeply\nfamiliar with prior work in this niche domain. A significant improvement\nwould be to make the writing more self-contained, which would broaden\nthe paper's accessibility and appeal to a wider audience beyond those\nspecifically focused on sketch-based systems.\n\n**Authors' Reply.** We agree that improving self-containment will\nbroaden accessibility. In the camera-ready version, we will (i) add\nconcise details in Subsection 3.1 titled \"Cognitive Framework: Bloom's\nTaxonomy in Sketch Understanding\" that defines the six levels and\nexplains how they operationalize our SRG nodes/edges, and (ii) introduce\n\"rubric concepts\" in Subsection 3.2 with a short example. We will also\ninclude a schematic figure that visually links the question to the\nrubric to SRG nodes/edges to Bloom levels to the feedback loop. These\nadditions aim to minimize prerequisite familiarity with sketch-based\nassessment literature.\n\n## **Questions**\n\n**Question 1:** The concept of \"Bloom level annotation\" would benefit\nfrom a more self-contained explanation. Including a brief description\nalong with a schematic diagram could significantly improve comprehension\nfor a broader readership.\n\n**Authors' Reply.** We appreciate this suggestion and have clarified\nBloom level annotation in Subsection 3.1 as follows:\n\n**Bloom-Level Annotation:** We annotate each node (concept) and edge (relation) in the Sketch Response Graph (SRG) with a cognitive level\n $\\ell \\in \\{1,\\ldots,6\\}$, following Bloom's revised taxonomy \\[a\\]:\n\n> 1.  **Remember**: Recall facts (e.g., \"Label the particles\").\n>\n> 2.  **Understand**: Explain concepts (e.g., \"Describe how particles\n>     move\").\n>\n> 3.  **Apply**: Use concepts in a context (e.g., \"Apply diffusion law\n>     to explain spread\").\n>\n> 4.  **Analyze**: Break down phenomena (e.g., \"Identify variables\n>     affecting diffusion\").\n>\n> 5.  **Evaluate**: Judge or justify (e.g., \"Assess explanation\n>     quality\").\n>\n> 6.  **Create**: Generate novel work (e.g., \"Design a new experiment\").\n\nWe implement $\\ell(\\cdot)$ by first extracting key verbs and criteria\nfrom rubric statements and mapping them via a curated Bloom-level\nlexicon \\[b\\], then validating with a fine-tuned classifier for\nambiguous cases. Each SRG component is thus paired with its cognitive\ndemand, supporting both nuanced evaluation and feedback generation.\nThese numeric levels are encoded as node/edge attributes and used\ndirectly in similarity scoring during assessment.\n\nFigure 2a in the revised manuscript will schematically illustrate the mapping: a rubric line (*e.g.,* \"describe the direction of dye\nmovement\") $\\rightarrow$ SRG node (*Dye Direction*) $\\rightarrow$\nassigned Bloom level (*Understand*, $\\ell=2$).\n\n**Question 2:** Similarly, the term \"rubric concept\" requires a clearer,\nstandalone explanation. Expanding on this would enhance the paper's\naccessibility to readers from diverse backgrounds.\n\n**Authors' Reply.** By \"rubric concept\", we mean an atomic idea or\nrelation explicitly described in the grading rubric (e.g., \"particles\nmove slower at lower temperature\"). Agent 1 parses the rubric to extract\nthese concepts, assign Bloom levels, and populate the gold SRG ($G_o$).\nFigure 1, demonstrates that how a rubric concept is converted into an\nSRG nodes. For example, a question in Figure 1 \"Red Dye Diffusion\"\nrubric line converts into node list Dye_Particles, Water_Molecules,\nDiffusion_Process with Bloom labels.\n\n**Question 3:** The representation of the visual suggestion H(t) is\nunclear. Further elaboration on how this is formulated and how step 5\n(line 175) is operationalised would be helpful.\n\n**Authors' Reply.** H(t) is the set of visual hints (e.g., \"draw an\narrow from hot water to steam\") generated via the reverse mapping\n$\\phi$. In step 5, [ScketchMind]{.smallcaps} transforms each hint into\nexecutable Python code (PIL overlay) to modify the canvas. We will\nexplicitly enumerate the inputs/outputs of step 5 and provide a short\ncode snippet in Appendix A showing overlay_hint().\n\n**Question 4:** A more detailed and standalone discussion of Bloom's\ncognitive hierarchy would strengthen the theoretical clarity and support\nthe framework's pedagogical grounding.\n\n**Authors' Reply.** Here is a brief overview of the discussion on\nBloom's cognitive hierarchy that will be included in the revised\nmanuscript.\n\n>**Bloom's Cognitive Hierarchy.** Bloom's Taxonomy \\[a\\] structures\n> learning objectives into six levels of increasing cognitive\n> complexity, from basic recall (*Remember*) to synthesis and innovation\n> (*Create*). This hierarchy is foundational in science education,\n> guiding both assessment design and instructional scaffolding \\[c\\]. In\n> our framework, each gold-standard SRG is labeled at the level of its\n> highest Bloom demand, and student sketches are evaluated by comparing\n> the cognitive depth of their annotated SRGs. This enables both\n> fine-grained diagnostics (identifying the highest achieved level) and\n> adaptive feedback: for example, if a learner demonstrates\n> *Understand*, the system generates visual hints nudging them toward\n> *Apply* (cf. Figure 3). Prior work demonstrates that aligning\n> AI-driven feedback with Bloom's hierarchy supports deeper learning\n> gains in STEM domains \\[c, d\\].\n\n**Question 5:** The caption for Table 1 could be more comprehensive,\noffering sufficient context and explanation to ensure that it is\ninterpretable without referring extensively to the main text.\n\n**Authors' Reply.** Here is a detailed caption for Table 1. \"Table 1.\nItem-wise accuracy (%) for six NGSS-aligned assessment items across\nmultiple MLLMs, comparing performance without SRG vs. with SRG. *Gain*\ndenotes absolute improvement. Items differ in their highest targeted\nBloom level (Analyze for R1-1/J6-1, Evaluate for J2-1/M3-1/H5-1, Create\nfor H4-1). The consistent gains across all models indicate that SRG\nsupervision improves both lower- and higher-order cognitive\nassessments.\"\n\n## **Limitations**\n\n**Limitation 1:** There exist some inconsistencies in the citations.\nPlease proofread carefully. For example, SketchFusion citation is wrong.\n\n**Authors' Reply.** We appreciate the pointer. We will run a full\ncitation audit (cross-checking author names, venues, page numbers) and\ncorrect errors. SketchFusion is now published at CVPR 2025; we will cite\nthe final version \\[e\\] in place of the arXiv entry.\n\n**Limitation 2:** I wonder if the codebase will be made public for the\nease of reproducibility.\n\n**Authors' Reply.** Yes. The anonymized repository link is currently\nprovided (footnote p.2). Upon acceptance, we will de-anonymize and\nrelease the full code (and, pending IRB approvals, the dataset) to\nensure reproducibility. We will add a \"Reproducibility and Resources\"\nparagraph to Appendix A with environment details and run scripts.\n\n**References:**\n\n\\[a\\] Elim, E. H. S. Y. (2024). Promoting cognitive skills in\nAI-supported learning environments: the integration of bloom's taxonomy.\nEducation 3-13, 1-11.\n\n\\[b\\] Reddy, Y. M., & Andrade, H. (2010). A review of rubric use in\nhigher education. Assessment & evaluation in higher education, 35(4),\n435-448.\n\n\\[c\\] Hui, E. S. Y. E. (2025). Incorporating Bloom's taxonomy into\npromoting cognitive thinking mechanism in artificial\nintelligence-supported learning environments. Interactive Learning\nEnvironments, 33(2), 1087-1100.\n\n\\[d\\] Gonsalves, C. (2024). Generative AI's impact on critical thinking:\nRevisiting Bloom's taxonomy. Journal of Marketing Education,\n02734753241305980.\n\n\\[e\\] Koley, S., Dutta, T. K., Sain, A., Chowdhury, P. N., Bhunia, A.\nK., and Song, Y. Z. (2025). SketchFusion: Learning Universal Sketch\nFeatures through Fusing Foundation Models. In Proceedings of the\nComputer Vision and Pattern Recognition Conference (pp. 2556-2567)."}, {"review_id": "ZalzE4smpu", "reviewer": "Reviewer_4rkr", "summary": "This paper proposes a multi-agent framework for evaluating student-drawn scientific sketches and providing automated feedback. The framework comprises four specialized agents:\n\nAgent 1 generates gold-standard sketch reasoning graphs (SRGs) for given scientific questions using multimodal large language models (MLLMs). \nAgent 2 constructs corresponding SRGs based on student-drawn sketches, also utilizing MLLMs. \nAgent 3 evaluates student performance by computing similarity scores between the gold-standard and student-generated SRGs, and additionally provides learning level assessments and feedback signals through rule-based logic. \nAgent 4 generates visual sketch hints to guide student learning using MLLMs.\n\nThe authors evaluate their proposed framework on the dataset from Zhai et al. (2022) [28], which consists of six scientific modeling assessment tasks. The evaluation employs both closed-source models (e.g., GPT-4) and open-source LLaMA models. Experimental results demonstrate that the proposed framework effectively automates the assessment of scientific sketches and provides feedback aligned with human assessment.", "questions": "The details regarding the human evaluation are insufficient and require clarification for reproducibility and validity assessment.\n- How many human evaluators participated in the study? Did they evaluate results from all baselines (e.g., GPT-4.1, O3…) across the 3,575 student-generated sketches, or was a subset used?\n- Are there established evaluation guidelines for human experts? What specific criteria do human experts use to assess sketch quality and feedback usefulness?\n- Could the authors provide cases where the framework's assessment aligns with human expert judgment, as well as cases where they diverge? This would help readers understand the quality of the alignment.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n- The paper proposes an effective multi-agent framework for evaluating students' responses to scientific questions, which could be valuable for developing AI-assisted tools in STEM education.\n- The paper is well-written.\n- The experiment is well-designed to demonstrates the effectiveness of the proposed framework. The authors utilizes both closed-source and open-source models and also includes the human evaluation studies.\n\nWeaknesses\n\n- Some details are missing about the human evaluation process. Specifically, the authors do not provide sufficient information about the number of evaluators and assessment criteria used by human experts.", "was_revised": true, "final_justification": "The authors have provided clear clarifications that address my main concerns. Updating my score accordingly.", "rebuttal": "Thanks for your valuable feedback! We are honored that you recognize our\neffective multi-agent framework's value for AI-assisted STEM education\ntools, find our paper well-written, and appreciate our well-designed\nexperiments utilizing both closed-source and open-source models with\nhuman evaluation studies. We address your questions below and would be\ngrateful if you could consider improving the rating after seeing our\nresponses!\n\n## **Weaknesses**\n\n**Weakness 1:** Some details are missing about the human evaluation\nprocess. Specifically, the authors do not provide sufficient information\nabout the number of evaluators and assessment criteria used by human\nexperts.\n\n**Authors' Reply.** We appreciate the reviewer's attention to the rigor\nof our human evaluation. As described in Section 4, our study involved\nfour domain-expert educators with graduate-level science education\nbackgrounds, each independently assessing the pedagogical quality of\nmodel-generated responses. Evaluation criteria and procedures were based\non an extended rubric from Zhai et al. \\[a\\], with calibration and\ninter-expert reliability (Quadratic Weighted Kappa (Qkappa = 0.83),\nconsistent with benchmarks for expert judgment in science assessment\n\\[b\\]) detailed below and in responses to Questions 1 and 2. These\ndetails will be reported in Section 4 of the revised manuscript.\n\n## **Questions**\n\n**Question 1:** The details regarding the human evaluation are\ninsufficient and require clarification for reproducibility and validity\nassessment. How many human evaluators participated in the study? Did\nthey evaluate results from all baselines (e.g., GPT-4.1, O3...) across\nthe 3,575 student-generated sketches, or was a subset used?\n\n**Authors' Reply.** As noted above, four expert raters participated in\nthe evaluation. Each independently assessed a stratified random sample\nof 890 student-generated sketches ( 25% of the dataset), ensuring\nbalanced representation across all baseline models (GPT-4o, GPT-4.1,\nO3), grade levels, and science tasks. All raters underwent a\ncollaborative calibration phase, jointly annotating and discussing a set\nof 10 representative examples to align rubric interpretation. They then\nindependently scored the same subset of 10 sketches, with (Qkappa =\n0.83) confirming high inter-expert reliability. For the main study,\nevery response was double-blindly rated by at least two experts,\nfollowing the protocol and criteria detailed in Question 2. These\ndetails will be reported in Section 4 of the revised manuscript.\n\n**Question 2:** Are there established evaluation guidelines for human\nexperts? What specific criteria do human experts use to assess sketch\nquality and feedback usefulness?\n\n**Authors' Reply.** Yes, all evaluators used a detailed, task-specific\nrubric adapted from Zhai et al. \\[a\\] and further tailored for formative\nscience assessment. For each model-generated response, experts rated:\n(i) *Clarity*, or whether the feedback, both textual and visual, was\nunderstandable and actionable; (ii) *Conceptual Accuracy*, i.e., whether\nthe suggested sketch modifications and explanations correctly reflected\nthe target scientific concepts; and (iii) *Instructional Value*,\nreflecting the feedback's potential to foster student learning and\nprogression along Bloom's taxonomy. Each dimension was scored on a\n5-point ordinal scale (1-5). Consistency was ensured by an initial\ncalibration session (see the response to Question 1), and all ratings were conducted\ndouble-blind to avoid bias. This rubric-driven approach supports\ntransparent, reproducible, and pedagogically meaningful evaluation of\nboth sketch quality and formative feedback. These details will be\nreported in Section 4 of the revised manuscript.\n\n**Question 3:** Could the authors provide cases where the framework's\nassessment aligns with human expert judgment, as well as cases where\nthey diverge? This would help readers understand the quality of the\nalignment.\n\n**Authors' Reply.** To illustrate alignment, A new table in Appendix E\nof the revised manuscript will present representative examples:\n\n-   **Aligned:** In 89% of the sampled cases, both the framework and\n    human experts agreed on sketch correctness and Bloom-level\n    annotation. For example, for a \"diffusion direction\" task, both\n    rated a student's labeled arrow and explanation as \"Apply\" (Level\n    3).\n\n-   **Divergent:** Divergence was observed primarily in cases of\n    ambiguous or unconventional student sketches. For instance, in 7% of\n    cases, the framework classified a response as \"Analyze\" (Level 4)\n    due to a complex graph structure, while human raters assigned\n    \"Understand\" (Level 2) based on missing explicit justifications.\n    These disagreements are discussed in Appendix D, with specific\n    examples provided.\n\nFramework-human agreement rates will be reported in Section 4 of the\nrevised manuscript, and all codebooks and samples are included in the\nsupplementary material.\n\n**References:**\n\n\\[a\\] Zhai, X., He, P., & Krajcik, J. (2022). Applying machine learning\nto automatically assess scientific models. *Journal of Research in\nScience Teaching*, 59(10), 1765-1794.\n\n\\[b\\] Hickling, E. M., & Bowie, J. E. (2013). Applicability of human\nreliability assessment methods to human--computer interfaces. Cognition,\ntechnology & work, 15(1), 19-27."}]

Author rebuttal:

Meta-review: This paper tackles an interesting problem in educational applications of AI, which is that of how to automatically assess student-drawn scientific diagrams in a way that is both accurate and pedagogically meaningful. They do so by introducing a graph representation grounded in both domain concepts and cognitive complexity levels (drawing on Bloom's canonical taxonomy), and a multi-agent architecture that decomposes the assessment protocol into well-described, specialized cognitive components.


All reviewers lean toward acceptance, remarking on the comprehensiveness of the empirical evaluation, cognitive grounding, and a variety of additional strengths. It is clear that this work has significant applied potential.


The most significant concerns expressed are primarily around (1) limited methodological novelty, (2) limited system-level validation, and (3) some limitations in analysis and reporting, e.g. of failure cases and experiment details.


In my view, the authors did a commendable job engaging in discussion with the reviewers, and I expect their comments to enhance the clarity of the paper significantly for the NeurIPS audience. I also do not think that these concerns fundamentally undermine the applied contribution here, and there is also the possibility of broader methodological influence in multimodal work. I thus recommend acceptance.
Decision: Accept (poster)