Code2Video: A Code-centric Paradigm for Educational Video Generation

## Abstract

Abstract While recent generative models advance pixel-space video synthesis, they remain limited in producing professional educational videos, which demand disciplinary knowledge, precise visual structures, and coherent transitions, limiting their applicability in educational scenarios.
Intuitively, such requirements are better addressed through the manipulation of a renderable environment, which can be explicitly controlled via logical commands ( e.g., code).
In this work, we propose Code2Video , a code-centric agent framework for generating educational videos via executable Python code.
The framework comprises three collaborative agents: (i) Planner , which structures lecture content into temporally coherent flows and prepares corresponding visual assets; (ii) Coder , which converts structured instructions into executable Python codes while incorporating scope-guided auto-fix to enhance efficiency;
and (iii) Critic , which leverages vision-language models (VLM) with visual anchor prompts to refine spatial layout and ensure clarity.
To support systematic evaluation, we build MMMC , a benchmark of professionally produced, discipline-specific educational videos. We evaluate MMMC across diverse dimensions, including VLM-as-a-Judge aesthetic scores, code efficiency, and particularly, TeachQuiz ,
a novel end-to-end metric that quantifies how well a VLM, after unlearning, can recover knowledge by watching the generated videos.
Our results demonstrate the potential of Code2Video as a scalable, interpretable, and controllable approach, achieving 40% improvement over direct code generation and producing videos comparable to human-crafted tutorials.
The code and datasets are available at https://github.com/showlab/Code2Video .

## 1 Introduction

“If you want to master something, teach it.” – Richard Feynman

Recent advances in natural video generation have made remarkable progress in pixel space. End-to-end solutions, including diffusion-based (Ho et al., 2022a; Weng et al., 2024b) and autoregressive architectures (Weng et al., 2024a; Yuan et al., 2025), can synthesize visually compelling videos directly from text prompts (i.e.,Text2Video), achieving fine appearance and short-form fidelity. Yet these models struggle when the task requires long-form reasoning or multi-entity interaction (Li et al., 2024a). To overcome these limitations, recent works have moved toward multi-agent pipelines, where complex video generation is decomposed into collaborative subtasks, allowing iterative refinement, temporal structuring (Yuan et al., 2024; Huang et al., 2024; Xie et al., 2024).

Educational videos that aim to teach subject-specific knowledge face unique challenges in the reasoning era.
Unlike short-form entertainment, educational content must integrate deep domain expertise (Clark & Mayer, 2023), carefully designed animations or transitions, and step-by-step reasoning (Bao et al., 2009) to support actual skill acquisition. This raises two fundamental challenges: (i) How to create high-quality educational videos that maintain both temporal coherence—concepts introduced, expanded, and reinforced in logical sequence—and spatial clarity—elements arranged legibly without occlusion; and (ii) How to evaluate educational videos beyond appearance, ensuring that they are educationally effective and semantically aligned with the intended learning topic.
Existing video generation pipelines rarely satisfy these requirements, leaving a critical gap for agentic methods that unify temporal planning, spatial organization, and educational assessment.

We are motivated by the intuition that code provides a uniquely suitable substrate for educational video generation. Unlike black-box models, code-centric pipelines are scalable, since new visualizations and external assets can be modularly integrated; interpretable, as every sequence, layout, and rendering decision is explicitly scripted and thus auditable; and controllable, enabling precise temporal sequencing and spatial organization through programmatic specification.

Figure: Figure 1: Overview of Code2Video. A code-centric paradigm for educational video generation, where Planner ensures temporal flow, Coder bridges instructions to executable animations, and Critic refines spatial layout. Evaluation is performed on MMMC with multi-dimensional metrics.
Refer to caption: x1.png

Building on these insights, we propose Code2Video, an agentic, code-centric framework for generating high-quality educational videos. The system decomposes the task into three agents: the Planner sequences concepts, examples, and recaps into a coherent lecture flow; the Coder translates structured instructions into executable Manim code, yielding precise, editable visualizations with consistent layout and timing; and the Critic leverages multimodal feedback and visual anchor prompts to refine spatial organization and ensure alignment with learning objectives.
This tri-agent design explicitly models the temporal and spatial structure of instruction, while grounding the entire pipeline in transparent, reproducible, and extensible code.

To evaluate this paradigm, we propose MMMC, a benchmark reflecting the distinct goal of educational videos: teaching new knowledge. It comprises professionally produced, discipline-specific Manim tutorials across 13 domains (e.g.,topology, physics).
Evaluation covers three complementary dimensions: (i) VLM-as-a-Judge aesthetic and structural quality;
(ii) code efficiency, measuring generation time and token consumption;
and (iii) TeachQuiz, a novel end-to-end knowledge-transfer metric that enforces unlearning of the target concept in a VLM, and then measures how effectively the generated video restores it.
This multi-dimensional protocol directly probes educational efficacy and grounds a code-centric paradigm for video generation.
Our results reveal clear trends: pixel-based models struggle with fine details and coherence, while direct code-centric generation improves TeachQuiz by 30%. Our full Planner–Coder–Critic pipeline further delivers a stable 40% gain. In human studies on TeachQuiz scores, agentically generated videos even outperform professional human-made tutorials, underscoring the effectiveness of code-centric, agentic generation.

Our contributions are summarized as follows:

- •
A New Paradigm for Video Generation. We introduce a new code-centric paradigm for educational video generation, positioning executable code as the unifying medium for temporal sequencing and spatial organization.
- •
Effective Designs for Visual Animation Agent.
We highlight a modular agent design with three key components: (i) Planner expands an external database for reference, enabling parallel yet consistent storyboard; (ii) Coder ensures compilable code via automatic debugging and scope-guided repair; (iii) Critic refines spatial layout and clarity using visual anchor prompts.
- •
New Benchmark with Well-designed Evaluation. We present MMMC, the first benchmark for code-centric educational video generation with multi-dimensional evaluation of efficiency, aesthetics, and end-to-end knowledge transfer.

## 2 Related Work

### 2.1 Video Generation

Early text-to-video generation methods (i) extend diffusion models into the temporal domain via space–time UNets and latent 3D VAEs (Weng et al., 2024b; Ho et al., 2022b), achieving strong perceptual fidelity and longer durations (Yang et al., 2024; Li et al., 2024a; Xing et al., 2024). However, their reliance on pixel-space synthesis limits controllability, making precise layout and symbolic alignment—both critical for educational videos—difficult to realize. Autoregressive and progressive schemes (Li et al., 2024b; Gu et al., 2025; Wang et al., 2024; Xie et al., 2025) have improved long-form generation (Lu et al., 2024; Zhou et al., 2024), yet still struggle with board-like composition and stepwise exposition required in educational contexts (Li et al., 2024a; Liu et al., 2024).
(ii) Recent advances in multi-agent collaboration show that decomposing tasks, coordinating tool use, and enabling iterative self-improvement can substantially enhance reasoning and generation (Yuan et al., 2024; Hu et al., 2024; Xie et al., 2024; Shen et al., 2024).
While multi-agent frameworks have proven effective in domains such as web interaction, their application to video generation remains unexplored (Ku et al., 2025; Wu et al., 2024b).
(iii) Building on this paradigm, we propose a code–centric animation framework for educational video synthesis. By elevating executable code as the generative substrate, our approach achieves symbolic layout, temporally structured exposition, and deterministic reproducibility—capabilities unattainable with pixel-level diffusion.

### 2.2 Coding Agents

Recent advances in LLM-based tool use have shown that agents can autonomously call APIs, retrieve specifications, and verify outputs, enabling neuro-symbolic modularity and robust task decomposition (Yao et al., 2023; Wang et al., 2025). By integrating code execution and tool invocation, representative methods extend language models beyond text-only reasoning, supporting complex workflows and project-level code generation (Patil et al., 2024; Liu et al., 2025; Gupta et al., 2024). Such developments demonstrate the potential of LLM agents to coordinate external retrieval, maintain memory across parallel processes, and incorporate feedback loops for iterative refinement (Li, 2025; Xu et al., 2025; Zhang et al., 2024).
In parallel, research at the intersection of coding and visual reasoning shows that generating and executing programs can yield structured perception and controllable rendering (Pang et al., 2025; Zhu et al., 2025; Lin et al., 2025). Visual programming and visual-to-code approaches leverage program synthesis for compositional reasoning and spatial arrangement, with benchmarks translating images or text into executable code for charts, plots, and graphical interfaces (Wu et al., 2024a; Zhao et al., 2025; Wei et al., 2025; Yen et al., 2025). While these works bridge symbolic and visual domains, they largely focus on static figures or localized visual tasks (Xing et al., 2025; Wen et al., 2024; Ye et al., 2025; Jain et al., 2025).
We advance this line by integrating code generation and visual synthesis for dynamic educational video creation.

## 3 MMMC Benchmark

Figure: Figure 2: MMMC overview. (1) Left: distribution of 13 subject categories with exemplar learning topics; ring width encodes video duration. (2) Middle: learning topic word cloud highlighting core concepts. (3) Right: average learning topic length per category.
Refer to caption: x2.png

### 3.1 Task Formulation

The task of code-centric educational video generation maps a learning query to executable Manim (Manim Community Dev, 2025) code whose rendering yields a tutorial video. The challenge lies in multi-step reasoning, precise temporal sequencing, and spatial coherence, where even minor syntax errors can nullify execution. We adopt Manim for its fine-grained spatiotemporal control, symbolic expressivity, and demonstrated effectiveness in expert-produced instructional videos.

### 3.2 Data Curation and Statistics

We construct MMMC, a benchmark for code-driven educational video generation, under two criteria: (i) educational relevance—each learning topic is an established concept worth teaching; and (ii) executable grounding—each concept aligns with a high-quality Manim reference, ensuring practical realizability.
We source from the complete 3Blue1Brown (3B1B) YouTube corpus, known for its instructional impact and expert Manim craftsmanship. After filtering out non-instructional items (e.g., Q&A), we curate 117 long-form videos spanning 13 subject areas, including calculus, geometry, probability, and neural networks. To enrich supervision, we segment videos using author-provided timestamps into 339 semantically coherent sub-clips, yielding 456 units in total. An LLM then extracts concise learning topics (avg. 6.3 words) from titles, descriptions, and metadata, producing a clean mapping from videos to educationally grounded units (details in §[A.1.5](https://arxiv.org/html/2510.01174v1#A1.SS1.SSS5)).
On average, a full-length video lasts 1014 seconds ($\sim$16.9 minutes), while a segmented clip spans 201 seconds ($\sim$3.35 minutes), thus balancing long-horizon reasoning with fine-grained supervision. Figure [2](https://arxiv.org/html/2510.01174v1#S3.F2) visualizes topical diversity with a hierarchical donut plot: the inner ring denotes 13 categories, while the outer ring shows individual topics with arc width proportional to cumulative duration. This structure highlights both the breadth of coverage and the temporal richness of MMMC, establishing it as a challenging and representative benchmark for educational video generation.

### 3.3 Evaluation Metrics

Unlike conventional video generation, educational videos are valued less for visual fidelity than for how effectively they convey knowledge. This makes standard synthesis metrics inadequate. We therefore design a three-pronged evaluation across aesthetics, knowledge convey, and efficiency:

##### VLM-as-Judges.

Since human judgments of video quality are inherently subjective, we adopt a VLM-as-judges protocol ([$\mathcal{P}_{\rm aesth}$](https://arxiv.org/html/2510.01174v1#A1.SS2.SSS1)) to approximate user perception across five axes:
(i) Element Layout (EL) — clarity and spatial arrangement of visual components.
(ii) Attractiveness (AT) — overall engagement and ability to capture learners’ attention.
(iii) Logic Flow (LF) — coherence in temporal presentation of concepts.
(iv) Visual Consistency (VC) — stylistic stability across frames and sections.
(v) Accuracy & Depth (AD) — correctness and richness of the presented knowledge.
Each dimension is rated on a 100-point scale.

Figure: Figure 3: TeachQuiz: score gap between Learning-from-Video and Unlearning stages.
Refer to caption: x3.png

##### TeachQuiz.

The goal of educational video generation is not merely visual plausibility, but effective knowledge transfer. To evaluate this, we introduce TeachQuiz, a two-stage protocol grounded in a quiz set $\mathcal{Q}(\mathcal{K})={(q_{i},y_{i})}_{i=1}^{N}$ for a given concept $\mathcal{K}$, and $Y$ denotes ground-truth answers. We consider multiple teachers $\alpha,\beta$, each producing a video $\mathcal{V}_{\alpha},\mathcal{V}_{\beta}$. A student model $\phi$ is tasked with watching the video and answering questions:

$$ $S(\mathcal{V}_{\alpha})=\mathbf{1}[\phi\left(Q,\mathcal{V}_{\alpha}\right)=Y]$ (1) $$

If $S(\mathcal{V}_{\alpha})>S(\mathcal{V}_{\beta})$, then teacher $\alpha$ is the stronger instructor.

However, a key challenge is that many quiz items are already be learn by top-performing VLMs (i.e.,answer correctly without watching the video). Thus, absolute accuracy alone does not measure teaching quality. Instead, a good educational video should improve knowledge acquisition relative to a controlled baseline. We enforce this through two steps:
(i) Unlearning. Apply [$\mathcal{P}_{\rm unlearn}$](https://arxiv.org/html/2510.01174v1#A1.SS2.SSS2) to block prior access to $\mathcal{K}$, yielding a knowledge-removed baseline.
(ii) Learning-from-Video. Expose the model to $\mathcal{V}$ under [$\mathcal{P}_{\text{learn}}$](https://arxiv.org/html/2510.01174v1#A1.SS2.SSS3), testing whether the video itself enables recovery of the knowledge.
The final TeachQuiz score measures relative improvement:

$$ $\widetilde{S}(\mathcal{V}_{\alpha})=S(\mathcal{V}_{\alpha})-S(\mathcal{V}_{\alpha}|\text{unlearn})$ (2) $$

which isolates the contribution of the video by subtracting the unlearned baseline. Higher $\widetilde{S}$ indicates stronger knowledge transfer induced by the generated video.

##### Token Cost and Generation Time.

Beyond output quality, an equally important dimension is how economically a model can generate effective videos. We measure efficiency by average code generation time and token usage per video, reflecting scalability and feasibility in large-scale or interactive educational settings where latency and resource costs are critical.

## 4 Method: Code2Video

Overview.
As illustrated in Fig. [4](https://arxiv.org/html/2510.01174v1#S4.F4), given a topic query $\mathcal{Q}$, Code2Video output a video $\mathcal{V}$, which is consists of three stages:
(i) Planner structures topics into storyboards with reference assets, (ii) Coder translates each section into executable Manim code using parallel synthesis and an effective debugging, and (iii) Critic refines rendered videos through a novel visual prompt and VideoLLM feedback to ensure spatial coherence and educational clarity.

Figure: Figure 4: Illustration of Code2Video. Given a user inquiry, Code2Video aims to render an educational video via Manim code writing: (i) the Planner converts a learning topic into a storyboard and retrieves visual assets; (ii) the Coder performs parallel code synthesis with scope-guided refinement to ensure efficiency and temporal consistency; (iii) the Critic uses visual anchor prompts to iteratively adjust spatial layout and clarity, yielding reproducible, educationally structured videos.
Refer to caption: x4.png

### 4.1 Planner: query to storyboard

Generating coherent educational videos requires careful organization of temporal structure. We design the Planner to decompose a topic $\mathcal{Q}$ into two stages: outline generation for high-level ordering, and storyboard construction for stepwise realization. This preserves logical flow while capturing cross-section dependencies.

(i) Outline Generation. Given a topic $\mathcal{Q}$, the Planner produces an outline $\mathcal{O}={o_{1},\dots,o_{n}}$, where each $o_{i}$ contains a unique identifier, section title, content summary, and illustrative examples. Crucially, the Planner also considers the intended audience (e.g.,trigonometric functions for middle school, Fourier’s law for undergraduates), ensuring level-appropriate structure. Formally, $\mathcal{O}\leftarrow\hyperref@@ii[p_outline]{\mathcal{P}_{\rm outline}}(\mathcal{Q})$, where $\mathcal{O}=\{o_{1},\cdots,o_{n}\}$ and each $o_{i}$ encodes the section-level metadata and educational intent. By explicitly specifying audience and structure, the outline establishes the temporal skeleton for the subsequent video, guiding both pacing and sequencing.

(ii) Storyboard Construction. The second stage converts the outline $o$ into a detailed storyboard $s$. Each section in $s$ includes title, lecture lines, and corresponding animations, with $s_{i}\leftarrow\hyperref@@ii[p_storyboard]{\mathcal{P}_{\rm storyboard}}(o_{i})$. The storyboard specifies the temporal sequence of lecture lines and paired animations, bridging high-level planning with concrete visual content.

External Database.
To enhance factual accuracy and visual fidelity, the Planner integrates an external database $\mathcal{D}$. It includes (a) reference images aligned with the topic to anchor complex concepts and reduce hallucination, and (b) visual assets (e.g.,icons, logos) that are difficult to generate from scratch. These assets $\mathcal{A}$ are automatically identified via a prompt [$\mathcal{P}_{\rm asset}$](https://arxiv.org/html/2510.01174v1#A1.SS2.SSS6) analyzing the storyboard, $a_{i}\leftarrow\mathcal{P}_{\rm asset}(s_{i})$, and stored in a persistent cache $\mathcal{D}_{\rm asset}$.
Caching enables reuse across sections, preventing redundant generation and ensuring visual consistency.
Please refer to § [A.1.6](https://arxiv.org/html/2510.01174v1#A1.SS1.SSS6) for more details and examples about $\mathcal{D}$.

### 4.2 Coder: Storyboards to Executable Code

The Coder $\mathcal{G}$ translates each section of the storyboard $s$ and the cached assets $a$ into executable Manim code $C=\{c_{1},\dots,c_{n}\}$, where each $c_{i}$ corresponds to a storyboard $s_{i}$.

(i) Parallel Code Generation.
A central bottleneck in full-code synthesis is generation time, as end-to-end Manim code production for a single educational video—including generation, debugging, and rendering—can exceed two hours.
To address this bottleneck, we parallelize the pipeline by decoupling serial steps—code generation, debugging, and refinement—so that each section is synthesized and fixed independently.
Each section is conditioned on its storyboard and shared assets $\mathcal{A}$: $c_{i}=\hyperref@@ii[p_coder]{\mathcal{P}_{\rm coder}}(s_{i},\mathcal{A})$.
Notably, asset sharing across sections ensures temporal consistency while retaining the efficiency benefits of parallelization.

(ii) Effective Debugging.
Even strong LLMs rarely produce fully executable code in one pass. Naïve strategies that concatenate all code with the full error log are costly in both time and tokens. We propose ScopeRefine (SR), a hierarchical, scope-guided repair strategy, as illustrated in Fig.[4](https://arxiv.org/html/2510.01174v1#S4.F4) middle bottom:
(a) Line scope: isolate the error line plus immediate context, $\mathcal{S}_{1}={\rm line}\pm 1$, attempt up to $K_{1}$ local fixes.
(b) Block scope: if unresolved, expand to the lecture-line block $\mathcal{S}_{2}=\mathcal{B}_{i,j}$ with up to $K_{2}$ repair attempts.
(c) Global scope: as a last resort, regenerate the full section $c_{i}$ from $s_{i}$.
This progressive “Go-to style” repair minimizes token usage and latency while ensuring high reliability, effectively bridging parallel generation with robust debugging.

### 4.3 Critic: Effective Visual Refinement

Figure: Figure 5: Illustration of visual anchor prompt ($\mathcal{P}_{\rm vis}$).
Refer to caption: x5.png

Even after debugging ensures executability, the generated code may still yield unsatisfactory visual outcomes. LLMs and VLMs often fail to provide actionable feedback due to limited spatial awareness (Cheng et al., 2024; Zha et al., 2025). In practice, models can identify issues (e.g.,“the cat icon is misplaced”) but struggle to provide actionable corrections. They often fail to indicate the direction or distance needed to adjust the element, which makes text-only refinement inadequate.

(i) Visual Anchor Prompt ([$\mathcal{P}_{\rm vis}$](https://arxiv.org/html/2510.01174v1#A1.SS2.SSS7)). We introduce $\mathcal{P}_{\rm vis}$, a textual prompt that discretizes the 2D canvas into a $6\times 6$ grid of predefined anchor points. Each grid cell is mapped to fixed Manim coordinates, allowing LLM-specified locations to be directly converted into executable code. Placement follows two granularities, as illustrated in Figure [5](https://arxiv.org/html/2510.01174v1#S4.F5): (a) point-level, where small elements (e.g.,symbols, short labels) occupy a single anchor; and (b) region-level, where larger elements are assigned to a bounding box spanning multiple anchors.
This discretization transforms the placement task from a continuous positioning problem into a discrete anchoring problem, serving as a visual debugging “go-to”, which substantially reduces the difficulty for LLMs to produce valid layouts.

(ii) VideoLLM for Code Feedback.
To detect violations and refine placement, the Critic inspects the rendered video $\mathcal{V}_{i}$ alongside its section code $c_{i}$. During parallel code generation, we maintain an occupancy table that records each element’s assigned anchors (point or region), scaling factor, and corresponding code lines. This design serves two purposes: (a) it makes all assets indexable, allowing the Critic to quickly trace a visual issue back to its source code; and (b) it reveals available anchors, enabling conflict-free reallocation.
With this structured view, the Critic efficiently detects three common issues: overlapping elements within a cell, lecture lines occluded by animations, and large unused regions creating visual imbalance. These findings are incorporated into a refinement prompt [$\mathcal{P}_{\rm refine}$](https://arxiv.org/html/2510.01174v1#A1.SS2.SSS9), yielding optimized code:
$\tilde{c}_{i}=\mathcal{P}_{\rm refine}(c_{i},\mathcal{V}_{i})$
and final video $\widetilde{\mathcal{V}}=\text{Render}(\{\tilde{c}_{i}\}_{i=1}^{n})$. By combining anchor-based guidance, indexable, occupancy-aware adjustment, and multimodal feedback, the Critic overcomes the limitations of text-only debugging.

## 5 Experiment

### 5.1 Implementation Details

Baselines. We compare four types of approaches:
$\diamond$ Human-crafted, expert-designed Manim videos as an upper bound;
$\diamond$ Pixel-based Diffusion, text-to-video models: OpenSora-v2 (Peng et al., 2025), Wan2.2-T2V-A14B (Wan et al., 2025), and Veo3 (Google DeepMind, 2025);
$\diamond$ CodeLLM Generation, where an LLM directly generates Manim code from a learning topic;
$\diamond$ Agentic Generation (ours), a Planner–Coder–Critic pipeline.
We evaluate across diverse models: Claude Opus 4.1 (Anthropic, 2025), GPT-4o, GPT-o4 mini, GPT-4.1, GPT-5 (OpenAI, 2025), Gemini-2.5 Pro (Imran & Almusharraf, 2024), with Gemini-2.5 Pro serving as Critic for refinement.
Evaluation. Aesthetics are judged by Gemini-2.5 Pro (VLM-as-a-Judge), and quantify knowledge transfer with TeachQuiz.
Resources. Reference images are retrieved from Google Images, and visual assets from Iconfinder(^1^11https://www.iconfinder.com).
All prompts are documented in § [A.2](https://arxiv.org/html/2510.01174v1#A1.SS2).

### 5.2 Main Results

**Table 1: Results across Efficiency, Aesthetics, and TeachQuiz (Quiz). Efficiency: Time (avg minutes per topic) and Token (avg token consumption per topic). Aesthetics: Element Layout (EL), Attractiveness (AT), Logic Flow (LF), Visual Consistency (VC), Accuracy & Depth (AD).**
| Method | Efficiency ($\downarrow$) | Aesthetics ($\uparrow$) | Quiz ($\uparrow$) |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Time | Token (K) | EL | AT | LF | VC | AD | Avg |  |  |
| Human-made 3B1B | – | – | 98.3 | 100 | 100 | 100 | 100 | 99.7 | 97.1 |
| Pixel-based Diffusion |  |  |  |  |  |  |  |  |  |
| OpenSora-v2 | 27.6 | – | 0.0 | 5.0 | 0.0 | 0.0 | 13.3 | 3.7 | 0.0 |
| Wan2.2-T2V-A14B | 17.4 | – | 0.0 | 10.0 | 0.0 | 0.0 | 20.0 | 6.0 | 0.0 |
| Veo3 | 2.3 | – | 0.0 | 15.0 | 0.0 | 5.0 | 25.0 | 9.0 | 2.5 |
| Code LLM |  |  |  |  |  |  |  |  |  |
| GPT-5 | 1.8 | 1.1 | 27.0 | 28.0 | 28.0 | 54.5 | 26.0 | 32.7 | 36.5 |
| GPT-4.1 | 2.1 | 1.2 | 30.5 | 34.5 | 39.0 | 42.0 | 24.8 | 34.2 | 37.0 |
| Claude Opus 4.1 | 2.8 | 2.3 | 47.5 | 40.0 | 26.5 | 56.6 | 18.4 | 37.8 | 40.0 |
| Code2Video Agent (Ours) |  |  |  |  |  |  |  |  |  |
| Code2Video Gemini-2.5 Pro | 15.5 | 41.8 | 70.3 | 60.3 | 44.3 | 37.6 | 74.7 | 57.4 | 72.0 |
| Code2Video GPT-4o | 14.1 | 32.7 | 70.3 | 58.3 | 54.6 | 48.5 | 68.3 | 60.0 | 44.0 |
| Code2Video GPT-o4 mini | 16.8 | 49.2 | 77.0 | 52.8 | 73.0 | 57.2 | 79.0 | 67.8 | 48.5 |
| Code2Video GPT-5 | 8.8 | 19.3 | 75.5 | 60.5 | 81.8 | 63.6 | 79.7 | 72.2 +39.5 | 80.0 +43.5 |
| Code2Video GPT-4.1 | 15.4 | 30.8 | 82.8 | 65.6 | 95.0 | 68.0 | 83.7 | 79.0 +44.8 | 82.0 +45.0 |
| Code2Video Claude Opus 4.1 | 13.8 | 43.1 | 90.6 | 79.7 | 93.3 | 84.2 | 91.9 | 87.9 +50.1 | 86.0 +46.0 |

Table [1](https://arxiv.org/html/2510.01174v1#S5.T1) compares Code2Video with human-crafted videos, pixel-based models, and code LLM baselines, evaluated on Efficiency, Aesthetics (AES), and knowledge transfer (TeachQuiz).
Our analysis yields several insights:
(i) Pixel-based models underperform. They obtain the lowest scores on both AES and TeachQuiz, particularly struggling with LF due to weak control over text grounding, animation timing, and cross-frame coherence.
(ii) Direct code-centric generation delivers clear improvements. Rendering videos from LLM-produced Manim code outperforms pixel-based models, underscoring code as an effective medium for controllable and coherent educational video generation.
(iii) Our agentic framework delivers stable and consistent improvements. Across different backbone LLMs, Code2Video achieves significant performance boosts. For instance, with Claude Opus 4.1, AES improves by 50% and TeachQuiz by 46%. These gains arise from distinct components: visual anchor points drive improvements in element layout, while the Planner enhances LF and AD. However, limitations remain in AT and VC, pointing to opportunities for refinement.
(iv) Human-made videos remain strong. Although Code2Video narrows the gap, professional videos still lead in storytelling, nuanced sequencing, and explanatory depth. This highlights the next frontier: advancing agentic pipelines toward professional-quality long educational videos.

##### Qualitative Analyses.

Figure [6](https://arxiv.org/html/2510.01174v1#S5.F6) illustrates that our code-driven pipeline produces videos with clear text and formulas, stable layouts without occlusions, and stepwise alignment with lecture lines. In contrast, the pixel-based model (Veo3) often generates blurry or corrupted text, inconsistent styles, and drifting visuals, weakening semantic grounding. Overall, code-driven synthesis ensures better spatial stability and clearer knowledge presentation. Additional cases are provided in § [A.1.7](https://arxiv.org/html/2510.01174v1#A1.SS1.SSS7).

Figure: Figure 6: Qualitative comparison between Code2Video and Veo3. Our approach generates videos with coherent logic flow, consistent semantics, and interpretable layouts.
Refer to caption: x6.png

### 5.3 Ablation Studies

##### Effects by Individual Components.

Table [7](https://arxiv.org/html/2510.01174v1#S5.T7) highlights several observations.
First, TeachQuiz is more sensitive than Aesthetics, revealing knowledge-transfer gaps even when videos still look visually acceptable.
Second, the Planner is essential: removing it collapses both metrics ($\approx$ 41 points), underscoring that high-level lecture planning and temporal sequencing are the backbone of effective teaching videos.
Third, other modules provide complementary gains: the External Database improves conceptual grounding, Visual Anchors stabilize layouts, and the Critic ensures refinement—each modest alone, but jointly essential for robustness.
These results highlight that structured visual guidance and iterative refinement are crucial for producing visually clear videos that effectively convey knowledge.

##### Efficiency Components.

Table [7](https://arxiv.org/html/2510.01174v1#S5.T7) evaluates efficiency-oriented modules. Removing parallel execution greatly increases latency ($15.4\to 86.6$ minutes).
Without ScopeRefine (SR), we test two alternative debugging methods: (i) Retry, which regenerates the section upon any error; (ii) Full-code Debug, which feeds the entire code and error log to the LLM to regenerate the section. In both cases, error correction is costly, highlighting the importance of SR’s localized, scope-aware repair.
Removing both mechanisms produces prohibitive overheads. These results underscore that parallel synthesis and scope-aware repair are essential for scalable, code-centric video generation.

**Table 8: Human study on Aesthetics, TeachQuiz (Quiz), Completion Willingness (CW), and Average Ranking (AR). Results align with VLM-based trends but show sharper score contrast, lower tolerance for layout errors, and reduced engagement in longer-duration videos.**
| Method | Duration | Aesthetics ($\uparrow$) | Quiz ($\uparrow$) | CW ($\uparrow$) | AR ($\downarrow$) |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| EL | AT | LF | VC | AD | Avg |  |  |  |  |  |
| Human-made 3B1B | 16.9 min | $98.9$ | 97.2 | 91.3 | 98.0 | 97.0 | 96.5 | $78.8$ | $36.2$ | $1.2$ |
| Pixel-based Veo3 | 8.0 s | 12.6 | 4.4 | 1.1 | 24.4 | 1.1 | 8.5 | 8.0 | 46.8 | $5.0$ |
| Code LLM Claude Opus 4.1 | 0.9 min | 16.1 | 41.1 | 55.6 | 71.1 | 72.2 | 51.2 | 56.6 | 15.0 | 3.9 |
| Code2Video Gemini-2.5 Pro | 1.6 min | $26.7$ | 68.3 | 78.1 | 90.2 | 81.0 | 68.9 | $65.3$ | 47.4 | 3.1 |
| Code2Video Claude Opus 4.1 | 2.0 min | $60.2$ | 89.3 | 84.6 | 92.0 | 83.1 | 81.8 | $\mathbf{80.3}$ | $\mathbf{64.0}$ | $1.8$ |

##### Human Study Evaluation.

We conduct a five-group user study (6 middle school, 2 undergraduate volunteers per group), where each participant watches one video type and answers 5 quiz questions for 20 learning topics. We measure Completion Willingness (CW, proportion finishing the video before answering, max score is 100) and Average Ranking (AR, mean preference across video types, 1 is the best).
Table [8](https://arxiv.org/html/2510.01174v1#S5.T8) reveals four patterns: (i) Clearer separation. Human ratings follow the same overall trends as VLM-based scores but with stronger contrast: high-quality videos are rated in the upper range ($>90$), while low-quality videos cluster near the lower bound ($<10$). (ii) Sensitivity to layout errors. Participants give lower layout scores (EL) to videos from Code2Video, as humans are highly sensitive to even brief occlusions, whereas VideoLLMs often miss such frame-level issues. (iii) Attention span limits. Human attention is inherently limited: to perform well on the quiz, participants must follow the full flow of knowledge details in the video. This requires not only strong logical coherence and engaging presentation but also a reasonable duration that allows sustained high attention for effective knowledge absorption.
(iv) Strong consistency. Aesthetics and TeachQuiz scores are strongly correlated($r=0.971$, $p=0.0059$): visually appealing videos keep students engaged, leading to higher learning outcomes.
Overall, the human study underscores that both structural clarity and visual appeal are decisive levers for learning efficacy, complementing the automated metrics.
Future work requires agent designs that explicitly account for human attention and patience, ensuring videos maintain fine-grained details while minimizing perceptual fatigue.

## 6 Conclusion

We have introduced a novel, code-centric paradigm for educational video generation, establishing executable code as the unifying medium for both temporal sequencing and spatial organization. Building on this paradigm, our tri-agent architecture Code2Video enables controllable and interpretable generation with multimodal feedback.
To systematically evaluate this paradigm, we introduce MMMC, targeting efficiency, aesthetics, and knowledge transfer.
Together, our paradigm, architecture, and benchmark chart a clear path for future research on leveraging code as a medium for high-quality, structured, and interpretable educational content generation.
Future work includes broadening the video scope and developing more lightweight, scalable agent frameworks.

## References

- Anthropic (2025)
Anthropic.
Claude opus 4.1.
[https://www.anthropic.com/claude](https://www.anthropic.com/claude), 2025.
- Bao et al. (2009)
Lei Bao, Tianfan Cai, Kathy Koenig, Kai Fang, Jing Han, Jing Wang, Qing Liu, Lin Ding, Lili Cui, Ying Luo, et al.
Learning and scientific reasoning.
*Science*, 323(5914):586–587, 2009.
- Cheng et al. (2024)
An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu.
Spatialrgpt: Grounded spatial reasoning in vision-language models.
*Advances in Neural Information Processing Systems*, 37:135062–135093, 2024.
- Clark & Mayer (2023)
Ruth C Clark and Richard E Mayer.
*E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning*.
john Wiley & sons, 2023.
- Google DeepMind (2025)
Google DeepMind.
Veo 3: Generative video model.
[https://deepmind.google/technologies/veo/](https://deepmind.google/technologies/veo/), 2025.
- Gu et al. (2025)
Yuchao Gu, Weijia Mao, and Mike Zheng Shou.
Long-context autoregressive video modeling with next-frame prediction.
*arXiv preprint arXiv:2503.19325*, 2025.
- Gupta et al. (2024)
Tanmay Gupta, Luca Weihs, and Aniruddha Kembhavi.
Codenav: Beyond tool-use to using real-world codebases with llm agents.
*arXiv preprint arXiv:2406.12276*, 2024.
- Ho et al. (2022a)
Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al.
Imagen video: High definition video generation with diffusion models.
*arXiv preprint arXiv:2210.02303*, 2022a.
- Ho et al. (2022b)
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet.
Video diffusion models.
*Advances in neural information processing systems*, 35:8633–8646, 2022b.
- Hu et al. (2024)
Panwen Hu, Jin Jiang, Jianqi Chen, Mingfei Han, Shengcai Liao, Xiaojun Chang, and Xiaodan Liang.
Storyagent: Customized storytelling video generation via multi-agent collaboration.
*arXiv preprint arXiv:2411.04925*, 2024.
- Huang et al. (2024)
Kaiyi Huang, Yukun Huang, Xuefei Ning, Zinan Lin, Yu Wang, and Xihui Liu.
Genmac: compositional text-to-video generation with multi-agent collaboration.
*arXiv preprint arXiv:2412.04440*, 2024.
- Imran & Almusharraf (2024)
Muhammad Imran and Norah Almusharraf.
Google gemini as a next generation ai educational tool: a review of emerging educational technology.
*Smart Learning Environments*, 11(1):22, 2024.
- Jain et al. (2025)
Vyoman Jain, Shiva Golugula, Motamarri Sai Sathvik, et al.
Manimator: Transforming research papers into visual explanations.
*arXiv preprint arXiv:2507.14306*, 2025.
- Ku et al. (2025)
Max Ku, Thomas Chong, Jonathan Leung, Krish Shah, Alvin Yu, and Wenhu Chen.
Theoremexplainagent: Towards video-based multimodal explanations for llm theorem understanding.
*arXiv preprint arXiv:2502.19400*, 2025.
- Li et al. (2024a)
Chengxuan Li, Di Huang, Zeyu Lu, Yang Xiao, Qingqi Pei, and Lei Bai.
A survey on long video generation: Challenges, methods, and prospects.
*arXiv preprint arXiv:2403.16407*, 2024a.
- Li (2025)
Xinzhe Li.
A review of prominent paradigms for llm-based agents: Tool use, planning (including rag), and feedback learning.
In *Proceedings of the 31st International Conference on Computational Linguistics*, pp. 9760–9779, 2025.
- Li et al. (2024b)
Zongyi Li, Shujie Hu, Shujie Liu, Long Zhou, Jeongsoo Choi, Lingwei Meng, Xun Guo, Jinyu Li, Hefei Ling, and Furu Wei.
Arlon: Boosting diffusion transformers with autoregressive models for long video generation.
*arXiv preprint arXiv:2410.20502*, 2024b.
- Lin et al. (2025)
Kevin Qinghong Lin, Linjie Li, Difei Gao, Zhengyuan Yang, Shiwei Wu, Zechen Bai, Stan Weixian Lei, Lijuan Wang, and Mike Zheng Shou.
Showui: One vision-language-action model for gui visual agent.
In *Proceedings of the Computer Vision and Pattern Recognition Conference*, pp. 19498–19508, 2025.
- Liu et al. (2025)
Kaiyuan Liu, Youcheng Pan, Yang Xiang, Daojing He, Jing Li, Yexing Du, and Tianrun Gao.
Projecteval: A benchmark for programming agents automated evaluation on project-level code generation.
*arXiv preprint arXiv:2503.07010*, 2025.
- Liu et al. (2024)
Yixin Liu, Kai Zhang, Yuan Li, Zhiling Yan, Chujie Gao, Ruoxi Chen, Zhengqing Yuan, Yue Huang, Hanchi Sun, Jianfeng Gao, et al.
Sora: A review on background, technology, limitations, and opportunities of large vision models.
*arXiv preprint arXiv:2402.17177*, 2024.
- Lu et al. (2024)
Yu Lu, Yuanzhi Liang, Linchao Zhu, and Yi Yang.
Freelong: Training-free long video generation with spectralblend temporal attention.
*Advances in Neural Information Processing Systems*, 37:131434–131455, 2024.
- Manim Community Dev (2025)
Manim Community Dev.
Manim community v0.19.0.
[https://github.com/ManimCommunity/manim](https://github.com/ManimCommunity/manim), 2025.
- OpenAI (2025)
OpenAI.
Chatgpt-series.
[https://openai.com](https://openai.com), 2025.
- Pang et al. (2025)
Wei Pang, Kevin Qinghong Lin, Xiangru Jian, Xi He, and Philip Torr.
Paper2poster: Towards multimodal poster automation from scientific papers.
*arXiv preprint arXiv:2505.21497*, 2025.
- Patil et al. (2024)
Shishir G Patil, Tianjun Zhang, Xin Wang, and Joseph E Gonzalez.
Gorilla: Large language model connected with massive apis.
*Advances in Neural Information Processing Systems*, 37:126544–126565, 2024.
- Peng et al. (2025)
Xiangyu Peng, Zangwei Zheng, Chenhui Shen, Tom Young, Xinying Guo, Binluo Wang, Hang Xu, Hongxin Liu, Mingyan Jiang, Wenjun Li, et al.
Open-sora 2.0: Training a commercial-level video generation model in 200 k.
*arXiv preprint arXiv:2503.09642*, 2025.
- Shen et al. (2024)
Leixian Shen, Haotian Li, Yun Wang, and Huamin Qu.
From data to story: Towards automatic animated data video creation with llm-based multi-agent systems.
In *2024 IEEE VIS Workshop on Data Storytelling in an Era of Generative AI (GEN4DS)*, pp. 20–27. IEEE, 2024.
- Wan et al. (2025)
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al.
Wan: Open and advanced large-scale video generative models.
*arXiv preprint arXiv:2503.20314*, 2025.
- Wang et al. (2025)
Hongru Wang, Cheng Qian, Manling Li, Jiahao Qiu, Boyang Xue, Mengdi Wang, Heng Ji, and Kam-Fai Wong.
Toward a theory of agents as tool-use decision-makers.
*arXiv preprint arXiv:2506.00886*, 2025.
- Wang et al. (2024)
Yuqing Wang, Tianwei Xiong, Daquan Zhou, Zhijie Lin, Yang Zhao, Bingyi Kang, Jiashi Feng, and Xihui Liu.
Loong: Generating minute-level long videos with autoregressive language models.
*arXiv preprint arXiv:2410.02757*, 2024.
- Wei et al. (2025)
Jingxuan Wei, Cheng Tan, Qi Chen, Gaowei Wu, Siyuan Li, Zhangyang Gao, Linzhuang Sun, Bihui Yu, and Ruifeng Guo.
From words to structured visuals: A benchmark and framework for text-to-diagram generation and editing.
In *Proceedings of the Computer Vision and Pattern Recognition Conference*, pp. 13315–13325, 2025.
- Wen et al. (2024)
Chao Wen, Jacqueline Staub, and Adish Singla.
Program synthesis benchmark for visual programming in xlogoonline environment.
*arXiv preprint arXiv:2406.11334*, 2024.
- Weng et al. (2024a)
Wenming Weng, Ruoyu Feng, Yanhui Wang, Qi Dai, Chunyu Wang, Dacheng Yin, Zhiyuan Zhao, Kai Qiu, Jianmin Bao, Yuhui Yuan, et al.
Art-v: Auto-regressive text-to-video generation with diffusion models.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pp. 7395–7405, 2024a.
- Weng et al. (2024b)
Zejia Weng, Xitong Yang, Zhen Xing, Zuxuan Wu, and Yu-Gang Jiang.
Genrec: Unifying video generation and recognition with diffusion models.
*Advances in Neural Information Processing Systems*, 37:108851–108876, 2024b.
- Wu et al. (2024a)
Chengyue Wu, Yixiao Ge, Qiushan Guo, Jiahao Wang, Zhixuan Liang, Zeyu Lu, Ying Shan, and Ping Luo.
Plot2code: A comprehensive benchmark for evaluating multi-modal large language models in code generation from scientific plots.
*arXiv preprint arXiv:2405.07990*, 2024a.
- Wu et al. (2024b)
Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, et al.
Autogen: Enabling next-gen llm applications via multi-agent conversations.
In *First Conference on Language Modeling*, 2024b.
- Xie et al. (2025)
Desai Xie, Zhan Xu, Yicong Hong, Hao Tan, Difan Liu, Feng Liu, Arie Kaufman, and Yang Zhou.
Progressive autoregressive video diffusion models.
In *Proceedings of the Computer Vision and Pattern Recognition Conference*, pp. 6322–6332, 2025.
- Xie et al. (2024)
Zhifei Xie, Daniel Tang, Dingwei Tan, Jacques Klein, Tegawend F Bissyand, and Saad Ezzini.
Dreamfactory: Pioneering multi-scene long video generation with a multi-agent framework.
*arXiv preprint arXiv:2408.11788*, 2024.
- Xing et al. (2025)
Guangming Xing, Tawfiq Salem, and Gongbo Liang.
Chartcode: A flowchart-based tool for introductory programming courses.
In *Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 2*, pp. 1665–1666, 2025.
- Xing et al. (2024)
Jinbo Xing, Menghan Xia, Yuxin Liu, Yuechen Zhang, Yong Zhang, Yingqing He, Hanyuan Liu, Haoxin Chen, Xiaodong Cun, Xintao Wang, et al.
Make-your-video: Customized video generation using textual and structural guidance.
*IEEE Transactions on Visualization and Computer Graphics*, 31(2):1526–1541, 2024.
- Xu et al. (2025)
Weikai Xu, Chengrui Huang, Shen Gao, and Shuo Shang.
Llm-based agents for tool learning: A survey: W. xu et al.
*Data Science and Engineering*, pp. 1–31, 2025.
- Yang et al. (2024)
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al.
Cogvideox: Text-to-video diffusion models with an expert transformer.
*arXiv preprint arXiv:2408.06072*, 2024.
- Yao et al. (2023)
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao.
React: Synergizing reasoning and acting in language models.
In *International Conference on Learning Representations (ICLR)*, 2023.
- Ye et al. (2025)
Hui Ye, Chufeng Xiao, Jiaye Leng, Pengfei Xu, and Hongbo Fu.
Mographgpt: Creating interactive scenes using modular llm and graphical control.
*arXiv preprint arXiv:2502.04983*, 2025.
- Yen et al. (2025)
Ryan Yen, Jian Zhao, and Daniel Vogel.
Code shaping: Iterative code editing with free-form ai-interpreted sketching.
In *Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems*, pp. 1–17, 2025.
- Yuan et al. (2025)
Hangjie Yuan, Weihua Chen, Jun Cen, Hu Yu, Jingyun Liang, Shuning Chang, Zhihui Lin, Tao Feng, Pengwei Liu, Jiazheng Xing, et al.
Lumos-1: On autoregressive video generation from a unified model perspective.
*arXiv preprint arXiv:2507.08801*, 2025.
- Yuan et al. (2024)
Zhengqing Yuan, Yixin Liu, Yihan Cao, Weixiang Sun, Haolong Jia, Ruoxi Chen, Zhaoxu Li, Bin Lin, Li Yuan, Lifang He, et al.
Mora: Enabling generalist video generation via a multi-agent framework.
*arXiv preprint arXiv:2403.13248*, 2024.
- Zha et al. (2025)
Jirong Zha, Yuxuan Fan, Xiao Yang, Chen Gao, and Xinlei Chen.
How to enable llm with 3d capacity? a survey of spatial reasoning in llm.
*arXiv preprint arXiv:2504.05786*, 2025.
- Zhang et al. (2024)
Zhehao Zhang, Ryan Rossi, Tong Yu, Franck Dernoncourt, Ruiyi Zhang, Jiuxiang Gu, Sungchul Kim, Xiang Chen, Zichao Wang, and Nedim Lipka.
Vipact: Visual-perception enhancement via specialized vlm agent collaboration and tool-use.
*arXiv preprint arXiv:2410.16400*, 2024.
- Zhao et al. (2025)
Xuanle Zhao, Xianzhen Luo, Qi Shi, Chi Chen, Shuo Wang, Zhiyuan Liu, and Maosong Sun.
Chartcoder: Advancing multimodal large language model for chart-to-code generation.
*arXiv preprint arXiv:2501.06598*, 2025.
- Zhou et al. (2024)
Yupeng Zhou, Daquan Zhou, Ming-Ming Cheng, Jiashi Feng, and Qibin Hou.
Storydiffusion: Consistent self-attention for long-range image and video generation.
*Advances in Neural Information Processing Systems*, 37:110315–110340, 2024.
- Zhu et al. (2025)
Qipeng Zhu, Yanzhe Chen, Huasong Zhong, Yan Li, Jie Chen, Zhixin Zhang, Junping Zhang, and Zhenheng Yang.
Uniapo: Unified multimodal automated prompt optimization.
*arXiv preprint arXiv:2508.17890*, 2025.

## Appendix A Supplementary Material

### A.1 Additional Implementation Details and Experiments

#### A.1.1 Unlearning Details and TeachQuiz

To probe whether generated tutorial videos genuinely transfer knowledge, we integrate a selective unlearning–relearning protocol into the TeachQuiz evaluation.

Model choice. We adopt Gemini-2.5 Pro (Imran & Almusharraf, 2024), one of the current state-of-the-art models in video understanding. Its closed-source nature precludes parameter-level interventions for unlearning; thus, we rely on a prompt-based strategy, a standard approach for steering proprietary models.

Unlearning stage.
We design a parameter-free pipeline $\mathcal{P}_{\text{unlearn}}$ tailored for closed-source models. Given a target concept $\mathcal{K}$, we define a shadow knowledge set $\mathcal{B}(\mathcal{K})$ consisting of canonical definitions, formulas, aliases, and exemplars associated with $\mathcal{K}$. During inference, $\mathcal{P}_{\text{unlearn}}$ enforces:
(i) contextual masking, where $\mathcal{B}(\mathcal{K})$ is silently identified and treated as inaccessible;
(ii) uncertainty injection, where the model must output “INSUFFICIENT EVIDENCE” whenever the reasoning chain depends on elements of $\mathcal{B}(\mathcal{K})$;
(iii) progressive forgetting validation, where queries of increasing difficulty $\{q_{i}\}_{i=1}^{N}$ are used to test suppression not only at recall-level but also across multi-step reasoning.
Formally, the model’s answer distribution is constrained to

$$ $f(q_{i}\mid\mathcal{P}_{\text{unlearn}})\in\big\{y_{i},\texttt{NULL}\big\},$ (3) $$

where NULL indicates blocked inference. This layered design obstructs both direct recall and indirect reconstruction, ensuring that performance degradation reflects genuine unlearning rather than prompt compliance artifacts.

Relearning stage.
We then expose the model to an educational video $\mathcal{V}$ and apply a relearning prompt $\mathcal{P}_{\text{learn}}$, which restricts evidence scope to $\mathcal{V}$ while maintaining the block on $\mathcal{B}(\mathcal{K})$. The answering constraint becomes

$$ $f(q_{i}\mid\mathcal{P}_{\text{learn}},\mathcal{V})\in\big\{y_{i},\texttt{NULL}\big\},$ (4) $$

with justification required to reference only cues present in $\mathcal{V}$. This ensures that any gain after relearning is attributable solely to video-grounded evidence rather than residual prior knowledge.

Evaluation setup. For each learning topic, we construct 10 multiple-choice questions with four options (A–D), each containing exactly one correct answer. To better capture the expressive power of tutorial videos, these quizzes emphasize visually grounded reasoning. For instance, rather than simply asking “What is the definition of a complex number?”, a question may ask “When a point moves on the complex plane, what visual transformation corresponds to multiplication by $i$?”. Such queries demand alignment between knowledge and its visual instantiation.

Metric.
Given a concept $\mathcal{K}$, we construct $N$ multiple-choice questions $\{q_{i}\}_{i=1}^{N}$ with ground-truth answers $\{y_{i}\}_{i=1}^{N}$. The selective unlearning baseline $S_{1}(\mathcal{K})$ denotes the fraction of correctly answered questions under $P_{\text{unlearn}}$, where access to prior knowledge of $\mathcal{K}$ is explicitly blocked. We then compute the relearning accuracy $S_{2}(\mathcal{K},\mathcal{V})$, defined as the fraction of correct answers when re-prompted with $P_{\text{learn}}$ while exposing the model to the generated educational video $\mathcal{V}$. Formally,

The *TeachQuiz* score is then defined as:

$$ $\mathrm{TQ}(\mathcal{K},\mathcal{V})=S_{2}(\mathcal{K},\mathcal{V})-S_{1}(\mathcal{K}),$ $$

which captures the relative gain in accuracy attributable solely to $\mathcal{V}$. Intuitively, $S_{1}$ reflects how well the model resists using forbidden prior knowledge, while $S_{2}$ reflects how much can be recovered from the video. A higher $\mathrm{TQ}$ thus indicates stronger video-induced knowledge acquisition.

Ablation on evidence sources.
To ensure that the observed gains are indeed attributable to the generated videos, we conduct an ablation study, shown in Table [9](https://arxiv.org/html/2510.01174v1#A1.T9).

**Table 9: Ablation on unlearning. Accuracy reports correct concept judgments; $\Delta={\rm TQ}$ denotes the improvement in TeachQuiz confidence from the Unlearn setting to the Relearn setting. Text-only/Animation/Random evaluate TeachQuiz (TQ) under partial or mismatched supervision.**
| Method | Accuracy | TeachQuiz (TQ) |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| Unlearn | Relearn | $\Delta={\rm TQ}$ | Text-only | Animation | Random |  |
| Code2Video GPT-5 | 5.0 | 85.0 | $80.0$ | 27.2 | 72.1 | 2.0 |
| Code2Video GPT-4.1 | 5.0 | 87.0 | $82.0$ | 22.1 | 75.0 | 5.0 |
| Code2Video Claude Opus 4.1 | 5.0 | 91.0 | $86.0$ | 24.0 | 76.6 | 4.0 |

First, when providing only Text-only lecture lines (akin to PDF-style slides without animation), performance improves moderately compared to the unlearn baseline but falls short of full video-based relearning, highlighting that textual scaffolding alone is insufficient.

Second, with Animation-only inputs (animations without accompanying lecture text), accuracy also rises above unlearn but remains lower than the full condition, suggesting that temporal visual cues contribute substantially but require textual grounding for maximum effect.

Finally, in the Random-video setting, where the VLM is paired with an unrelated topic video, performance collapses to the unlearn level (or lower), confirming that improvements do not stem from superficial video exposure but rather from semantically aligned educational content.

Overall, these results provide evidence that the generated videos drive knowledge reacquisition: text and animation are complementary, and their synergy yields the strongest TeachQuiz gains.

#### A.1.2 Human Study: Middle School vs. Undergraduate Comparison

Table [10](https://arxiv.org/html/2510.01174v1#A1.T10) compares middle school and undergraduate participants on Aesthetics, TeachQuiz, and Completion Willingness (CW). As TeachQuiz measures knowledge acquisition, middle school students—closer to a true “unlearned” state—benefit more from effective videos, showing substantial TeachQuiz gains (e.g., Code2Video boosts middle school TeachQuiz to 88.1 versus 55.0 for undergraduates). Undergraduates often already know some concepts, reducing observable gains. Across both groups, Code2Video achieves high Aesthetics and CW, outperforming pixel-based models by large margins. Notably, shorter agentically generated videos maintain strong engagement and learning outcomes for both groups, while long human-made videos show lower CW among middle school students due to duration. Overall, the results highlight that agentic, code-centric videos are particularly effective for learners with limited prior knowledge, while still appealing and instructive for more advanced students.

**Table 10: Comparison of middle school and undergraduate participants on Aesthetics, TeachQuiz, and Completion Willingness (CW).**
| Method | Duration | Middle School | Undergraduate |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Aesthetics | TeachQuiz | CW | Aesthetics | TeachQuiz | CW |  |  |
| Human-made 3B1B | 16.9 min | 96.3 | 86.3 | 34.9 | 97.5 | 56.0 | 40.2 |
| Pixel-based Veo3 | 8.0 s | 10.7 | 6.0 | 55.6 | 2.0 | 14.0 | 20.5 |
| Code2Video Claude Opus 4.1 | 2.0 min | 81.7 | 88.1 | 76.0 | 82.2 | 55.0 | 58.2 |

#### A.1.3 Ablation on Visual Anchor Point Granularity

**Table 11: Ablation on anchor point granularity $\mathcal{P}_{\rm vis}$. Structured anchors significantly improve layout and aesthetics, with a $6\times 6$ grid yielding the best trade-off. Finer grids (e.g., $8\times 8$) cause clutter, while unconstrained (Self-directed) placement underperforms due to inconsistent spacing.**
| # Anchor Points | AES | AES Avg |  |  |
| --- | --- | --- | --- | --- |
| Element Layout (EL) | Attractivenss (AT) | ( EL + AT ) / 2 |  |  |
| w/o Visual Anchor Prompt | 45.2 | 54.7 | 50.0 | 69.2 |
| Center Point | 49.0 | 56.4 | 52.7 | 69.7 |
| $4\times 4$ | 76.1 | 63.0 | 69.6 | 76.9 |
| $6\times 6$ | 82.8 | 65.6 | 74.2 | 79.0 |
| $8\times 8$ | 77.2 | 60.6 | 68.9 | 76.0 |
| Self-directed | 48.8 | 57.3 | 53.1 | 70.3 |

We further study the impact of anchor point design in $\mathcal{P}_{\rm vis}$, which governs where visual elements are placed on the canvas. Table [11](https://arxiv.org/html/2510.01174v1#A1.T11) reports results under the AES framework, focusing on Element Layout (EL) and Attractiveness (AT), the two most placement-sensitive dimensions.

Setup. We compare six variants: (i) w/o $\mathcal{P}_{\rm vis}$, i.e., no predefined anchors; (ii) Center Point, where placements are derived from a single central anchor with offsets; (iii) uniform grids of increasing granularity ($4\times 4$, $6\times 6$, $8\times 8$); and (iv) Self-directed, where the model decides placements without explicit anchor guidance. All variants above are instantiated with ChatGPT-4.1.

Findings. Three observations emerge.
(1) Structured anchors substantially improve layout quality. Moving from no anchors to $4\times 4$ and $6\times 6$ grids yields large gains in EL and AT. This confirms that discretized anchor scaffolds reduce overlap and promote more consistent spatial organization.
(2) Moderation is key. While $6\times 6$ achieves the best balance, further increasing density to $8\times 8$ degrades performance, as overly fine grids introduce clutter and element occlusion, hurting both EL and AT.
(3) Unconstrained placement is suboptimal. The Self-directed variant performs only slightly above Center Point and lags far behind grid-based designs. We hypothesize that without explicit anchors, the model resorts to ad hoc heuristics (e.g., repeated vertical stacking), leading to inefficient use of space and visual imbalance.

Overall, the results highlight that *anchor granularity acts as a structural prior*: moderate discretization (here, $6\times 6$) provides sufficient flexibility while preventing crowding, thereby offering the best trade-off between precision and aesthetics.

#### A.1.4 Evaluation on TheoremExplainBench

Beyond our primary benchmark, we further test Code2Video on TheoremExplainBench (Ku et al., 2025), originally proposed to evaluate LLMs’ capacity for visualizing abstract mathematical concepts. Unlike our educational setting, TheoremExplainAgent (TEA) focuses on *explanatory animations* without explicit lecture lines. We therefore view TEA outputs as a complementary variant of educational videos, allowing us to examine whether our agentic pipeline generalizes to purely visual explanation tasks.
Table [12](https://arxiv.org/html/2510.01174v1#A1.T12) reports the results, and the comparison yields three key findings.

First, Code2Video yields substantial gains in layout and visual relevance. With GPT-4o, Element Layout improves from 0.59 (TEA) to 0.91, and Visual Relevance from 0.79 to 0.91, with consistent gains across backbones. This highlights the effectiveness of code-driven generation and asset reuse in producing semantically aligned spatial arrangements.

Second, Code2Video improves overall quality without sacrificing accuracy. Overall scores rise by 0.06–0.10 over TEA, while Accuracy & Depth remains comparable or better. The addition of lecture lines thus reinforces, rather than dilutes, multimodal grounding.

Third, model-specific trade-offs remain. For example, Gemini-2.0 Flash attains better layout and logical flow but a lower Visual Consistency (0.70 vs. 0.87). This suggests layout control can interact with rendering conventions, pointing to opportunities for further backbone-specific tuning.

These gains can be attributed to several design choices in Code2Video. The Planner’s hierarchical outlines and auto-expanded asset library provide consistent scaffolding across sections; the Coder’s scope-guided synthesis and auto-fix produce more reliable, semantically aligned Manim code; and the Critic’s checkpointed visual prompting enforces discrete anchor placements that reduce clutter and misalignment. Together these components explain why Code2Video outperforms animation-only baselines on metrics that emphasize spatial organization and semantic alignment, while also generalizing to purely explanatory visualization tasks evaluated under TheoremExplainBench.

**Table 12: Comparison on TheoremExplainBench (Ku et al., 2025). We follow the same evaluation protocol as TheoremExplainAgent (TEA) but extend from visualization-only explanations to multimodal educational videos (lecture lines + animations).**
| Method | Accuracy<br>and Depth | Visual<br>Relevance | Logical<br>Flow | Element<br>Layout | Visual<br>Consistency | Overall |
| --- | --- | --- | --- | --- | --- | --- |
| Human made Manim videos | 0.80 | 0.81 | 0.70 | 0.73 | 0.87 | 0.77 |
| TEA Gemini 2.0 Flash | 0.79 | 0.75 | 0.84 | 0.58 | 0.87 | 0.76 |
| TEA o3-mini | 0.76 | 0.76 | 0.89 | 0.61 | 0.88 | 0.77 |
| TEA GPT-4o | 0.79 | 0.79 | 0.89 | 0.59 | 0.87 | 0.78 |
| Code2Video Gemini 2.0 Flash | 0.81 | 0.80 | 0.92 | 0.88 | 0.70 | 0.82 |
| Code2Video o3-mini | 0.76 | 0.86 | 0.92 | 0.90 | 0.93 | 0.87 |
| Code2Video GPT-4o | 0.82 | 0.91 | 0.86 | 0.91 | 0.92 | 0.88 |

#### A.1.5 Details of MMMC

##### Data Collection.

Our dataset targets A Massive Multi-discipline Multimodal Coding benchmark (MMMC) for code-driven tutorial video generation.
Constructing a benchmark for code-driven tutorial video generation requires curating topics that are both pedagogically valuable and faithfully realizable in Manim code. Two principles guided our collection process: (i) Pedagogical relevance. Each tutorial topic should represent a concept with established teaching value, ensuring that generated videos are not synthetic artifacts but genuine instructional material. (ii) Executable grounding. Each tutorial topic must admit a high-quality reference video created by practitioners with substantial Manim expertise, guaranteeing that the underlying visualization is not only theoretically possible but also practically realizable. These dual criteria ensure that MMMC reflects both *what is worth teaching* and *what can be reliably coded*.

To satisfy these requirements, we turned to the 3Blue1Brown (3B1B) repository (^2^22https://www.3blue1brown.com/), which uniquely balances pedagogical impact and Manim craftsmanship. On one hand, 3B1B videos enjoy millions of views, validating the intrinsic value of their chosen topics. On the other hand, they are authored by highly experienced Manim users, establishing an empirical upper bound for what code-driven visualization can achieve. Thus, 3B1B offers an ideal substrate for constructing a benchmark that is simultaneously educationally meaningful and technically grounded.

Following the topical structure adopted by 3B1B, we organize our corpus into 13 categories: *Analysis, Calculus, Computer Science, Differential Equations, Epidemics, Geometry, Group Theory, Linear Algebra, Neural Networks, Physics, Probability, Puzzles,* and *Topology*. From YouTube (^3^33https://www.youtube.com/@3blue1brown/videos), we scraped the complete collection of 3B1B videos, then manually filtered out off-topic items such as Q&A sessions or non-instructional content, resulting in a curated set of 117 long-form videos.

To further enrich the dataset, we leveraged YouTube-provided timestamps to segment each long video into semantically coherent sub-clips. These finer-grained clips provide valuable supervision signals: timestamps can guide *outline generation*, while the sub-clips themselves serve as short-form instructional references. Finally, we distilled tutorial topics from both long videos and their sub-clips by prompting an LLM $\mathcal{P}_{\rm topic}$ with titles, descriptions, and metadata, yielding a clean mapping from videos to pedagogically grounded knowledge units.

##### Dataset Statistics.

Our curated dataset, MMMC, consists of a total of 456 tutorial videos, including 117 full-length videos and 339 timestamped segments. On average, a full-length video lasts 1014.41 seconds ($\sim$16.9 minutes), while a segmented clip spans 201.13 seconds ($\sim$3.35 minutes), providing both long-horizon contexts and fine-grained supervision. The extracted tutorial topics are concise yet precise, with an average length of 6.28 words per point.
Figure [2](https://arxiv.org/html/2510.01174v1#S3.F2) visualizes the distribution of the dataset with a hierarchical donut plot: the inner ring represents 13 high-level categories (e.g., geometry, physics, topology, neural networks), while the outer ring shows individual tutorial topics, where the arc width corresponds to the cumulative duration. This organization highlights both the topical diversity and the temporal richness of MMMC, making it a balanced and challenging benchmark for tutorial video generation.

Figure: Figure 7: Sample reference images and visual assets from the external database, illustrating the types of visual materials used to enhance aesthetics, maintain consistency across sections, and support the depiction of complex concepts.
Refer to caption: x7.png

#### A.1.6 External Database

Figure [7](https://arxiv.org/html/2510.01174v1#A1.F7) illustrates sample reference images and visual assets retrieved by our system. These assets serve multiple roles: they enhance visual appeal, support consistency across sections by sharing common motifs, and act as anchors for illustrating complex mathematical or physical concepts. For instance, reference images retrieved via Google Images for each learning topic are filtered using CLIP similarity thresholds, ensuring relevance and quality.

Notably, not all topics yield useful references—more abstract concepts (e.g., Topology) lack clear visual counterparts, limiting the benefit. Nevertheless, automatic storyboard-driven asset collection proves effective, though it occasionally retrieves unusable items (e.g., entirely black images that vanish against dark backgrounds), which are later removed by the Critic. Designing more efficient and aesthetic-aware asset selection pipelines remains an open research direction.

#### A.1.7 Qualitative Analyses

Figure: Figure 8: Showcase of generated tutorial videos across diverse topics. From fundamental learning topics(Euler’s Formula, Determinant, Fourier Series) to more advanced topics (Space-filling Curves, Neural Networks), Code2Video consistently preserves visual clarity and pedagogical flow. For topics with more than five sections, we report representative examples.
Refer to caption: x8.png

Figure: Figure 9: Comparison with diffusion-based text-to-video models. Videos generated by Veo3 and Wan2.2-T2V-A14B (<8s) under the topics The Determinant and Space-filling Curves. Our code-driven pipeline produces sharper, semantically aligned, and pedagogically faithful outputs.
Refer to caption: x9.png

We provide qualitative case studies in Figure [8](https://arxiv.org/html/2510.01174v1#A1.F8) and Figure [9](https://arxiv.org/html/2510.01174v1#A1.F9).
Figure [8](https://arxiv.org/html/2510.01174v1#A1.F8) showcases generated videos across diverse learning topics, including Euler’s Formula, The Determinant, Pure Fourier Series, Space-filling Curves, and Neural Network Learning and Backpropagation. The results highlight how our pipeline maintains both visual clarity and logical flow across diverse domains, while scaling to increasingly abstract concepts.
Figure [9](https://arxiv.org/html/2510.01174v1#A1.F9) further compares our approach with diffusion-based text-to-video models (Veo3 (Google DeepMind, 2025), Wan2.2-T2V-A14B (Wan et al., 2025)) under the topics The Determinant and Space-filling Curves. Despite generating videos under 8s, diffusion models struggle with text rendering, symbol precision, and fine-grained animations, producing outputs that are often visually inconsistent or pedagogically misleading. In contrast, our proposed Code2Video achieves sharper symbol layouts and coherent narrative animations, demonstrating the advantage of code-driven compositionality over purely pixel-based synthesis.

### A.2 Prompts of Code2Video

#### A.2.1 Prompt of VLM-as-judegs for aesthetics

#### A.2.2 Prompt of Unlearning

#### A.2.3 Prompt of Learning-from-Video

#### A.2.4 Prompt of Outline

#### A.2.5 Prompt of Storyboard

#### A.2.6 Prompt of Assets

#### A.2.7 Visual Anchor Prompt

The Visual Anchor Prompt $\mathcal{P}_{\rm vis}$ not only consists of a textual prompt fed into the LLM to guide object placement, but also encodes the predefined mapping between grid cells and corresponding coordinates, as illustrated in the code snippet below. Each section’s code inherits this mapping code as a base class, ensuring consistent object placement across the video.

#### A.2.8 Prompt of Coder

#### A.2.9 Prompt of VideoLLM Refinement

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: see weakness
soundness: 2
strengths: 1. The paper is well organized.
2. This paper introduces a code-centric paradigm for educational video generation, positioning executable code as the unifying medium for temporal sequencing and spatial organization. This seems to be able to generate longer videos than Veo3.
summary: This paper proposes a code-centric agent framework for generating educational videos via executable Python code.
weaknesses: 1. The paper's task seems to be quite similar to PPT generation. Could you discuss the differences between them?
2. Where did you collect your data from?
3. The generated video is still not long enough; real educational videos are usually over 30 minutes.
4. Large language models suffer from hallucinations, and the knowledge they provide may not necessarily be accurate, which is a fatal weakness in teaching.
5. In Table 1, the author's method is six times slower than Veo3. Generating a 40-minute educational video is estimated to take 5 days, which makes the efficiency unacceptable.

---

confidence: 3
contribution: 2
initial_rating: 4
presentation: 3
questions: 1. On Novelty: Given that generating video from Manim code is an existing human workflow, and LLM-based coding agents are an established research area, could the authors please clarify what precisely is the fundamental novelty of this work beyond applying existing agent techniques to the Manim library?

2. On TeachQuiz: How can the authors provide any empirical guarantee that the prompt-based $\mathcal{P}_{unlearn}$ protocol results in "genuine unlearning" rather than simple instruction-following? Without this validation, how can the TeachQuiz scores, and the 40% improvement claim, be considered reliable?

3. On Feasibility: How does the team reconcile the claim of "scalability" with generation times of up to 43 minutes and ~50K token costs for a single short video?

4. On the Critic: If the human study confirms that the VLM-Critic "misses" layout issues that humans find critical, does this not invalidate the Critic as a viable path to achieving "professional" quality? Why should we trust an automated critic that is demonstrably misaligned with human perception?
soundness: 3
strengths: 1. Problem Identification: The paper correctly identifies a significant limitation of current pixel-based video generation models, which struggle with the logical coherence, precise layout, and domain-specific knowledge required for professional educational videos.

2. Metric Conceptualization: The idea behind the TeachQuiz metric is commendable. Attempting to move beyond simple aesthetic scores to measure actual knowledge transfer is a crucial direction for this field. The "unlearn-relearn" concept  is a novel approach to addressing the confound of prior knowledge in VLMs.

3. Modular Architecture: The proposed tri-agent framework (Planner, Coder, Critic) is logical, modular, and clearly structured, with each component addressing a distinct part of the video creation pipeline.
summary: This paper presents Code2Video, a framework for generating educational videos by adopting a "code-centric paradigm" instead of traditional pixel-based synthesis. The authors argue that for structured educational content, generating executable Python code (specifically for the Manim library) provides greater controllability, interpretability, and scalability. The method employs a tri-agent pipeline: a Planner to create a storyboard, a Coder to write and debug Manim code using a "ScopeRefine" strategy, and a Critic to refine the spatial layout using a VLM and a "Visual Anchor" grid system. To evaluate this, the paper introduces the MMMC benchmark, sourced from 3Blue1Brown's YouTube channel , and a novel metric, TeachQuiz, which assesses knowledge transfer by using a prompt-based "unlearning" protocol on a VLM before showing it the generated video.
weaknesses: 1. The paper's central claim of introducing a "new paradigm" is a significant overstatement. Generating educational videos from Manim code is the standard, existing workflow for creators like 3Blue1Brown, the paper's own data source. The paper's actual contribution is the automation of this existing workflow using an LLM-based agentic pipeline. This is more of an application of the well-established "coding agent" paradigm to a specific library (Manim) rather than a fundamentally new paradigm for video generation.

2. TeachQuiz is Unverified: The paper's primary metric, TeachQuiz, lacks experimental robustness. It relies entirely on a prompt-based "unlearning" protocol ($\mathcal{P}_{unlearn}$) applied to a closed-source VLM (Gemini-2.5 Pro). There is no validation that this prompt induces "genuine unlearning" as claimed. It is equally, if not more, likely that the VLM is simply adhering to a complex negative constraint ("do not use knowledge X to answer") rather than truly "forgetting" the concept. The paper's headline claim of a 40% improvement rests entirely on this fragile, unverified, and opaque prompt-engineering artifact.

3. Benchmark is a Monoculture: The experiments are not thorough enough to support the paper's broad claims. The MMMC benchmark is a monoculture, sourced exclusively from 3Blue1Brown. This means the system is only evaluated on a single, niche aesthetic (Manim, dark background, math topics). There is zero evidence of generalization to any other style of educational video (e.g., whiteboard animations, slide-based lectures, software tutorials) or other domains (e.g., history, literature, biology).

4. Baselines are Non-Competitive: The comparison to pixel-based models (e.g., Veo3) is a strawman. It is widely known that these models cannot render coherent text or perform complex, long-form logical reasoning. A more thorough and meaningful comparison would have been against other state-of-the-art agentic coding frameworks, tasked with the same goal of writing Manim code.

5. Prohibitive Cost: The method's feasibility is highly questionable. The paper's own data (Table 1) shows generation times of 15-43 minutes and token consumption up to 49.2K tokens per video. This directly contradicts the claims of being "scalable" and efficient. These costs make the system impractical for "interactive educational settings"

6. VLM-Critic is Flawed: The feasibility of the Critic agent is fundamentally undermined by the paper's own human study. The study explicitly states that humans are "highly sensitive" to layout errors that the VLM-as-a-Judge (and thus the VLM-Critic) "often miss". This means the Critic is optimizing for a flawed, non-human-aligned objective. This misalignment makes it incapable of achieving the "professional" quality it claims to target.

---

confidence: 3
contribution: 2
initial_rating: 4
presentation: 3
questions: See the weaknesses section.
soundness: 3
strengths: - The paper is well organized, with clear and visually rich figures, making it easy to read and follow.
- The paper explores an understudied area in education, which adds meaningful community value to the research.
summary: The paper proposes a pipeline for generating educational videos using the Manim software, built on top of VLMs. It also introduces MMMC, a benchmark designed to assess the effectiveness of the proposed pipeline.
weaknesses: - The paper lacks novelty. It mainly presents a code-generation pipeline without introducing any fundamentally new algorithmic ideas. Most components of the work are combinations of existing (V)LLM techniques, rather than newly developed methods. As such, the contribution resembles more of a workshop or community project rather than a work suitable for a serious machine learning conference.
- The expression is inappropriate. The paper emphasizes that a code-centric approach can produce better videos than generative model–based methods, which is somewhat inappropriate: as is well known, generative models and virtual engines represent fundamentally different paradigms of visual content creation. The paper should not treat generative models as a central motivation and baseline (e.g., Fig. 1 and Tab. 1), nor should it refer to the proposed approach as 'video generation' (e.g., line 99), a term that conventionally refers to generative model-based synthesis.

---

confidence: 4
contribution: 3
initial_rating: 2
presentation: 1
questions: 1. Has the owner of the 3Blue1Brown been contacted and informed that their videos have been used to construct a dataset for research?
2. Is there a reference or principled explanation for how the knowledge unlearning step can be used to guarantee that the model is in fact not using specific concepts?
3. Has a correlation between the proposed metrics utilising the VLM as judge and the results of the human study been conducted?
4. Are the low scores for the human videos in Table 4 purely because of the length?
5. generation of manim code can exceed two hours, is this using the method as described without parallelising it?
soundness: 2
strengths: * The idea of using code-centric generation for educational videos is a nice contribution and is clearly validated when comparing to the pixel based methods in the results both quantitatively and qualitatively.
* The reasoning behind why new evaluation metrics are needed and that the previous metrics which only capture visual fidelity is strong.
* The results show the benefits of the proposed method, whilst there is a tradeoff between the proposed method and the time and token requirements from an efficiency point of view, the results more than make up for it when compared to the Code LLM (i.e. baseline) methods with large improvements when the Code2VideoAgent is used.
summary: The paper focuses on generation of educational videos via using a code-centric paradigm. The paper claims that by using pixel based methods, the results are not robust, explainable, and require a large amount of pre-training data. Whereas using a code-centric paradigm solves these issues and allow for longer and better videos. Accordingly, a new benchmark dataset is curated which consists of videos from the 3Blue1Brown YouTube channel with a corresponding novel evaluation scheme which utilises the knowledge that educational videos should not just be visually correct but represent good teaching resources is also proposed. A new method, named Code2Video, is presented which incorporates a planner, coder, and critic components to render an educational video via writing of code. Results show that pixel based methods indeed fail at the task and the proposed method outperforms naive LLM usage. A human study further verifies the claims.
weaknesses: ## Weaknesses
* Has the owner of the 3Blue1Brown YouTube channel been contacted and informed that their videos have been used to construct a dataset for research? Have they licensed their use? Upon looking on their website (https://www.3blue1brown.com/faq#licensing) the use for research/AI is not specified as one of the use cases that do not require explicit permission from the owner.

* Sections 3 and 4 are missing a lot of details, especially within the main paper, and are very hard to read. Some of the information is given in the appendix, but this is not linked to and is disparate enough that it is hard to understand what the details of the paper are. Examples include:
  * $\mathcal{P}$ has not been defined within the main paper, there are links to the appendix but these do not specify what these are and how they are utilised in the main paper.
  * The reasoning behind how aesthetics, knowledge convey, and efficiency map to the evaluation metrics given is not clear and not well argued.
  * Figure 3 has not been referenced within the text of the paper.
  * $S$ and $\tilde{S}$ are not defined within the method section.
  * On line 293 it says that generation of manim code can exceed two hours, is this using the method as described without parallelising it?
  * Figure 4 is not clear from the accompanying text, for example, the effective debugging section does not seem to match well with figure 4 middle bottom, which showcases the error and then two scopes for the code with arrows.
* As far as I can tell, the method to cause the model to forget certain knowledge is just apply a prompt and does not require any fine-tuning which does not seem to be a principled way of evaluating a model in this respect. This is expanded on very briefly in the appendix, but I am not convinced that the VLMs will not be biased in some way/unlearning the data. 
* Numbers within Table 1 and Table 4 are not bolded consistently throughout the table (i.e. only Avg and Quiz are bolded in Table 1), it's nto clear why.
* The correlation values given on line 471, it sounds as if these are checking the correlation of human participants' scores on the aesthetics and the quiz scores whereas a separate correlation value could be calculated between the human study scores and the VLM as judge scores within Table 1, was this calculated?
* The discussion regarding human attention and patience is interesting, and could be expanded to include the results presented in table 4. For example, the human made videos score worse for both quiz and completion willingness (the latter by quite a large margin) is this purely because of the length of the human-made video or is there something else going on here?
* There are a lot of spelling and grammar mistakes within the paper and it could benefit from another round of proof-reading and corrections. A incomplete list is given below.


## Corrections
* Line 174: We source from the complete 3Blue1Brown (3B1B) YouTube corpus. This implies that there is a completed corpus and not videos from a channel? If the former there's no reference.
* Figure 2 (1) is quite hard to read at the scale
* consider knowledge conveyance instead of knowledge convey
* Line 241: "are already be learn" -> "have already been learned" or similar
* line 242: missing space between i.e., and answer.
* Line 259: "which is consists of three stages" -> "which consists of three states"


Decision: Reject