ORIGAMISPACE: Benchmarking Multimodal LLMs in Multi-Step Spatial Reasoning with Mathematical Constraints
Authors: Rui Xu
Dakuan Lu
Zicheng Zhao
Xiaoyu Tan
Xintao Wang
Siyu Yuan
Jiangjie Chen
Xu Yinghui
Venue: NeurIPS 2025 spotlight ()
Keywords: spatial reasoning
multimodal large language model
benchmark
Primary area: evaluation
TL;DR: We introduce ORIGAMISPACE, a new origami dataset and benchmark to evaluate MLLMs in Multi-Step Spatial Reasoning with Mathematical Constraints.

Abstract: Spatial reasoning is a key capability in the field of artificial intelligence, especially crucial in areas such as robotics, computer vision, and natural language understanding. However, evaluating the ability of multimodal large language models (MLLMs) in complex spatial reasoning still faces challenges, particularly in scenarios requiring multi-step reasoning and precise mathematical constraints. This paper introduces ORIGAMISPACE, a new dataset and benchmark designed to evaluate the multi-step spatial reasoning ability and the capacity to handle mathematical constraints of MLLMs through origami tasks. The dataset contains 350 data instances, each comprising a strictly formatted crease pattern (CP diagram), the Compiled Flat Pattern, the complete Folding Process, and the final Folded Shape Image. We propose four evaluation tasks: Pattern Prediction, Multi-step Spatial Reasoning, Spatial Relationship Prediction, and End-to-End CP Code Generation. For the CP code generation task, we design an interactive environment and explore the possibility of using reinforcement learning methods to train MLLMs. Through experiments on existing MLLMs, we initially reveal the strengths and weaknesses of these models in handling complex spatial reasoning tasks.

ORIGAMISPACE: Benchmarking Multimodal LLMs in Multi-Step Spatial Reasoning with Mathematical Constraints
Rui 123
Dakuan Lu Siyu Yuan
Zicheng Zhao
Xiaoyu Tan 3x Yinghui Xu
Xintao Wang'
Jiangjie Chen'
FFudan University
SII 'INF Technology
rxu24@m.c fudaa wud.anedu..c.m edu. cn
xuyinghui @fudan edu. cn
Abstract
Spatial reasoning is key capability in the field of artificial intelligence, especially crucial in areas such as robotics, computer vision, and natural language under- standing. However, evaluating the ability of multimodal large language models (MLLMs) complex spatial reasoning still faces challenges, particularly in scenar- ios requiring multi-step reasoning and precise mathematical constraints. This paper introduces ORIGAMISPACE, new dataset and benchmark designed to evaluate the multi-step spatial reasoning ability and the capacity to handle mathematical con- straints MLLMs through origami tasks. The dataset contains 350 data instances, each comprising strictly formatted crease pattern (CP diagram), the Compiled Flat Pattern, the complete Folding Process, and the final Folded Shape Image. We propose four evaluation tasks: Pattern Prediction, Multi-step Spatial Reasoning, Spatial Relationship Prediction, and End-to-Fine CP Code Generation. For the CP code generation task, we design interactive environment and explore the possibility of using reinforcement learning methods to train MILMs. Through experiments on existing MLLMs, initially reveal the strengths and weaknesses of these models in handling complex spatial reasoning tasks.
1 Introduction
Spatial reasoning is a core component of artificial intelligence (1.2), with wide applications in robotics 33. autonomous driving (4) and geographic information systems (5). Although multimodal large language models (MLLMMs) demonstrate outstanding performance in various vision and lan- guage tasks 66 [] they face challenges imagining spatial transformations and grasping spatial relationships in image and text spaces. Evaluating their spatial reasoning ability has become an important task.
Multi-step reasoning and constraints are critical yet underexplored areas in spatial intelligence. Current spatial reasoning benchmarks typically focus understanding static images simple scenes (8) Some studies are dedicated to comparing and reasoning about spatial relationships between image pairs, but lack attention continuous spatial transformations [9] 10]. Some studies propose multi-step spatial reasoning but do not involve interaction with the environment and lack constraints found in real-world tasks (17.. These limitations indicate current need for
benchmark to more comprehensively evaluate the capabilities MILMMs complex spatial reasoning scenarios.
Origami art offers an ideal platform for evaluating complex spatial reasoning abilities 112] Origami involves a sequence of ordered folding operations, where each step depends on the result of the
"Corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

previous one, embodying the essence of multi-step step reasoning. Furthermore, the origami process governed by explicit geometric constraints, such folds must occur along straight lines, and the paper cannot be torn or separated; all origami operations are defined by strict mathematical constraints (Kawasaki's Theorem, Huzita-Haton axioms, etc.) 13 14]. The transformation from two-dimensional crease pattern (CP diagram) through multiple folding steps to three-dimensional folded shape image requires strong spatial imagination and reasoning abilities.
CP Diagram
Compiled Flat Pattern
Folded Shapo Image
CP Code
[1 11
Telaes 1A0077
11:
Folding Process
22
1,
Figure 1: An example data instance from ORIGAMISPACE includes: CP Diagram, Compiled Flat Pattern, Folded Shape Image, and Folding Process, where the CP Diagram can be represented the form of Code.
To bridge the gap of existing benchmarks, this paper introduces the ORIGAMISPAN dataset and benchmark. This dataset contains 350 meticulously collected origami data instances, including a CP diagram, its corresponding compiled flattened pattern, illustrations of the complete folding process, and the final folded shape. The diversity and complexity of the data cover various origami types. We improve the existing origami compiler, enabling it to output detailed flattened diagrams that include crease locations and stacking relationships, support interactive simulation with MLLMs, and provide more comprehensive error feedback. Based on this dataset, we design four challenging evaluation tasks: pattern prediction, spatial relationship prediction, multi-step spatial reasoning, and end-to-end CP code generation, which comprise 1,500 multiple-choice questions and 120 code generation questions. For the code generation task, we meticulously design a comprehensive evaluation strategy to measure the quality of the generated CP code across multiple dimensions.
The core advantages of ORIGAMISPACE lie its authenticity (derived from real origami designs), multi-step reasoning characteristics (reflecting the inherent process of origami), and rigorous mathe- matical constraints (precisely verifiable through origami theorems). We evaluate the performance of various MLLMs on ORIGAMISPACE. and introduce environmental learning and reinforcement learning methods for the code generation task, which opens up new perspectives and effective avenues for assessing and enhancing the spatial reasoning abilities of MI..Ms..
The main contributions of this paper include:
We introduce ORIGAMISPACE, a dataset containing 350 high-quality origami data instances, and optimize the existing origami compiler, enabling it to provide more comprehensive feedback.
We design four challenging tasks centered around spatial reasoning, including 1,500 multiple- choice questions and 120 CP code generation questions, which is the first benchmark to evaluate the multi-step spatial reasoning ability of MLLMs under mathematical constraints.
We conduct comprehensive evaluation of existing MILMs and develop complete interactive environment for the end-to-end CP code generation task, and explore environmental learning and reinforcement learning methods through this environment.

---

2 Related Work
2.1 Spatial Reasoning Benchmarks
Evaluating the spatial reasoning abilities of MLLMs is crucial for advancing their application in real-world scenarios, but existing benchmarks have certain limitations 88 [5]. CLEVR [9] and Visual Genome [16] focus static scene understanding
single-step reasoning and often operate in synthetic environments, making it challenging to reflect the complexities of the real world. NLVR2 [7] concentrates comparative reasoning through image pairs but struggles to measure a model's s ability to understand and execute tasks involving multiple spatial state transitions. StepGame (18) and LEGO-Puzzie to pure text models or do not sufficiently emphasize precise geometric and physical constraints. Furthermore, interaction with the environment and understanding physical manipulation are also weak points in current evaluation methods. Many benchmarks primarily rely on static inputs and less frequently involve tasks that require models to predict guide sequence of physical actions. To address these challenges, we propose ORIGAMISPACE. By introducing origami, structured and complex multi-step physical task, ORIGAMISPACE directly targets the shortcomings of existing benchmarks. It leverages origami's 's inherent precise geometric constraints and sequence of operations, aiming to comprehensively and deeply evaluate the capabilities of MLLMs in complex, dynamic spatial reasoning.
[11] explore multi-step processes, but they either limited
2.2 Computational Origami
Computational origami is emerging field within computer science that focuses studying algorithms for solving origami-related problems [19 20]. This field covers two main aspects: origami
design 200 and origami foldability [21] 22 Origami design involves the development of algorithms
to generate origami crease patterns with specific shapes or functionalities [23] Origami foldability,
on the other hand, investigates how to determine whether given crease pattern can be folded into
particular shape, especially flat-oodability (24). Our work does not focus on designing new origami models; instead, we leverage the characteristics of origami to evaluate the spatial reasoning abilities of MILMs. Drawing upon knowledge from computational origami regarding crease patterns, folding processes, and mathematical principles, have optimized an existing origami compilation system and evaluation functions for crease patterns, thus establishing a benchmark designed to test the capabilities of MLLMs in multi-step spatial manipulation and constraint satisfaction.
ORIGAMISPACE Dataset
3.1 Data Collection
We collect 350 sets of origami data, These data originate from various online resources, including origami tutorial website forum and origami books As depicted Figure each complete data entry comprises the following four parts:
CP Diagram The CP diagram is standardized format, representable by code, that displays all the creases of an origami model. It is typically a two-dimensional planar drawing where different line styles indicate different types of folds (e.g., mountain fold, valley fold). Subject constraints, CP diagram uniquely determines a folded shape image. The format of the CP diagrams in our dataset adheres strict requirements, ensuring their correct parsing by our compiler.
Compiled Flat Pattern Through the compiler, the final folded state of the CP diagram under all constraints be computed, and the output compiled flat pattern represent the wwoddimensional state the origami model after complete folding.
httpe://origmmi-cattaassss com/
https://mitani.co www.kaaaaa.......aaaaa....a..... jp.jpaa.ipp.
httpe://www.giaddos. wwtpeaaaa.....maoo cwm/ coan/rrigami database. php
https://oriwiii.comm
3

---

Folded Shape Image Different from the strictly compiled flat pattern, the folded shape image provides a direct, intuitive visualization of the final origami shape. is typically a photograph or 3D rendering.
The folding process refers to the multi-step sequence of transforming the original paper into the final shape. This folding process is gathered from various origami tutorials and cannot be represented in a standardized format, existing only natural images.
Folding Process
We manually check and verify all data to ensure that 1) all CP diagrams can be compiled into the compiled flat pattern and correspond to the folded shape image; 2) the names of all origami data correspond to the folded shape image, with no potential for confusion (such as indistinguishable birds); and 3) all folding processes are feasible. In addition to this part of the data, we also collect 471 groups of data without intermediate folding processes for the subsequent training of the model.
3.2 Compiler
The current origami compiler computes the final state achievable by a CP diagram under all math- ematical constraints, thereby compiling the compiled flat pattern. We have optimized this process: 1) During compilation, we mark each crease, allowing us to locate the position of every crease in the compiled image. 2) We further compute the paper stacking order information, clarifying the top-bottom relationship of different paper regions in the compiled flat pattern. 3) We construct an interface for direct interaction between MLLMs and the compiler, enabling the model to call this system more conveniently to complete origami simulations. 4) We improve the error feedback system of the compiler. Specifically, it returns four types of errors:
CP Code Syntax Error (CSE) Validates the existence, format, and validity of inter-references of core data structures in the CP code (such as vertex coordinates vertices. coordds, edge-vertex verticeee ces, and face-vertex relationships faces_verticn vertices). It also checks crease types (e.g., 'B', 'M', 'V', 'F', 'U') are predefined characters, and verifies if Euler's formula for planar graphs is satisfied: V - E + 2, where V, E, and represent the number of vertices, edges, and faces, respectively.
relationships
Geometrically Impossible Fold (GIF) Refers to cases where the CP code geometrically violates fundamental origami principles, making the fold physically unrealizable. For example, violating local flat fat-foldabylity conditions at a vertex (such as Mackawa's 's theorem |M - V| = 2 or Kawasaki's
theorem Σ a; 2m), specified crease angle combinations would require the paper to be stretched or torn.
(PSI) Occurs when logically incompatible situations are found while deducing the relative positions and layering order of different paper sections after folding. This may manifest as cycle in the calculated paper layering relationships (e.g.. layer A is above layer B, layer B is above layer C, and layer C is, in turn, above layer A), or in a 2D unfolded representation, different paper regions are assigned to overlapping positions that would cause physical penetration.
Paper Self- wew.-iesteetionPeeetreee
Ambiguous Folding State (AFS) This error occurs when a given CP code, due to its inherent under-con conttraied nature (e.g., allowing multiple valid mountain-vall valley assignments for creases, or lacking critical information such crease types or angles), can be compliantly folded into multiple different stable geometric structures, or prevents the compiler from uniquely determining the layering order when processing complex overlapping paper regions.
3.3 Dataset Statistics
In ORIGAMISPACE, the distribution of different types of origami is relatively even. To ensure data diversity, we choose origami models covering different levels of complexity and types of folds, such as animals, plants, geometric shapes, etc. The average number of folding steps for origami models 8.2, but the variation between different models varies greatly, ranging from minimum of 3 steps to maximum of 25 steps. Appendix A presents more detailed data analysis, including the themes and names of all origami data and the proportion of different folding steps.

---

Task1: Pattern Prediction
Task2: Mutti-step Spatial Reasoning
Task3: Spatial Relationship Prediction
Q: Which the following origami sequences cornect?
Q: What pattern will diagram form after being folded?
A: 3-4-1-66---2
After folding, sailboat model, which two original points have the largest distarice change?
B: 8-31-66-4-5-2
C: 3-1-4-6-5-2
A: snail
C: teacup
D: 3-1-4-5-66-2
A: and B pt
B: pT and
B: somural hot pencil
C: And Ct
At and D
Task4: End-to-End CP Code Generation +>
In-context Learning
(vertices_ coliidsssccco:::::.oo 0J, LL. 0J, [1. 11, [0,
edges, vertices: [0, 11 [1
01,
Environment Learning
+
edges aiges_accigmmen: [8, 8, B, M]
vertice::: [0, 2). [0, 3|F
Reinforce Learning
Figure 2: Data examples the four tasks. The first three tasks are in a multiple-choic format, and the fourth task is code generation task.
4 Task
Based on ORIGAMISPACE, we propose four tasks evaluate the spatial reasoning capabilities of MLLMs comprehensively.
4.1 Pattern Prediction
This task evaluates the model's ability understand the folding process from the CP diagram and imagine the final 3D shape. For this task, the input is the CP diagram, and MLLMs are required to predict the resulting folded shape image based it. To enable better quantitative evaluation, structure this task For the incorrect options, three origami enthusiasts design three options for each diagram, adhering to criteria that require them to be easily distinguishable from the correct option; not be variations of the same concept (e.g., if the correct option is cat, incorrect options are lions, leopards, etc.); and be close to potential folded states based on the CP diagram (e.g., removing a few key creases makes a boat's CP diagram similar a hat). We create 350 questions for this task. See Appendix B. for the specific annotation rules.
multiple-choice question. The correct option is the name of the target shape.
4.2 Multi-step Spatial Reasoning
This task evaluates the model's ability to understand the dynamic origami process and the logical relationships between steps. The input for the task is set of images that collectively show several key steps of complete origami process. However, the order of these images randomly shuffled. MILMS need to infer the correct chronological order in which these steps occur, based their un- derstanding of the geometric state changes in the images. To better quantify the model's performance, we structure this task as multiple-choic question. The correct option is the sequence of steps that represents the unique correct folding process (for example, "1-2-3-4"). For the incorrect options, we generate multiple logically incorrect sequences of steps (for example, "1-2-4-3". "4-1-2-3", etc.). These incorrect sequences may contain partially correct local orders but contain errors in the overall flow, in order to test the model's grasp of the complete, coherent process. We design 250 such questions, and the average number of steps per question is 7.5.
4.3 Spatial Relationship Prediction
This task evaluates the model's ability to predict spatial relationships and geometric properties after the folding process is complete. For this task, the input is the CP diagram. The model required to predict specific spatial relationships between parts of the origami model after it is fully folded. The task comprises three types of multiple-choice questions designed to test this ability: 1) Spatial Pose Localization: Determining the specific 3D position of point from the original paper in the final
5

---

model, including its pose within a reference frame (e.g., on table. facing upwards). 2) Layering Relationship Analysis: Determining the paper stacking order after folding, requiring analysis of covering relationships during the folding process and identifying how many paper layers form specific region (e.g., the thickest region). 3) Geometric Change Analysis: Predicting how specific geometric features (such as angles, distances, areas, etc.) change from the flat CP diagram to the final folded state. For example, predicting the relative angle or spatial distance between two original line segments after folding. The correct answers for all three question types are obtained using our optimized compiler. Incorrect options are then manually designed. We design 900 multiple-choice questions (300 for each type) for this task. See Appendix B.2 for specific annotation rules.
4.4 End-to-End CP Code Generation
This task requires the MILM to generate corresponding CP code based on compiled flat layout and an image of the folded shape. Ideally, this CP code should compile into a folded pattern identical to the target shape. To comprehensively evaluate the quality of the generated results, we have designed a multidimensional evaluation framework.
Compilation Attempt and Evaluation The CP code generated by the model will first be attempted to be compiled using our origami compiler (see Section 3.2 for details). If the compilation fails, the model will return is syntactically valid, geometrically foldable, and free of self-intersections, and produces a definite folded state, the system will compare the compilation result with the reference result across the following four dimensions:
more error types. If the compilation succeeds, meaning the CP code
1) Topological Structure Similarity (TSS) This dimension evaluates similarity at the graph theory level by comparing the compiled output. compares the number of vertices of successfully compiled
-0.5 mint ggen Vref)) ) edge connectivity (e.g., similarity of degree distribution.
(Vgen Veef!
patterns (score Sg = E
number of connected components), face relationships (c.g., number of faces, distribution of face sizes), and the distribution similarity of crease types ("M", "V", "B", etc.).
2) Geometric Similarity (GS) This dimension focuses on the spatial characteristics of the compiled model. It evaluates point position similarity by calculating the bidirectional Hausdorff distance HH between the normalized 3D point sets of the generated and reference compiled models (score Sp = -k-dh where k is a sensitivity coefficient, e.g., 5). It assesses angular similarity by comparing the distribution of dihedral angles at the creases, and evaluates size and proportion similarity by comparing the aspect ratios of the overall bounding boxes of the models.
3) Constraint Satisfaction (CS) This dimension evaluates whether the successfully compiled CP code, beyond the basic foldability ensured by the compiler, further adheres to the physical and mathematical constraints of origami. This includes comparing the presence and matching degree of critical constraint types (Taco-Taco, satisfaction of fundamental theorems of local flat-foldability, such as Maekawa's 's theorem (the difference between the number of mountain creases M and valley creases V around a vertex is M - Vl = 2) and Kawasaki's theorem (the sum of the angles a; of creases around a vertex is Σa; 2T 0).
Tortill,,,,,,, transitivity constraints) and checking for
4) Final Folded State (FFS) This dimension directly compares the final 3D model shape compiled from the generated CP with the reference compiled 3D model. It primarily evaluates overall shape similarity by calculating the Hausdorff distance of the point sets, and where possible (if the model provides layering information), compares the layering relationships between facets, including paper stacking order information that may be obtained during the compilation process.
Total Score: The final total score Stotal is a weighted average of the scores Saim from each evaluation dimension: Stotel = Edin Wdli 8dim- By default, each of the four dimensions accounts for 25% of the weight (Waim = 0.25), and Σ Waim = 1. This score ranges from 0 to 1 (Sentat E [0, 1). reflecting the overall quality of the generated CP code. For more details on the evaluation process, please refer to Appendix

---

Pattern Prediction
Multi-step Spatial Reasoning
Spatial Relationship Prediction
Model
Spatial Pose
Layering Relationship
Geometric
Localization
Change
Open-source
Models
MiniCPM-o 2.6
26.9910.42
30.11 1.554
10.98
30.50
23.75
1lva-1.5-7bb
27.23
19.0511..0
29.06.12.71
30.94
25.51
deepseek-v12
28.40
30.01 #0.06
26.71
29.05=0.23
24.30
NVILA-15B
28.33=1.09
32.51
30.60+1.22
31.00=1.53
26.48=0.76
Videol.LaMA3-7B
29.01
30.86:10.14
29. 29.0610.02
28.74=1.04
27. 27.80
Qwen2.5-VL-7B
28.40
31.51
28.43
28.05=0.01
28.83
Qwen2.5-VL-32B
30.19
36.82#0048
33.51
32.59=0.18
10.11
Qwen2.5-VL-72B
36.29
39.10#0.88
35.68+1.69
38.04=0.70
31.89
IntemVL22.5-8B
36.76=0075
38.55#0.08
38.01+0011
37.66=0.18
30.48
Close-source Models
Claude-3.--Sonnet
3148
45.07=0.64
39.55+0063
40.19=0.11
39.73=0.10
GPT-4o
40.61
#0.48
48.2441.73
50,42=0.59
46.72.1..0
Gemini2.5-Flas
35.01
48.92:10.18 53.45
40.15
39.91 11.09
40.01 11.63
Gemini2.5
42.6
49.06:10.07
47.68=0.07
47.10
Human Performance
human(common) 51.18 98.45
88.52
55.12
50.55
50.15
human(expert)
100.00
96.44
92.10
85.38
Table I: Accuracy (%) of various MILMSs different spatial reasoning tasks. Bold underlined values indicate best performance across open-source models and all models, respectively.
Experiments
5.1 Models
evaluate multiple representative MLLMs. For open-source models, we evaluate MiniCPM-o 2.6 23J.NVILA-155 (26). 1.5-7
(27). VideoLLaMA3 (28). Oween2.5VVL-17B/32BB777
22)
deepseek-viD (30] IntermVL2.5-78B [31]. For proprictary models, we evaluate Claude-3.5.
Sonnet (32). gpt-4o (33). ( [34]. For all these models, we adopt the original
model and official instruction formats.
5.2 Baseline
We recruit two categories of people to complete the first three tasks. The first category consists of five laypersons recruited via crowdsoureing platform, and the second category comprises three experts with extensive origami experience. Specific details of the human evaluation are provided in Appendix For the CP code generation task, we adopt the following settings:
B.3
In-context learning In this setting, we provide the model with detailed task instructions and a set of CP code examples. The instructions will introduce the meaning represented by each part of the CP code and all the constraints that must be followed. MILMMs need to generate the complete CP code in one go based on these instructions and examples.
Environmental learning this setting, MLLMs no longer attempt to generate the complete CP code in one go, but instead engage in iterative interaction with the compiler. Specifically, the MLLM will first perform planning, then generate CP code. The compiler will return its compilation result, and the model then performs inference based the returned compilation result, subsequently choosing to add or delete creases, iterating in this manner. We set the upper limit of interaction rounds 10.
Reinforcement learning Through constructed compilation environment, we explore reinforcement learning approach. We utilize the 471 sets of data mentioned in Section 3.1 for training, sampling data in the same process as in environmental learning. The reward mechanism is set as follows: (1)

---

Compilation
Model
TSS
GS
CS
FFS
Total
CSE
GIF
PSI
AFS
CPR
In-context learning
Qwen2.5-VL-32B
78.32
42.09
38.11
34.52
10.18
35.04
28.51
30.93
26.26
30.19
Qwen2.5-VL-72B
80.85
44.51
40.93
37.01
14.55
37.11
31.65
33.08
28.90
32.68
Intern VL2.5-78B
74.12
42.17
36.01
33.91
12.84
35.04
30.67
31.95
29.04
31.68
Claude3.5-Sonnet
87.36
57.94
50.12
41.62
20.73
44.02
38.99
39.21
36.87
39.77
GPT-4o
95.03
61.13
48.28
45.25
28.56
50.06
40.57
41.58
39.06
42.82
Gemini2.5-Flash
83.60
50.24
46.89
40.77
18.93
42.61
40.86
37.13
36.91
39.38
Gemini2.5-pro
94.47
60.06
53.41
46.01
30.03
51.51
43.71
42.68
37.28
42.80
Environmental learning
Qwen2.5-VL-32B
88.87
70.13
65.02
63.92
39.08
45.61
33.51
38.28
29.51
36.72
Qwen2.5-VL-72
90.58
77.90
68.91
66.92
43.81
48.72
38.03
40.86
34.74
40.58
Intern VL2.5-788
85.23
68.92
65.81
60.01
38.34
48.91
35.37
36.72
35.02
39.00
Claude3.5-Sonnet
98.05
85.89
81.74
78.52
52.90
55.82
50.21
52.15
43.66
50.46
GPT-4o
100
92.55
88.25
82.56
66.92
58.29
51.52
54.81
46.07
52.67
Gemini2.5-Flash
92.51
82.81
80.03
79.93
51.94
53.01
48.86
50.95
44.91
49.43
Gemini2.5-pru
100
90.74
92.57
84.27
65.89
60.18
52.23
56.99
45.24
53.66
Reinforcement learning
Qwen2.5-VIL-328
91.03
72.84
70.42
68.92
45.17
49.55
39.91
42.78
38.07
42.57
Table 2: Results of different MLLMs and methods on the code generation task. Compilation indicates whether compilation is successful, including the probability of no occurrence of the four compilation errors 3.2). as well as the overall compilation pass rate (CPR). When compilation successful, the similarity in four dimensions(4.44 and the total score calculated. This score is scaled to [0,100] for ease of presentation.
Intermediate reward: After modifying the code, if compilation is successful, reward is given based on the quality progress of the current partial CP code (Sportiat - Spartial_ prev- where Spartial is a quickly evaluated partial quality score), plus small basic compilation success reward. If compilation fails, fixed negative penalty is given. (2) Step penalty: A small negative reward received for each action taken encourage efficiency. (3) Final reward: After the interaction ends, the result of the evaluation function defined in Section 4.4. serves as the main reward. We adopt TRICO (35) for training on gwen2.55v1-32B, which is a PPO-based (36). more efficient MLLMs multi-turn reinforcement learning algorithm. Specific training settings and parameters can be found in Appendix
5.3 Main Results
Tasks to primarily focus on spatial analysis and prediction. The results shown in Table are the average of three runs for different MLLMs, from which we observe that: 1) For MLLMs. ORIGAMISPACE is challenging task; the performance of poor-performing models is close to random guessing (25%), and even for best-performing models, there a significant gap compared to human performance, especially in multi-step spatial reasoning. 2) Despite the different task types, the relative performance ranking various models largely remains consistent, with Gemini 2.5 2.5-pro and GPT-4o demonstrating the best spatial reasoning ability. 3) Human experts perform well all tasks, demonstrating the task's upper bound. 4) MLLMs perform worst on the Spatial Relationship Prediction task, especially the sub-tasks involving Geometric Change, indicating significant difficulty for models in understanding fine-grained, internal spatial structures.
Table 2 presents the results of different methods and models on Task 4. We observe the following: 1) Impact of learning settings: The results clearly indicate the significant impact learning settings on performance. In-context learning shows relatively limited performance. Environmental learning brings significant performance improvements, demonstrating that through iterative interaction with the compiler, planning, and trial-and-errou based feedback, models overcome the limitations

---

of one-shot generation. Reinforcement learning shows potential, as the trained Qwen2.5-VL-32B surpassed the performance of 72B model. 2) There are significant performance differences among different models, with top-tier closed-source models exhibiting the best spatial reasoning capabilities.
qWen2.5-vI332B(L)
Swwn22.-VVL322
GPT4o
Gemini-2.5pro
claude-3.5
55
60
50
50
Total Score
65
40
Compitation
30
20
10
10
16
10
13
16
Round
Round
Figure 3: The impact of interaction rounds the compilation pass rate and total score of different models.
5.4 Impact of Mathematical Constraints
Mathematical constraints present a primary challenge in generating valid CP codes for the ORIGAMIS- PACE task. Table indicates that failingg satisfy constraints is the main bottleneck for compilation failures; even when provided with detailed instructions, models struggle to strictly adhere to these complex rules, leading to persistently high compilation failure rates. Interactive processes with the environment enhance models' ability to follow constraints, demonstrating that models can learn and internalize rules from feedback. Compared environmental learning, reinforcement learning also shows improvement in constraint satisfaction, proving the effectiveness of specific reward mecha- nisms. However, even with interactive learning, precisely satisfying all mathematical constraints remains significant challenge for top-tier models (such as GPT-4o and Gemini 2.5-pro, whose constraint satisfaction score only 56.99% under environmental learning settings). This reveals MILMs' deficiencies in deep multi-step geometric and layering reasoning and highlights the value of the fine mne-graieed feedback and constraint satisfaction evaluation introduced in this study.
5.5 Impact of Interaction Rounds in Environmental Learning
Figure 3 illustrates the impact of interaction rounds on model performance across different dimensions under the environmental learning setting. We observe that as the number of interaction rounds increases, model performance improves in various aspects, particularly the compilation pass rate. However, performance tends to saturate after 8-10 rounds, indicating that interaction primarily helps overcome initial learning obstacles but struggles to break through the model's inherent bottlenecks. Weaker models, limited by their understanding capabilities, reach their upper limit in fewer rounds. The reinforcement learning-trained Qwen2.5- also follows similar trend, but due to policy optimization, it may reach its performance ceiling in fewer rounds.
Conclusion
In this paper, we introduce ORIGAMISPACE, a novel benchmark specifically designed to address the underexplored areas of multi-step spatial reasoning and constraint adherence in Multimodal Large Language Models (MLLMs). Leveraging the inherent complexities of origami, ORIGAMISPACE provides 350 meticulously curated data instances and an enhanced compilation program to facilitate in-depth evaluation. The benchmark features four challenging tasks, including pattern prediction, spatial relationship prediction, multi-step spatial reasoning, and end-to-end code generation, making the first to assess MILMs multi-step spatial reasoning under rigorous mathematical constraints. Our comprehensive evaluation of existing MILMs and exploration of reinforcement learning methods for code generation highlight the utility of ORIGAMISPACE in not only assessing current capabilities but also in paving new ways to enhance the spatial intelligence of MLLMs.

---

Acknowledgments and Disclosure of Funding
We wish express our sincere gratitude several individuals and communities for their invaluable contributions to this research. We are deeply grateful for the enthusiastic support and engagement from the global origami community, particularly the many knowledgeable members active on Discord servers dedicated to origami and Baidu Tieba's 's origami forums. Their discussions and shared resources were highly beneficial. Special thanks are extended to Niels Stoermer, founder of the Origami Database, for his generosity in providing foundational data and for his instrumental assistance in connecting us with wider network of origami experts. We are also immensely thankful to QingLiang and Jason Ku for offering their expert advice and constructive feedback the development and refinement of our origami compiler; their insights were crucial.
References
[1] B. Chen al. "SSpatiallm: Endowing vision-language models with spatial reasoning ca- pabilities". In: Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition. 2024.
[2] I. Stogiannidis, S. McDonagh, and S. A. Tsaftaris. "Mind the Gap: Benchmarking Spatial Reasoning in Vision-Languaan Models". In: arXiv preprint arXiv:2503. 19707 (2025).
[3] C. H. Song al. "RoboSpatial: Teaching Spatial Understanding 2D and 3D Vision-Language Models for Robotics". In: arXiv preprint 1rXiv:2411. 16537 (2024).
[4] S. Yang et al. "Lidar-Ilm: Exploring the potential of large language models for 3d lidar understanding". In: Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. 2025.
[5] M. Aliman, S. Sumarmi, and S. Marni. "Developing Spatial Thinking through the Earthcomm Learning Model: Exploring the Role of Earth Science in the Community". In: Journal of Social Studies Education Research (2024).
[6] D. Zhang et al. "Mm-llms: Recent advances in multimodal large language models". In: arXiv
preprint arXiv::2011 1360 (2024).
[7] D. Caffagni et al. "The revolution of multimodal large language models: survey". In: arXiv preprint arXiv:2402. /245/ (2024).
[8] Z. Li, X. Wu, H. Du, H. Nghiem, and G. Shi. "Benchmark evaluations, applications, and challenges of large vision language models: A survey". In: arXiv preprint a/Xiv:2501.02189 (2025).
[9] J. Johnson et al. CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning. 2016
[10] Z. Li et al. Super-CLEVR: Virtual Benchmark to Diagnose Domain Robustness in Visual
Reasoning. 2023.
[11] K. Tang et al. arXiv preprint arXiv:2503. 19990 (2025).
How Good Are MLLMs Multi-Step Spatial Reasoning?" In:
[12] D. Misseroni et al. "Origami engineering". In: Nature Reviews Methods Primers (2024).
[13] D. Carberry, S. Williams, G. Wang, E. Sevick, and D. J. Evans. "The Kawasaki identity and the fluctuation theorem". In: Journal of Chemical Physics (2004).
[14] A. Kasem, F. Ghourabi, and T. Ida. "Origami axioms and circle extension". In: Proceedings of the 2011 ACM symposium on applied computing. 2011.
[15] W. Zhou, Y. Zeng, S. Diao, and X. Zhang. "Vlue: A multi-task benchmark for evaluating vision-language models". In: arXiv preprint arXiv:2205. 15237 (2022).
[16] R. Krishna al. Visual Genome: Connecting Language and Vision Using Crowdsourced Dense
Image Annotations. 2016.
[17] A. Suhr and Y. Artzi. NLVR2 Visual Bias Analysis. 2019.
[18] Z. Shi, Q. Zhang, and A. Lipani. StepGame: A New Benchmark for Robust Multi-Hop Spatial
Reasoning in Texts. 2022.
[19] E. D. Demaine and M. L.. Demaine. "Recent results in computational origami". In: Origami3:
Third International Meeting of Origami Science, Mathematics and Education. 2002.
10

---

[20] R. J. Lang. "A computational algorithm for origami design". In: Proceedings of the twelfth annual symposium on Computational geometry. 1996.
[21] T. Tachi and T. C. Hull. "Self-fokdability of rigid origami". In: Journal of Mechanisms and Robotics (2017).
[22] S. Li, H. Fang, S. Sadeghi, P. Bhovad, and K.-W. Wang. "Architected origami materials: how folding creates sophisticated mechanical properties". In: Advanced materials (2019).
[23] J. L. Silverberg et al. "Using origami design principles to fold reprogrammable mechanical metamaterials". " In: science (2014).
[24] T. Tachi. "Generalization of rigid-foldable quadrilaterl--mesh origami". In: Journal of the International Association for Shell and Spatial Structures (2009).
[25] Y. Yao al. "MiniCPM-V: A GPT-4V Level MLLM on Your Phone". In: arXiv preprint
arXiv:2408. 01800 (2024).
[26] Z. Liu et al. NVILA: Efficient Frontier Visual Language Models. 2024.
[27] B. Li al. LLa Vision: Easy Visual Task Transfer. 2024.
[28] Z. Cheng et al. "VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Under- standing in Video-LLMs" In: arXiv preprint arX14:2406.07776 (2024).
[29] S. Bai, K. Chen, and X. Liu. Qwen2.5-VL Technical Report. 2025.
[30] Z. Wu et al. DeepSeek-VL2: Mixture-of-Expert. Vision-Language Models for Advanced Multi- modal Understanding. 2024.
[31] 7. Chen et al. Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling. 2025.
[32] Anthropic. Introducing Claude 3.5 Sonnet. http:://sst...attrrpc...oooaaaaa claude-
3-5-sonnet: Accessed: Maat-Montt-Daay 2024.
[33] OpenAL. Introducing GPT-4o, Intp:::tp//// /totpp://openi.... com/nnde////ee/eeeoggttt..
Accessed:
Yearr-Mothh-Dyy.. 2024.
[34] Google and G. DeepMind. Gemini 1.5 Pro and Flash: A step forward in maltimodal un- derstanding. hhtpps /ttpg//llg..ommo gwwgle/tooooooooooo technology//c/// ai harp/ooggyy//ogglleeeeeeoooooo/e google gemini- update io
2024/ Accessed: [Ycar-Month-Dayy 2024.
[35]
Wang* al. VAGEN: Training VLM Agents with Multi-Tun Turn Reinforcement Learning. 2025.
[36] J. Schulman, F. Wolski, P. Dhariwal, Radford, and O. Klimov. "Proximal policy optimization
algorithms". In: arXiv preprint arXiv: 1707063345 (2017).

---

A Dataset
The ORIGAMISPACE comprises a total of 350 data entries, covering various types of origami. We have categorized these based on the required number of folding steps into Easy (3-9 steps), Medium (10-19 steps), and Hard (20-30 steps). Tables 3 and respectiivll display all the data for these three difficulty levels, including the origami design name, its category, and the number of folding steps required. All our data are public data or authorized by the original websites and data sources, with no potential infringement risks.
Table 3: Easy Origami Models (3-9 Steps)
Easy Origami Models (3-9 Steps)
1. Triangle- Geometry 33
2. Square base fold Geometry -4
3. Mountain - Nature 4
4. Letter 1- Alphabet- 3
5. Number 1 Numbers- 3
6. Minus Sign Symbols-
7. Bird Beak- Animals-
8. Letter L- Alphaber-4
9. Number 7- Numbers- 44
10. Cross MareXX - -Symbols- 44
11. Plus Sign- - Symbols
12. Diamond shape- Geometry 5
Water Drop Nature- 5
Trapezoid Geometry- -
15. Lucky Star strip prep- Decorations 5
Comma symbol Symbols 5
17. Single French 5 5
18. Letter C- A---ahhett- 5
19. Fish Fin Animals
20. Check Mark SSymols 5
21. Nail Tools
Simple Envelope- Items 6
23. Small Flag. Decorations 6
24. Simple Leaf Plants- 6
25. Arrow Symhols - 6
26. Band aid Items 5
27. Screw Tools 6
28. Letter F Alphabbe
24. Number 2- Numbers
Number 4 Numbers
31. Bread
6
32. Plate Irems- 6
33. Simple Cloud Nature 6
34. Simple Ring band Accessories -6
35. Ice Lolly/Popsicle Stick Food -
36. Simple Conster Iiems
37. Pointed Bookmark- ltems- 7
Paper Dart- Durt-Toys-1
39. Simple Heart Decorations-?
40. Fox- AAiimlls7
41. leeberg- Nature 7
42. Bone -ire--Itmss-- 7
Simple Pen/Penciil outline liems
Simple Screwdriver outline Tools
45. Letter E- 7
46. Number 3- -Numbers-7?
47. 1. Letter Z.- 7-Alphhhet- 7
48. Simple Fish Animals 7
49. Simple Mushroom Plants
50. Simple Radish/Carrot top - Plants 7
House Outline- Items- 7
Simple Temt/Teepee liems 7
53. Ice Cream Cone hase Food 7
54, Pointy Hat Clothing-
53. Crescent Moon - Nature- 7
36. Candle - simple Items- -
57. Simple Ghost Decorations 7
58. Boomerang - simple V TToss
Choese Slice Food 7
Simple Shovel outline- Tools
61. Simple Cup liems 88
62. Simple Boat Iiems 88
63. Simple Dog Face - Animals- 8
64. Simple Cat Face - Animals 8
65. Simple Pig Face Animals-
66. Traditional Cup/Masu Box base Traditionl 88
Apple Core -hope-Fod-- 8
68. Simple Necktie -Cothing 8
69. Dinosaur Footprint Animals 8
70. Clover/Shamrock- Plants 88
71. Simple Butterfly- - Animals 8
72. Computer Mouse - simple- Items-8 8
73. Simple Crown band Clothing -8
Letter A Alphabet-
75. Number O- Numbers 8
76. Frishae flat circle Toys
77. Croissant shipe- very simple - Food 8
78. Figg shape flat Food
79. Sandwich- triangle cut Food 8
80. Onigin//Rice Ball shape Food
Lollipop- circll on stick Food 8
Simple Hammer outline- Tools 8
83. Simple Saw outline- Tools 8
84, Tadpole Animals- 8
85. Simple Bow- Decorations 8
86. Simple Pinwheel hase Thys 9
87. Simple Book - Items- 9
88. Suail Shell - Animals
Simple Snake- Animals 9
Tulip Head Plants
91. Simple Slueld Toys 9
92, Bird Silbowette- very sumple- Animals- 9
93. Square Coaster - Items 9
94. Hlat Christmas Lree Plants 9
95. Number 8- Numbers 9
96. Fishbone Animals -9
97. Bamboo Shoot Plants 9
98. Lemon slice- -ood
99. Donut flat with hole Food-9
100, Pretzel shape- very simple- Food
101. Fried Egg - flat -
102. Hot Dog in bun - flat Food
103. Sushi Roll simple cylinder end Food
104. Tea Rag with string Food .
105. Simple Vuse outline Items-9
106. Simple Wrench outline - Tools
107. Simple Axe outline Tools 9
108. Dinosaur Fig Animals -
109. Bow Tie Clothing -y
110, Candy Cane Food
111. Letter J- Alphabel- 9
112. Stop Sign octagon shape- Symbols
12

---

Table 4: Medium Difficulty Origami Models (10-19 Steps)
Medium Difficulty Origami Models (10-19 Steps)
1. Dog body- Animals- 10
2. Pig body- Animuls- 10
3. Swan Animals 10
4. Golddish Animals 10
5. Butterlly - common- - Animals- - 10
6. Starfish Animals 10
7. Sun with rays Nature 10
8. House with roof
9. Photo Frame Items 10
10. Sword Toys 10
11. Sailboat- liems- - 10
12. Classic Glider- Toys - 10
13. Trisngular Box base Items- 10
14. Shuriken single piece- Traditional 10
Kimono- fla Traditional 10
Strawberry - Food 10
17. Watermelon Slice Food 10
18. Banana - Food 10
19. Shirt - Clothing -10 10
20. Simple Tree- - flat Plants 10
21. Acom - Plants 10
22. Witch Hat- Clothing 10
23. Sock/Stocking- CCltticninn--Cohingg---- 10
24. Ring with simple gem Accessories 10
25. Letter B A1pphlbbtttt 10
26. Modular Box Corner Unit simple Modular 10
27. Easter Egg Stand - Decorations 10
28. Thernometer- simple - ltems - 10
29. Letter Alphbbeet
Number 6- Numbers- - 10
31. Number 9- Numhers- 10
32. Caterpillar- simple segments- Animals 10
33. Mitten - Clothing 10
34. Letter K --Aphhhe- 10
35. Simple Sola front view Furmiture 10
36. Pigeon Animals-
Duck Animals- 11
Pinwheel fimetional- TToss- 11
39. Pouch- simppe Items- II
40. Dress simple- Clothing- II
41. Pear - Food 1I
42. Seagull simple flying Animals 1I
43. Slipper Ilat Clothing II
44. Mobile Fhone- flat Items -11
Popsicle- Food I1
Number 5 Numbers II
47. Signpost - Irems- II
48. Scart - CCoohhngg- II
49. Simple Bed - top view - Furniture 11
30. Cat body - AAnmals- 12
31. Angelitsh- Animals- 12
52. Ladybug - Animals- 12
33. Crane--raditional -traditional
Animals- 12
Bat- Animuls- 12
55. Chair Chair-Furniture Furniture- 12
56. Lanter - simple flat Items 12
37. Airplane - durt style- style-1oss-111
58. Masu Box - Traditional 12
Wallet/Coin Purse simple Items
Samurai Helmet/Kabuto. Traditional
61. Apple shape- Food 12
62. Pants Clothing
63. Cupcake paper- Food 12
64. Lour-Leaf Clover Plants 12
65. Lily Hower simple- Plants 12
66. Cactus - simple - Plants 12
Wheat stalk simple- Plants- 12
Glasses Accessories-
69. Gingerhread Man flat- Food 12
70. Geometric Pattern file- Geometry 12
71. Cake Slice flat - Food 12
72. Fish Bowl flat simple Items 12
73. Letter G Alphabet 12
74. Pirate Hal - simple flat Clothing 12
Letter M- Alphabet-
Simple Street Lamp post- Iiems- 12
17. Jumping Frog base Animals- 13
78. Fan Iiems 113
79. Ice Creum with scoop- - Food 13
80. Scal - Animmlls-1
81. Monkey Face Animals- 13
82. Koala Face -Animals-
83. Cherries puir- Food Prair-Food111
84, Key Items-
85. Tent- A-frame- - Items - 13
86. Medal Decorations 13
87. Easter Bunny Face Decorations - 13
88. Coffee Mug - Items 13
89. Traffic Light- - Items
Chef Hat simple flat Clothing 13
91. Flying Saucer simple- Toys 13
92. Bear Face Animals- 14
93. Maple Leaf Plants 14
94. Chicken simple Animals - 14
95. Grapes simple bunch Food 14
96. Tulip with stem Plants
Crown Fuller- Clothing
Ginlt Box
Decorations
99. Baseball Cap Maa- Clothung 14
100. Computer Monier- flat liens - 14
101. Finger Puppel Bear TToys 14
102. Simple Tree Ornament Decorations 14
103. Hamburger- simple layers - Food 14
104. Dog House simple front - Items
105. Mailbox- simple- Ieess-
106. Top Hat simple Clothing
107, Anbmll-Animals- Animals- - 13
108. Penguin - Animals- 13
109. Snake Coiled Snake - Animals 13
110. Lion Face Animals 15
111. Tiger Face - Animals- 15
112. Table - simple Furniture
113. Rocket- simple- Toys-
114. Pumpkin flat Food -5
115. Rose easy flat Plants 15
116. Woodpecker simple hend - Animals- 15
117, Duisy -asyy-ooi simple- - Plants 15
118. Boot simple- Clothhng - 15
119. Diamond shape faceted look Decorations-. 15
120. Modular Star 3 simple points Decorations
121, Halloween Bat hunging Decorations 15
122. Pen Holder very simple cylinder- Items 15
123, Bird House simple front - Items- 15
124. Ladies Hat wide brim simple Clothing 15
125. Drigonily- simple- Animals 16
126. Sunflower- simple face - Plants 16
127. Puncapple- simple- Food
128. Winged Heart Decorations 16
129. Lock simple- liems
130, Woven Mat small 2x2 strip Geometry 16
131. Teapot simple dat - liems 16
132. Compass Rose 4 points Symbols- - 16
133, Bench Furniture 16
134, Guitar - simple Ilat Musical Instruments 17
133. Caraatinn simplined -laptiindddlaatts---
136. Snowllake -siple 6-point- Decorations 17
137, Bucket/Pall Items - 17
138. Firefighter Helmet simple- Clothing 17
139. Cicada Cicada-simple- - Animals- 18
140, Squirrel - simple- - Amimals- - 18
141. Lotus Flower- simple- - Plants 18
142. Car side view, simple- Vehicles 18
143, Piano simple upright- Musical Instruments- 18
144, Poinsettia- simple layer- Plants 18
145. 3D Star simple module- - Decorations 18
146. Tissuc Box Cover simple sleeve- Items 18
13

---

Table 5: Hard Origami Models (20-30 Steps)
Difficult Origami Models (20-30 Steps)
1. Blephant- standing Animals 20
2. Bookend simple L-shape, thick liems 20
3. Drum simple Musical Instruments 20
4. Torch with flame Items 20
5. Giraffe - standing Animals 22
6. Hedgehog - Animals- 22
7. Lidded Bux separate & base, simple- liems 22
8. Spaceship- - simple rocket style- Vehicles 22
9. Binoculars Items 22
10. Phone Stand functional - Items 22
11. Pyramid more detailed base Architecture 22
12. Scroll open - Items - 22
13. Bear - standing Animals 23
14. Shrimp/Prawa Animals 23
Lighthouse Architecture- 23
Saibbott more detailed Vehicles 23
17. Hourglass shage liems 23
18. Dog Toy squuaky bone shape - Toys 23
19. Floor Lamp- - Furniture- -23
20. Camel Aniimals - 24
21. Crab Animals- 24
Hot Air Balloon - simple 3D- Velicles 24
23. Camera simple body ltems-
24. Trophy Cup Items 24
25. Bridge simple arch- Architecure 24
26. Vase - with some shaping Items-
27. Shoji Screen - simple panel Traditional 24
28. Horse - standing Animals 25
29. Hippopotamus Animals- 25
Shark Animals- 25
31. Bee- detailed wings- Animals 25
32. Owl with features Animals- 25
33. Treasure Chest simple- - Items 25
34. Church simple front - Architecture - 25
35. Robot boxy- Toys -25 25
36. Microphone with stand base liems 25
Tower/Rook
piece shape Toys 25
Kettle Items 25
39. Photo Frame standing type liems 25
40. Sola more detailed - Furmiuue - 25
41. Panda sitting- Animals- 26
42. Kangaruo with jocy pouch outline Animals 26
43. Scahorse- Animals-
44. Hlamingo - Amimals - 266
Truck simple profile- Velucles- 26
Eaftel Tower simpliflied Ilat Landmarks 26
47. Harp- simplified pronle- Musical Instruments- 26
48. Tont more complex dome like Items 26
49. Snowman- - Decorations -26
50. Wolt howling pose - Animals - -27
31. Turile with shell detail Animals
Eagle - spread wings Animals-
33. Windmull building with vanes Architecture-21
Violin simpliftedd profile- Musical Instruments 22
55. Backpack - with strups Items 27
56. Dragonily more detailed Animals 27
37. Sports Car - simple profile- - Vehicles-2 27
58. Potted Plant Pimatt-sipe-elee Plants
Lion standing Animals
Deer/Stag Animals
61. Crocodile/Alligate simple form- Animals 28
62. Spider legs Animals- 28
63. Parrot- on perch Animals- 28
64. Pentagonal Box simple - Items 28
65. Train Engine - simple profile- - Vehicles - 28
66. Castle simple fromt Architecture - 28
Telephone- receiver and body- liems 28
Saxophone simplifted profile- Musical Instruments 28
69. Accordion simplified Musical Instruments 28
70. Butterfly
detailed - Animals 28
71. Christmas Wreath- simple modular- Decorations - 28
72. Unicorn- simple standing - Animals- 28
73. Laptop open liems 28
Rhinocerus Animals- 29
Peacock- simplifted tail Animals 29
Picrodactyl -imple Animals -2
77. Fire Truck basic shape- Vehicles 29
18. Police Car basic shape Vehicles 29
79. Ambulance basic shape- Vehicles 29
80. Grund Piano simplified top view new-Bi Musical Instruments 29
81. Clownfish- Animals- 29
82. Ice Cream Truck- simple profile- Vehicles - 29
83. Lotus mulii-petal- -9
84, Octopus with 8 simple tentacles- Animals- 30
85. Scorpion - Animals-30 30
86. Dinosaur T-Rex simple standing Animals 30
87. Hexugonal Box simple Items - 30
88. Bicycle - very simplified profile Vehicles 30
Motorcycle- very simplified profile- Vehicles 30
Pirate Ship - simplified Vehicles 30
91. Double Decker Bus simple profile- Vehicles 30
92. Reindeer simple standing Animals- 30
B
Manual annotation
B.1 Annotation Rules for Pattern Prediction Task
The primary goal of this annotation task is to create challenging yet fair incorrect options for multiple- choice questions (MCQs). For each given Crease Pattern (CP) diagram and its known correct folded 3D shape, annotators are required to design three distinct incorrect shape options. These options, along with the correct one, will form an MCQ designed to evaluate a model's 's ability to predict the 3D shape from the CP. The following rules must be strictly adhered to when designing these incorrect options:
B.1.1 Rule 1: Ensure Visual Distinguishability
Each incorrect option must be easily and clearly distinguishable visually from the correct folded shape. The purpose is prevent ambiguity where an incorrect option might be confused with the correct one due to only subtle visual differences.
Guideline:
14

---

The overall silhouette, major components, and general form of the incorrect option should significantly different from those the correct option.
Avoid creating incorrect options that merely slight modifications, re-orientations, minor proportional changes of the correct shape.
Example:
If the correct shape is an origami crane:
incorrect option that is another bird in a very similar pose (e.g., a crane with wings slightly more elevated versus wings fully spread, if the overall form remains highly similar) might be unsuitable it's not clearly visually distinct at glance.
suitable incorrect option would be an origami box, an origami boat, or an origami star, these are visually very different from crane.
B.1.2 Rule 2: Maintain Conceptual Distinctness
Incorrect options should not be variations the same concept or fall within the same narrow semantic category the correct option. They should represent fundamentally different objects ideas. This rule ensures the task tests the prediction the specific shape, not fine-grained classification within a single conceptual group.
Guideline:
If the correct option is specific type of animal, incorrect options should not be other animals that are closely related (e.g., from the same family) or share very similar overarching characteristics.
Strive for incorrect options that belong to different conceptual categories than the correct option (e.g., animal vs. inanimate object vs. geometric form).
Example:
If the correct shape an origami cat:
Incorrect options such Lion, Tiger, or Leopard are unsuitable because they all felines and thus variations the same core concept ("large cat" or "wild cat" as opposed to "domestic cat").
Suitable incorrect options could be origami airplane, origami hat, origami fish (assuming the "fish" is a distinctly different concept from "cat' within the context of common origami figures).
B.1.3 Rule 3: Ensure Crease Pattern Plausibility
While incorrect, the alternative shapes should be plausible outcomes that could potentially be folded from a Crease Pattern that bears some relationship the given CP diagram. This means an incorrect option might be a shape that could result from misinterpreting some creases, omitting few key folds, or simplifying the original pattern. The objective create distractors that are not arbitrary but reflect potential, albeit erroneous, folding paths from a CP similar to the one provided.
Guideline:
Consider what alternative, simpler, or related shapes might emerge if certain folds in the CP are ignored, if mountain and valley folds are confused, if common base derived from the CP completed into a different known figure.
The incorrect option's 's implied CP should not be drastically more complex or entirely unrelated to the structural elements suggested by the given CP. It should ideally represent shape that an intermediate folder might erroneously produce when attempting the correct. model or a related one.
Example:
Given CP diagram for relatively simple origami boat:
15

---

A suitable incorrect option could be an origami hat (e.g., traditional paper hat like "samurai helmet" or simple party hat). Many simple hats share foundational folds or bases (like the water bomb base or preliminary fold variation) with simple boats, or their CPs can be derived by altering or omitting a few creases from a boat's CP.
An unsuitable incorrect option might be a highly complex origami insect or multi- piece modular origami ball if the provided CP is for a simple, single-sheet boat. The CP for such complex figures would likely be vastly different and far more intricate, making them implausible alternatives based on the given simple CP.
Summary for Annotators Creating Incorrect Options: For each CP diagram and its corresponding correct folded shape, you are to design three unique incorrect shape options. Before finalizing these options, please verify each one against the following three criteria:
1. Visual Distinguishability: Is the incorrect option clearly visually different from the correct shape?
2. Conceptual Distinctness: Is the incorrect option conceptually different from the correct shape, avoiding mere variations of the same theme?
3. Crease Pattern Plausibility: Is the incorrect option shape that could plausibly (even if incorrectly) be derived from the provided CP or closely related CP (e.g., through simplification or common error)?
Adherence to these rules is crucial for creating high- and effective multiple-choice questions for the Pattern Prediction evaluation task.
B.2 Annotation Guidelines for Incorrect Option Generation in Spatial Relationship Prediction Task
This section outlines the rules for annotators tasked with designing incorrect options for the Spatial Relationship Prediction task. For each Crease Pattern (CP) diagram, questions are posed about the spatial properties of the final folded origami model. While correct answers are generated by an optimized compiler, annotators must manually create three plausible yet incorrect options for each question to form a multiple-choice question (MCQ). The aim to generate distractors that effectively test model's nuanced understanding of 3D spatial relationships post-folding.
The task comprises three types of questions. Below are specific guidelines for designing incorrect options for each type:
B.2.1 Type Spatial Pose Localization
This question type requires predicting the specific 3D position and/or pose (orientation) of designated point (or feature) from the original flat paper once the model is fully folded. The pose might be described relative to a global reference frame (e.g., on table, with a specific part facing upwards).
Guidelines for Designing Incorrect Options:
Plausible Positional Errors:
Offer coordinates that are slightly offset from the correct 3D position (e.g., incorrect by small delta in one or more axes, located an adjacent quadrant, or on wrong but nearby surface).
Suggest position that would be correct if a key fold were made inaccurately (c.g., mountain fold treated as a valley, an incorrect fold angle, or slight misalignment of layers).
Propose the final position of a different, perhaps nearby or symmetrically opposite, salient point from the original CP.
Plausible Pose Errors (if orientation is part of the question):
Provide options with the correct 3D position but an incorrect orientation (e.g., correct (a,, y, z) coordinates, but the point/surface faces downwards instead of upwards, or rotated 90 incorrectly).
16

---

Offer an orientation that is common simplification (e.g., aligned perfectly with major axis when it's actually slightly tilted).
Symmetry-basee Errors: For CPs/models exhibiting symmetry, an incorrect option could be the symmetrical counterpart of the correct position or pose.
Reference Frame Confusion: Offer position or pose that is correct relative to a local part of the origami model but incorrect within the specified global reference frame, or vice-versa.
Example: Suppose specific vertex 'P' on the CP is queried for its final 3D coordinates (I, y, and the direction its local paper surface is facing (e.g.. 'upwards'), relative to a table sits on. The correct answer (from compiler) is (10, 3), local surface facing 'upwards'.
Suitable Incorrect Options could be:
(10, 5, 0). local surface facing 'upwards' (Incorrect Z-coordinate, perhaps implying it's on the table surface when it's elevated).
(10, 5, 3). local surface facing 'downwards' (Correct position, but incorrect orientation).
(-10. 5, 5) local surface facing 'upwards' (A symmetrical position if the model has
YZ plane symmetry and origin is centered).
The final coordinates and pose of an adjacent vertex 'Q' from the CP.
Unsuitable Incorrect Options: Random coordinates or orientations with no plausible
relation to the model's scale, structure, or folding process.
B.2.2 Type 2: Layering Relationship Analysis
This question type focuses on the internal structure of the folded model, specifically the stacking order of paper layers the number of layers particular region (e.g.. identifying the thickest region or counting layers specific point).
Guidelines for Designing Incorrect Options:
For Number of Layers Questions:
Offer layer counts that are slightly off from the correct number (e.g., correct count + or + layers).
Propose the layer count of an adjacent or visually similar region in the folded model.
Suggest count that might result from overlooking some hidden internal layers or, conversely, double-counting some visible folded edges as separate layers.
- If the question asks to identify the "thickest region" from set of options, incorrect options should be other regions that also thick, but not maximally so, regions that appear thick but are not.
For Stacking Order Questions:
Provide plausible but incorrect permutations of the layer sequence. For example, if the correct top-to-bottom order of layers (referenced by their original CP surface labels like $1, S2, S3) is S1-S3-S2, an incorrect option could be S1-S2-S3 or S2-S1-S3,
Suggest order that would result if specific flap were tucked differently during folding (e.g., flap going over another flap instead of under it).
Offer an incomplete order (e.g., missing one or more layers from the sequence in that region) or order that incorrectly includes layers not present in that specific stack.
Example: Question: "How many layers of paper form the central part of the crane's 's body?" Correct answer (from compiler): layers.
Suitable Incorrect Options could be:
layers (Plausible underestimation, perhaps missing some internal folds).
layers (Close, but incorrect).
10 layers (Plausible overestimation, perhaps counting edges).
4 layers (Number of layers in the crane's wing, different region).
17

---

Question: "Consider point X on the wing of folded paper airplane. Starting from the externally visible top surface X, what is the order the original paper surfaces (labeled S1, S2, S3, S4 on the CP) one would pass through if drilling perpendicularly downwards through all layers at X?" Correct answer (from compiler): $1, $4, S2.
* Suitable Incorrect Options could be:
S1, S2, S4 (A common misremembered or simplified stacking). -S4, S1, S2 (Incorrect starting layer or internal order).
-
-
S1, S4 (Incomplete, missing the bottom layer S2).
B.2.3 Type 3: Geometric Change Analysis
This question type involves predicting how specific geometric features (e.g., angles between lines, distances between points, areas of surfaces) change from their state in the flat CP diagram to their state in the final 3D folded model.
Guidelines for Designing Incorrect Options:
Value from Original CP: A very common and effective incorrect option is to offer the original geometric value as it was on the flat CP diagram (e.g., an angle is 90º on the CP but becomes 45 in 3D, then 90 is strong distractor). This tests whether the model understands that geometric properties transform during folding.
Plausible Estimations or Miscalculations:
For angles: Provide common angles (e.g., 35" ,5",55600 60%, ,909, 180") that might appear correct upon a cursory visual inspection of the folded form, or angles that result from assuming a simplified 3D configuration (e.g., assuming perpendicularity or parallelism where it doesn't 't exactly exist).
For distances: Offer distances measured along the paper surface instead of the true Euclidean distance through 3D space (or vice-versa, depending on the question's phrasing). Suggest distances that might result from slight errors in visualizing the 3D form, such as ignoring foreshortening or using dimensions from 2D projection.
For areas: Propose areas that don't 't account for overlaps of paper in the folded state, or the area of a 2D projection rather than the true 3D surface area (if the latter is specified). An area that results from a miscalculation of how shape transforms (e.g., halving an area when it should be less or more).
Qualitative Change Errors: If the question is about the nature of change (e.g.. "Does distance X increase, decrease, or stay the same?"), incorrect options could be the opposite type of change, or "stays the same" when there is indeed a significant change.
Values from Unrelated or Different Parts: Offer a geometric value (angle, distance, area) that is correct for different feature or part of the folded model, or for a different but related origami model.
Example: Question: "Two line segments L1 and L2 are parallel on the CP diagram and are 5 cm apart. In the final folded model, these segments become two adjacent edges of wing. What the approximate angle between the segments LI and L2 in the folded state?" Correct answer (from compiler): 60º.
Suitable Incorrect Options could be:
0" (Implying they remain parallel, i.e., no change from CP state regarding their relative orientation).
90 (A common angle man-made objects and some origami steps, could be plausible guess).
45 (Another common angle, plausible visual estimate).
Question: "A defined square region on the CP has an area of 16 cm? After folding, this region forms part of a curved surface. What is the approximate surface area of this region in the 3D model?" Correct answer (from compiler): 16 cm2 (assuming no stretching/shrinkee surface area remains the same, though its projected area might change).
of paper, the intrinsic
18

---

Suitable Incorrect Options could be:
cm3 (Perhaps confusing with a projected area that is halved).
12 em2 (A value less than original, implying shrinkage or significant overlap not intrinsic to the region itself).
20 em² (A value more than original, implausible without stretching). (Note: If the question was about *projected area*, then 16 cm2 could be an incorrect option if the projection foreshortens it).
Designing Incorrect Options: For each question across these three types, remember the following overarching principles when designing your three incorrect options:
General Summary for Annotators
1. Understand the Query: First, be absolutely clear about what the question is asking regarding the folded CP and what the compiler-generated correct answer is.
2. Plausibility is Key: Incorrect options should appear as reasonable possibilities to someone who might have slight misunderstanding of the folding process, 3D geometry, or spatial reasoning. Avoid options that are trivially wrong, absurd, or completely random.
3. Ensure Clear Incorrectness: While plausible, each incorrect option must be demonstrably wrong upon careful analysis based on the correct folding sequence and 3D geometry.
4. Introduce Variety in Errors: The set of three incorrect options should ideally probe different potential misunderstandings types of errors (e.g., one based CP value, one on slight miscalculation, one on conceptual error).
5. Maintain Consistency: Ensure that the format of your incorrect options (e.g., units, preci- sion of numbers, terminology) is consistent with the format of the correct answer.
By following these guidelines, you will help create high-quality multiple-chol questions that rigorously and fairly evaluate a model's capabilities in spatial relationship prediction for origami.
B.3
Human evaluation
For the manual evaluation of the first three tasks, we recruited evaluators from two different categories. The first category included five non- proppoffessionas recruited through a crowdsoureing platform; the second category comprised three experts with extensive experience in the field of origami. Participants these evaluations were compensated according to the prevailing local minimum hourly wage standard.
C Detailed Explanation of Origami Compiler Error Feedback System
The following is detailed supplementary explanation of the origami compiler error feedback system, including more specific error types, possible error messages, relevant parameters, and their underlying principles.
C.1
CP Code Syntax Error
This type of error occurs in the initial phase when the compiler parses the Crease Pattern (CP) code provided by the user, if the code does not conform to predefined syntax rules.
C.1.1 More Details
Example Error Codes:
CCP_SYNTAX__,C SYNTAXXINVALIID INVALID SWTAALID_PARRA.....EA MARAM_COUNT: "Instruction "COMMANT has an insuf- ficient excessive number of parameters. Expected X, got Y."
E__PP_ SYNTAX_UNKNUR COMMAND: "Unrecognized instruction 'COMMAND'.
Please check spelling or the instruction set."
-E_PP SYNTAX INVALID PARAM TYPE: "Parameter PAARAM_N N for instruc- tion 'COMMAN D' has an invalid type. Expected type EEXPECTED_TYPE but received value VV ALUE of type AYPEE
19

---

E_CP_SYNTAX, _SSNTAXAA BATTAAAVALUEEOUUTOOO OFAVECE
"Value 'VALUE" for parameter PPARAM NAME of instruction 'COMMAND' is out of the allowed range [MIN_ WIIN_VAL AT, M MAX_VAZ]]
E_CP SPPTTXX UNWUPEETEEDDEEEEE TOKEN: "Unexpected symbol/character 'TOKEN en- TOOKE:
countered at line number !ine_number)], column [col_nemberr] while parsing instruc- tion (OOMAND)
"Instruction 'COMMAND' is missing a re- quired delimiter. For example, the expected EXPECTED. DELIMITER EXPECTED.ELLIITER was not found."
A__PSSSTTA,,,,_ SYNTAX_MISSINA MISSINGG MISSIINGDELLIMIER DELIMITER:
CLCCP_SYNTAA. IIV_LL INVALID __LI_____R__E____ RENFREEEEE "Instruction 'COMMAND" refer- ences a non-existent line ID "LINE or point ID "POINTID""
faulty_cpp Saulty_e_ccod__eeee _p_ numbers: cene__umberr] occurred.
The specific code line where the error
Saulty_token ficken_o__coiie or__ommand: (Optional) Indicates the specific instruction token that caused the error.
C.1.2 Underlying Principles
Formal Language and Grammar: CP code is treated as formal language with precisely defined lexical and syntax rules.
Parsing Stages:
1. Lexical Analysis: Code text is broken into "tokens."
2. Syntax Analysis: Token sequence is checked against grammar rules, often building Abstract Syntax Tree (AST).
Error Detection: Errors are reported if tokens or their sequence violate rules, preventing further compilation.
C.2 Geometrically Impossible Fold
This error indicates that some defined folding operations are physically geometrically unfeasible.
C.2.1 More Details
Example Error Codes:
GEOM_TOOL MANY_LAYERS: "Folding near (z, would result in N paper layers, exceeding the limit of M layers."
max max_ max__alloaable_11 max_aallowable_layersss Maximum allowed layers.
caleulaaeed_
Calculated layers the point.
E_ GEOM_ ANNCLENCCNSSAAAAEEA VIL.LITTTTIOOOAIIOO "Target angle [Otarget] of crease [id] con-
flicts with existing angles [01,.... (22m] at vertex parteercooodinatess
* Specific reasons may include:
"Maakawa-Justnn: |M 2."
"Kawasaki-Justin E(-1)'a; + or alternate sums + x for flat-folds."
"Angle sum around vertex 2T (internal) or i (boundary)."
"Single crease angle is too large or small."
E. wwa.llictinggcressee__e. cwwaseeeeiiieeeeee _id___dd________________ and angles: IDs and angles of conflicting creases. CEEICEEEETTINVVAALALE
* conflicting.
"Endpoints of crease (id] outside paper, or crease illegally intersects boundary."
E_ BEOM_LENGTH COTTTROOOSTTTTTTTSS CONSSTRIINTLVOOLATION:: "Operation requires points [A] and [B]
to coincide, but original distance d1 # required de (usually 0), implying stretching."
faulty_crease_ide ids: Lislofcrease! creasel Dscausinglhecmm
faulty vertex ids or point coordinates: (ong
problematic_coordinttes wow.dinates..cooooom regions: (Proleenttcrriooosssooottttteeee
20

---

C.2.2
Underlying Principles
Non-strefchability of Paper: Paper is inextensible; folding is isometric transformation.
Local Developability: Paper must be locally developable onto plane (zero Gaussian curvature except at singularities).
Flat-foldability Conditions: For flat folds:
Mackawa-Jusinn Theorem: |M - - 2.
Kawasaki-Justin Theorem: Σi=1 022-1 E ii= =
Big-Little-Big Angle Constraint: a; S 0;-1 0:+1-
.
Layer Thickness Limitation: Real paper has thickness, limiting layer stacking.
C.3
Paper Self-Intereeciio/Pennttatii
This error means different paper parts occupy the same 3D space.
C.3.1
More Details
Example Error Codes:
E_ P_PPHS_SSEI SELF INTERSECTION: "After crease [id], facet [facet_A_id (region
[coordinates A)) penetrates facet [facet_BBiii B_id__________ (region (coordinates
"During folding of [id], time t [time], region [region_AA A] collides with [region_ BB."
LWWS_BOUNDARY_ "Folded part [facet_id] penetrates defined container boundary."
faulty_crease_ids (CreasseDDs))anne (CreessssDssyyossssrorrrrderreep
popllmmatic_oordinnaat Baapilessorrregggeeee orgreooons:
[Penerrationorea
pointsets, bowndingbozes facetl
intersecting layer ids intersecting facet ids: (Optional) [layer_idj, Layer_iiee idg] or
facet_idi, facet_idy focet__dd,,fett_eeeeee specifying penetrating parts.
penetration depth: (Optional) Estimated penetration depth/volume. E.g., 0.5mm.
C.3.2 Underlying Principles
Volumetric Exclusion: Physical objects cannot occupy the space.
Collision Detection: Algorithms detect intersections between paper parts (meshes/facets),
Discrete Collision Detection: Checks static geometry at time steps.
Continuous Collision Detection (CCD): Detects collisions between time steps to prevent "tunneling."
Data Structures: Spatial partitioning (Octrees, BVHs) for efficient detection.
Layer Ordering and Penetration: Incorrect layer order in flat folds can cause penetration.
C.4 Ambiguous Folding State
Indicates that the CP code and constraints do not uniquely determine the folded form.
C.4.1
More Details
Example Error Codes:
E_ EEIIGUOUUSSTATEE "CP code is insufficient for a unique state. N possible configu- rations in region (coordinales] (or vertex [verlea_id))
F_AAMBGUUOU__LAA LALESS__L____LL _ODEER: "Insufficient constraints determine stacking order of layers [layer_ Lar_____ii id] and [layer_Bi ___BB_ _d] in region [coordinates). At least two valid orders."
E_ AMAMIIGUUU SERUUUUSSTUUU__USE CHOICE: "Operation 'tuck" at [coordinates) has multiple valid TUCK_CHOICE:
insertion methods; CP unspecified."
21

---

AMAMICUOUS E_AMBICIOUSS_OOUTTAAAI MOOTTAAIINNNINNNNA VALLEY ASSIIMMENT: "For crease [crease id], multiple valid M/V assignments satisfy local constraints but yield different global forms."
ambiguous_creet ids or vertex ids: [Rggionorrereeeeeeeeemmminnngugu
samaber_of_posiblle states (Optional) Number of possible states detected (N).
suggested diirriedddiaabnnuationn (Optional) "Suggestion: Add layer order constraint (e.g..
LAYER (ABVE) or specify crease direction."
C.4.2 Underlying Principles
Non-uniqueness of Solution Space: A CP may correspond multiple valid configurations.
Local vs. Global Information: Local constraints may be met, but global form can vary.
Symmetry: Symmetric CPs or operations can lead to multiple equivalent results.
Branching Points in Configuration Space: Folding path may have bifurcations. Implicit vs. Explicit Instructions: Unstated conventions can lead to ambiguity for the compiler.
Solver Behavior: Solvers for underdetermined systems might not find unique solution.
D Crease Pattern evaluation system
This section introduces the complete evaluation process of the Crease Pattern. The final score is a weighted average of the scores from the different dimensions. Each of the four main dimensions is assigned an equal weight:
Topological Similarity: Wtopological 0.25
Geometric Similarity: Wgeometrie 0.25
Foldability Constraint Satisfaction: W foldability 0.25
Final Folded State: state 0.25
The total score Stotet calculated as:
Stotal
Σ dim
Wdim
Since Σ Weem with these weights, this simplifies to:
Stotal 0.25 Stopotogical + 0.25 Sgeometric + 0.25 ffoddbiiltty + 0.25 Sfold state
where 8dim is the score for particular dimension.
D.I CP Structure Validation dtalliddte_cp_strructt
This initial step ensures the generated CP data (cp_data) is well-formed and meets basic criteria for a valid crease pattern.
Presence Basic Elements: Checks if "vertices_coords "edges_ and "eaces_vertces" keys exist in the input.
Vertex Coordinates: Each vertex in vertices_coorda must be a list of two numerical coordinates (e.g., [x, y]).
Edge Definitions: Each edge in indices (c.g., [v1, v2]). These indices must be valid and within the bounds the vertex list.
must be list of two integer vertex
Crease Assignments (Optional): If "edges assi assignment" is present, each assignment must be one of the valid types: "B" (Boundary), "M" (Mountain), "V" (Valley), "F" (Flat), "U" (Unassigned).
22

---

Face Definitions: Each face in sectt_e must be a list of at least three integer vertex indices. These indices must be valid.
Euler Characteristic: For a planar graph, the Euler characteristic must satisfy V E+F== 2, where V is the number of vertices, E is the number of edges, and F is the number of faces.
Flat-Folder Validation (Optional): If the Flat-Folder compute module is available, its validate_ capeeeeppppppeeeeeee up_uttuuture (cp_data) API is called to check if the CP can be compiled into valid origami model. If not, the CP is considered invalid.
any of these checks fail, the function returns {"valid": False, "reason": "error message"). Otherwise, it returns ("valid": True).
D.2 Topological Similarity
This dimension assesses the similarity of the graph-theoretical structure of the generated CP (gen_cp) and the reference CP (ref_cp) It combines scores from four sub-metrics, after extracting basic topological information using extract_tore topoooog (cp_data), which retrieves vertices, edges, edge assignments, and faces.
The overall topological similarity score Stopological is weighted average defined within the method:
cal calculate_topollle
Stopological = 0.2 Svertex + 0.3 Sedge + 0.3 8face + 0.2 Screase
D.3 Vertex Count Similarity Compare_vertex__oom
Compares the number of vertices (Vgens Vres).
If gen Vref. score s, 1.0.
Otherwise, the score calculated using an exponential decay function:
Vgen Veagl 0.5 muulVgen..Keg
(Note: The code implements this
exp(-0.5
(max(Vges)) Vref)
min (Vgens Vres))/ min(V V).
D.4
Edge Connectivity Similarity (compare_edge_coom edge__coneettvvigg
Compares the edge structures based on degree distribution and connected components.
Adjacency List Construction saaildd_djaceccy... _iist) Adjacency lists are built for both CPs from their edge-vertex relationships.
Degree Distribution Similarity:
calculate @alculate__dgrre__.. degree distriiiuttt ibttion: Computes the distribution of vertex degrees (number of edges connected to each vertex).
calculate_ calculate_wasserssteee wasserstein distance: : A simplified Wasserstein distance (dw) is calculated between the degree distributions of the generated and reference CPs. The score for degree similarity is 8degree
dw.
Connected Components Similarity:
Count_ crant__conecteddc seredddoooooeeeeeeeeeeeeeee The number of connected components (Cgens
-
Cref) is determined for each CP graph using Depth First Search (DFS).
If Cgen Cref Sconn 1.0.
Otherwise, Sconn (Cgan Over!
The final edge connectivity score Sedge weighted average: Sedge = 0.7 8degree + 0.3
5 Sconn
23

---

D.5
Face Relations Similarity (compare_faco_relationn
Compares properties of the faces in the two CPs.
Face Count Similarity (sy_cnnt))
PFoen Feorl _ccuntt mmaint raf)1
where gee and Fref are the number of faces.
Average Vertices per Face Similarity (#g_ang_v): augV and augVees be the average number of vertices per face.
Wayygenn ave ovaleedl _angg_ max(1,mintovxgV (enn
Face Size Distribution Similarity (s dist)::: The distribution of face sizes (number of vertices per face) is computed for both CPs. A simplified Wasserstein distance (dw) is
calculated between these distributions using cal aalcclaaewwaserrsteeee wwsserstein direeiinddittneeeeeee The score is 8 ffsdist
dw.
The final face relations score face is a weighted average: face = 0.3 _fcountt 0.3
$f_avg_v + 0.4 _f_dist-
D.6 Crease Assignment Similarity (compare, crease_assigneee
Compares the distribution of crease types (M, B) if "edges_aasss aassssssssiinnnsssss " is available.
If either CP lacks edge assignments, low score of 0.2 is returned.
Crease Type Counts (count_ wwwwseeeeeeppeeeeceeeeeee ttype): Counts the occurrences of Mountain ("M'), Valley ("V"), Boundary ("B'), Flat ("F',, and Unassigned ("U") creases.
Proportion Similarity: For Mountain, Valley, and Boundary creases, the similarity of their proportions (prop) in the generated (gen) and reference (ref) CPs calculated:
Mountain: SM
prop M.en prr
Valley: sy 1 propv.gen propy.refl
Boundary: 8B = pprop B.gen propBrref!
where proportion is count of type total number of assigned edges for that CP.
Length Penalty (Pt)): penalty is applied if the total number of assigned edges differs:
min(Lgens Lres)
Pr. max(Lgens Lref)
where the total number of assigned edges.
The final crease assignment score Screase is a weighted average of the proportion scores, multiplied by the length penalty:
Screase = (0.4- sM + 0.4-sy sy + 0.2 - sB) PL
D.7 Geometric Similarity
This dimension evaluates the similarity of the spatial characteristics of the compiled/folded models. requires compiling the CPs into 3D models.
CP Compilation (compile_cp.. :
If the Flat-Folder compute. compute_folde www.foollddatteeee (cp_datcppaaaa)) API is available, it's used get the folded model data (typically including 3D vertex coordinates "P" and crease edges "SP").
If Flat-Folder is unavailable, simplified_foldeeeeeiieeeeeeeeee method is used, which essen- tially returns the original 2D vertex coordinates as "p" and edges as "SP". This is significant simplification.
24

---

If either CP fails to compile (or provide simplified data), low score of 0.2 is returned by culculaac...emmmtriiiiiiiii..aia
The overall geometric similarity score Sgeometric is a weighted average defined within
geometric 0.4 Spoint + 0.3 Sangle 0.3 8size
D.8 Point Position Similarity (compare_point_posttee
Compares the 3D point clouds of the folded models.
Coordinate Normalization (normalize cotiiiiattcrrddattess Vertex coordinates (from "P") of both models are normalized. If points are 2D, a Z-coordinate of 0 is added. Points are then translated their centroid is at the origin and scaled the maximum distance from the origin to any point is (i.e., normalized to a unit sphere).
Bidirectional Hausdorff Distance (calculate
The max (suPae inf e d(a, supseB infae d(a, 6))
ietirectiinal_haudddeo
Hausdorff distance (A, BB is calculated between the normalized point sets of the generated (P gen) and refer- ence (Pref) models. d(a. b) is the Euclidean distance. This is achieved by calling calculate saattee_huuddofff_seeeeee distance twice.
The point position similarity score Spoint is calculated using an exponential decay function:
point e k-d
where is sensitivity coefficient.
D.9 Angle Similarity (compare_ang]ess
Compares the distribution of dihedral angles along creases in the folded models.
Crease Edge Extraction (extract_cresl cereacc_rreeeeeeee :: Crease edges extracted from the folded model data (typically from "SP").
Dihedral Angle Calculation (calculate aaacculaaeddhhedral__
Note: In the provided eval. py, if Flat-Folder unavailable, this function returns a list
of random angles placeholder. A proper implementation would calculate actual dihedral angles between faces sharing a crease.
Angle Histogram Comparison
create histogram: Histograms of dihedral angles are created for both models. Angles are typically in [0, 180%), binned into 18 bins (10 degrees per bin).
calculate__ coulate_cosineeeee simi iimilarit:: The cosine similarity between the two angle his- togram vectors calculated. This value serves as the angle similarity score Sangle-
If creases cannot be extracted angles cannot be calculated for either model, default score 0.5 returned by compare, saapare_angle..
D.10 Size and Proportions Similarity (compare_size cimpare__ize_ante caanparossiee_an__proorrrt pro_prrpp)
Compares the overall dimensions and aspect ratios of the folded models' bounding boxes.
Bounding Box Calculation couuulaatebbudddingg box): The axis-aligned bounding box (min/max coordinates along X, Z) computed for the point clouds of both models. 2D points are padded with Z=0.
Proportion Calculation: The dimensions (length, width, height) of the bounding boxes are calculated. These dimensions are sorted in descending order and then normalized by dividing by the largest dimension (e.g., [1, L2/ L1, LI])..
Similarity Score:
The cosine similarity between the normalized proportion vectors of the two models is calculated using calculate, cosilaateccsinneeeeee cwwioecsieeeeeccc www.sooesssseeeniiiiiioooeeeeeee similarity. This value the size
and proportions similarity score Saize:
If either point set is empty, a default score of 0.5 returned by compare_sit siiare_siee___e and_prroossions.
25

---

returns a score D.11 Foldability Constraint Satisfaction (aa.aulacc..ffoalll....s.iii.iia
This dimension assesses whether the generated CP adheres to known origami foldability constraints, beyond basic geometric foldability.
Basic Foldability Check (Optional):
Folders compute. check_ Check_.ooldabllittyyq_...aa. API available, it's used to check if both CPs are foldable.
If the reference CP is foldable but the generated CP is not, the score for htwttte_oooabblttysiiiieiie
returns 0.2.
The overall foldability score foldability is weighted average defined within foc.llate_oodaaility.....to
Sfaddabitity = 0.3 8TT + 0.3 STTO + 0.2 8Trans + 0.2 flotfold
an exception occurs during calculation, follolltte_ollddblitt..... of 0.3.
D.12 Specific Origami Constraint Comparison
This involves extracting and comparing critical origami constraints.
Constraint Extraction (extract_coostraintts)
This method aims extract Taco-Taco (TT), Taco-Tortilla (TTo), and Transitivity (Trans) constraints by calling helper methods like
Note: In the provided eval. py. Flat-Folder's constraints module unavailable, the extraction methods are simplified and return empty lists. A full implementation would identify these constraints from the CP geometry and crease assignments.
(compare_ Coomparrr_ttoo__tcocooootttttat compare_ taco tortilla, coaillaanntttantss....mo compare wtw.sstttittiinneee ctt.riiiiityyycottriattttttt)) For each constraint type (TT, TTo, Trans):
Constraint
Set
Comparison
If both CPs have no such constraints, similarity is 1.0.
one has constraints and the other doesn't 't similarity 0.3.
Otherwise:
Constraint Overlap (Soverlap): Calculated using Jaccard similarity
the sets of constraints (constraints are stringified for comparison via calcuulate.costrrattt.ooorre
AAnD B AUB B
JAA
* Count Similarity (Scount):
[N N Scount Mo
where N is the number of constraints of that type. * The score for that constraint type (e.g., STT) is 0.7 Soverlap + 0.3 Scount-
D.13 Local Flat-Foldability Conditions (compare flat_flllllaaaae foldability
Checks for adherence local flat-folding theorems around vertices.
Kawasaki's Theorem Check
States that for flat-foldable vertex, the sum of alternating angles around the vertex is 180% or equivalently, Σ az (or 0, depending on how angles are measured like Σ--1) 0).
26

---

Note:
The mock implementation in eval, py always returns True. A full implementa- tion would iterate internal vertices and check angles.
Maekawa's Theorem Check (check_ Batecck_aekawaa____ maekewa_thhorem)
States that for a flat-foldable vertex, the number of mountain creases (M) and valley creases (V) must differ by two: |M -
Note: The mock implementation in eval py always returns True. full implementa- tion would check crease assignments around internal vertices.
Scoring:
Kawasaki score (: 0.2 if reference theorem status is True and generated is False, 1.0 otherwise.
Maekawa score (s 0.2 if reference theorem status is True and generated is False, 1.0 otherwise.
The final flat-folaability score flat 0.5 8K 8M--
D.14 Final Folded State Similarity (compare f saall__olldd__aee
This dimension directly compares the 3D geometry of the final folded shapes compiled from the generated and reference CPs.
CP Compilation: Similar to geometric similarity, compile_gr cp_ to_model is used. If compilation fails for either (returns falsy), compare_ folded_ state returns score of 0.3.
Point Cloud Extraction: 3D vertex coordinates ("P") are extracted from the compiled models. If point clouds are missing for either, score of 0.3 is returned.
The overall final folded state score Sfinal_state is a weighted average defined within faaerr__inal__fodde _ttte:
Sfinai-state 0.7 Sshape 0.3 Slayer
If an exception occurs during calculation, compare final_folded stade__tttddddeteeeeeeeeee state returns 0.3.
D.15 Overall Shape Similarity
Calculated using the bidirectional Hausdorff distance du between the (normalized) point clouds of the generated and reference folded models, identical to the method in compare compare_poine wwwppponooiniioeeeeeeeeepp
The shape similarity score Sshape is:
8 Sshope = e 33d
D.16 Layering Similarity (compare_layee)
Compares the stacking order of faces/layers in the folded state.
This relies on layering information being present in the compiled model, typically under key like "CF" (face assignments or configuration).
Note: The compare compare_layers function in the provided eval. py is simplified placeholder and returns default score of 0.5. A full implementation would require detailed comparison of the layer graph or face ordering.
The score is Stager-
E Training setting
For the reinforcement learning method, we adopt TRICO [35 training on qwen2.55v1-322B, which
is a PPO-based [36). more efficient MILMs multi-turn reinforcement learning algorithm. Specifically, we trained for 10.2 hours on 16 H100 GPUs, with the following hyperparameter settings: Tium = 0.95,
Tioken 1.0, KL penalty 0.001, Actor LR=1 x 10-6, and Critic LR=1 x 10-5.
27

---

Limitation
While the ORIGAMISPACE benchmark and dataset offer novel approach to evaluating multi-step spatial reasoning in MLLMs, we acknowledge certain limitations that provide avenues for future work. Firstly, although our dataset comprises 350 meticulously collected origami instances, the overall scale is relatively modest compared to some large-scale benchmarks in other vision and language domains. Future efforts could focus on expanding the dataset size and further diversifying the range origami types and complexities included, potentially through anii-automated generation techniques, to ensure even broader coverage and statistical power. Secondly, while origami provides an excellent structured environment with clear mathematical constraints, the direct transferability of MLLM performance and the specific reasoning mechanisms learned on ORIGAMIS to other, less constrained or visually distinct spatial reasoning tasks (c.g.. understanding dynamic real-world scenes interpreting abstract diagrams from different fields) warrants further investigation. Exploring this generalization gap could be valuable direction for future research. Finally, current of evaluation tasks, though designed to be challenging, focuses on specific facets of spatial reasoning highlighted by origami. There may be other subtle aspects of spatial intelligence different interaction modalities with the origami compilation process that could be explored in future iterations to provide an even more holistic assessment of MILM capabilities.
28

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The contributions and scope of the paper are provided in the introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of the paper in Appendix F
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]

---

Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide complete data, evaluation code, and model training code, which can be accessed via GitHub in the public version.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
30

---

Answer: [Yes]
Justification: We provide complete data, evaluation code, and model training code, which can be accessed via GitHub in the public version.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (https
hetpolliicggiddeeCoooooo woweeeeeeeooeSbumiiioooe ssionlolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide all the details of data(Appe A evaluation(Appennu ManddixCaann Cland D) and training(Appendix Eb in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
We report the possible errors in Table
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
31

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: In Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics httph:
Answer: [Yes]
Justification:
Our research complies with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work primarily explores the performance of MLLMs (Multimodal Large Language Models) in origami scenarios and has no potential societal impact.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
32

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: As discussed in A all data public data authorized by the original websites and data sources, with no potential infringement risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have complied with all licensing and usage terms and acknowledged the data owners.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version the asset used and, possible, include URI
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
33

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper introduces new assets, and all data and code are publicly avail- able. Details about training, license, limitations, etc., are documented in compliance with submission guidelines.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/cood/mmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [Yes]
Justification: All details of manual annotation, including annotation instructions and com- pensation descriptions, provided in
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [Yes]
Justification:
The corresponding content is described in
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
34

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification:
We are working on evaluating MILMS and have described the MLLMs used.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy battps://neurre http://rris.cc//oofferrreeee///e Cc/Connferenccs/2225//LL. for what should or should be described.
35


Reviews:
[{"review_id": "20YOtK0UlK", "reviewer": "Reviewer_DNJx", "summary": "This paper introduces ORIGAMISPACE, a novel benchmark designed to evaluate the complex spatial reasoning capabilities of MLLMs. The authors argue that existing benchmarks fall short in testing multi-step reasoning under precise mathematical constraints. To address this, they use the domain of origami, which naturally involves sequential folding (multi-step reasoning) and adherence to strict geometric principles. The benchmark includes a dataset of 350 origami instances, each with a formal crease pattern (CP), folding process, and final shape. Based on this, they propose four challenging evaluation tasks, including multiple-choice questions and an interactive code generation task. Their evaluation of current MLLMs reveals significant challenges in this domain and demonstrates the potential of interactive and reinforcement learning methods to improve performance.", "questions": "1.  The use of origami is highly innovative. Could you elaborate on how you see the specific skills tested here (e.g., understanding crease patterns, mathematical constraints) transferring to other complex spatial reasoning domains like robotic manipulation or architectural design?\n\n2.  The manual collection and verification process for the 350 models must have been very labor-intensive. Have you considered semi-automated methods for expanding the dataset, perhaps by procedurally generating valid CPs and using the compiler to filter them?\n\n3.  The reinforcement learning results are very promising. Could you provide more insight into the policy the agent learns? For example, does it learn to satisfy certain geometric constraints (like Kawasaki's theorem) first before attempting more complex folds?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "The primary strength of this paper is its highly innovative and well-motivated choice of origami as a domain for evaluating spatial reasoning. This is a significant step beyond existing benchmarks. Origami inherently combines multi-step procedural understanding, 2D-to-3D spatial transformation, and strict, verifiable mathematical constraints (e.g., Kawasaki's and Maekawa's theorems), providing a rich and challenging testbed that current models struggle with. The comprehensive design of the benchmark is another major strength; it is not a single task but a suite of four distinct evaluations that probe different facets of spatial intelligence, from holistic shape prediction to fine-grained generative control.\n\nFurthermore, the technical contribution extends beyond a static dataset. The authors have improved an origami compiler and built an interactive environment for the code generation task. This is a crucial element that elevates the work, as it provides a platform not just for evaluation but for actively training and improving models via environmental feedback and reinforcement learning. The initial results from this RL exploration are very promising and open up a new research direction.\n\nWhile the work is excellent, a few minor points could be considered. The dataset size of 350 instances is modest, though this is understandable given the high complexity and manual verification required for each origami model, emphasizing quality over quantity. The direct transferability of skills learned on this benchmark to other real-world spatial reasoning domains like robotic manipulation remains an open question for future research, which is a common characteristic of specialized benchmarks.", "was_revised": true, "final_justification": "I thank the authors for their detailed rebuttal. They have provided sufficient answers to my questions regarding the transferability of the skills tested by ORIGAMISPACE, their future plans for dataset expansion, and the specifics of the policy learned by the reinforcement learning agent. \nMy assessment of this work remains unchanged. The paper's merits—such as its novel use of origami as a testbed for spatial reasoning and the comprehensive design of the benchmark—are clear. The authors' responses have adequately addressed the minor points I raised. Therefore, I will maintain my original positive score.", "rebuttal": "We sincerely appreciate your thorough review and valuable suggestions. We have provided detailed responses to all of your comments and questions. We hope that our clarifications will help you to better understand the specifics of our work and findings.\n\n> The use of origami is highly innovative. Could you elaborate on how you see the specific skills tested here (e.g., understanding crease patterns, mathematical constraints) transferring to other complex spatial reasoning domains like robotic manipulation or architectural design?\n\nWe believe that the core competency tested by ORIGAMISPACE—multi-step spatial reasoning under strict constraints—is universal across multiple domains. This ability is transferable because it simulates common spatial challenges. For example:\n\n* In **robotic manipulation**, the sequential folding and mathematical constraints of origami correspond to the multi-step manipulation planning for deformable objects and their physical limitations, respectively.\n* In **architectural design**, predicting a 3D model from a Crease Pattern (CP) is analogous to conceptualizing a building from a 2D blueprint, while the code generation task shares similarities with the parametric design logic in CAD.\n\nTo transfer this capability from origami to other domains, we believe the following technical aspects require attention:\n\n*   **Mapping of Representations:** Build a bridge to map problems from the new domain (e.g., point clouds for robotics, CAD files for architecture) into code or symbolic sequences that the model can better understand.\n*   **Generalization of Constraints:** Enable the model to understand and apply equivalent physical or design rules in the new domain (e.g., material non-stretchability, structural load-bearing requirements for buildings). This may involve constructing similarly strict constraints and interactive environments across different fields.\n*   **Unified Action Space:** For generative tasks like robotic manipulation, the \"folding\" action in origami needs to be abstracted into more universal \"manipulation primitives,\" allowing the model to learn a unified decision-making policy in the new domain.\n\n> The manual collection and verification process for the 350 models must have been very labor-intensive. Have you considered semi-automated methods for expanding the dataset, perhaps by procedurally generating valid CPs and using the compiler to filter them?\n\nThank you for your valuable suggestion; this is indeed a key focus for our future work.\n\nWe did attempt semi-automated methods in the early stages but found that the usability rate of Crease Patterns (CPs) generated by the model was extremely low (less than a 1% success rate) after manual review, while the cost of model generation and human verification was prohibitive. We believe that the capabilities of current Multimodal Large Language Models (MLLMs) are not yet sufficient for high-quality, semi-automated generation tasks. This will likely become feasible only after continued iteration and optimization of model versions in the future.\n\nTherefore, our current strategy is to collaborate with established origami communities and databases. We have reached cooperation intentions with global origami communities and the creators of several origami resource websites, such as the Origami Database. Through official partnerships, we plan to access their larger-scale, manually verified, and curated origami model data. This approach not only ensures the quality and accuracy of the data but also allows us to expand our dataset more efficiently beyond the current 350 instances.\n\nLooking ahead, we see these two approaches as complementary. As MLLM capabilities improve, we will revisit and deploy a semi-automated generation pipeline. We can also involve our community experts in a \"human-in-the-loop\" process to filter and refine the procedurally generated models, thereby balancing efficiency and quality.\n\n> The reinforcement learning results are very promising. Could you provide more insight into the policy the agent learns? For example, does it learn to satisfy certain geometric constraints (like Kawasaki's theorem) first before attempting more complex folds?\n\nIn the early stages of training, the model's compilation success rate (CPR) is low, and the primary feedback is a fixed negative penalty for compilation failures. During the middle phase, rewards related to compilation increase rapidly, especially for avoiding CP Code Syntax Errors (CSE) and Geometrically Impossible Folds (GIF). We believe this is because these two constraints are relatively simple, and the model learns to pass the compilation stage more effectively. In the later stages, the model's CPR gradually stabilizes, while the code quality scores, particularly Topological Similarity (TSS), show more significant improvement. The change in Geometric Similarity (GS) is less stable, possibly because its feedback signal is less refined and the required learning is more abstract. Additionally, influenced by the step penalty, the average number of interaction rounds required to complete a task slightly decreases. Overall, similar to what you suggested, the model first learns simpler constraints, such as Euler's formula for planar graphs and Maekawa's theorem, before attempting to learn more complex strategies.\n\nWe will open-source all our data, environments, training code, models, and complete training details in the short term.\n\n\nThank you again for your recognition of our work, as well as for your constructive comments and suggestions!"}, {"review_id": "tewzxsRv6C", "reviewer": "Reviewer_VSem", "summary": "This paper introduces ORIGAMISPACE, a novel benchmark dataset aimed at evaluating the multi-step spatial reasoning capabilities of multimodal large language models (MLLMs), particularly under mathematical and geometric constraints. The benchmark leverages origami-based tasks and consists of 350 instances, each including rich multimodal artifacts such as crease patterns, flat patterns, folding sequences, and final folded shapes. The authors define four tasks: Pattern Prediction, Multi-step Spatial Reasoning, Spatial Relationship Prediction and End-to-End CP Code Generation. They also propose an interactive environment for CP code generation and explore reinforcement learning (RL) techniques to enhance model training.", "questions": "1. Could you explain further how the mathematical constrains contribution in the proposed multi-step geometric and layering reasoning? \n2. What is the impact of mathematical constrains? During each step, will be possible of precisely satisfying all mathematical constraints? if not, what is the meaning of adding these constrains? \n3. Could you elaborate how RL is set up in the experiments?", "limitations": "1. data sets limitations.\n2. Error analysis", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: \n1. The use of origami as a testbed is creative and well-motivated. It naturally involves complex spatial transformations, offering a rich ground for evaluating MLLMs beyond typical vision-language tasks.\n2. The four distinct tasks reflect a thoughtful design that targets various dimensions of spatial reasoning.\n3. Mathmatical constraint is a key strength compared to existing loosely defined visual reasoning.\n\nWeaknesses:\n1. The dataset contains only 350 instances, which may limit statistical robustness and hinder the training of larger models.\n2. The mention of RL is promising, but it fails to offer details on the task structure, reward design, or preliminary performance. It is hard to evaluate the feasibility and contributions of RL here.\n3. The mathematical constraints are not well illustrated in the paper. I think it is one of you biggest contributions while the papers fails to convince reader how these constraints affect the overall performance of this problem.", "was_revised": true, "final_justification": "Thank you for efforts made during Rebuttal. I am quite satisfied with the revision and explanation given, especially the importance of the math constraints and RL details.", "rebuttal": "We sincerely appreciate your thorough review and valuable suggestions. We have provided detailed responses to all of your comments and questions. We hope that our clarifications will help you to better understand the specifics of our work and findings.\n\n> The dataset contains only 350 samples, which may limit statistical robustness and hinder the training of larger models.\n\nWe believe that for this task, the quality of the data is more important than the quantity. The process of origami is extremely rigorous, especially since it requires strict compilation via CP code, where even a minor error can lead to a vastly different result. We previously attempted a semi-automated approach with manual filtering, but the success rate was very low.\n\nTherefore, we have partnered with established origami communities and databases. We plan to access their larger-scale, manually verified, and curated origami model data through official collaborations. This approach not only ensures data quality and accuracy but also allows us to expand our dataset more efficiently beyond the current 350 instances.\n\n> Regarding Mathematical Constraints\n\nCertainly, we will further elaborate on the impact and function of mathematical constraints to hopefully resolve your questions.\n\nFirst, you can think of mathematical constraints in two forms. The first is **\"hard constraints,\"** as described in Section 3.2 of the paper. These include CP Code Syntax Error (CSE), Geometrically Impossible Fold (GIF), Paper Self-Intersection (PSI), and Ambiguous Fold State (AFS). These four hard constraints must be satisfied; otherwise, the compilation will fail. The second is **\"soft constraints,\"** which refers to the Constraint Satisfaction mentioned in Section 4.4. At this stage, when the CP Code is compilable, the constraints evaluate the ability to achieve more detailed and self-consistent spatial intelligence. This primarily includes two parts: the verification of specific advanced origami constraints and local flat-foldability conditions, as described in Appendices D.11, D.12, and D.13. The former includes Taco-Taco (TT), Taco-Tortilla (TTo), and Transitivity constraints. During evaluation, the system extracts these advanced constraints from both the generated model and the reference model and then compares the similarity of the two sets. The similarity is calculated as a weighted sum of the overlap of the constraint sets (Jaccard similarity) and the similarity of the constraint counts. The latter assesses a more fine-grained folding state through local vertex metrics.\n\nTo make this easier to understand, here are two examples:\n\n* **Paper Self-Intersection (PSI) - Hard Constraint:** This violates the physical principle that \"different parts of an object cannot occupy the same space simultaneously.\" When the compiler deduces the relative positions and stacking order of the paper's parts, it will report an error if it finds any regions that would overlap or penetrate.\n* **Taco-Tortilla (TTo) - Soft Constraint:** \"Taco\" refers to a folded flap of paper, while \"Tortilla\" represents a flat, unfolded surface. The Taco-Tortilla constraint is used to specify this \"one-folded, one-flat\" stacking order. This decision is critical to the final model's appearance and function. For instance, it might determine whether a pocket is on the outside or inside of a garment, or whether a part of a model is convex or concave. A model that correctly handles TTo constraints demonstrates that it understands more refined spatial operations like \"hiding\" and \"covering.\"\n\nNow, to answer your two questions directly:\n\n**How exactly do the mathematical constraints work?**\n\nYou can understand it as the compiler performing complex mathematical derivations to calculate whether the CP code can conform to various constraint states and then returning relevant information. This provides a clear, computable feedback signal and evaluation result.\n\n**What is the impact of the mathematical constraints? Is it possible to satisfy all of them at each step?**\n\n* The impact of **hard constraints** is that a CP diagram that fails to meet them cannot be folded at all.\n* The impact of **soft constraints** is that a CP diagram that fails to meet them will have an incorrect folding state in some spatial areas compared to the reference.\n\nThe role of hard constraints is to assess whether the current state is foldable. The role of soft constraints is to provide a partial reward signal *after* foldability is confirmed. Therefore, in a multi-step folding process, it is not necessary to satisfy all constraints at every step, as adjustments can be made based on the compiler's feedback. The significance of the constraints is twofold: 1. They provide our evaluation with clear standards and environmental feedback. 2. They provide a simulation that is perfectly consistent with real-world physics.\n\n> Reinforcement Learning (RL) Setup\n\nOf course. We will provide more details and plan to open-source all our data, environment, training code, and models in the short term.\n\n**Task Structure**\n\n* The agent (MLLM) engages in iterative, multi-turn interactions with the environment (the compiler). The compiler returns either a text-based success message or specific error feedback.\n* The model first performs planning to generate initial CP code, then reasons through interaction with the environment and decides on the next action, such as adding or deleting a crease.\n* This \"generate-feedback-modify\" loop continues for a maximum of 10 interaction rounds, as set in our experiments.\n\n**Training Algorithm & Data**\n\n* We employed the **TRICO algorithm**, an efficient multi-turn reinforcement learning algorithm based on PPO, designed for MLLMs.\n* The training data consisted of 471 independent origami datasets.\n\n**Training Hyperparameters**\n\n* Appendix E provides detailed training parameters, including: training for **10.2 hours on 16 H100 GPUs**, with Actor and Critic learning rates of **1×10⁻⁶** and **1×10⁻⁵**, respectively, among other relevant parameters.\n\n**Performance Change**\n\n* In the **early stages of training**, the model's compilation success rate (CPR) is low, with the primary feedback being a fixed negative penalty for compilation failures.\n* In the **middle phase**, compilation-related rewards increase rapidly, especially for avoiding CP Code Syntax Error (CSE) and Geometrically Impossible Fold (GIF). We believe this is because these two constraints are relatively simple, and the model becomes proficient at passing the compilation stage.\n* In the **later stages**, the model's CPR gradually stabilizes, while code quality scores show more significant improvement, particularly Topological Similarity (TSS). The change in Geometric Similarity (GS) is less stable, possibly because its feedback signal is less refined and the required learning is more abstract. Additionally, influenced by the step penalty, the average number of interaction rounds required to complete a task slightly decreases.\n* Overall, the model first learns simpler constraints, such as Euler's formula for planar graphs and Maekawa's theorem, before attempting to learn more complex strategies.\n\nWe will open-source all our data, environments, training code, models, and complete training details in the short term.\n\nThank you again for your recognition of our work, as well as for your constructive comments and suggestions!"}, {"review_id": "GVb7jweGE8", "reviewer": "Reviewer_mktL", "summary": "This paper introduces a spatial reasoning benchmark for MLLMs grounded in origami folding. The benchmark defines four tasks: final shape prediction, multistep spatial reasoning, spatial relationship prediction, and CP Code prediction, which symbolically encodes crease patterns for origami compilation. The authors also introduce four evaluation directions for CP Code quality and use the benchmark to evaluate several foundational models using in-context learning, environment learning, and reinforcement learning.", "questions": "Questions can be found in the weakness section.", "limitations": "No", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths \n\nThe use of origami as a grounding mechanism for spatial reasoning is novel and well-motivated. It provides a mathematically constrained yet visually interpretable setup for multistep reasoning, which avoids the overly synthetic feel of many spatial datasets. The benchmark is thoughtfully constructed with attention to dataset realism and bias reduction. The paper also explores multiple prediction setups (in-context, environment learning, RL), which makes the evaluation broad and informative for understanding MLLM capabilities across paradigms.\n\nWeaknesses\n\n1. The authors are urged to add OpenAI’s recent reasoning-capable multimodal models (o3)  here, given their strong performance on comparable tasks. Even a qualitative comparison would be useful.\n\n2.  Explanations for what each crease type (mountain, valley, etc.) does or what the full syntax rules are for compiling a valid code are unclear This makes the CP Code prediction task hard to interpret. The paper would benefit from a more detailed breakdown or a concrete example of CP Code semantics.", "was_revised": true, "final_justification": "I raised some questions regarding missing relevant comparisons and more analysis of the parts of the presented benchmark. The authors' response addresses the questions satisfactorily. And I have raised my rating to \"Accept\".", "rebuttal": "We sincerely appreciate your thorough review and valuable suggestions. We have provided detailed responses to all of your comments and questions. We hope that our clarifications will help you to better understand the specifics of our work and findings.\n\n> Given that OpenAI's recent multimodal model (o3), which has reasoning capabilities, has performed exceptionally well in similar tasks, it is recommended that the authors include this model here. Even a qualitative comparison would be helpful.\n\nThank you for pointing this out. We have added the experimental results of o3 on different spatial reasoning tasks, as shown in the tables below:\n\n| Pattern Prediction | Multi-step Spatial Reasoning | Spatial Pose Localization | Layering Relationship | Geometric Change |\n|:------------------:|:----------------------------:|:-------------------------:|:---------------------:|:----------------:|\n| 44.01 | 52.89 | 49.23 | 51.86 | 46.95 |\n\n| | **Compilation** | | | | | **Quality** | | | | |\n|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n| **Setting** | **CSE** | **GIF** | **PSI** | **AFS** | **CPR** | **TSS** | **GS** | **CS** | **FFS** | **Total** |\n| **In-context learning** | 94.27 | **62.55** | 51.26 | **46.95** | **31.12** | **52.09** | 43.26 | **44.87** | **39.11** | **44.20** |\n| **Environmental learning** | **100** | **93.51** | 91.29 | 83.50 | **69.44** | **61.75** | 51.88 | 55.08 | **46.92** | 53.65 |\n\nAs can be seen, o3 performs very well on ORIGAMISPACE, comparable to gemini-2.5-pro. We will add these results in the next version.\n\n> The function of each crease type (mountain fold, valley fold, etc.) and the complete syntax rules for writing valid code are not clearly explained. This makes the CP code prediction task difficult to understand. The paper would be more helpful if it could break down the semantics of CP code in more detail or provide a specific example.\n\nCertainly. Below is an introduction to the syntax rules for CP code:\n\nA complete CP code consists of four parts: vertex coordinates (`vertices_coords`), edge-to-vertex relationships (`edges_vertices`), edge type assignments (`edges_assignment`), and face-to-vertex relationships (`faces_vertices`).\n\n* **`vertices_coords`**: This defines the positions of all key points on the paper, represented by the 2D coordinates of each point.\n* **`edges_vertices`**: This defines the line segments, including the paper's boundaries and the internal creases. Each edge is defined by connecting the indices of two vertices.\n* **`edges_assignment`**: This is the most critical step, representing the property of each \"edge.\" This includes \"B\" (Boundary), \"M\" (Mountain fold), \"V\" (Valley fold), and \"F\" (Flat fold).\n* **`faces_vertices`**: This defines all the 2D \"faces\" that are partitioned by the edges and creases. Each face is defined by the sequence of vertices that make up its contour.\n\nIn the next version of the paper, we will provide a more detailed explanation of the CP code, along with an analysis of specific examples.\n\nThank you again for your recognition of our work, as well as for your constructive comments and suggestions!"}, {"review_id": "1waERBu2Pd", "reviewer": "Reviewer_GYhc", "summary": "This paper introduces ORIGAMISPACE, a novel benchmark designed to evaluate the multi-step spatial reasoning capabilities of Multimodal Large Language Models (MLLMs), with a particular focus on their ability to adhere to precise mathematical constraints. The authors argue that existing benchmarks often focus on static scenes or simple spatial transformations, lacking the complexity of sequential, constrained, and dynamic reasoning found in real-world tasks. To address this gap, they propose using the domain of origami, which inherently involves a sequence of dependent folding operations governed by strict geometric rules.\n\nExperiments on a wide range of state-of-the-art open and closed-source MLLMs (including GPT-4o and Gemini 2.5-pro) demonstrate that ORIGAMISPACE is highly challenging. All models perform significantly below human expert levels, and the results highlight the profound difficulty MLLMs have with fine-grained geometric reasoning and constraint satisfaction. The work also shows that interactive learning with compiler feedback dramatically improves performance over standard in-context learning.", "questions": "Dataset Scale and Robustness: The dataset size of 350 test instances is modest for a benchmark of this nature. Could the authors comment on the statistical significance of the reported performance gaps between models? Have you considered or are you planning methods to scale the dataset, perhaps through procedural generation of simpler origami CPs and their corresponding folded forms, to further improve the benchmark's robustness?\n\nInput to Code Generation Task: For the End-to-End CP Code Generation task (Task 4), the input is described as a \"compiled flat layout and an image of the folded shape.\" Providing the compiled flat layout seems to give the model a significant amount of information that is not typically available in a real-world \"reverse engineering\" scenario. Could you clarify the motivation for this choice? Would it not be a more challenging and perhaps more realistic task to generate the CP code from only the folded shape image, possibly augmented with natural language instructions?\n\nGeneralization of RL Agent: The reinforcement learning results are very promising, showing that an agent can learn to interact with the compiler to improve its output. Could you provide more insight into the generalization capabilities of the trained Qwen-VL-32B agent? For instance, how well does it perform on origami designs of a complexity level or type not seen during its training? Is the learned policy mostly correcting minor syntax errors, or is it demonstrating a deeper, more generalizable understanding of geometric constraints?\nCompiler and Environment Availability: A benchmark's utility is greatly enhanced by its accessibility. Will the optimized origami compiler, the interactive environment, and the full evaluation suite be made publicly available alongside the dataset?\n\nA positive response to these questions, particularly clarifying the design choice in Question 2 and discussing the potential for scaling in Question 1, would further strengthen this already excellent paper.", "limitations": "Yes, the authors have adequately addressed the limitations of their work in Appendix F. They correctly identify the modest dataset scale, the open question of transferability to other domains, and the potential to expand the suite of evaluation tasks as avenues for future work. This is a thoughtful and honest assessment.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\nSignificance and Originality: The paper identifies a clear and critical weakness in current MLLM evaluation: the lack of benchmarks for multi-step, mathematically-grounded spatial reasoning. The choice of origami as a domain is both highly original and exceptionally well-suited to this problem. It provides a task that is intuitive to understand yet programmatically verifiable, procedurally complex, and rich with implicit and explicit constraints. This is a far more rigorous test of spatial \"imagination\" than what is offered by most existing benchmarks.\n\nQuality of the Benchmark Design: The benchmark is exceptionally well-designed.\n\nThe four tasks are diverse, probing different facets of spatial reasoning from high-level pattern recognition to low-level geometric prediction and generative construction.\n\nThe use of an origami compiler is a major strength. It provides a formal, non-negotiable ground truth for evaluation, which is a significant step up from relying on human consensus or simple metric comparisons. The detailed error feedback system (CSE, GIF, PSI, AFS) is very well-thought-out and enables the novel interactive evaluation setting.\n\nThe evaluation metrics for the CP code generation task (TSS, GS, CS, FFS) are comprehensive, covering topological structure, geometric similarity, constraint satisfaction, and the final folded state. The level of detail in Appendix D is commendable and demonstrates a deep engagement with the problem.\n\nQuality of the Experimental Evaluation: The empirical study is thorough and compelling.\n\nThe authors evaluate a wide and very current set of both open-source and proprietary SOTA MLLMs, making the results highly relevant.\nThe inclusion of both layperson and expert human performance provides essential context and clearly demonstrates the difficulty of the tasks and the significant gap that remains for AI.\n\nThe comparison between in-context learning, environmental learning (with interaction), and reinforcement learning provides a key insight: that iterative refinement based on structured feedback is a powerful and necessary paradigm for solving such complex tasks.\n\nClarity and Reproducibility: The paper is extremely well-written, logically structured, and easy to follow. The extensive appendices are a model of good practice; they provide exhaustive detail on the dataset, annotation rules, compiler errors, and evaluation metrics, which will be invaluable for any researcher wishing to build upon this work.\n\nWeaknesses:\n\nDataset Scale: The primary weakness is the relatively modest scale of the dataset (350 test instances, with an additional 471 used for RL training). While the quality of each instance is clearly very high due to meticulous curation, this size might raise concerns about statistical power and the potential for models to overfit, particularly in the reinforcement learning setting.\n\nScope and Transferability: While origami is an excellent proxy for constrained spatial reasoning, the paper could benefit from a slightly broader discussion on the expected transferability of skills developed on ORIGAMISPACE to other domains like robotics (e.g., cloth folding, assembly), molecular biology (protein folding), or architectural design. This is briefly acknowledged in the limitations but could be expanded upon in the main text to strengthen the paper's impact statement.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your thorough review and valuable suggestions. We have provided detailed responses to all of your comments and questions. We hope that our clarifications will help you to better understand the specifics of our work and findings.\n\n> Regarding Dataset Scale and Robustness\n\nRegarding the statistical significance of the performance differences between models, we report in Table 1 the average of three independent runs, and each model's performance score is appended with a \"±\" value. Regarding the data regularity, thank you for your valuable suggestions. This is indeed a focus of our future work.\n\nWe have attempted semi-automated methods but found that the usability of CP (Crease Pattern) generated by the model is extremely low when reviewed by humans (success rate less than 1%). The cost of using the model to generate and then having humans verify is also very high. We believe that the current MLLM's capabilities are not yet sufficient to handle high-quality semi-automated generation tasks, which will require continuous iteration and optimization of the model version to be realized in the future.\n\nTherefore, our current strategy is to collaborate with established origami communities and databases. We have reached cooperation intentions with creators from several resource websites, including Global Origami Community and Origami Database. We plan to officially cooperate with them to bring in a larger volume of origami model data that has been verified and organized by humans. This approach not only ensures the quality and accuracy of the data but also allows us to expand the dataset more efficiently on the basis of the existing 350 examples.\n\nLooking forward, we believe these two methods are complementary. With the improvement of MLLM capabilities, we will re-examine and deploy semi-automated generation processes, which will allow our community experts to participate in the \"human-in-the-loop\" process, screening and optimizing the procedurally generated models, thereby balancing efficiency and quality.\n\n> Input to Code Generation Task\n\nWe agree with your point of view. Providing the model with a \"Compiled Flat Pattern\" compared to a \"Folded Shape Image\" does indeed reduce the \"real-world\" inverse engineering difficulty of the task. However, our goal is to provide a clearer final result, because a \"Folded Shape Image\" is usually a photo or a 3D render, which itself carries a large amount of **perceptual uncertainty**, such as viewpoint, lighting, shadows, material texture, and non-rigid deformation. Requiring a model to directly generate precise CP code from such an image would mix the core **spatial reasoning** challenges with complex **3D visual perception and reconstruction** challenges.\n\nWe completely agree with your view that generating CP code from a \"Folded Shape Image\" (possibly with natural language instructions) is a more challenging and more intuitive inverse engineering task. We believe this represents a more difficult and more valuable future research direction.\n\nOur ORIGAMISPACE benchmark can be seen as a **key intermediate step** in achieving this grand goal. It pioneers the systematic evaluation and improvement of the ability of Multi-modal Large Language Models (MLLM) in core **geometric constraints and multi-step spatial reasoning** in a well-controlled, perception-free environment. Once a model demonstrates sufficient capability on this foundational task, the next step is to challenge more open and realistic ultimate tasks.\n\n> Generalization of RL Agent\n\nThank you for pointing this out. Regarding generalization, our experimental setup itself is a kind of test. The RL agent is trained on a dataset containing 471 examples. The final results (Table 2) are evaluated on the ORIGAMISPACE benchmark test set, which contains 350 examples. All the origami designs in the test set are **completely new** to the RL agent during the training process.\n\nRegarding the strategy, in the early stages of training, the model's compilation success rate (CPR) is low. The main feedback comes from the negative penalty for compilation failures. During the middle stage of training, the compilation-related rewards increase rapidly, especially for CP Syntax Error (CSE) and Geometrically Impossible Fold (GIF). We believe this is because these two types of constraints are relatively simple, allowing the model to learn to pass compilation relatively well. In the later stages, the model's CPR gradually stabilizes, but the code quality score, especially the topological similarity (TSS), shows a more noticeable increase, while the geometric similarity (GS) does not change much. This may be because the feedback for the latter is not well-designed, and the required capabilities are more abstract. Furthermore, constrained by the penalty for step length, the number of interaction rounds required to complete the task will decrease. Overall, the model will first learn simpler strategies, such as Euler's formula for planar graphs and Maekawa's theorem, and then try to learn more complex strategies.\n\n> Accessibility of Compilers and Environments\n\nYes, we will publicly release all data, environments (compilers), training and evaluation code, and models in the short term.\n\nThank you again for your recognition of our work, as well as for your constructive comments and suggestions!"}]

Author rebuttal:

Meta-review: This paper introduces a novel dataset designed to evaluate the ability of multimodal large language models to perform multi-step spatial reasoning. The authors propose several benchmark tasks, including Pattern Prediction, Multi-step Spatial Reasoning, Spatial Relationship Prediction, and End-to-End CP Code Generation, and assess the performance of multiple foundational models on these tasks.

All the reviewers strongly appreciate the creation of the benchmark and agree that it evaluates a uniquely challenging ability of foundation models. Some minor issues were raised by the reviewers, but these were well addressed by the authors during the rebuttal. One only concern is the size of the proposed dataset is small.

Given the strong agreement among the reviewers, AC recommends an accept for this paper.
Decision: Accept (spotlight)