Error Notebook-Guided, Training-Free Part Retrieval in 3D CAD Assemblies via Vision-Language Models
Authors: Yunqing Liu
Nan Zhang
Zhiming Tan
Venue: ICLR 2026 Poster (2026)
Keywords: Vision–Language Models
Error Notebook
Specification-Aware Part Retrieval
CoT Reasoning
Human-Preference Dataset
Primary area: applications to robotics, autonomy, planning
TL;DR: Training-free framework that uses Error Notebooks + RAG to guide VLMs for specification-aware part retrieval in CAD assemblies, yielding significant accuracy gains (e.g., +23.4% on human-preference data).

Abstract: Effective specification-aware part retrieval within complex CAD assemblies is essential for automated engineering tasks. However, using LLMs/VLMs for this task is challenging: the CAD model metadata sequences often exceed token budgets, and fine-tuning high-performing proprietary models (e.g., GPT or Gemini) is unavailable. Therefore, we need a framework that delivers engineering value by handling long, non-natural-language CAD model metadata using VLMs, but without training. We propose a 2-stage framework with inference-time adaptation that combines corrected Error Notebooks with RAG to substantially improve VLM-based part retrieval reasoning. Each Error Notebook is built by correcting initial CoTs through reflective refinement, and then filtering each trajectory using our proposed grammar-constraint (GC) verifier to ensure structural well-formedness. The resulting notebook forms a high-quality repository of specification-CoT-answer triplets, from which RAG retrieves specification-relevant exemplars to condition the model's inference. We additionally contribute a CAD dataset with human preference annotations. Experiments with proprietary models (GPT-4o, Gemini, etc) show large gains, with GPT-4o (Omni) achieving up to +23.4 absolute accuracy points on the human-preference benchmark. The proposed GC verifier can further produce up to +4.5 accuracy points. Our approach also surpasses other training-free baselines (standard few-shot learning, self-consistency) and yields substantial improvements also for open-source VLMs (Qwen2-VL-2B-Instruct, Aya-Vision-8B). Under the cross-model GC setting, where the Error Notebook is constructed using GPT-4o (Omni), the 2B model inference achieves performance that comes within roughly 4 points of GPT-4o mini.

Published as a conference paper at ICLR 2026
ERROR NOTEBOOK-GUIDED. TRAINING-FREE PART RETRIEVAL IN 3D CAD ASSEMBLIES VIA
VISION-LANGUAGE MODELS
Yunqing Liu, Nan Zhang & Zhiming Tan'
Fujitsu Research & Development Center
Shanghai, China (liuyanqing, zhangnan, zhmtan| (fujissu..uuuuuooo
The problem in question:
Retrieve (symbolic) text from long, non-nsturalllanguag metadata using a naturakkanguage specification.
Long, non-eatunal-annuuaee metadata
Assembly 3D CAD Model
Specification
VLM
CAD Part Identifler
Natural-taguuage specification
RAG
(Symbolic) text
Example:
Specification
Assembly
The cybediicall protnarie dused the recue leasing
Assembly CAD Model (luctratio))
part identifier
ENOSEC:
DacD0uckd50k
18066226-07500 1eeob----
$3C-0NEEMOnR-- AL (hORON sacies Heeosnnie KON werre NA MONN DEA4LIOONU
0ac00a041000
IDBBUCES
225.022--..... IOSOUITES #501//500/3110
NOT HITML LOCFINED, TRANSFORMAT ONIS. MONAAGGAA5 (AAKHORMWAI 2005s///
41:00SH CILDUR inset Akummmas Aneillen Mimee (twp://00//2240000000000
Figure 1: Scope of work. The goal is retrieve symbolic part identifiers from long, non-natural- language assembly CAD model using natturalllanguage specification. Our two-stage VLM pipeline first converts CAD model part information into geometric descriptions (Ist VLM), then performs specification-aware reasoning (2nd VLM) assisted by Error Notebook with RAG.
ABSTRACT
Effective specification-awaare part retrieval within complex CAD assemblies is essential for automated engineering tasks. However, using LLMs/VLMs for this task challenging: the CAD model metadata sequences often exceed to- ken budgets, and fine-tuning high-performing proprietary models (e.g.. GPT or Gemini) is unavailable. Therefore, need a framework that delivers engineer- ing value by handling long, non-natural--anguaar
CAD model metadata using VLMs, but without training. We propose 2-stage framework with inference- time adaptation that combines corrected Error Notebooks with RAG to sub- stantially improve VLM- part retrieval reasoning. Each Error Notebook built by correcting initial CoT's through reflective refinement, and then filtering each trajectory using our proposed grammar--costrrint (GC) verifier to ensure structural foll-formed.ess. The resulting notebook forms high-quality reposi- teighttualttycpeeeee tory of specification-Col Careiiio-------wweeecc triplets, from which RAG retrieves specification- relevant exemplars to condition the model's inference. We additionally contribute
"Corresponiing author.

---

Published as a conference paper at ICLR 2026
CAD dataset with human preference annotations. Experiments with pro- prietary models (GPT-40, Gemini, etc) show large gains, with GPT-4o (Omni) achieving up to +23.4 absolute accuracy points on the human- human-preference bench- mark. The proposed GC verifier can further produce up to +4.5 accuracy points. Our approach also surpasses other training-free baselines (standard few- shot learning, self-consisteccy) and yields substantial improvements also for open-source VLMs (Qwen2-VL-22--nstruus
Sya--ision--B). Under the cross-model GC setting, where the Error Notebook is constructed using GPT- 4o (Omni), the 2B model inference achieves performance that comes within roughly 4 points of GPT-40 mini.
INTRODUCTION
Recent efforts demonstrate the promise of LLMs/VLMS in the engineering design and manufac- turing domain. For instance, Alrashedy et al. (2025) applied LLMs to generate Computer-Aided Design (CAD) code from natural language descriptions, which can then be executed to render 3I) objects. Such approaches show that general-purpose models can automate the CAD modeling pro- cess. Additionally, WW et al. (2021) developed deep generative models to create 3D CAD structures directly (e.g., by modeling sequences of CAD operations), hinting at the potential of combining lan- guage and vision for CAD design tasks. Recent work has also shown that LLMs can assist in design ideation and automation, such as guiding parametric modeling, generating shape descriptions, and integrating CAD workflows with natural language instructions (Li et al. (2025); Zhhan et al. (2025); Kumar et al. (2055). These studies further highlight the versatility of LLMS in supporting cre- ative and engineering tasks within CAD environments. Despite this progress, a critical task remains challenging for LLMs/VLMs: ppecificationddriee part retrieval within complex CAD assemblies. Each CAD assembly (e.g., stored as STEP file) can contain dozens of parts described by lengthy, non-natural language metadata. Retrieving specific parts that match a given design specification or relational description is essential for automated design verification and other downstream tasks, yet directly prompting LLMs or VLMs for this often yields poor results. A primary obstacle is the extreme sequence length of assembly data, which can exceed current model token limits. Even if the STEP data is processed, for example, into images, we found that off-the-shelf models still fre- quently misidentify parts because the task requires fine-grained reasoning about part relationships and attributes.
tuninn a model on this task could improve performance, but it sometimes impractical: many models (e.g., GPT or Gemini) are proprietary or lack fine-tuning access, and training a custom model would demand significant computational resources. However, certain training techniques for LLMs and VLMs may serve as inspiration for enhancing the performance of methods that do not require training. For example, in the mathematical domain, Pan et al.) (2025) fine-tuned a model on spe- cial dataset of erroneous reasoning chains paired with corrected solutions. This taught the model to reflect on and fix its own errors during generation. More broadly, research on reflection and self- correction in LLMs highlights several strategies that could inspire training- free framework. One line of work leverages external critics or verifier models to provide feedback on intermediate rea- soning steps, guiding the model away from incorrect trajectories (An et al.) (203) Li et al. (2023), Shinn et al. (2023). Another line explores intrinsic self-correction, where models are fine-tuned on specially constructed datasets that pair erroneous reasoning trajectories with their corrections
(Weng et al. (2023); Yang et al. 2025); y; Zhang et al.] (2024); Han et al.] (2024); Yan et al. (2024); Tong et al. (2024): Renze (2024)). To collect such data, prior studies often introduce errors by raising the sampling temperature or by sampling across multiple models, ensuring that the training set contains both flawed and corrected reasoning paths (Xi et al. (2024)). These approaches enable models to revise their reasoning, and prevent error accumulation. Although our method does not involve weight updates, we draw inspiration from these techniques. In particular, the idea of cou- pling flawed reasoning with reflection and correction motivates our Error Notebook design. Instead of using fine-tuning to encode these revision patterns into the model parameters, we operationalize them at inference time: by retrieving analogous past samples and their corrections, we provide the model with direct exemplars of reflection, thereby encouraging more reliable reasoning without any additional training cost.
2

---

Published as a conference paper at ICLR 2026
As shown in Figure we introduce novel inference-phasee strategy for vision-language part trieval in 3D CAD assemblies. Rather than training or fine-tuning new model, our approach en- hances reasoning on-the-fly through retrospective error analysis and serrevallaugmentee guidance. Central to our method the Error Notebook, a mechanism that refines model reasoning at inference time by recording and organizing corrected reasoning trajectories. For each new assembly query, we retrieve analogous cases from the Error Notebook and provide them as few-shot exemplars to guide the model's s chain-of-thought (CoT) using a eetrieall-augmente generation (RAG) strategy. The grammar conssraatt (GC) verifier leads to further performance gains on the part retrieval task. We evaluate several state-of-the afrtthe-rrt VLMs (including GPT-4 variants and Gemini models) and open- source small VLMs on our benchmark. In summary, our contributions are as follows:
(1) We propose training-free reasoning framework that combines the Error Notebook and RAG for VLM inference. Importantly, our method surpasses traditional training-free inference-time ap- proaches (standard few-shot, self-consistency) and further demonstrates strong improvements even on open-source models (c.g., Drren2-VL-2B-Imae
and Sya-Vision-8BB.
(2) We introduce grammar-constraint (GC) verifier to ensure the structural validity of corrected CoTss used in the Error Notebook. This consistently improves the quality of retrieved exemplars, yielding further gains across all evaluated VLMs.
(3) We reconstruct multimodal CAD assembly dataset with relational specifications and human- preference annotations, consisting of 752 assemblies with part counts ranging from 2 to 249.
(4) From the perspective of the engineering value, we design an effective two-stage VLM strategy that first generates part descriptions and then uses these descriptions for retrieval, thereby overcom- ing the challenge of processing extremely long STEP file inputs.
2
METHODOLOGY
Section 2.1 provides the problem formulation and two-stage (Ist and 2nd VLM) process of our part retrieval framework, illustrated in Figure Section 2.2 describes the dataset construction and human preference annotation process. We introduce the construction of the Error Notebook in Section 2.3 which is used for the 2nd VLM reasoning. We further describe the GC mechanism for refining the Error Notebook in Section 2.4 Finally, the complete inference procedure that integrates the Error Notebook with RAG presented in Section 2nd summarized in Figure (b).
2.1 PROBLEM OVERVIEW AND PROPOSED PART RETRIEVAL FRAMEWORK
Given a 3D CAD assembly A consisting of n parts (P1, P2....,Pall and natural language as- sembly specification S, our goal to retrieve the subset of parts Pr C (Pi..... Pn that satisfy the specified relation described in S. The retrieval process is formulated two-stage VLM reasoning pipeline:
Stage 1 (Ist VLM): Part Description Generation. For each part P. in assembly, we provide both the image of the complete assembly Tassembly and the image of the individual part Ip, as input to model (-). The decomposition of the assembly into individual components is implemented using Python libraries, as illustrated in Figure A.7 The model is prompted to generate a concise and discriminative sentence d; describing P; with respect to the assembly context:
d, = faese (Tassembly Ir,,
p prompt
(1)
where promptaese is a designed instruction that encourages the model to focus on salient geometric and functional features. Repeating this process for all parts yields set of pert-level descriptions (d which are organized into a structured JSON with one entry per part, where the number of
entries equals the number of parts in the assembly.
Stage (2nd VLM): Specification-Aware Part Retrieval via CoT Reasoning. We first aggregate the part-level outputs from Stage into a structured mapping (JSON) from part filenames (identi- fiers) to their corresponding descriptions, denoted as D = {filename: : de H- Given the assem- bly image Isssembly, the aggregated description mapping D. and the specification S, we prompt the model freu(.) to identify the relevant parts:
p freat (Iussenbly D, S, prompt
(2)
3

---

Published as a conference paper at ICLR 2026
Specificaton
Descriptions part given aesembbly
modal Info
and suarch sccldd
Arc sarcal not with proclar base
STEP procssaing
book with thaepute bas draaurrccee Ane calic
BOJB
Pant
Assomibly
Viualasionn amsteaoo
assembly
comppn
RAG
Error Notebook
The helice cal rud be secuale sepad and cartardd stbbe fig mestt
patta Baiggss
Nervic
Output
Figure 2: Overview of the (a) dataset construction pipeline and (b) Error Notebook RAG- based inference process. (a) For each assembly, VLM is used to generate concise and discrimina- tive natural language descriptions for every part. Subsequently, the model generates assembly-level specification sentences describing the required relationship. To support human annotation, the spec- ified parts are merged and visualized as CAD model image. (b) Following the Ist VLM, at the 2nd stage, given the assembly specification, the system retrieves the most relevant examples from the Error Notebook according to the assembly specification, incorporates these as few-shot exemplars, and then performs see-bby-ttep reasoning to generate the final answer.
where prompt requires the model reason step-by-step (CoT) and produce both an interpretable rationale and the final answer in the form of subset of part filenames.
2.2 DATASET CONSTRUCTION AND HUMAN ANNOTATION
Our study is based on the Fusion 360 Gallery Dataset (Willis et al. 202 a: Lambourne et al., 202T Specifically, we utilize the Assembly Dataset, subset of the Fusion 360 Gallery Dataset, which comprises multi part CAD assemblies enriched with detailed information regarding joints, contact surfaces, and holes. For this work, we focus on archive al.0.0000 _0,00, which contains 752 assemblies (the Fusion 360 Assembly Dataset is divided into multiple sets whose assembly counts, and CAD model types are highly consistent). Each assembly project within this archive includes a single assembly and the corresponding part information (such as PNG images, STEP files, and additional metadata). The PNG files provide 2D image representations of the 3D models. STEP files (Standard for the Exchange of Product model data), as defined by ISO 10303, are neutral file formats that facilitate the exchange of 3D model data across different CAD software platforms, preserving geometry, structure, and other essential attributes.
Figure (a) shows the overview of the dataset construction pipeline. To begin, we catalog all part names and count the number of parts per assembly. Next, we utilize the GPT-4o (Omni) to generate concise and descriptive noun phrases for each individual part. For each part, we provide both the overall assembly image and the part image input, so that the model can generate the part description with full awareness of the assembly context. Each phrase is intended to succinetly describe the part's primary shape and distinguishing features, thereby allowing it be differentiated from other parts within the same assembly. We provide several few-shot examples to guide the model toward generating higher-quality descriptions. Figure presents the prompt.
Subsequently, we leverage GPT-4o (Omni) again to further generate high-level specifications for the 3D assemblies. Each specification is focused on relationships between selected parts within the assembly. The process is as follows: First, the model reviews the assembly image and the corre- sponding list of part descriptions. then selects two part descriptions that are most likely to exhibit a direct physical, spatial, or functional relationship (e.g.. fit, mounting, alignment, or coupling). For each pair, the model generates a specification sentence that articulates the relationship. fit, or assem- bly condition between the two parts. The resulting set of filenames, fi. is subsequently adopted as the ground truth for downstream part retrieval tasks. Figure preesenss the prompt for this process.
Finally, to facilitate the construction of human preference database, incorporate human anno- tation stage. Each annotation bundle includes the merged part image, the original assembly image, and the relevant specification sentence. Professional annotators review and filter items according to the following procedure:
(1) Examine the assembly image gain comprehensive understanding of the overall structure.
4

---

Published as a conference paper at ICLR 2026
(2) Items with overly similar part descriptions are discarded, such cases can lead to ambiguity and multiple possible answers during part retrieval.
(3) Assemblies in which the overall structure is nearly indistinguishable from more of its constituent parts are also filtered out.
(4) Any other scenarios that may introduce ambiguity permit multiple correct answers in part retrieval are excluded.
2.3 ERROR NOTEBOOK CONSTRUCTION
To further improve model reasoning, we construct Error Notebook for the 2nd VLM that lever- ages the ability of VLMs to self-reflect and correct mistakes within their step-by-step reasoning process. Given, for each assembly, the assembly image Tassembly. the mapping from part filenames to their descriptions D, specification S, the previous CoT reasoning Re and the ground-truth filenames p=()0) The goal is generate corrected reasoning trajectory Roor that leads to the correct solution, shown in Figure
In theory, we formalize the step-by-step reasoning process as a trajectory = (8), 8 ................................ 8n, a), where 8; are intermediate reasoning steps and is the predicted answer. A suboptimal trajec- tory, Roccy may contain both correct steps and erroneous steps, ultimately leading to an incor- rect prediction. Models are expected identify and revise the first erroneous step in prrey We define a corrected reasoning trajectory Homr as the concatenation of: 1) all steps up to the first error, 2) natural language reflection that pinpoints and transitions from the error, and 3) the corrected reasoning steps that ultimately yield the ground-truth answer prig), Formally, Rewe (oi...., &h si....; ... a²), where sf are correct steps and 5 are erroneous, we extract the subsequence ending at the first error, pprey = .......... sh. s4). The corrected trajectory is then constructed as:
R
R + TR + Ro,
where TR is transition phrase, and R is the correct trajectory from the correction point to the ground-truth answer palgu
In our approach,
no Scoo (Jassembly, D, Rerey pp-(g)) prompt
The prompt instructs the model to: (1) Read and follow the previous reasoning Herey step by step.
(2) Upon encountering the first logical or factual error, stop and explicitly articulate the transition. (3) From that point onward, independently correct the error, reasoning step by step until reaching P*gg (4) If errors detected, simply reproduce the previous correct reasoning and answer.
2.4 VERIFYING CORRECTED REASONING: GC FILTERING
To ensure that the corrected trajectories included in the Error Notebook are logically well- formed, we introduce grammar-constrain (GC) check / filtering mechanism. This procedure serves a deterministic verifier that inspects each corrected reasoning trace and determines whether it satisfies a set of structural and semantic validity conditions.
Given a corrected reasoning trajectory Reor and the set of allowable part filenames P, we check whether the final segment of hearr contains a well-defined and valid answer. Concretely, the verifier searches for line beginning with the phrase Finul Answer: and extracts the predicted filenames. A reasoning trace is accepted if and only if (1) such a line exists, (2) at least one filename is provided, and (3) every predicted filename appears in the allowed set P. In practice, evaluate two variants of this filtering rule:
Strict grammar constraint (sGC). This variant requires the explicit presence of a Final Answer: line and accepts a corrected trajectory only it satisfies all structural validity rules.
Relaxed grammar constraint (rGC). To accommodate models whose corrected reasoning is log- ically sound but omits the explicit Final Answer: marker, we introduce a relaxed variant that addi- tionally accepts trajectories that are identical to sGC except for missing this indicator.
5

---

Published as a conference paper at ICLR 2026
Reference formation
Error Notebook Construction Process
fewllfiaiin
Results: Name1, Named
Rulqunce
Grammar Constraints (sGC) rGC)
JSON
CoT: First Lchack the diecrptions of perts
Revision
hameel Nared
ndd VLM Output
Filter
notebook
(c) CoT Revision
Original CoT up first (0) Transition (T) New reasoning path
parts. The specification states that the rectangular panel. The second part, Name1, is ae the panet the of arm:
(0)
First, check the descriptions
(T) But, walt, let's pause and exemine this more carefully.
(N) description mentions the absence of holes could initially seem problematic. However, given this specific Instruction, there may be another mounting method or the holes could be added later. Let's think about this logically. The second part, arm
panel "without holes."
panel
mounted securely with fasteners,
Name1, described as
integrated restangular block face.
structure,
(GT) Final answer: Name1, Nameé.
Figure 3: Error Notebook construction. We define corrected reasoning trajectory as the con- catenation of: 1) all steps up to the first error, 2) a natural language reflection that pinpoints and transitions from the error, and 3) the corrected reasoning steps that ultimately yield the ground-truth answer. The proposed GC check is further employed to improve the quality of the Error Notebook.
2.5 ERROR NOTEBOOK + RAG-BASED INFERENCE
In the inference stage, we adopt a RAG strategy that leverages examples from the Error Notebook few-shot exemplars. Specifically, it retrieves the top-n relevant samples from the Error Notebook based on their similarity to the current assembly specification, using the corrected CoT trajectories from these entries guide the model's reasoning. Figure (b) shows the overview of the overall inference process based on VLMs.
Given an instance defined by the assembly image Inwsembly the mapping from part filenames to descriptions DD and specification S, the RAG-based inference proceeds as follows:
Let & {e1..... ... eM } denote the set of entries in the Error Notebook, each comprising specification Sj. part descriptions Dj. and corrected CoT trajectory R For the current query, compute the similarity sim Sj) between S and each S in E. To avoid data leak- age, the current query instance ceur is excluded from retrieval and will never appear among its own few-shot exemplars. The top-n most similar samples are selected:
(1) Sample Retrieval:
{eky..... ..... Ck. arg max sim 8,),
(5)
where Ccur denotes the current query instance.
(2) Few-Shot Prompt Construction: For each retrieved sample ck construct prompt block con- taining the assembly context, part descriptions, specification, and the corrected CoT Rş.. with the corresponding final answer. These prompt blocks concatenated for the current query.
few-shot exemplars
The final model input consists of (i) the few-shot exemplars constructed above and (ii) the current query context, which includes the assembly image Inssembly. part descrip- tions D, and specification S. The model is prompted to perform step-by-step reasoning, leveraging the retrieved exemplars as references.
(3) Main Query Prompt:
Formally, let F denote the few-shot prompt constructed from the top-n retrieved entries. The model's 's output is given by:
R = (F, Inseembly: D,S, prompt
(6)
where R is the model's answer, and prompt provides the instructions for the inference task.

---

Published as a conference paper at ICLR 2026
3 EXPERIMENTS
3.1 IMPLEMENTATION DETAILS
Our pipeline interacts with VLMs (e.g., GPT-4o, Gemini) via API endpoints. For each inference call, images encoded as base64 data URLs. We implement emror handling with exponential backoff and up to retries in the event of API errors. To process the dataset efficiently, all major computation steps are parallelized for asynchronously executing functions using multiple threads. Each assembly is processed as an independent unit. The generated part descriptions, which serve as intermediate outputs, stored in ISON format. For fair comparison, both/all experiments the same model/group (experiments w/o E-Notebook and w/ E-Notebook) employ identical description JSON files. Unless otherwise specified, the value of for RAG's top-k retrieval is equal to the number of exemplars in the part retrieval stage, which defaults to 2.
3.2
MAIN RESULT
Accuracy comparison of general models with and without Error Notebook-RAG integration on self-generated and human preference datasets. The best result highlighted in bold. We divided the data from both datasets into 4 groups based on the number of parts in each assembly, reflecting the varying difficulty levels.
Table I:
Self-generated dataset
Human preference dataset
Strategy
Overall
10
10
20
20
50
50
Overall
10
10
20
20
50
50
GPT-4o (Omni)
w/o E-Notebook
28.5
40.7
22.4
15.3
5.0
41.7
47.9
32.4
26.5
0.0
w/ B-Notebook
48.3
66.8
35.9
29.7
16.3
65.1
75.5
42.6
41.2
21.4
w/ E-Notebuok+sGCC
48.5
67.0
36.5
32.2
12.5
66.8
75.5
48.5
50.0
21.4
GPT-4o mini
w/o H-Notebook
13.6
20.5
10.9
4.2
1.3
19.3
24.8
10.3
0.0
0.0
w/ E-Notebook
24.9
34.9
25.0
5.9
7.5
35.4
41.5
29.4
8.8
7.1
B-Notcbook+sGGC
25.9
37.7
20.5
11.0
5.0
36.4
42.6
29.4
11.8
7.1
Gemini 2.5 Pro Non-streaming
w/o E-Notebook
36.5
55.1
25.6
14.4
6.2
$4.0
65.2
35.3
20.6
0.0
w/ B-Notebook
42.2
60.9
30.8
21.2
11.3
59.5
69.5
42.6
29.4
14.3
E-Notebook++GGC
42.9
64.8
28.8
62.1
74.1
38.2
32.4
Gemini 2.0 Flash Non-streaming
w/o E-Notebook
30.9
46.8
21.2
12.7
5.0
44.2
53.5
23.5
20.6
14.3
w/ NoNeobbook
40.4
58.2
31.4
19.5
8.7
56,8
67.0
39.7
23.5
14.3
E-Notebook+ssGC
40.3
57.3
30.1
19.5
57.0
66.3
39.7
29.4
Gemini 1.5 Pro Non-streaming
E-Notebook
29.9
44.3
21.2
13.6
6.2
43.0
52.1
23.5
17.6
14.3
w/ Notebookk
32.4
49.3
22.4
11.9
6.2
46.7
57.1
25.0
17.6
14.3
w/ B-Notchook+sSCC
36.2
51.8
26.3
17.8
50.3
60.6
30.9
14.7
21.4
Cloud Vision (Image) Gemini 2.0 Flash Non-streaming
E-Notebook
35.0
51.8
25.0
14.4
50.0
58.9
38.2
17.6
w/ E-Notebook
40.4
59.3
30.1
16.1
11.3
$7.8
66.3
47.1
29.4
7.1
w/ B-Notebook+sGG
43.2
63.2
32.7
17.8
11.3
62.3
73.0
48.5
20.6
14.3
(1) Our experimental results demonstrate that the proposed Error Notebooks with RAG frame- work enhances retrieval accuracy across all evaluated models and assembly complexities, as summarized in Table [ The performance gains are particularly pronounced on the human preference dataset. For example, GPT-4o (Omni) improves from 41.7% to 65.1% overall on the human preference dataset, marking absolute gain of 23.4%, while its performance on the self- generated dataset also rises from 28.5% 48.3% (+19.8%). Similar trends are observed for other models: GPT-4o mini increases from 19.3% 3% to 35.4% (+16.1%) 1%).. Gemini 2.0 Flash Non-streaming

---

Published as a conference paper at ICLR 2026
Table 2: Ablation study on the number of exemplars retrieved from the Error Notebook. We also analyze the effect of excluding explicit CoTT reasoning in each exemplar. CoT Group indicates that each retrieved exemplar includes explicit step-by-ste reasoning, while Non- Col Group omits such reasoning in the exemplars and includes ground truth only. The data from both datasets are divided into four groups based on the number of parts in each assembly, reflecting varying difficulty levels.
Self-generated dataset
Human preference dataset
Number of Exemplars
Overall
<
10
10
20
20
50 > 50
Overall
10
10
20
20
50
> 50
Non-CoT Group
49.4
69.5
37.8
27.1
13.8
69.3
80.5
50.0
38.2
14.3
5
50.1
70.4
38,5
29.7
11.3
69.1
79.8
51.5
41.2
7.1
50.6
69.8
37.8
32.2
16.3
70.4
79.4
55.9
44.1
21.4
20
50.8
69.3
42.3
32.2
11.3
69.1
77.7
60.3
38.2
14.3
52.7
72.0
42.3
32.2
16.3
72.9
83.0
57.4
41.2
21.4
CoT Group
49.7
68.4
39.7
30.5
12.5
67.8
77.7
54.4
38.2
7.1
49.4
67.0
38.5
32.2
16.3
67.8
75.5
52.9
50.0
28.6
10
49.4
66.5
42.3
29.7
15.0
68.8
76.2
61.8
44.1
14.3
20
51.7
69.0
42.3
35.6
16.3
71.1
79.8
57.4
52.9
7.1
50
49.5
67.9
37.8
33.1
13.8
68.1
77.0
51.5
52.9
7.1
from 44.2% to 56.8% (+12.6%), and Gemini 1.5 Pro Non-streaming from 43.0% to 46.7% (+3.7%) Another clear trend is that improvements not limited to small assemblies: while the largest abso- lute gains often appear in cases with fewer parts (e.g., < 10 parts, GPT-4o Omni rises from 47.9% to 75.5%), consistent accuracy improvements are observed across all part-count intervals, including the more challenging > 50 parts group. These results highlight the effectiveness and generality of the proposed Error Notebooks + RAG strategy, which enhances inference across different proprietary (GPT, Gemini) models, without requiring additional training.
The effect GC check on Table 1 We then rebuilt the Error Notebook using entries that passed this strict grammar constraints (sGC) check, and re-ran inference with the same RAG pipeline. And this trick further produces up to 4.5 points of improvement on the human preference dataset.
While Table demonstrates the performance gap between models with and without Error Note- books, Table 2 further shows that once Error Notebooks are incorporated, the number of exemplars retrieved by RAG has only a minor effect on final accuracy. For instance, on the self-generater dataset, the overall accuracy the Non-CoT group varies only slightly between 49.4% (1 exemplar) and 52.7% (50 exemplars). A similar trend holds for the CoT group, where performance remains stable in the narrow range of 49.4% to 51.7%. Consistent patterns are observed on the human prefer- ence dataset. These results indicate that the key factor driving improvements is the presence of Error Notebooks themselves, and the effect of the specific number of exemplars sampled negligible.
(2) The results in Table 2 and Figure show show that incorporating CoT reasoning from the Er- for Notebook is particularly valuable for challenging cases with higher part counts (> 10). For assemblies with fewer parts (< 10), the Non-CoT group, where only final answers are given, often performs comparably or even slightly better, suggesting that in simple scenarios, direct access to the final correct solution sufficient. By contrast, for complex assemblies with 10-50 parts, the CoT group consistently outperforms the Non-CoT group across nearly all exemplar sizes, confirming that step-by-stey reasoning provides crucial guidance for harder queries. This trend is observed across all exemplar group sizes, with one notable exception: when using 50 exemplars, the CoT group shows a drop in accuracy. We attribute this to excessively long prompts caused by concatenating many CoTs, which may interfere with the model's judgment. A second important observation (Fig- ArelA.I is that for simple assemblies, increasing the number exemplars has little effect, regardless of whether CoT is used. In contrast, for complex assemblies, accuracy improves the number of exemplars increases, up to around 20 exemplars.
(3) The ablation experiments Table show that method outperforms two traditional mhining-free, inference-time approaches. We conducted ablation experiments to compare our Error Notebook method with representative training-free, inference-time approaches. The
8

---

Published as a conference paper at ICLR 2026
Table Ablation comparison between training-free baselines and our proposed method.
Self-generated dataset
Human preference dataset
Strategy
Overall
<
10-20
20-50
50
Overall
10
10-20
20-50
50
Standard few-shot
26.6
37.4
19.2
16.9
6.2
37.7
42.9
29.4
17.6
21.4
w/o F-Notebook
28.5
40.7
22,4
15.3
5.0
41.7
47.9
32,4
26.5
0.0
Self-consistency
38.9
54.6
30.1
21.2
11.3
54.8
61.7
42.6
29.4
35.7
w/ E-Notebook (ours)
48.3
66.8
35.9
29.7
16.3
65.1
75.5
42.6
41.2
21.4
perimental settings are as follows. For standard few-shot learning, we use GPT-4o (Omni) with 2 API endpoints, and adopt two GPT-generated exemplars as few-shot examples (aligned with the 2-eemplaar setting in Table T]. We keep the full two-stage pipeline: the 1st VLM generates part descriptions from the assembly and part images; the second VLM performs reasoning. Standard few-shot is applied to the 2nd VLM (reasoning stage). For self-consistency, we keep the same two- stage VLM pipeline. The 1st VLM generates part-level descriptions exactly as in main method. For the 2nd VLM, we replace the Error Notebook with self-consistency strategy: GPT-4o (Omni), temperature 0.7, independent samples, followed by majority voting. Across both datasets, our method consistently outperforms those baselines.
Table 4: Results of Qwen2-VL-22 -L---LL---st-tuee We report both accuracy and the number of correctly solved cases (in parentheses) under identical settings Table
Self-generated dataset
Human preference dataset
Strategy
Overall
10 (361)
10-20 (156)
Overall
10 (282)
w/o E-Notebook
0.8 (6)
1.7 (6)
0.0 (0)
1.5 (6)
2.1 (6)
w/ E-Notebook
6.4 (46)
12.5 (45)
0.6 (1)
10.8 (43)
15.2 (43)
Improvement
+5.6 (+40)
+10.8 (+39)
+0.6 (+1)
+9.3 (+37)
+13.1 (+37)
w/ E-Notebook+sGCC
3.6 (26)
7.2 (26)
0.0 (0)
6.0 (24)
8.5 (24)
w/ E-Notebook+rGCC
6.6 (47)
12.7 (46)
0.6
10.8 (43)
15.2 (43)
w/ gB- Notebook+sGC**
8.4 (60)
16.6 (60)
0.0
14.6 (58)
20.6 (58)
Improvement (#-w/o)
+7.6 (454)
+14.9 (+54)
+0.0 (+0)
+13.1 (+52)
+18.5 (+52)
(4) Our method also demonstrates strong performance on open-source models. We further evaluated our approach on two open-source VLMs, Vwen2-VLL-2B-Imae
(Wang al. 2024)
and Aya- Vision-8B (Dash et al. 2025). All experimental settings (prompting format, RAG retrieval, and evaluation protocol) were kept identical to those used in Table [ For Powen2-VL222 2B-Inutruc, experiments were conducted on 8x A40 GPUs for approximately days. A detailed breakdown of its performance is reported in Table and the results for Aya- va-Vision-88B appear in Appenidx (A.2
During the grammar-ched filtering evaluation, we compared three variants. The E-Notebook+ configuration applies the same strict rule used for proprietary models. However, we found that the 2B model frequently produced otherwise valid reasoning traces that lacked the explicit Final An- swer: marker, causing many acceptable traces to be discarded. This substantially reduced the size of the Error Notebook and degraded performance. The E-Notebook++GG variant therefore relaxes this requirement, leading to improved accuracy compared to the basic E-Notebook setup. Finally, the gE- Notebook+ variant uses an Error Notebook constructed entirely from GPT-40 (0mi)) while still performing inference with the 2B model, reinstating the strict grammar rule under this cross-model setting. Strikingly, the cross-model variant (gE-Notebook+sGCC) achieves the strongest perfor- mance across all configurations. On the numan-preferenee dataset, the 2B model equipped with DG- performs only 4.2 points below GPT-40 mini in the <10 group. These results indicate that lightweight open- source model, when paired with high- Error Notebook and appropriate grammar-checl strategies, can closely approach the performance of substantially stronger proprietary VL.Ms.

---

Published as a conference paper at ICLR 2026
Overall, these findings confirm that the Error Notebook framework provides substantial and meaningful gains for open-source VLMs. Moreover, the improvements achieved through cross- model distillation show that the Error Notebook serve as effective mechanism for transferring high-quality reasoning traces from powerful proprietary models to compact open-source ones with- out any finetuning additional training.
3.3 EFFICIENCY ANALYSIS
Token Usage and Latency. We conducted a runtime and token-cost evaluation on 100 samples under: GPT-4o (Omni), a single API endpoint, one worker, and no batching. Jummariss the results. Although using the Error Notebook increases prompt tokens, inference does not become slower (8.04s vs 6.50s). Corrected exemplars may improve reasoning coherence and reduce internal search depth. The one-time correction step is lightweight (7.39 per sample). Also, 1st VLM latency is high since it depends on the number the CAD model's parts. Overall, the Error Notebook introduces prohibitive overhead, and RAG-enhanced inference remains efficient.
Table 5: Lateney and token usage for constructing the Error Notebook and performing inference.
Setting
Avg Time (s)
Prompt Tokens
Completion Tokens
1st VLM (part description)
78.32
2nd VLM (w/o E-Notebook)
8.04
967.7
235.4 .4
2nd VLM (w/ E-Notebook)
6.50
278.7 1815.3
CoT Correction Step
7.39
1328.7
377.5
API Call Cost. The total number of VLM calls required to construct the Error Notehook over samples is:
Σ (part count, + 1) + n
For each sample part count, VLM calls used to generate part-level descriptions, plus call for the (initial) part retrieval result. Then, new CoTs must be generated for correction, adding one more call per sample.
4 CONCLUSION
In this work, we introduced novel Error Notebook-guided, training-free part retrieval approach for complex 3D CAD assemblies. Our framework leverages retrospective error analysis and RAG to en- hance VLM reasoning without additional training or fine-tuning, By systematically constructing Er- ror Notebooks that capture and correct flawed reasoning trajectories, and by retrieving specification- similar exemplars at inference time, method consistently improves accuracy proprietary VLMs. Importantly, our method surpasses traditional training-fiee inference-time ap- proaches (standard few-shot, self-consistency) and further demonstrates strong improvements on open-source models (c.g., Qwen2-VL- VL-2B-Instruct and Aya-Vision-81y Vision-8B).
multiple
Future work will explore cross-domain applications of Error Notebooks, aiming establish more general paradigm for training- ffre reflective reasoning in multimodal AI.
5 THE USE OF LARGE LANGUAGE MODELS (LLMs)
We used LLMs as the experiment subject to study the improvement of our method on existing LLMs. We also used LLMs polish writing.
ETHICS STATEMENT
Our dataset construction process relies professional human annotators, who compensated fairly and provided clear annotation guidelines. Care was taken to exclude ambiguous or misleading cases avoid introducing bias into the dataset. No personally identifiable information sensitive
10

---

Published as a conference paper at ICLR 2026
data is involved. The proposed methods are intended for engineering and design applications, such as automated verification in CAD workflows, and do not pose foresecable risks of misuse.
REPRODUCIBILITY STATEMENT
All code for dataset preprocessing, part description generation, Error Notebook construction, and inference experiments is provided in the Supplementary Material. Detailed descriptions of dataset construction (including filtering and annotation protocols) are provided in Section 2.2 Experimental settings, including API interaction details, hyperparameters, and error-handling mechanisms, documented Section 3.1 Reproduction our results requires access to the Fusion 360 Gallery Assembly dataset and VLM APIs (e.g., GPT-4o, Gemini).
REFERENCES
Kamel Alrashedy, Pradyumna Tambwekar, Zulfiqar Haider Zaidi, Megan Langwasser, Wei Xu, and Matthew Gombolay. Generating CAD code with vision-language models for 3d designs. In The Thirteenth International Conference on Learning Representations, 2025.
Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou, and Weizhu Chen. Learning from mistakes makes IIm better reasoner. arXiv preprint arXiv:2310. 20689, 2023.
Saurabh Dash, Yiyang Nan, John Dang, Arash Ahmadian, Shivalika Singh, Madeline Smith, Bharat Venkitesh, Vlad Shmyhlo, Viraat Aryabumi, Walter Beller-Morales, Jeremy Pekmez, Ja- son Ozuzu, Pierre Richemond, Acyr Locatelli, Nick Frosst. Phil Blunsom, Aidan Gomez, Ivan Zhang, Marzieh Fadaee, Manoj Govindassamy, Sudip Roy, Matthias Gallé, Beyza Ermis, Ahmet Üstün, and Sara Hooker. Aya vision: Advancing the frontier of multilingual multimodality. arXiv preprint Stt.::250500855511 2025.
Haixia Han, Jiaqing Liang, Jie Shi, Qianyu He, and Yanghua Xiao. Small language model can self-correct. Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 2024.
Sumit Kumar, Sarthak Kapoor, Harsh Vardhan, and Yao Zhao. Generative for cad automation: Leveraging large language models for 3d modelling. arXiv preprint arXiv:2508. 00843, 2025.
Joseph G. Lambourne, Karl D.D. Willis, Pradeep Kumar Jayaraman, Aditya Sanghi, Peter Meltzer, and Hooman Shayani. Brepnet: topological message passing system for solid models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Jiahao Li, Weijian Ma, Xueyang Li, Yunzhong Lou, Guichun Zhou, and Xiangdong Zhou. Cad- Ilama: Leveraging large language models for computer-aided design parametric 3d model gener- ation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 18563-18573, 2025.
Ming Li, Lichang Chen, Jiuhai Chen, Shwai He, and Tianyi Zhou. Reflection-tuning: Recycling data for better instruction-tuning In NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following, 2023.
Zhuoshi Pan, Yu Li, Honglin Lin, Qizhi Pei, Zinan Tang. Wei Wu, Chenlin Ming, H. Vicky Zhao, Conghui He, and Lijun Wu. Lemma: Learning from errors for mathematical advancement in llms. arXiv preprint arXiv:2503.1 17439, 2025.
Matthew Renze. The effect of sampling temperature on problem solving in large language models. In Findings of the Association for Computational Linguistics: EMNLP. 2024.
Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. arXiv preprint arXiv:2303.1 1366, 2023.
Yongqi Tong, Dawei Li, Sizhe Wang, Yujia Wang, Fei Teng, and Jingbo Shang. Can 1lms learn from previous mistakes? investigating lims' emrors to boost for reasoning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL.), pp. 3065-3080, 2024.

---

Published as a conference paper at ICLR 2026
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision- -anguge model's perception of the world at any resolution. arXiv preprint arXiv:2409. 12191, 2024.
Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao. Large language models are better reasoners with selfiveriiication. Findings of the Association for Computational Linguistics: EMNLP, pp. 2550-2575, 2023.
Karl D. D. Willis, Yewen Pu, Jieliang Luo, Hang Chu, Tao Du, Joseph G. Lambourne, Armando Solar-Lezama, and Wojciech Matusik. Fusion 360 gallery: dataset and environment for pro- grammatic cad construction from human design sequences. ACM Transactions Graphics
(TOG), 40(4), 2021a.
Karl DD Willis, Pradeep Kumar Jayaraman, Hang Chu, Yunsheng Tian, Yifei Li, Daniele Grandi, Aditya Sanghi, Linh Tran, Joseph G Lambourne, Armando Solar-Lezama, and Wojeiech Ma- tusik. Joinable: Learning bottom-up assembly of parametric cad joints. arXiv preprint
arXiv:2111.12772. 202Ib.
Rundi Wu, Chang Xiao, and Changxi Zheng. Deepead: A deep generative network for computer- aided design models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 6772-6782, 2021.
Zhiheng Xi, Dingwen Yang. Jixuan Huang, Jiafu Tang, Guanyu Li, Yiwen Ding, Wei He, Boyang Hong, Shihan Do Wenyu Zhan, al. Enhancing I1m reasoning via critique models with test-time and training-time supervision. arXiv preprint arXiv:2411 11112241 2024.
Yuchen Yan, Jin Jiang, Yang Liu, Yixin Cao, Xin Xu, Mengdi Zhang, Xunliang Cai, and Jian Shao. S3c-math: Spontaneous step-level self-correction makes large language models better mathemat- ical reasoners. arXiv preprint 24Xi::209900122 2024.
Zhe Yang, Yichang Zhang, Yudong Wang. Ziyao Xu, Junyang Lin, and Zhifang Sui. Confidence v.s. critique: A decomposition self-corection capability for Ilms. In Proceedings of the 63rd Anmual Meeting of the Association for Computational Linguistics (ACL), 2025.
Licheng Zhang, Bach Le, Naveed Akhtar, Siew-Kei Lam, and Tuan Ngo. Large language models for computer-aided design: A survey. arXiv preprint arXiv:2505.08137, 2025.
Yunxiang Zhang, Muhammad Khalifa, Lajanugen Logeswaran, Jaekyeom Kim, Moontae Lee, Honglak Lee, and Lu Wang. Small language models need strong verifiers to self-correct rea- soning. In Findings of the Association for Computational Linguistics: ACL, pp. 15637-15653, 2024.
12

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 ABBREVIATIONS
VLM
Vision-Language Model
LLM
Large Language Model
CAD
Computer-Aided Design
STEP
STandard for the Exchange of Product model data (ISO 10303)
CoT
Chain-of-Thought
RAG
Retrieval Hurreval-Augmentee Generation
API
Application Programming Interface
GT
Ground Truth
GPT
Generative Pre-trained Transformer
GC
Grammar Constraint
sGC
strict Grammar Constraint
IGC
relaxed Grammar Constraint
E-Notebook
Error Notebook
w/o
without
w/
with
A.2 SUPPLEMENTARY RESULTS
(1) Retrieval relevance T We report the retrieval relevance results as shown in Table A.
Define TP |GT n Pred|, FP = Pred GT,, FN = |GT - Pred], then:
TP
Recall(R)
(A.1)
TP + FN'
TP
Precision (P) TP + FP
(A.2)
2PR P R
F1
(A.3)
We report both global averaged Recall/F1 and per-group Recall/F1 based on the number of parts (< 10, 10-20, 20-50, > 50), evaluated on the self- self-generated dataset. In A.ble we can see that the proposed Error Notehook method consistently yields clear and meaningful improvements in retrieval relevance.
(2) Our method is highly sensitive the specific retrieval scoring function. In Table I the Error Notebook relies on churacter-level similarity retriever, which computes a normalized character-level matching score between textual specifications. To further examine whether our method is sensitive to the retrieval scoring function, additionally implemented new retriever based on token-level Jaccard similarity as shown Table A.2 This new version tokenizes cach specification and measures the overlap between the resulting token Jaccard retriever yields slightly higher accuracy (approximately +2% on the self-generated dataset). Importantly, for both retriever methods, the Error Notebook consistently provides large and ro- bust gains over the baseline.
Overall, the token-level
13

---

Published as a conference paper at ICLR 2026
Table A.I: Retrieval relevance evaluation.
Per-group Recall
Strategy
Global Recall
Global F1
< 10
10-20
20-50
> 50
GPT-40 (Omni)
w/o E-Notebook
0.406
0.532
0.520 0,664
0.362
0,481
0.277
0.370
0.171 / 0.239
E-Notebook
0.692
0.686
0.828 0.837
0.644
0.629
0.557 0.534
0.367 /0336
GPT-4o mini
w/o E-Notehook
0.261
0.385
0.344 0.494
0.218 0.325
0.179 / 0.266
0.089 / 00.44
w/ B-Notebook
0.500
0.523
0.619 / 0.675
0.500 0.504
0.289 / 0.288
0.272 / 0.275
Gemini 2.5 Pro Non-streaming
w/o B-Notebook 0.627
0.607
0.778 / 0.781 0.571 0.532
0.451 / 0.416
0.316 / 0.304
w/ B-Notchook
0.662
0.595
0.815 0.796
0.590 / 0.569
0,472 / 0.444
0.392 / 0.225
Gemini 2.0 Flash Non-streaming
w/o E-Notebook 0.552
0.573
0.681 / 0.722
0.529 0.531
0,400 0.392
0.241 / 0.255
E-Notchook 0.630
0.777 / 0.784
0.583 0.584
0.468 0.446
0.297 0.29
Gemini 1.5 Pro Non-streaming
w/o E-Notebook 0.565
0.554
0.717 0.727
0.522 0.497
0.366 0.340
0.253 00247
w/ E-Notebook 0.575
0.557
0.745 / 0.738
0.474/ 0.456
0.396 / 0.362
0.272 0.261
Cloud Vision (Image)+ + Gemini 2.0 Flash Non-streaming
w/o E-Notebook
0.617
0.604
0.750 / 0.776 0.583 0.553
0.438 / 0.398
0.342 / 0.318
w/ E-Notebook
0.636
0.622
0.788 / 0.794 0.577 0.562
0.447 0.412
0.342 / 0.326
Table A.2: Comparison between character-level and token-level retrieval scoring functions.
Self-generated dataset
Human preference dataset
Strategy
Overall
< 10
10-20
20-50
50
Overall
10
10-20
20-50
50
w/o E-Notebook (Table 1)
28.5
40.7
22.4
15.3
5.0
41.7
47.9
32.4
26.5
0.0
w/ E-Notebook (Table I, character-level)
48.3
66.8
35.9
29.7
16.3
65.1
75.5
42.6
41.2
21.4
w/ E-Notebook (New, token-level)
50.2
68.4
39.7
31.4
16.3
68.1
77.3
50.0
47.1
21.4
(3) The results in Figure A.1 show that incorporating CoT reasoning from the Error Notebook particularly valuable for challenging cases with higher part counts (10 (0).
(4) We demonstrate the effectiveness of the proposed two-stage pipeline. As shown in Fig- ure A.2] the proposed two-stage pipeline for part retrieval in 3D CAD assemblies achieves signif- icantly higher accuracy compared to the image-only reasoning baseline. In the image-only setup, both the assembly image and individual part images are directly fed the VLM in a single infer- ence step, relying solely visual input. In contrast,
proposed method first utilizes the VLM to generate concise part descriptions within the assembly context, and then performs part retrieval second reasoning step with the assistance of these textual deseriptions. This design introduces leading to consistent performance im-
an additional layer of interpretability and context-awarenn provements across all part count groups We slightly modified the prompt content to tailor it for this ablation study, using GPT-4o (Omni) as the model. Quantitatively, the image-only baseline yields an overall accuracy of 15.0% (107/715) The proposed pipeline achieves overall accuracy of 33.6% (240/715), with 51.2% (185/361) /361) for < 10 parts, 23. 23.7% (37 156) for 10-20 parts, 11.9% (14/118) for 20-50 parts, and 5.0% (4/80) for > 50 parts. These results demonstrate the effectiveness of incorporating part descriptions as intermediate representations.
14

---

Published as a conference paper at ICLR 2026
(a)
For simple assemblies (<10 parts)
(b) For challenging assemblies (10-50 parts)
244
405
203
5.0
124
23
20
50
Number of Exemplars
Number of Exemplars
Non-Co Group
CoT Group
Figure A.I: Effect of CoT reasoning and exemplar number retrieval accuracy across differ- ent assembly complexities and datasets. Top row: results on the self-generated dataset; bottom row: results the human preference dataset. (a) For simple assemblies (< 10 parts). (b) For more complex assemblies (10-50 parts). The r-axis indicates the number of exemplars retrieved from the Error Notebook, where each exemplar consists of either (i) the final corrected answer only (Non-CoT group) or (ii) the corrected CoT reasoning steps plus the final answer (CoT group).
Image only reasoning
Proposed pipeline
10
1.%
Overall
Part count
Figure A.2: Accuracy comparison between proposed pipeline and image-only reasoning. Per- formance is shown for the proposed pipeline, which leverages part descriptions as intermediate references, versus the one that directly reasons over images.
(5) Our method can demonstrate strong performance on open-source models. The results of Aya- Vision-8- is shown Table A.3 For efficiency, we used 7x A40 GPUs for around 36 hours, and additional run 3x H20 GPUs for around 12 hours. All experimental settings (except the model itself) remained identical to those in T Therefore, for open-source VLMs, our Error Notebook method still brings substantial and meaningful gains.
15

---

Published as a conference paper at ICLR 2026
Table A.3: Results of Aya-Vision-88
Self-generated dataset (666 cases)
Human preference dataset (370 cases)
Strategy
Overall
< 10
10-20
Overall
<
10
10-20
w/o E-Notehook
16
16
0
14
14
0
w/ E-Notehook (ours)
54
53
51
50
1
Improvement
+38 (3.4x)
+37
+1
+37 (3.6x)
+36
+1
A.3 PROMPTS
You are an expert mechanical engineer. Given Image 1 (the assembly) and Image 2 (an individual part from the assembly). please generate concise and descriptive noun phrase (not full sentence). The phrase should briefly describe the part's main shape and any key features, in a way that clearly distinguishes it from the other parts in the assembly. Avoid generic names like 'part' or "component'. Be specific about the shape and any holes, slots, or functional features. Your output should be a single noun phrase.
For example:
conical mount with a forked top;
A cylindrical pin;
Two plates with each having holes;
flat round disk with three small holes;
rectangular bracket with two mounting slots.
Figure A.3: Prompt used to generate part-level descriptions in the dataset construction pipeline.
You are expert mechanical engineer. Given image of assembled product (assembly) and list of its part descriptions below: Part descriptions: (desc _is__lssssrrr__
Your task:
1. Review the assembly image and the list of part descriptions.
2. Choose any two part descriptions that are most likely to have a direct physical, spatial, or functional relationship in the assembly (such as fit, mounting, alignment, or coupling).
3. Generate one specification sentence (inspection/check item) that describes the required relationship, fit, or assembly condition between these two parts, as would appear in manufacturing or assembly checklist.
4. Your specification should be clear, specific, and professional, mentioning both se- lected part descriptions explicitly.
5. Output only one specification sentence. Do explain your reasoning.
6. Output format: The selected two part descriptions (exactly as shown above, sepa-
rated by a semicolon), then a line break, then the specification sentence.
For example, given descriptions like:
1. cylindrical pin
2. A flat plate with holes
Output:
A cylindrical pin;A flat plate with holes
The cylindrical pin must fully inserted into one the holes the flat plate.
Figure A.4: Prompt used to generate specification for each assembly in the dataset construction pipeline.
16

---

Published as a conference paper at ICLR 2026
You are expert mechanical engineer with a sharp analytical mind. You are given the assembly image, the descriptions of all parts (each as 'filename: description), the inspection specification, and a previous reasoning process (including its step-by-step thoughts and its Final Answer).
Your job:
1. Carefully read the previous reasoning step-by-step. Follow along and reproduce the steps until you encounter the first error or mistake.
2. Once you spot the first mistake, stop following the previous reasoning and use natural transition phrase (such as: "But, wait, let's pause and examine this more carefully." or "Wait, something seems off. Let's pause and consider what know far.") point the error and correct
3. From that point on, continue the reasoning process your own words, step-by- step, until you reach the correct answer (lee., the filenames consistent with the correct ground-trutii solution).
4. Do not mention "previous attempt" "ground-truth solution" explicitly. Make your reasoning sound like a student discovering and correcting their own mistake in real time.
5. If the previous reasoning is already correct, simply reproduce the previous reason- ing and the final answer is.
6. End your output with a "Final Answer:" line followed by the filenames (from the keys above), separated by semicolons (:), with no extra words or punctuation.
Figure A.5: Prompt used to revise CoTs.
Now, for the following question, use the above reasoning as reference and answer step-by-step:
Assembly image:
[image attached]
Part descriptions:(desc lines)
Specification:(spe)]
Your task:
1. Think step by step (Chain-of-Thought) and explain how you identify the required part(s).
2. In the last line, write 'Final Answer:' followed by only the selected part filenames
(from the keys above), separated by semicolons (:), with no extra words or punctuation. Example output:
First, check the descriptions of all parts. Only part1 .png and part2.png are described as cylindrical pins. Therefore, the required parts are part1. .png and part2.png. Final Answer: part1. portt1ppn::prr2.pgg
Figure A.6: Prompt used to generate the part retrieval results.
17

---

Published as a conference paper at ICLR 2026
A.4 FULL ENGINEERING PIPELINE ILLUSTRATION
To clarify the broader engineering context of our method and help better understand the meaning of parl relrieval in practical CAD assembly analysis, we provide in Figure A.7 a complete overview of our proposed pipeline in an engineering setting. Specifically, the left side depicts the STEP pro- cessing stage: an input assembly (in STEP format) decomposed into its constituent parts using freecad, and subsequently rendered into 2D images using the pythonoce library. This generates inter- mediate representations (part-level STEP files and rendered images) that provide concrete references for the VLM-based retrieval process. On the right side, a textual specification is provided, and the VLMs enhanced with Error Notebook + RAG reasoning produce candidate part identifiers. These are then fused back into the assembly using freecad, and the resulting structure can be visualized with pythonocc.
Assembly STEP
Specification
The cylindhical motor coupling with external gear
STEP processing
Assembly STEP shards part STEPs using freecad
Output:
VLMs
Notebook RAG
STEP file for the assembly)
STEP files for each part & its Part name (ID)
STEP to IMAGE using pythonocc library
STEP fusion based oulpul using freecad. Visualization using pythonocc.
Output:
Output
An assembly image
Parts images
(Eg. (8..65.7...... ..00........
Figure A.7: Full engineering pipeline for specification-driven part retrieval. The assembly STEP file is first decomposed into part-level STEP files using freecad, and both the assembly and part images are generated via pythonoce. Given a textual specification, VLMs enhanced with Error Notebook RAG output candidate part identifiers, which are then fused back into the assembly with freecad for visualization.
A.5
CASE STUDIES
18

---

Published as a conference paper at ICLR 2026
Table A.4: Case studies of assembly-level part retrieval by GPT 4o (Omni) with Error Notebook. Each row shows the assembly image, the part count, the specification, and the retrieved results in image format.
ID
Assembly Image
Specification
Retrieval Results
Count
The cylindrical protrusion on the vertical plate must align and curely into the curved channel the rectangular housing.
16
The concave plate with central cir- cular hole short cylindrical base must be securely seated the cylin- drical base with radial grooves, en- suring proper alignment and fit.
The curved tapered arm with de- tailed thumb and fingers must fit snugly within the arm- shaped cav- ity the curved block, ensuring full contact proper alignment.
The semi-cylindrical block must fit securely onto the circular grid's s cen- tral hub without obstructing the ra- dial struts.
4
10
The cylindrical cap with integrated spout and loop handle must be curely screwed onto the threaded top collar of the cylindrical bottle body, ensuring leak- ofoof seal.
The curved cylindrical shackle must be securely fitted into one of the lat- eral round holes the cylindrical body.
19

---

Published as a conference paper at ICLR 2026
Part Count
ID
Assembly Image
Specification
Retrieval Results
A flat rectangular plate with diago- nal cutouts and rounded corners; rectangular plate with larger cut- out featuring stylized raspberry sign
10
LAXT
The helical coil must be securely seated centered on the cylindri- cal rod with a flat circular base to en- sure stable alignment.
The threaded shaft of the knurled cylindrical knob must be securely fastened into the threaded hole of the curved lever arm ensure proper functionality and alignment of the assembly.
The long, curved cylindrical tube must be snugly inserted into per- forated cylindrical opening of the elbow-shaped casing for secure fit without gaps.
cylindrical with must be fully inserted into the in- ternal square socket of the cylindri- cal housing, ensuring secure attach- ment.
11
4
The hollow cylindrical cap must he securely fitted over the central cir- cular protrusion of the curved base block, ensuring no gaps between the mating surfaces.
12
20


Reviews:
[{"review_id": "dxhZDq4DcG", "reviewer": "Reviewer_vpJW", "summary": "This paper considers a specific sub-problem in CAD tasks automatization, mainly part retrieval in the assembly. For this task a new approach, based on error notebooks on RAG is proposed, which can be used on top of any LLM model without fine-tuning or adaptataion. Experimental validation shows validity of the proposed approach.", "questions": "Please address the weakness #2 and comment on problem statement and significance/impact of your work.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "1) The proposed method can be used without fine-tuning on top of any LLM model", "weaknesses": "1) Very limited scope of work due to selected problem\nAs far as I understand, the considered problem is retrieval of specific part in assembly by using its textual description. Each assembly in question consists of very limited amount of parts, less then 50. So the problem is like text retrieval from corpus of up to 50 texts. Yes, texts are very specific (CAD description of parts), but nonetheless the problem is small. This limits both the significance and impact of the work.\n\n2) No related works or similar methods or comparision with existing methods. \nHow such problems are solved now? Are there any approaches for part retrieval, other then LLMs? Has anyone considered the same problem before, or it is a first time?\n\n3) Unclear description and presentation. \nUp till section 2.2 i have been struggling to understand what the problem in question is. Overall the presentation is very unclear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sUq4D9mKod", "reviewer": "Reviewer_f2Wn", "summary": "The authors propose a part retrieval framework that uses error notebooks + RAG for refined prompt engineering. Furthermore, they contribute a human-in-the-loop CAD dataset for evaluating the proposed approach. Evaluations are performed on closed source models and achieve upto a 23% accuracy improvement on human-preference benchmarks.", "questions": "1. why was the dataset size limited to using a single set (752 assemblies) of Fusion 360 Assembly Dataset?\n2. i might have missed this, but, how are the corrected reasoning trajectories verified?\n3. are there scenarios where the model is not able to self-correct with prompting?\n4. what kind of assemblies does the method struggle with?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- error notebooks idea is a creative solution for test-time reasoning augmentation\n- contribution of human-in-the-loop dataset", "weaknesses": "- comparisons are primarily on proprietary models. It would be great to see them on open-source models as well.\n- the eval metrics are somewhat under-developed. For example, there is not analysis on retrieval relevance.\n- baseline comparisons against conventional CAD part retrieval methods are missing", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uT3YOhk3DP", "reviewer": "Reviewer_eRWS", "summary": "This paper introduces a training-free framework for 3D CAD part retrieval using Vision-Language Models (VLMs). Its core contribution is the Error Notebook, a mechanism that uses Retrieval-Augmented Generation (RAG) to guide VLM inference with corrected reasoning exemplars, avoiding costly fine-tuning.\n\nKey contributions: i) A training-free reasoning framework (Error Notebook + RAG) that significantly improves accuracy (up to 23.4% absolute gain) without fine-tuning. ii) A new, human-annotated CAD assembly dataset with relational specifications, based on the Fusion 360 Gallery. iii) A two-stage VLM pipeline (part description generation, then retrieval) to handle long, complex STEP file inputs.", "questions": "- What specific function is used for RAG retrieval? How sensitive are results to this choice?\n- How can the Error Notebook be constructed or applied in settings where ground-truth labels are unavailable or expensive to obtain?\n- How does the Error Notebook compare to other training-free methods like inference-time scaling (for example self-consistency)?\n- What are the computational overheads (API calls, tokens, latency) for notebook construction and RAG-enhanced inference compared to the baseline?\n- How often do the VLMs successfully generate valid reasoning corrections given the ground truth? What happens when the correction generation itself fails?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The Error Notebook is a creative solution for improving proprietary, \"black-box\" VLMs that cannot be fine-tuned. It effectively adapts self-correction concepts to an inference-time strategy.\n\nThe evaluation is comprehensive, testing multiple VLMs (GPT-4o, Gemini) across various assembly complexities. The method shows robust and consistent improvements for all models, such as boosting GPT-4o (Omni) from 41.7% to 65.1% accuracy.\n\nThe two-stage pipeline is a practical engineering solution for processing lengthy, non-natural language STEP file metadata, making the approach viable for real CAD workflows.\n\nThe new human-preference dataset is a significant contribution, enabling more human-centric evaluation by filtering out ambiguous cases with multiple correct answers.\n\nThe ablations provide valuable insights, such as CoT reasoning being most beneficial for complex assemblies (>10 parts). The finding that performance is similar for 1 to 50 exemplars is also a key practical takeaway.", "weaknesses": "The paper is empirically strong but lacks a theoretical justification for why the Error Notebook works. There is no formal analysis of its properties or potential failure conditions.\n\nThe approach requires ground-truth labels to build the Error Notebook. This reliance limits its generalizability and scalability to new domains where labels are unavailable or expensive to acquire.\n\nThe primary comparison is against the same models without the Error Notebook. The paper needs comparisons against other training-free methods (e.g., standard few-shot learning, self-consistency) to isolate the specific benefit of the Error Notebook.\n\nThe computational overhead (API costs, token usage, latency) of constructing the notebook and performing RAG-enhanced inference is not discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FAAzDsTb50", "reviewer": "Reviewer_Hfy1", "summary": "This manuscript discusses the specification aware part retrieval in complex 3D CAD assembly, which is a task that leads to VLM failure due to long context and poor reasoning ability. The author proposes a novel, non training framework that utilizes Error Notebook and RAG to enhance proprietary VLM. This notebook is constructed by collecting initial erroneous thought chains (CoT) inference from VLM, and then using VLM self reflection to generate corrected CoT trajectories. During inference, RAG retrieves these corrected examples as minority shot examples to guide VLM's inference. This article also contributes a new manual annotation CAD dataset and a two-stage pipeline to solve the problem of long context. Experiments have shown that GPT-4o has improved accuracy by 23.4% on a human preference benchmark", "questions": "1. This paper notes that CoT performance drops with 50 exemplars due to long prompt lengths. Does this suggest a fundamental limitation? Have authors explored strategies to mitigate this, such as summarizing the retrieved CoTs or dynamically selecting only the most relevant steps from each CoT?\n\n2. What is the average token length and inference time per query? This is quite important for practical applications, as it is difficult to apply if the output result is too slow due to the long time of doing CoT.\n\n3. Could authors please quantify the computational cost of building the Error Notebook? For example, how many VLM calls are required per sample to generate the corrected CoT?\n\n4. Are the geometric constraints required for part assembly essentially obtained from the retrieved STEP files, rather than being calculated by the LLM/VLM directly?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The authors contribute a new multimodal CAD dataset that includes human preference annotations. This would be  a valuable resource for the community if it can be made public.\n\n2. The concept of leveraging corrected reasoning exemplars as reusable Error Notebooks is novel and elegant.\n\n3. This work provides a new approach for solving the problem of excessively long STEP file inputs, and the part retrieval in assemblies that it focuses on is also a broad application in CAD design.", "weaknesses": "1. Some typos need further correction, such as missing commas after some equations and missing spaces between some words.\n\n2. The latency and resource overhead of RAG-based prompting are not quantified (e.g. the time for doing one complete inference).\n\n3. The core viewpoint is that retrieving corrected error is crucial. However, the experiment (Table 1) only compared \"w/E-Notebook\" (its method) with \"w/o-E-Notebook\" (zero sample). Missing a key baseline: a standard few shot method based on RAG that only retrieves correct examples (i.e. (query, correct_CoT, correct_answer)). Without this comparison, it is hard to know whether the performance gain comes from  error corrections or solely from known benefits provided by RAG with any relevant contextual examples.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)