CAD-Tokenizer: Towards Text-Based CAD Prototyping via Modality-Specific Tokenization
Authors: Ruiyu Wang
Shizhao Sun
Weijian Ma
Jiang Bian
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Models
Computer-aided Design
CAD Generation
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Computer-Aided Design (CAD) is a foundational component of industrial prototyping. 
where models are defined not by raw coordinates but by construction sequences such as sketches and extrusions. 
This sequential structure enables both efficient prototype initialization and subsequent editing. 
Text-guided CAD prototyping, which unifies Text-to-CAD generation and CAD editing, has the potential to streamline the entire design pipeline. 
However, prior work has not explored this setting, largely because standard large language model (LLM) tokenizers decompose CAD sequences into natural-language word pieces, failing to capture primitive-level CAD semantics and hindering attention modules from modeling geometric structure.
We conjecture that a multimodal tokenization strategy, aligned with CAD’s primitive and structural nature, can provide more effective representations. 
To this end, we propose CAD-Tokenizer, a framework that represents CAD data with modality-specific tokens using a sequence-based VQ-VAE with primitive-level pooling and constrained decoding.
This design produces compact, primitive-aware representations that align with CAD’s structural nature. 
Applied to unified text-guided CAD prototyping, CAD-Tokenizer significantly improves instruction following and generation quality, achieving better quantitative and qualitative performance over both general-purpose LLMs and task-specific baselines.

Published as a conference paper at ICLR 2026
CAD- TOWARDS TEXT-BASED CAD PRO- TOTYPING VIA MODALITY-SPECITH TOKENIZATION
Ruiyu :ang'; Shizhao Sun Weijian Ma², Jiang Bian?
University of Toronto, *Microsoft Research Asia, Fudan University rwangecs. toronto. edu, chizsudmicrosoff
ABSTRACT
Computer-Aided Design (CAD) is foundational component of industrial prototyp- ing, where models are defined not by raw coordinates but by construction sequences such as sketches and extrusions. This sequential structure enables both efficient pro- torype initialization and subsequent editing. Text-guided CAD prototyping, which unifies Text-to-CAD generation and CAD editing, has the potential to streamline the entire design pipeline. However, prior work has not explored this setting, largely because standard large language model (LLM) tokenizers decompose CAD se- quences into natural- aatural-language word pieces, failing to capture primitive-level CAD semantics and hindering attention modules from modeling geometric structure. We conjecture that multimodal tokenization strategy, aligned with CAD's primitive and structural nature, can provide more effective representations. To this end, we propose CAD-Tokent Tokenneee framework that represents CAD data with modality- specific tokens using a sequence- sequence-based VQ- VAE with primitive-level pooling and constrained decoding. This design produces compact, arimitive-aware represen- tations that align with CAD's structural nature. Applied to unified text-guided CAD prototyping. CAD-Tokenizer significantly improves instruction following and generation quality, achieving better quantitative and qualitative performance over both general-purped purpose LLMs and task-specific baselines.
INTRODUCTION
Computer-Aides Design (CAD) plays a central role in industrial prototyping and production. CAD models are constructed as sequences of operations-succ as sketches and extrusions-tha. CAD kernels compile into machine-interpretablee 3D objects. Unlike other 3D) representations, CAD explic- itly encodes design history, which makes it particularly suitable for both creating initial prototypes and performing subsequent modifications. Automating CAD sequence generation, especially from natural language instructions, could substantially accelerate the design workflow.
Recent work has begun to explore text-guided CAD subtasks, including CAD editing Yuan et al. 2025) and text-to-CAD generation (Wang al., 2025at] Li al. [222bb] Khan et al.) (202b)). These studies not only demonstrated the feasibility each individual task but also showed that large language models (L.LMs) can effectively operate CAD sequences. However, real-world design is rarely one-shot process: engineers iteratively refine prototypes by alternating between creation and modification. An effective text-based CAD system should therefore support both tasks in unified mnaner--gnneratngg CAD objects from instructions while also allowing editing of existing designs.
Handling both tasks within a single unified model has never been studied previously. The difficulty lies in the distinct requirements of the two subtasks. Text-to-CAD demands strong generative capability to produce high-quality sequences from scratch, whereas CAD editing requires precise instruction understanding and alignment with existing geometry. A unified framework must therefore master two complementary but oveellella skills, which places significantly higher demands on the backbone than solving either task alone.
"Wor done during the authors' internship at Microsoft Research Asia. Cooresponndecce Author.

---

Published as a conference paper at ICLR 2026
1ine
30
"GENERATE WITH A...
TEXT-TO-CAD
UNIFIED CAD FOR
PROTOTYPING
"ADD... "
line
18
curve
end
sketch
and
ccurve_ert end
<curve ends
scurve_ends
<sketch_ens
Dua TORENIOK SPLIS CAO INTO PRIGITINCS
Figure I: Overview of the unified text-based CAD prototyping task, which includes the Text-to-CAD generation and text-based CAD Editing. Our approach introduces a CAD-specific tokenization method that produces primitive-level tokens instead word pieces. This paradigm allows LLMs cap- ture relationships among CAD primitives more effectively, improves compression and performance.
We introduce CAD-specific tokenization. Our key insight that the default tokenizers used in LLMs are poorly matched to CAD data. They decompose CAD programs into natural-language word pieces, frag- menting operations and numeric parameters into arbitrary substrings (e.g., [extru] [ [-sion] "). This causes attention layers to focus on punctuation or partial tokens instead of meaningful primitives, limiting the model's 's ability to capture structural and geometric dependencies. To address this, we use VAE to compress CAD sequences into primitive-level tokens before training, enabling LLMS to directly model inter- tritt--rimitive relations and predict the next operation rather than the next character or word piece (Figure []. This formulation makes unified training possible, leading to substantial improvements in both efficiency and quality.
the first framework to tackle unified text-based CAD prototyping with
a VQ-VAE
Our contributions are as follows:
We develop primitive-level VQ-VAE tokenizer that maps sketch-extrusion pairs into multiple discrete tokens. To integrate with LLMs, we train adapters that align the pretrained tokenizer vocabulary with the backbone's embeddings.
2. We fine-tune LLM these CAD-specific tokens under unified setup covering both Text-to-CAD generation and CAD editing. This design improves training efficiency and surpasses strong task-specific baselines.
3. We leverage the formal grammar of CAD by introducing finite-state automaton (FSA)-based sampling strategy, which enforces valid syntax in generation and improves quality.
4. Through extensive qualitative, quantitative, and ablation studies, we demonstrate that CAD- Tokenizer improves both efficiency and performance across unified CAD prototyping tasks.
RELATED WORKS
2.1 CAD MODELING
In general, CAD modeling tasks take different types of user inputs and generate sequential CAD output that renders into 3D objects by 3D kernels.
Classical CAD Tasks. Classical CAD tasks focus on generating CAD sequences from non-textual inputs. This includes hand sketches (Seff et al. 2022 Xu et al. 2025), point clouds (Khan et al.
2024a Ma al. 2024), sequence prefixes (Xu et al. 2222 2023), even random noise (Wu et al.
2021). There are also tasks involving masked variation, where some primitives are masked out
and converted into random variants (Zhang al. 2025 Xu et al. 2022 2023). Our work differs
2

---

Published as a conference paper at ICLR 2026
from these works in different ways. First, our work focuses CAD generation from textual inputs. Second, while works in these categories often use transformer encoders to extract features and decode specific IDs, they do not unify the encoding procedure into model due to technical limitations. Third, while some of the previous works (Xu et al. 2022 2023) train transformer modules using their codebooks, they do not involve texts natural language imputs and thus have need to leverage pretrained LLMs. This prevents the need to add alignment modules in the classic paradigms.
Text-based CAD Tasks. Text-based CAD tasks focus on generating CAD sequences based on textual instructions. Inputs for these tasks fall into two main types: the first type includes only textual instructions on how to generate an initial CAD model, and the second type includes a CAD sequence (of the same representation format) and a textual instruction on how to modify the given model.
The former is the Lext-to-CAD task (Wang et al.) 2025a Li et al. 2024 2025b Khan et al. 2024b) Govindarajan et al.] 2025; Guan et al. 2025 Wang et al. 2025b) and the latter is the CAD-editing task (Yuan et al., 2025). Our work differs from these tasks as we propose the text-based CAD prototyping
task, which merges them. Given that initiating and editing CAD models is standard procedure in industry, a single-model solution both necessary and promising for the prototyping pipeline.
2.2 LLM TOKENIZATION
LLMs operate over discrete tokens that are mapped into hidden embeddings. In natural language, these tokens are defined by vocabularies (Pennington et al. 2014 Mikolov et al. 2013) or obtained via algorithms such as byte-pair encoding (BPE) (Gagel 1994). In multimodal tasks where inputs often lie outside the space of predefined vocabularies, external encoders and decoders such as VAEs and VQ-VAEs (Kingma & Welling, 2022 vaa den Oord et al. 2018), vision transformers (Dosovitskiy et al. 2021 Radford et [0.2 or diffusion- beafus algorithms (Podell et al., (023) are commonly used map continuous modalities into token sequences that LLMs can process.
Tokenization strategies vary across domains. For visual tasks, pretrained VAE/VQ-VAE or CLIP/VIT encoders are typically used, sometimes paired with custom decoders when generative outputs are
required (Zhou et al. 2024; Ge et al. 2023] [025) Liu et al. 2023). In robotics, many works bypass neural tokenizers, instead stringifying actions for direct use with default LLM tokenizers (Brohan
et al. 2023b]a), a). or applying lightweight a algorithmic encodings (Pertsch et al. 2025)). Time-series modeling has explored both neural tokenizers based on attention architectures (Nie et al. 2023) and
classic tokenization methods (Li et al. 2025a).
Prior CAD works are task-specific on tokenizations. Classical CAD generation methods train separate encoders and decoders for different inputs, such as sketches and extrusions (Xu et al.) 2022 2023
Khan et al. 2024b a), resulting in multi-encoder pipelines. Recent text-based CAD efforts typically rely the-theeshelf LLM tokenizers, directly applying LLM tokenizations to CAD sequences without adaptation (Wang et al. 2025at Yuan et al. [025 Zhang et al. 2025). In contrast, our work introduces the first encoder-decoder tokenizer specifically designed to integrate with LLMs. We employ single
VQ- VQ-VAF-vvrriant encoder-decoder pair to tokenize all CAD inputs and align its vocabulary with the LLM backbone. This unified tokenization allows the model to reason over CAD primitives as discrete units, addressing the fragmentation issues inherent in natural- lanuual-anguuge tokenizers.
3 METHODOLOGY
3.1 APPROACH OVERVIEW
The text-based CAD prototyping task unifies Text-to-CAD generation and CAD editing. Formally, given prompt = (I, Corig) consisting of an instruction and an optional original sequence Cozig. the goal is to learn mapping f(-) that outputs a parametric sequence Cgen- For Text-to-AAD, Corig = 0. while for editing tasks, Cosig is a valid CAD sequence. The generated sequence should reflect the design intent expressed by the instruction and, when applicable, the original shape.
CAD-Tokenizer addresses this unified task by fine-tuning LLMs with CAD-specific tokens produced by pretrained VQ-VAE (Figure 2). Specifically, (I) primitive-basse VQ- VAE is trained on CAD sequences to compresses the sketch-extrusion pairs into discrete tokens, and adapters are deployed to align these tokens with the backhone's embedding space (Section 3.2). (2) the backbone is fine-tuned
3

---

Published as a conference paper at ICLR 2026
1. VQ PRETRAINING
2. VQ-LLM ALIGNMENT
REcONSTRUTTION
SEQUENCE RECONSTRICTION LOSS
CAO Seo.
VQ NEDAREST RE PREEIAL
OUTPUT Sea
CAO Seo.
QUTNUT Sco
LAMA EXIEDDIIG
LINEAR ADAPTER
EXCOOEER
RRMNSFORMER
DEcooFR
VQ ENCOPER
DQ DECOPER
loGit LAYER
castraiioe Posccos
Proe THAINIM-S
3. LLM FINE-TUNING WITH CAD TOKENS
4. INFERENCE
FONIE--STATE CAD AUTONATON
Exconen ADAPTER
ADAPTER DECODER
LANGUAGE Mones
PRETRAINED LLM BACHBONE
FINETUNED BALKSONE
-FINITS SAMPLING
CAU Si4t.
TOWENS
GOLPRATION
Figure 2: Overview of the CAD- Tokeeienr framework. (1) A primitive-based VQ- VAE tokenizes CAD sequences into discrete primitive tokens. (2) Adapter modules align these tokens with the LLM embedding space. (3) The backbone LLM is fine-tuned using CAD-specific tokens. (4) At inference, an FSAAguidedd sampling strategy ensures syntactically valid CAD generation.
on the encoded CAD data (Section 3.3 and (3) an FSA-based decoding strategy enforces CAD grammar during inference (Section We describe each step below.
3.2 PRIMITIVE-BASEL VQ-VAE AS CAD TOKENIZER
Pretraining Primitive VQ-VAE, To convert CAD sequences into compact symbolic representations, we modify and pretrain a VQ- variant that outputs primitive-level tokens. Each input sequence C' needs be decomposed into sketch-extrusion pairs SE (4,,..,,4, and encoded into latent vectors (P',-..,P'k) with 1 < in pairs by transformer encoders. Unlike standard VQ-VAEs that pool the entire sequence into single latent vector, we introduce primitive-speciff pooling layer, producing multiple discrete representations for each both local and contextual information from the surrounding sequence. Training follows a sequence reconstruction objective with an accumulated VQ loss across all pooled tokens:
pair. This design captures
(DD((Decoder(("))))t----)))) + EVQTT PT). JEK
LvQ-Prim
IEn
where EMD is the squared Earth Mover's Distance Loss, {r/} denotes the set of all primitive latent vectors (r',..., p/)) generated by the encoder and pooling layer, 1, denotes the input data's one- hot property, and T denotes the pooled vector the tokens construct up the j-th primitive. We demonstrate this method in Figure 1) and provide additional details such as the detailed pooling procedure in Appendix B
Aligning CAD Tokenizer with LLM Embeddings. The raw primitive vectors {2G}} have dimension dug. while the LLM requires tokens its embedding space of dimension diok- Typically, this adaptation is performed during co-training with LLM backbones (Liu et al., (023), which is compute- intensive and does not guarantee alignment in the reverse direction. We introduce novel use of adapters that map VQ-VAE outputs and LLM embeddings bi-directionally. In detail, we leverage the frozen embedding and logit layers of the pretrained LLM. Adapters are trained with a vector reconstruction loss that encourages the mapped tokens {p> argmax [togit (Whog faithful to their raw representations:
diok (f/)))} remain
Lresun
-' p9112 + [[eembeedpp)) (p}) Whow (1/))12-
4

---

Published as a conference paper at ICLR 2026
P) Widue (P))) is the reconstructed embedding corresponding to {p/}- The latter term the reconstruction of the post-embedding layer tokens and the pre-logit tokens ensure both adapter layers are trained. This alignment produces native L.M-rrcognizablee primitive IDs, enabling seamless integration without modifying the backbone. The procedure illustrated by Figure 2
3.3 INSTRUCTION TUNING WITH CAD TOKENS
With aligned tokens, CAD sequences be encoded into compact symbolic forms and directly used for instruction tuning. Given an input prompt (I, Corig) and target sequence Cgen. we define and y CAD-Ence Eneodercoeeer (Cgea) We fine-tune the pretrained LLM by minimizing the standard cross-entropy loss between the generated tokens ÿ f(x') and ground-truth y' as described below:
x' (I, HADD-Ecoder(Cazigg
T
LSrT (xr)"))))
log p(ū 1/12²')
T
Because CAD sequences significantly compressed, i.e., |a'| < |arl. fine- fine-tuning is not only more effective but also more computationally efficient. Figure 3(3 illustrates this stage.
3.4 MODEL INFERENCE AND TOKEN DECODING
During inference, naive autoregressive sampling may produce invalid CAD sequences, as standard L.M decoding strategies (e.g., top-p, beam search) are unaware of CAD grammar. However, unlike natural language, CAD sequences are formal languages and can be fully described by states and transitions. Therefore, we can improve the sample quality by designing finite-state automaton (FSA) that formalizes valid CAD construction rules (Figure 4)).
At each step, the FSA provides series of masks restricting the candidate logits grammar-comee commaar-complint tokens, ensuring syntactic validity. The automaton state evolves with the decoder, whose choice for the next token also passed to the automaton deciding a transition going to take place. Despite some context-dependent geometric constraints are unable to fully catched, this method significantly reduces syntactic errors and improves sampling robustness by preventing obvious grammatical mistakes. We demonstrate the detailed process by Algorithm I and the FSA design in Figure Further implementation details are given in Appendix B.2
Algorithm 1 The FSA-driven decoding algorithm.
Given: LLM-generated primitive tokens T
Given: Max length of generation mar_len Stale + 'init'
M + (iintiallmmsstt
SSeq <- (
while SSgl < max_len do n.xxt()
logits + Decoder(Ser) (Seq, T)
choice am logits)
Seq.aapeen(chhice)
if M () then
new new_state, new M new_M FSA(State, choice) State <- new state
M +- new M
end if
end while
5

---

Published as a conference paper at ICLR 2026
Zloop.
<face cpua
lend-at-loopi jrrmtve-star]
Sop_enn epua
[end-ot-tae]] [prmithe-ssartt
(end-ot-skatehj- (primivv-star)
<curve_ ends
<kketch end>
line
circle
STATES
[6] X 1,
[numbers] x (end of curve]
[ummbers] x (end cf-cuve)
[rumbers] x (end of curve]
TRASITIONS
(0) x
<>> DECODED TOKENS
[end-ol-extn.slon)
[M] xm: SERIES
line
arc
circle
pad
<extrusion_ent
Init
[PAD]
Figure 3: An overview our FSA design. At each step, the FSA receives an input action, transitions to the corresponding new state (i.e., updates its internal state), and returns the mask(s) associated with that new state (node). The FSA map is specific to our CAD sequence representation, which can be found in Appendix
EXPERIMENTS
4.1 EXPERIMENTAL SETUPS
Datasets. For training the VQ-VAE, we convert the SkexGen (Xu et al. (022) dataset into the formatting used by CADFusion (Wang et al.) 2025a) and CAD-Editor (Yuan et al. 2025). We include the formatting details in the Appendix A seen and unseen data, we use half of the training data for VQ-VAF training, ensuring the L.M also encounters samples not seen by the VQ-VA VAE encoder during its training. training the backbone LLM,, we use the datasets of CADFusion and -dD-oor Although both originate from SkexGen, the two datasets have different content, so we merge them for training our model without further deduplication. Since CAD-Editor uses the original split of SkexGen but the entire CADFusion dataset split from its training set, we consolidate the CADFusion dataset and additionally annotated 200 pairs for our test set. We also balance this combined dataset by randomly selecting five EAD-Editing data points for every Text-to-CAD data point. Ultimately, this results in approximately 100k training pairs and 1000 testing pairs.
To prevent distribution gap for the tokenizer between
Implementation Details. The CAD sequence formalization and primitive selection detailed in Appendix A The VA-V is trained with 5 encoder and 5 decoder layers, codebook size 2048, and an internal dimension of 512. It is trained with a learning rate of 3e-4 for up to 250 epochs, using early stopping with a patience of 25. The logit and embedding layers inserted are from LLaMA 3- (109)-dim,, which is also used as our LLM backbone. The aligmment uses exactly the same setup, but the encoder, decoder and LLM layers are frozen. and only the two adapter layers are trained. For LLM training, the backbone is configured with maximum token length of 1024 and a hidden dimension of 4096. We adopt Low-Rank Adaptation (LoRA) (Hu et al.) 202T) for efficient fine-tuning, with rank of 32 and an value of 32. The model is trained for 20 epochs with learning rate of 1c-4 and the AdamW optimizer. The VQ- TQ-VAE training is run on a single NVIDIA A100-80G GPU, and the LLM fine-tuning is run by four NVIDIA A6000-48GB SMX GPUs using Distributed Data Parallel (DDP).
We select two types of baselines. First, we include CADFusion and CAD-Editor as state-of-te -ftthh-rtttttttt methods in Text-to-CAD and CAD-editing tasks, respectively. However, since these are for individual text- bast- tasks only and no existing works address text-based CAD prototyping, propose additional baselines for the unified task. First, train vanilla LLM model using
Baselines.

---

Published as a conference paper at ICLR 2026
Methods
F1-Skt
F1-Ext
CD
COV
JSD
MMD
IR
VLM
HE
CAD-Editor
73.3
82.6
40.7
51.1
1.63
1.25
1.50
4.28
1.63
GPT-40
80.0
78.1
42.9
51.8
4.00
2.14
47.9
1.94
Vanilla-LLaMA
78.8
84.9
42.4
48.6
3,90
2.37
48.6
4.31
2.64
CAD-Tokenizer
88.6
94.8
13.5
52.4
2.54
1.97
8.38
5.09
1.72
(a) CAD-Editing quantitative results, "CAD-Editor is the task-specific model on it.
Methods
FI-Skt
F1-Ext
CD
COV
JSD
MMD
IR
VLM
HE
CADFusion
68.8
80.1
38.5
54.4
10.6
3.86
22.5
5,41
1.91
GPT-40
66.7 7
66.8
79.8
52.6
62.0
13.5
90.5
1,47
Vanilla-LLaMe
66.4
80.9
48.4
53.8
27.4
72.6
80.5
3,45
2.58
CAD-Tokenizer 77.9
84.7
26.7
54.5
7.26
3.74
1.50
3.82
1.62
(b) TexD-to-CAD quantitative results. CCADFusion is the task-specific model on it.
Table 1: Results in (a) and (b) include scores for sketches (F1-skt f) and extrusions (F1-ext f),
Chamfer Distance (CD 1), Coverage(COV 1), Minimum Matching Distance (MMD 4), Jensen- Shannon Divergence (JSD 4), Invalidity Ratio (IR 4), the VLM Score (VLM 1), and average rank from Human Evaluation (HE 4). GPT-4o generations were not included in the human evaluation process. (f) marks the higher the better, and ()) marks the opposite. The best results are marked in bold, and sccond-best results underlined.
the native LLaMA tokenizer (VanillaLLLaMA), with all other aspects of the setup identical to CAD-Tokenizer. Second, we prompt a five-shot GPT-4o to examine how modern commercial LLMs perform the text-based CAD prototyping task. Details including baseline implementation and prompting samples can be found the Appendix C.
Metrics. Our evaluation focuses different aspects of the generated CAD models. First, measure the pairwise correspondence between generated outputs and the ground truth sequences. For this criterion, we use F1 scores and Chamfer Distance (CD). We follow CADFusion's (Wang) et al. 2025a) measurements of F1-Sketch and F1-Extrusion for simplicity. Second, we measure distributional similarity. Between the generated samples and the ground truths, we follow (Xu et al. 2022 Yuan et al. 2025 Wang et al. 2025a) and measure: Coverage (COV), which quantifies how well generated distribution covers the ground truth: Minimum Matching Distance (MMD), which evaluates similarity by finding the closest matches between samples from the two distributions; and Jensen-Shannon Divergence (JSD) for distributional similarity. Third, we measure the Invalidity indicator of sequence robustness; and fourth, we measure the generated outputs' aligmment with instructions using a VLM-based score (VLM) and Human Evaluation (HE). The VLM is prompted to rate instruction following a scale of 0 to 10, and human judges are asked to rank the outputs from different models based on the same criterion. Details on VL.M prompting and human evaluation methodology can be found in Appendix C.I
Ratio (IR)
For the clarity of the presentation, the results of the distribution scores (COV, JSD, MMD) and chamfer distance in all records are multiplied by 10?.
4.2
MAIN RESULTS
Quantitative Evaluation. Table I reports results on CAD editing (a) and Text-to-CAD generation (b). On CAD editing, CAD-Tokenizer achieves the best results across almost all metrics. It improves the Fl scores by around 10 points each. Compared to the task-specifi CAD-Fditor, CAD-Tokenizer leads in F1 metrics while also achieving stronger VI.M and human evaluation scores. Moreover, discoveries detailed in the supplementary results of Appendix C.2. suggest that our model is best intended to edit the original object despite being disfavored by the distributional metrics by doing so.
On Text-to-CAD generation, CAD-Tokenizer again shows clear improvements. It significantly improves the Fl scores, indicating strong matching in CAD sequential information. Chamfer Distance is also decreased by large margin. Although CADFusion attains slightly higher VLM score (5.41 vs. 3.82),
surpasses in nearly every other metric, including lower human evaluation rank that indicates preference.
7

---

Published as a conference paper at ICLR 2026
COMPRESSION RATIO (%)
Methods
F1-Skt (1)
COV (1)
JSD (D)
HNC-CAD
85.5
57.5
29.8
CAD- Tok-oeenierrr (curve)
94.1
64.5
8.19
CAD-Tokenizer loop
91.5
59.5
18.4
HNC
CAD-Tokenizer single
76.5
54.0
35.9
BPE CURVE
Table 2: The ablation study the reconstruction quality of the CAD-Tokenizer variants and HNC-CAD. Only the sketch scores are reported for the F1 score because the CAD- Tokenizer encodes objects by cketch-extrusion pairs and al- ways achieves full score for this sub-metric. CAD-Tokenizer (curve) is the default variant that we reported in main results.
Figure 4: The Compression ratio of the different tokenization algo- rithms. The compression ratio is 100% for no compression.
Besides the advantage of our framework over task-specific baselines, is noteworthy that Vanilla- LLaMA almost collapses on the majority of metrics. This supports our conjecture that text-based CAD prototyping inherently a challenging task for conventional tokenizers, and highlights that our improvements the vanilla tokenizer and training paradigm are substantial.
Qualitative Evaluation. Figurrsjillustrates typical generations of our model and selected baselines. In Text-toCCAD, CAD-Tokenizer produces more faithful multi-face objects compared to Vanilla- LLaMA, which often drops faces or misaligns structures. Also, the outputs are natural and balanced, which resemble human designs. Its qualitative performance is competitive with CADFusion, the task-specific baseline.
For CAD editing, CAD-Tokenizer demonstrates stronger instruction-foolloingg it modifies existing shapes as required rather than persisting the original form, a failure mode described in Appendix C.2.3 Compared the vanilla tokenizer, CAD-Tokenizer better respects structural templates and achieves higher-quality modifications.
Additional results including more complex prompts and designs can be found in Appendix C.2.2
4.3 ABLATION STUDIES
We analyze the contribution of each component in CAD-Tokenizer across three stages: pre-SFT tokenization, in-SFT integration with the LLM, and SFS sampling.
Before LLM Finetuning: the Quality of Tokenizers. The tokenizer quality critical to downstream (i) reconstruction quality, in terms of selected sequence and distribution metrics, and (ii) compression ratio for efficiency. We compare CAD- Tokenizer against the following: 1. HNC-CAD (Xu et al.) 2023), a strong neural coding baseline on conditional and unconditional CAD generation, 2. BPE Gage 1994), the Byte-pair Encoding, standard tokenization algorithm, 3. CAD-Tokenizer (curve), our default tokenizer with curve-based pooling, 4. CAD- Tokenizer (loop), variant pooling loops instead, and 5. CAD-Tokenizer (single), which omits primitive pooling and encodes each sketch exttcc-inttsionn pair as a single unit.
LLM performance. We evaluate two aspects:
Table and Figure presents the results. Except that BPE is not involved in the reconstruction evaluation as guarantees returning original vectors algorithmically. the exploration is three-fold: variants outperform prior methods on both reconstruction quality and
First, all compression. The CAD-Tokenizer variants perform uniformly better than other tokenization methods, while having better compression rate. Second, within CAD-Tokenizer, the results also highlight the trade-off between compression rate and the reconstruction accuracy, as the variants that compress more tokens into embedding tend to have worse quality after reconstruction. Third, the sharp degradation of the single variant underscores the importance of primitive-level pooling.
In LLM Finetuning: the Performance of Different Tokenizion Methods. We next evaluate the tokenizers during supervised fine-tuning (SFT) with the LLM backbone. All methods are included except HNC-CAD, which was not designed for LLM training.
As shown in Table 3 both BPE and the single variant CAD-Tokenizer perform poorly, with high invalidity ratios and consistently weak metrics. Their setups justify the unsatisfactory performance:
8

---

Published as a conference paper at ICLR 2026
Pencorte endin arrer
riam coron en rat iaval de
TEm--O-CAA INSTUCTTION
X
CADFUSION
VANILLA-LLMA
dtalis aut bul
meuume tolt algriie seccia copert.
Head Late Honerin
eooddro Roald PutLe shaeed ont
QUALITATIVE DEMONSTRATION TEXI-BASUD CAD EDITING
Oaso SHoPe
EarTING DNSTFLCTION
Grouna TRUTH
Figure 5: Qualitative results. The upper and lower sections display results for the Text-to-CAl CAD and CAD editing tasks, respectively. In each subfigure, the top-left corner shows the input instruction and CAD object (or only the instruction in Text-to-CAD), the top-right corner shows the ground truth, and the bottom three show outputs from CADFusion/CAD--Edoor Vanilla-LLaMA, and
the single variant performed already poor enough during the pre-finetuning reconstruction evaluation; and BPE does not align the post-tokenization tokens with the LLM vocabulary, which adds up additional burden during finetuning. The loop variant achieves performance close to the default curve tokenizer, with slightly lower reconstruction but higher compression. This not only indicates that primitive pooling-Whether whether curve- or loop-based is essential, but also offers flexibility and alternatives in trading off accuracy against efficiency.

---

Published as a conference paper at ICLR 2026
Methods CAD- (curve)
F1-Avg (1)
CD (1)
COV (1)
JSD (J)
IR (J)
86.5
20.1
53.4
4.90
4.94
AAD-ooknizerr loop
86.3
32.6
53.6
7.05
4.91
CAD-Tokenizer single
78.3
59.1
43.4
48.5
70.7
BPE
76.2
56.8
48.8
48.3
88.5
Table The ablation study of each tokenizer when integrated and trained with the backbone LLM. CAD-Tokenizer (curve) is the default variant which we reported in the main quantitative results, while loop is variant on pooling loop primitives and single does not perform specific pooling.
Methods
F1-Avg (1)
CD (4)
COV (1)
JSD (4)
IR (L)
MAD--Tokeiizer(+FSAA
86.5
20.1
53.4
4.90
4.94
CAD-Tokenin WADD-Tokenizer+top-22
80.4
66.4
30.3
61.2
17.2
JAD-Tokenizer++bam search
82.8
49.8
51.2
46.7
45.2
Table The ablation study on the generation quality of each sampling method after training. The base model used for evaluation is the fine-tuned CAD-Tokenizer. CAD-Tokenizerr<#5S. is the default variant which we reported in the main quantitative results.
Different Sampling Strategies. Finally, we assess the impact of FSA-based decoding strategy described Section 3.4 We fix the backbone to the fine-tuned CAD- Tokenizer and apply different sampling strategies on top of it, and compare our method with the two mainstream approaches: top-p sampling and beam search.
After LLM Finetuning:
Table shows that FSA-guided sampling consistently outperforms both alternatives across all metrics. Its FAD-specific design leverages formal sequence constraints, enabling it to utilize structural formatting information that purposs decoding methods do not take into account. Also notably, beam search improves the model performance in all aspects comparing to the top-p sampling, but encounters a significant tradeoff in invalidity ratio.
LIMITATIONS
Our work's limitations are two-fold. First, the quality gap between the open-source and private- sector CAD data limits us from training further complex shapes. Second, mentioned and referred, we observe a gap between the distributional metrics and the actual performance in CAD Editing, which stems from the failure to evaluate the 'intent to keep the original shape." We therefore anticipate both better datasets and metrics that can create a better ground for future works to improve. Additionally, our failure study in Appendix C.2.3 suggests better spatial or commonsense reasoning capabilities, which can only be improved by better pretrained backbones. We include some of the related diseussions, well the detail of our failure study, in Appendix
CONCLUSION
In this work, we presented CAD-Tokenizer, the first framework to address the unified text-based CAD prototyping problem, a task that requires a unified model on both Text-to-CAD generation and CAD editing. Key CAD-Tokenizer's approach is pretrained transformer-based VQ- VQ-VAE module that enables CAD-specific tokenization by converting sequential CAD inputs into meaningful primitive tokens, instead of the word pieces produced by vanilla tokenizers. LLM backbones that were finetuned on the CAD specific tokens demonstrated better performance and training efficiency. Our extensive experiments validated these core design choices, including the primitive-specine pooling mechanism within the VQ-VAE and the FSA-driven sampling, and demonstrated the advantage of over both task-specific and general-puryy purpose basclines. We also improved inference time decoding quality by introducing novel finite-state automaton eirivve sampling method, designed to enforce sequence formatting and hence improve generation quality.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
The data used in this work is tailored for CAD generation. Due to its specialized nature, the misuse risk is naturally minimized, ensuring the developed methods primarily benefit CAD development.
REPRODUCIBILITY STATEMENT
We have included sufficient information on our all aspects of our experimentations including dataset resources and setups, model hyperparameters and LLM backbones, computational resources and setups, and metric setups. Relevant detail can be found in Section 4.1 and Appendix C.I
THE USE OF LARGE LANGUAGE MODELS
The use of LLMs is restricted in revising the word choices and grammar only. No LLMs are used for research ideation contributive enough be regarded contributor.
ACKNOWLEDGEMENT
We would like to thank Lexin Zhou, Rushuai Yang. Yun Pan for insightful discussions. We appreciate the discussion with Zhuoyu Jiang and Yuheng Lu during the rebuttal phase. We also acknowledge the support from the MSR Asia Industry Innovation Center for motivating real-world examples and guidance industrial pipelines.
REFERENCES
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choro- manski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, Karol Hausman, Alexander Herzog, Jasmine Hsu, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Isabel Leal, Lisa Lee, Tsang-Wei Edward Lee, Sergey Levine, Yao Lu, Hen- ryk Michalewski, Igor Mordatch, Karl Pertsch, Kanishka Rao, Krista Reymann, Michael Ryoo, Grecia Salazar, Pannag Sanketi, Pierre Sermanet, Jaspiar Singh, Anikait Singh, Radu Soricut, Huong Tran, Vincent Vanhoucke, Quan Vuong, Ayzaan Wahid, Stefan Welker, Paul Wohlhart, Jialin Wu, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Tianhe Yu, and Brianna Zitkovich. Ri- 2: Vision-language-action models transfer web knowledge to robotic control, 2023a. URL https::/arxiis
ab/p//////22// 15818
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Tomas Jackson, Sally Jesmonth, Nikhil Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Isabel Leal, Kuang-Huei Lee, Sergey Levine, Yao Lu, Utsav Malla, Deeksha Manjunath, Igor Mordatch, Ofir Nachum, Carolina Parada, Jodilyn Peralta, Emily Perez, Karl Pertsch, Jornell Quiambao, Kanishka Rao, Michael Ryoo, Grecia Salazar, Pannag Sanketi, Kevin Sayed, Jaspiar Singh, Sumedh Sontakke, Austin Stone, Clayton Tan, Huong Tran, Vincent Van- houcke, Steve Vega, Quan Vuong, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Tianhe Yu, and Brianna Zitkovich. Rt-1: Roboties transformer for real-world control at scale, 2023b. URI. https::/arxiv... wwwp:/iwwwrg.....ooo wtw.g/bb///a. 22t.s/22222cc 06817
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image worth 16x16 words: Transformers for image recognition scale, 2021. URL htt.itttss://arttto wrg.
Philip Gage, A new algorithm for data compression. The C Users Journal. 12(2):23-38. 1994.
Yuying Ge, Yixiao Ge, Ziyun Zeng, Xintao Wang, and Ying Shan. Planting a seed of vision in large language model. arXiv preprint arXiv:2307.088. 00.00041,, 2023.

---

Published as a conference paper at ICLR 2026
Yuying Ge, Sijie Zhao, Jinguo Zhu, Yixiao Ge, Kun Yi, Lin Song, Chen Li, Xiaohan Ding, and Ying Shan. Seed-x: Multimodal models with unified multilgrauularity comprehension and generation, 2025. URL httpwaaxvvv...////.//e.. org/ abs
Prashant Govindarajan, Davide Baldelli, Jay Pathak, Quentin Fournier, and Sarath Chandar. Cadmium: Fine-tuming code language models for text-driven sequential cad design, 2025. |
//arxiv. org/ http:///abb22270999999
Yandong Guan, Xilin Wang, Ximing Xing, Jing Zhang, Dong Xu, and Qian Yu. Cad-coder: Text- to-cad generation with chain-of-thought and geometric reward, 2025. URL httpt:t/ss//iii
org hag/abs/ee 2505
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. URL https
// //arxiv.org///oooo abs /ttp:/.1.....000
Mohammad Sadil Khan, Elona Dupont, Sk Aziz Ali, Kseniya Cherenkova, Anis Kacem, and Djamila Aouada. Cad-signet: Cad language inference from point clouds using layer-wise sketch instance guided attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4713-4722, June 2024a.
Mohammad Sadil Khan, Sankalp Sinha, Sheikh Talha Uddin, Didier Stricker, Sk Aziz Ali, and Muhammad Zeshan Afzal. Text2cad: Generating sequential cad designs from beginner-to-exper level text prompts. In The Tiihty- Annual Conference on Neural Information Processing Systems, 2024b. URL httpt::////// batttps://ppenreeeeee
net forum? id 5k3XeHIK3I
Diederik P Kingma and Max Welling. Auto-encoding variational bayes, 2022. URL https
6114
Hao Li, Yuhao Huang. Chang Xu, Viktor Schlegel, Renhe Jiang, Riza Batista-Navarg Goran
Nenadic, and Jiang Bian. Bridge: Bootstrapping text to control time-series generation via multi- agent iterative optimization and diffusion modelling, 2025a. URL
org/
abs/2503, 02445
Jiahao Li, Weijian Ma. Xueyang Li, Yunzhong Lou, Guichun Zhou, and Xiangdong Zhou Cad-llama: Leveraging large language models for computer-aided design parametric 3d model generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2025b.
Xueyang Li, Yu Song, Yunzhong Lou, and Xiangdong Zhou. CAD translator: An effective drive for text 3d parametric computer-aided design generative modeling. In ACM Multimedia 2024, 2024.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. URL httts::/aarxtt
httg//b///23000 08485
Weijian Ma, Shuaiqi Chen, Yunzhong Lou, Xueyang Li, and Xiangdong Zhou. Draw step by step: Reconstructing cad construction sequences from point clouds via multimodal diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 27154-27163, June 2024.
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation word representa- tions in vector space, 2013.
wrw. htsp//bs/13300 37811
Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-tern forecasting with transformers, 2023. URL htllttttp:///xxi.vt wwwpaaxii..rrggaaaa org/abs/////..oooo...
14730
Jeffrey Pennington, Richard Socher, and Christopher Manning. GloVe: Global vectors for word
representation. In Alessandro Moschitti, Bo Pang, and Walter Daclemans (eds.), Proceedings of
2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532-1543, Doha, Qatar, October 2014. Association for Computational Linguistics. doi: 10.3115/w 162.
URL https: / / antho log.. ogy.org/p14-11122
12

---

Published as a conference paper at ICLR 2026
Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for vision- ssion-languagg--cttin models, 2025. httthtt////aakeet wtt. abs /ww.aa/w......... 09747
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. 2023. URL hwt.hhttps/eeee http///axvvvvv.. org/ abs/ 2tt.//307.000 01952
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. URL https
//arxiv. wwtaiiv.rrgaaae abs /tab///2100
Ari Seff, Wenda Zhou, Nick Richardson, and Ryan P. Adams. Vitruvion: A generative model of parametric cad sketches, 2022. URL wwwptsss//aaatt.e wwtp://///l.a......//////// 14124
Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learning, 2018. Bttppt:ss////
abs 00937
Ruiyu Wang, Yu Yuan, Shizhao Sun, and Jiang Bian. Text-to-cad generation through infusing visual feedback in large language models, 2025a. bttps:s:://axxi wwtp.bs////abb/////// 19054
Siyu Wang. Cailian Chen, Xinyi Le, Qimin Xu, Lei Xu, Yanzhou Zhang, and Jie Yang. Cad- gpt: Synthesising cad construction sequence with spatial reasoning-enhancee multimodal lmms. Proceedings of the AAAI Conference Artificial Intelligence, 3988):888--8888 April 2025b. ISSN 2159-5399, doi: 10. 10.1609/aaii.v3900
URL htt.:htp://// dx doi org/ 1609/
aaai. w3918..1.... 32849
Rundi Wu, Chang Xiao, and Changxi Zheng. Deepead: deep generative network for computer- aided design models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6772-6782, October 2021.
Jingwei Xu, Chenyu Wang, Zibo Zhao, Wen Liu, Yi Ma, and Shenghua Gao. Cad-mllm: Unifying mulimoddalty-condiionee cad generation with mllm, 2025. URI. htttps://arxix.orrg
abs/2411 04954
Xiang Xu, Karl DD Willis, Joseph G Lambourne, Chin- Yi Cheng. Pradeep Kumar Jayaraman, and Yasutaka Furukawa. Skexgen: Autoregressive generation of cad construction sequences with disentangled codebooks. In International Conference Machine Learning, pp. 14699822472 PMLR, 2022.
Xiang Xu, Pradeep Kumar Jayaraman, Joseph G Lambourne, Karl DD Willis, and Yasutaka Furukawa. Hierarchical neural coding for controllable cad model generation. arXiv preprint 4:X1022307.00119, 2023.
Yu Yuan, Shizhao Sun, Qi Liu, and Jiang Bian. Cad-editor: A locate-then-infill framework with automated training data synthesis for text-based cad editing, 2025. URL https::/arxix: orq/ heq/abs/// 2502.00000000 03997
Zhanwei Zhang, Shizhao Sun, Wenxiao Wang, Deng Cai, and Jiang Bian. Flexcad: Unified and versatile controllable cad generation with fine-tuned large language models, 2025. URL https
7/arxiv. org/ abs/ /odb//22411 2411.1110 05823
Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model, 2024. URL BRRLhttps: pettps::/arxiie org 2408 11039
13

---

Published as a conference paper at ICLR 2026
A
ADDITIONAL INFORMATION OF THE CAD REPRESENTATION
A.1 SEQUENCE REPRESENTATION
We follow the practice of Zhang al. (2025); Yuan al. (2025); Wang et al. (2025a). In detail, CAD object is defined by series of sketches and extrusions, termed the Bhetch-and-Extrutt Modeling (SEM) format.
Each sketch consists of multiple faces, with each face typically containing Primitives in each loop include lines, arcs, and circles. They are defined by an identifier and, for their geometric properties, one, two, four key coordinates sets of parameters, respectively.
more loops.
Each extrusion represented as a BYVTTTRERRRERESSOO sequence. refers to the boolean operation selected from add, cut, intersect; the two values define the respective displace- ments of the top and bottom extrusion planes from a reference plane; the three T values form the translation vector; the nine R values represent rotation parameters; S is the scaling factor; and the two O values identify the origin of scaling.
Figure illustrates the components of the sketch and extrusion operations.
OVERVIEW OF SKETCH SEQUENCE
line, <curve _ndd
<loop_ end>
<face end>
<sketch end>
OVERVIEW AN EXTRUSION SEQUENCE add, 16, 31, 31, 31, 31, 1,0,0,0,0,1,000 1,0, 29, 31, 48, <extrude_end>
END
Figure 6: An overview of the CAD sequence structure. Top: sketch components; bottom: extrusion components.
In our VQ- VAE training, we tokenize CAD sequences into their smallest semantic units. These units defined in Table This tokenizer differs from native LLM tokenizers in that it does not further split these semantic units into word pieces.
Token Name
Vocabulary Index
Description
pad
Padding token
line are circle _nteeeendd>>d
1
Identifier token of curve: line
2
Identifier token of curve: are
3
Identifier token of curve: eircle
<curve <curve_end>
4
End token of curves End token of loops End token of faces End token of sketches
<loop_ends
5
<face_ende _nde__nddddd
6
<sketch end> add cut intersect <extrusion end> -1 0-63
8
Identifier token of extrusion operation: add Identifier token extrusion operation: cut Identifier token of extrusion operation: intersect End token of extrusions Number -1 Numbers from 0 to 63
10
11
12
13-76
Table Token vocabulary for the CAD sequence representation used in training our VQ- VAEE..
A.2 PRIMITIVE DEFINITION
We define our primitives differently for sketches and extrusions. For sketches, we define one primitive curve and any succeeding end tokens, if present. For extrusions, split each into three parts:
14

---

Published as a conference paper at ICLR 2026
STAGE 1. PRETRAINING VQ-VAE
STAGE 2. ALIGNING VQ-VAE
DECODER
FROM
line 10
add /ext
SEQUENET RESOOTTTTIOO LOCK
TRANSFORMER DECOBER LAYERS
ADAPTER
Vescee BUCTION
ENCODER
LLAMA ENBEDDING
CAO Toand
COVESOO
VQ NEAREST RETRIEVAL
Loos
ADAPTER
PROEINETRAND (UAMA,
LLAMA LOCIT
LINEAR ADAPTER LAYER
FROM PAETRANED & mazEN
PRIMETIVE POOLING
CON-SOO
VQ NEAREST RETREVA
TRANSI ORNER ENCODER LAYERS
ENCODER & PRINITIVE POOLING
PRIMITIVE
PRIMETIVE
PRIMITIVE N
PREMITIVE
PRIMITIVE
PRIMUTIVE N
Figure 7: A detailed overview of the VQ-VAE pretraining and finetuning.
B, V, T, R, and S, O, END, and split R into distinct tokens. This partitioning is designed to prevent the VQ-VAE from having to summarize overly long sequences, which can be difficult to learn effectively.
Our primitive design offers several advantages. First, limiting the scope the loop level restricts the number tokens the VQ-VAE model generates for the LLM to a manageable quantity. Generating token per curve would provide the LLM with too many tokens; conversely, token per face or per sketch would yield too few for the LLM to extract meaningful correlations and, in the case of per-sketch tokenization, might also represent effectively into a single token. Second, compared to works that do not pool primitives but instead train on smaller 'particles' (Xu et al. 2022; 2023), our approach improves representation quality by embedding all primitive representations into unified subspace (which is not guaranteed by methods training separate models for different CAD pieces) and by posing contextualization. Furthermore, those approaches may not adequately address the challenge of long extrusion sequences, which are difficult for VQ-VAE to compress meaningfully into single token.
much information for VQ-VAE to compress
B IMPLEMENTATION DETAILS
PRELIMINARY: TERMINOLOGIES AND NOTATIONS
We the term Encoder and Decoder to refer the transformer encoder and decoder layers in with in the VQ module. We refer the term VQ Encoder to the entire module before token extraction, iee. the transformer encoder layers, the primitive pooling, and the VQ layer. The same applies VQ Decoder. but since no additional modules are added, it is in fact equivalent to the Decoder. CAD Encoder and CAD Decoder describe the combination of the VO Encoder Decoder and the corresponding adapter layers.
B. VQ AND PRIMITIVE POOLING
classic VQ- VAE for sequential data consists of transformer encoder layers, a VQ layer to quantize vectors, and transformer decoder layers. Input tokens encoded, pooled into single hidden
15

---

Published as a conference paper at ICLR 2026
INPUT SEQUENCE OF
TOKENS AND K PRIMITIVES
<LOOP> <LOOP>
<EXTRUSION>
LATENT REPRESENTATION
GENERATE
PASSING ENCODER
PRUNITIVE
K RAW PRIMITIVE TOKENS
MASKS
REPRESENTATION
PABDINNS
EMBED_DIN
ELEVENT-
POOLINGe
PooLING
FooLING+
SPLIT INTO PRIMITIE-SSECIFIC MASKS
FIND SEIITIVE-SPECIFEC TOKENS
3. MAX POOL SELECTED TOKENS
Figure 8: An overview of our primitive pooling procedure for each sequence.
vector, and then mapped to codebook vectors. These VQ vectors are then fed into the decoder, which typically reconstructs the output sequence via a cross-attention mechanism, A traditional training procedure includes reconstruction loss between the decoder outputs and the initial inputs, well as a quantization loss.
This traditional approach of relying single hidden vector be insufficient for representing complex sequences like those in CAD. To obtain richer primitive representations, previous CAD works (Xu et al. 2022 2023) separated CAD objects into multiple primitives and trained individual VQ models on them. We argue that this approach has limitations because training separate models can introduce alignment issues, and primitives may not receive contextualized information that could be beneficial for their encoding. In contrast, we split each CAD sequence into its constituent sketch-extrusion pairs and propose primitive-sppeciee pooling layer replacing standard global pooling to generate distinct representations for these pairs.
Our primitive pooling is achieved by masking; i.e., we generate masks for each imput sketch-extrusiee pair to isolate each primitive within it. [ our method. In detail, we extract pooling masks from the input sequence that reflect primitive corresponding to each token. Subsequently, perform element- wise multiplication between each individual mask and the encoded representation output by the encoder. At this point, the resulting matrix contains information related only to the specific primitive isolated by the mask, allowing us to perform max pooling to obtain the pooled representation for that primitive. By concatenating these pooled primitive representations, we obtain sequence of primitive-spede speecffc representations used for the decoder's cross- atts-ntttnnioon mechanism and for subsequent LLM alignment.
We the maximum number of primitives in method to 12. This corresponds 9 flexible slots for loop primitives and 3 fixed slots for extrusion primitives. The limit for loop primitives is generally restrictive, as the average number of loops per sketch-extrusion pair in our dataset is approximately 4, and during VQ-VAE training, we filtered out only around 900 sequences (out of 90,000) due to exceeding this limit.
B.2
FSA SAMPLING
Our FSA sampling is based on a peecially-designed Finite State Automaton (FSA) that determines the set of permissible next tokens (referred to logit options). The system generates token (or tokens) based on these allowed options, and then updates the FSA by inputting the current FSA state and the last generated token action, to determine the options for the subsequent step.
16

---

Published as a conference paper at ICLR 2026
The algorithm is described in Algorithm I and the FSA's detailed design is illustrated in Figure The specific logit masks employed by the FSA are listed in Table
Mask Name
Mask Content
Description
Init
[line, arc, circle]
Enforces that the first token eurve type. Restricts selection to numerical values (0-63).
Numbers
[0-63]
<ccrve_ends>
Enforces the <curve suure__end>>>> token.
Primitive-start
[line, arc, circle]
Enforces new curve; identical to Init.
End-of-loop
[<loop_end>]
Enforces the dloop_end> token.
End-of-face
<<fac__end>>
Enforces the <face _nnd__nddd token.
End-of-sketch
(<sketch_endee [add, cut, intersect]
Enforces the <sketch_ennde _endeeddb token.
Boolean extrusion operations. Numerical values for V parameter.
[0-63]
[0-63]
Numerical values for T parameters.
R
[-1, 0, 1]
-1, O, or 1 for rotation parameters.
[0-63]
Numerical values for the parameter. Numerical values for O parameters.
O
[0-63]
End-of-extruse
[<extrusion_end>) [pad]
Enforces <extrusion_ _etioon___dd____
Pad
Restricts selection to the padding token.
Table 6: Details of the logit masks provided by the FSA to guide token generation during decoding.
Methods
COV
JSD
MMD
VLM(left)
GPT-40 <-> Original Seq.
60.8
6.75
2.39
1.94
CAD-Tokenizer <-> Original Seq.
52.4
2.54
1.97
5.09
Original Seq. Ground Truth
55.6
6.07
2.54
2.02
Table 7: Distributional measures for pairwise sequence comparisons from the CAD editing task. VLM scores of the left-hand-side component are reported.
C EXPERIMENTAL RESULTS
C.I EVALUATION SETUP
Instruction of Human Judges. Five human judges were provided with 50 generation outputs for evaluation (40 from editing tasks and 10 from Text-toCAAD tasks). All participants had completed college-level to graduate-level education. The judges were given specific instructions on the evaluation task and were shown visual examples similar those in the main qualitative results, but without model identifiers.
The following are series of pictures. The upper half is the instruction and the standand answer. You need to rank the following three pictures based on their response quality to the instruction.
For example, if in picture you like the most, following, and the worst, rank them as (2 1).
Listing 1: Instructions of human judges.
Instruction of VL.M. The prompts we use to generate VL.M scorings are listed Listing and 4.
The following is an original image of CAD instance, text description editing and an image of the edited result. Measure if the figure corresponds the gi ven description, and give score in the scale
17

---

Published as a conference paper at ICLR 2026
of 10. Only return the score. Do comment on issues such texture, smoothness and colors. description: (description) An
Listing Instructions of VLM on CAD Editing
The follewing is text descripticn of 3D CMD rigure and image of instance. Measure if the figure corresponds to the given description, and give a score in the scale of 10. Only return the score. Do not comment on issues such as texture, smoothness and colors. An description: :deccription An
Listing 3: Instructions of VLM on Text-toCAAD generation
C.2 EXPERIMENTAL RESULTS
C.2.1 ELABORATION ON CAD EDITING RESULTS ON THE DISTRIBUTIONAL METRICS
In the CAD Editing task, we observe that models such GPT-4o can achieve high scores on distributional metrics for the CAD editing task, despite poor actual performance in editing. We hypothesize that a model can frequently fail apply edits but still achieve satisfactory distributional scores by simply reproducing the input CAD sequence without modifications. To investigate this matter further, we provide additional distributional measures for three comparison types: between GPT-4o outputs and the original input sequences, 2. between outputs from CAD-Tokennier and the original input sequences, and 3. between the original input sequences and the ground truth edited sequences.
The results are displayed Table As shown the third row (comparing original inputs to ground truth), the original sequence yields high distributional scores. However, such outputs should be considered failed edits, as they do not incorporate any of the specified editing instructions. VLMs capture mnstruction-folloiin capabilities much better by assigning low score when output merely returns the original sequence. the original input sequences and the outputs from GPT-4o and our CAD-Tokenizer. It found that GPT-4o's 's outputs are much closer to the original sequences than those from CAD-Tokenizer, suggesting that our model more consistently attempts modify the original sequence and perform the requested edits. This pattern, along with our model's 's overall advantages, is more accurately reflected by the difference in VLM scores presented in the main quantitative results.
light of this, we measured the distributional similarity between
We draw the conclusion that the interpretation of the distributional metrics should be extra careful as it may not always reflect the true performance of the model. The analysis the CAD Editing performance should take all metrics (especially the qualitative ones) into consideration in order to get unbiased information.
C.2.2 ADDITIONAL QUALITATIVE RESULTS
We present additional qualitative results for CAD editing and Text-to-CAD generation in Figure and 10 respectively.
C.2.3 FAILURE CASES
In this section, discuss failure cases observed with method. As shown in Figure I1] classify these failures into three categories: overcomplicated testing entries, misleading instructions, and lack of spatial commonsense reasoning. We list our findings below:
1. For the first category, the model faces difficulty in generating correct shapes from compli- cated instructions. For example, extrusion resembling the number '4" poses difficulty, especially when the instruction phrases as "four polyhedra" without stating its resemble with the number.
18

---

Published as a conference paper at ICLR 2026
The 1D stape metargrannee net with lange ooe the thick features forr proteer every speced holes farctiona This adding commaniy uses moustigg poos, oftering anchtactoraal appt catton.
The dhape is dodecagoral prien merremerres long bud rully and para dodocagon espee enca recamnu right acjpowe
TEXT CAD INSTRUCTION
ang marta rang the prisiss symmenry trrtughout
GROUND TRUTH
CAD-TOKENIZEN
conere and holoww rectangstar cutur the corsiitant antteess creating unitom bonder around the cutout design ghoweisfe
diouur surfice tha incuudes central ewenty spaced cytiddical hoke sumourdin hole wrappes with indertir iedebattion
cicclar holes are it's fat faces The laggr lacatdd and sights sise the large maimanting ti diace mex oo aupkoarars, whcre prcoss
suter
support or ds mouning bracket.
The 30 duase consits twige rectangu prim vertisilty from one and creating L-inased form. The design iocosorate simple geaments mairtain unitee smucture scll ane ongula sestretk. etc.
The 3D shape ReCtanIAAT 018m win everty speced tole combi both
The dhase coneats of two sdenical rylindrict Each haatures consitert crouier cross sectins elengated, shapss emphatoe simpliity uni kodd Shucturl apel cation, suth mashinery..
retang promn
ahoughh ionper presente streamt
etments Jenodnami: ppoo aroe. The struttre anturcas is belatced and functional anstmete.
ofte
Figure 9: Additional qualitative results for CAD editing.
2. The second category describes instances where the instructions do not always accurately reflect the relationship between the original and target shape. This issue can arise because the CAD-Editor dataset was generated by prompting VLMs, for which achieving absolute robustness and trustworthiness remains open problem; this, in turn, poses instruction- following challenges for our model.
3.
The last category reflects lack of common sense or spatial reasoning, likely originating from limitations in the backbone LLM's capabilities. Generating concrete shapes such "key" or letter shapes, as well as arranging multiple objects an organized way, remains challenging.
19

---

Published as a conference paper at ICLR 2026
Remove the centra)
ASS a central CROUUS
EDITING INSTRUCTION
ORIG SHAPE
GROUND TRUTH
CAD-TOKENIZER
Replacs cylndor sole.
out firm a sold
curved przm vorically
ferove legs, dh taketie.
Reziace vettoll rectonguatr proo.
stoer Oynde rectinguaar
femrve pro..
hallow dil a crubee hole
though oues trp
Figure 10: Additional qualitative results for Text-to-CAD generation
20

---

Published as a conference paper at ICLR 2026
OVERCOMPLICATED TESTING ENTRIES
with syrineticiilyy amanged pollinucte.
ronlove one sirll athid сytieatcaaty
untellly risaigging fom bas
4
MISLEADING INSTRUCTIONS
Add fur symmattically
Facce smaller centered cytnder
Edennd he ght nemove L-hased chande prict.
tap
LACK OF SPATIAL REASONING OR COMMON SENSE REASONING
The 3D shaze featuees eorgabed recterpuler
The 3D shape reuerbrs capiaa letter le' with Stucture conizts of lirg semidimuor ait conozttdd Herisnttall strightscgnt fellow dedgr belanced and may applledd aierent archtectural industrill centexts.
The 3D -huce corn nite of thve seperste oyl tnstt cal cquorr configration nation around on trai cylindrr rusura daindaueed
arongoment of redtaeguu bicoss at the other
creaie
lyyuu
opp
astiritaall oentr The daaly
michection
jnzorpondes cammiweicn acgue
terting alemects.
structure
Figure 11: An overview of our model's failure cases. The layout follows that of Figures and 10
21


Reviews:
[{"review_id": "JAjph3pmie", "reviewer": "Reviewer_uche", "summary": "Authors proposed a new tokenizer to unify text-based CAD generation / editing. More specially they first train VQ-VAE to represent CAD sketch and extrude as CAD-specific DSL tokens. They then align those token embeddings with pretrained LLM tokens, effectively converting DSL tokens into LLM text indices. Then the base LLM is SFT on the new sequence of token indices. Authors demonstrate good results compared to using HNC-cad and the default BPE tokenizer.", "questions": "(1) Please make fair comparison to hnc-cad and espeically other text-based CAD generative models. Directly applying text tokenizer on abstract DSL is not a sensible comparison. BPE baseline should be considered in the setting when CAD DSL is converted to text or python code etc.\n\n(2)  After finetuning, would the LLM still maintain its original text generation and reasoning capability? Have authors did any ablation study on this? This could potentially be a huge disadvantage for this approach compared to text-based CAD generation, since for them the grammar and code structured are aligned with the pretrained model. \n\n(3) Visual editing results are very simple, much more simple than what I am seeing in other works using pretrained LLM. I highly suggest authors compare to those more recent methods (cad-llama, ca-mllm etc).\n\n(4) Have authors tried different LoRA settings? Recent experiments show that when data distirbution is different from training, it is ususally beneficial to use larger rank and sometimes full training is even better.", "limitations": "", "rating": 2, "confidence": 5, "soundness": 1, "presentation": 3, "contribution": 2, "strengths": "(1) The proposed VQ-VAE approach with max pooling is more token efficient than HNC-CAD.\n\n(2) Aligning CAD DSL to pretrained LLM embedding is interesting and works surprisingly well.\n\n(3) Authors demonstrate better result using the new LLM aligned CAD tokenizer.", "weaknesses": "(1) Sketch seqence and extrude representation largely follows previous work. Authors should make this more obvious as opposed to categorying those relevant works as \"classical cad tasks\". The CAD primitive-level parsing is actually a more compressed verison of hnc-cad with masking and single VQ-VAE as opposed to multiples. \n\n(2) The main comparison to hnc-cad is not sound. hnc-cad encodes deepcad data into a sequence of discrete codes. So the claim that it is not designed for LLM training is not fair. Anything that representat data as discrete indices can be used for LLM training. In fact an autoregressive Transformer was trained from scratch for hnc-cad. Perhaps authors should more clearly state that a pretrained LLM was not trained on those discrete codes in that paper. But it should be easy to train one for ablation study, which the author did not do.\n\n(3) The main comparison to BPE tokenzier is not correct. Authors directly apply text-based tokenizer on the parsed sketch-and-extrude sequence, which is very abstract dsl and different from text. Some works might have did this but it is very obviously bad practice. In this case the performance could be bad because the compatiblity issue with data format and pretrained LLM model, not because of the BPE tokenizer. A more fair comparision is to convert CAD DSL into text like CADQuery, JSON, or python-like code. Then after finetuning a LLM or Code-LLM with default text tokenizer and test if the model understand CAD constructions. Actually there are already many works in this direction (cad-llama) and has proved that this works well with low invalid ratio. This should be much better than the 88% BPE invalid ratio reported in the paper. \n\n(4) Failure to compare to more recent text-based CAD generation / editing works such as CAD-Llama, CAD-MLLM and etc. Text generation and editing visual results also look much more simplier than those in related works. It will be more convincing to show that taken out the default text encoder in those models and switching to the one from the paper also leads to improvements. \n\n(5) Unclear if the alignment step is still necesary if the LLM vocabulary was expended to include the new CAD tokens, and LoRA finetuning uses larger rank and also update the embedding and logits layer.\n\nOverall, when it comes to something as fundemental as tokenizer, I hope the authors can be more careful about the comparision and make sure it is not a model or training issue that affects the final result. Right now it lacks comparsion to text-based CAD generation works and some important settings are not sounds. I feel liket this will have a high impact on the community. If the claims are true it will basically disencourge future works to directly train on DSL or text / code data paired with default text tokenizer. This is a huge change and should be better validated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RAmu7hmZuW", "reviewer": "Reviewer_A1Ms", "summary": "This paper introduces CAD-Tokenizer, a unified framework for text-based CAD prototyping that integrates both Text-to-CAD generation and CAD editing within a single model. The authors argue this joint formulation better reflects real industrial design workflows that are inherently iterative.\n\nThe core contribution is a tokenization method that replaces standard LLM tokenizers. Using a VQ-VAE, CAD operations are converted into discrete primitive-level tokens, yielding representations aligned with the structural and semantic nature of CAD data. An LLM is fine-tuned on CAD tokens to perform both generation and editing. This joint training improves performance and computational efficiency.Finite-State Automaton is used during decoding to enforce syntactic validity, ensuring outputs adhere to the CAD grammar. Experiments show CAD-Tokenizer outperforms task-specific baselines (CADFusion, CAD-Editor) and general models like GPT-4o across diverse metrics, including geometry, instruction following, and human evaluation.", "questions": "- How does the tokenizer’s 2048-codebook design scale with increased CAD complexity? Would a larger codebook affect alignment or fine-tuning efficiency?\n- How adaptable is CAD-Tokenizer to other CAD formats beyond Sketch-and-Extrude Modeling? Would retraining be necessary or could components be reused?\n- What types of geometric errors (e.g., self-intersections, open sketches) persist despite FSA-based enforcement?\n- Can you provide concrete comparisons of training or inference time relative to Vanilla-LLaMA?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Unifying CAD generation and editing is an important conceptual step that aligns research with practical design workflows.\n\nThe insight that standard tokenizers fail to capture CAD’s structured semantics is compelling, and the proposed primitive-aware tokenizer addresses this gap effectively.\n\nThe combination of VQ-VAE tokenization, adapter-based embedding alignment, and FSA-guided decoding is well-justified. Ablation studies clearly demonstrate each component’s benefit.\n\nThe experimental design is rigorous, using rich metrics and strong baselines. Results show consistent and convincing gains.", "weaknesses": "Due to limited open-source CAD datasets, experiments focus on simple shapes. The model struggles with complex geometries requiring advanced spatial reasoning.\n\nStandard distributional metrics (COV, JSD) are shown to be unreliable for editing tasks, as they can reward unchanged outputs. Although alternative VLM and human metrics mitigate this, the issue underscores a broader challenge in evaluation.\n\nAdditionally, some errors stem from the LLaMA-3-8b model’s limited spatial reasoning, capping overall performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iX5Ri5S5fn", "reviewer": "Reviewer_GPRz", "summary": "This paper presents a unified LLM-based model for text-driven generation and editing of Computer-Aided Design (CAD) models. The authors design a CAD VQ-VAE that compresses CAD operations into compact representations, which can tokenize CAD sequences into discrete codes. In addition, they also introduce linear adapters that align VQ embeddings with the LLM through an additional training phase. Experimental results demonstrate that the proposed method achieves state-of-the-art performance.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper demonstrates thoughtful design of the CAD tokenizer to enable seamless integration with the LLM framework, effectively bridging the gap between continuous CAD representations and discrete language model inputs.", "weaknesses": "1. What are the typical lengths of CAD sequences in your dataset? I have concerns about the scalability of the proposed pipeline. While the transformer encoder can handle variable-length inputs and capture relational information, the primitive pooling operation fuses all tokens within each primitive into a single latent vector before VQ encoding. As CAD sequences grow longer, the number of operation types remains constant, but the complexity of inter-operation relationships increases significantly. It is unclear whether this pooling-then-quantizing approach can adequately capture these increasingly complex relationships in longer sequences.\n2. Each extrusion operation contains precise parameters such as translation and rotation values. However, the VQ tokenizer maps continuous embeddings to a finite discrete codebook, which may result in quantization loss. This raises concerns about whether fine-grained geometric details are preserved, potentially leading to inaccuracies in the reconstructed parameter values.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wwVz4C7Nvq", "reviewer": "Reviewer_9hkP", "summary": "The paper introduces a novel framework for unified text-guided CAD prototyping that jointly handles both Text-to-CAD generation and CAD editing.\n\nThe authors propose a primitive-level VQ-VAE pooling strategy to compress sketch–extrusion pairs into discrete, compact and semantically meaningful tokens.\n\nThese tokens are aligned with the LLM’s embedding space via lightweight adapters, enabling efficient fine-tuning.\n\nAdditionally, a finite-state automaton(FSA) guides decoding to enforce syntactic validity.\n\nExperiments show that CAD-Tokenizer compresses and reconstructs sequences well across quantitative metrics. Ablation study proves the effectiveness of designed FSA-based sampling.", "questions": "The reviewer acknowledges the technical contribution of primitive-specific pooling layer and FSA-based sampling. \n\nHowever, if the authors insist that the main focus of the paper is the “unification of Text-to-CAD generation and CAD editing”, the reviewer would have some essential concerns/questions:\n\n1. Are the primitive-level VQ-VAE tokenizer and FSA-based sampling genuinely helpful for the **CAD-Editing** task? If so, how can this be demonstrated through ablation studies?\n\n2. Is the \"unification of Text-to-CAD generation and CAD editing\" beneficial to both **CAD-Editing** and **Text-to-CAD generation** if we exclude the improvements introduced by the primitive-level VQ-VAE tokenizer and FSA-based sampling?  How to prove it through experiment?\n\nSome additional important questions:\n\n1. In Figure 2(4), why is the instruction fed into the Encoder+Adapter? In contrast, in Figure 2(1), only the CAD sequence is fed into the encoder.\n\n2. Why does Figure 2(1) include “PRIM. TOKENS(RAW)”? It appears that both the VQ latent tokens and raw primitive tokens are input to the decoder for sequence reconstruction—can this be clarified?\n\n3. Why does Figure 2(2) also include “PRIM. TOKENS(RAW)”? What does the arrow labeled “For Training” signify?\n\n4. Although FSA-based sampling's ablation study is convincing, the reviewer is curious about why IR is not an exact 0% because invalid sequence may not be sampled due to masking in Figure 8 from Appendix.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The proposed primitive-specific pooling layer is thoughtfully designed and its effectiveness is convincingly demonstrated through the reconstruction experiments reported in Table 2.\n\n2. The evaluation across different levels of primitives—curves, loops, and single primitives—in Table 2 and Figure 4 clearly illustrates the trade-off between reconstruction quality and compression rate, making this key insight readily interpretable.\n\n3. The utility of the finite-state automaton (FSA) is well supported by the ablation study in Table 4, which compellingly demonstrates its contribution to overall performance.", "weaknesses": "While the technical contribution of the paper is sound, the writing suffers from several issues that hinder readability and clarity:\n\nMajor writing weaknesses:\n\n1. The formula at Lines 191-194 is really confusing. \n\nSeveral notational and conceptual ambiguities:\n\n- The term $\\mathbf{l}_i$ on the right-hand side of the EMD is not explicitly defined; while it likely represents a target distribution or point set at step $i$ in a sequential modeling context, this assumption requires clarification.\n\n- The notation { $p'$ } (set-bracketed) is weird, which are typically indexed without set notation. And its subscript seems to be missing.\n\n- The definition of $E$ (the superscript of $T$) is unclear.\n\n- What is $t_{1,i-1}$?\n\n2. The reviewer recognizes that the main technical contributions of the paper are the primitive-specific pooling layer and FSA-based sampling, supported by thorough experiments on these components.\n\nHowever, the details of the proposed primitive-specific pooling are entirely placed in the Appendix. The reviewer believes that this component should be elaborated in the main body of the paper.\n\n> Meanwhile, the Abstract and Introduction emphasize the main contribution as the unification of Text-to-CAD generation and CAD editing. The reviewer does not observe an obvious technical novelty that directly enables this unification. (Notably, the primitive-specific pooling layer demonstrates its value by enabling a trade-off between reconstruction quality and compression rate, while FSA-based sampling appears to significantly reduce the Invalidity Ratio.)\n\nMinor writing weaknesses:\n\n1. The abbreviation “FSA” is introduced in line 156 without prior definition, which may confuse readers encountering the term for the first time.\n\n2. Lines 88–89 introduce the finite-state automaton (FSA) for the first time, yet it is neither mentioned in the abstract nor in any preceding section.\n\n3. Some recent works on CAD generation are missing from the related work section. LLM-based methods are evolving rapidly, including a more comprehensive coverage of recent work would strengthen the paper’s survey:\n\n- CAD-MLLM: Unifying Multimodality-Conditioned CAD Generation With MLLM, Xu et al. (https://arxiv.org/abs/2411.04954)\n\n- CAD-GPT: Synthesising CAD Construction Sequence with Spatial Reasoning-Enhanced Multimodal LLMs, Wang et al. (https://arxiv.org/pdf/2412.19663)\n\n- CAD-Coder: Text-to-CAD Generation with Chain-of-Thought and Geometric Reward, Guan et al. (https://arxiv.org/abs/2505.19713)\n\n- CADmium: Fine-Tuning Code Language Models for Text-Driven Sequential CAD Design, Govindarajan et al. (https://arxiv.org/abs/2507.09792)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)