CAD Translator: An Effective Drive for Text to 3D Parametric Computer-Aided Design Generative Modeling

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-08 14:31:22

Abstract: Computer-Aided Design (CAD) generative modeling is widely applicable in the fields of industrial engineering. Recently, text-to-3D generation has shown rapid progress in point clouds, mesh, and other non-parametric representations. On the contrary, text to 3D parametric CAD generative modeling is a practical task that has not been explored well, where its shape can be defined with several editable parametric command sequences. To investigate this, we design an encoder-decoder framework, namely CAD Translator, for incorporating the awareness of parametric CAD sequences into texts appropriately with only one-stage training. We first align texts and parametric CAD sequences via a Cascading Contrastive Strategy in the latent space, and then we propose CT-Mix to conduct the random mask operation on their embeddings separately to further get a fusion embedding via the linear interpolation. This can strengthen the connection between texts and parametric CAD sequences effectively. To train CAD Translator, we create a Text2CAD dataset with the help of Large Multimodal Model (LMM) for this practical task and conduct thorough experiments to demonstrate the effectiveness of our method.

Main paper: 
 introduction: 
 Computer-Aided Design (CAD) generative modeling plays a crucial role in the fields of design and engineering, providing strong support for manufacturing, visualization, and data management, which drives the advancement of modern design and engineering practices [26, 39, 48, 49] . The CAD model shape design or drawing process can be defined as a parametric CAD sequence of command operations (e.g., line, arc, circle). This kind of representation is called parametric CAD models and can be quickly edited to construct 3D shape. Given its flexibility and practicality, various studies recently have focused on different applications of CAD generative modeling, such as random CAD generation [49] , machining segmentation [17] , CAD assembly suggestions [16, 47] , shape parsing [40] , and classification [15] . The parametric CAD model inherently involves two modalities of representation, as it combines textual information (CAD commands and parameters) and (implicit) visual/shape information simultaneously. This means that the execution order of command operations would indicate the process of 3D shape generation. By changing the values of these parameters, the size and shape of the model can be automatically adjusted. Under the current trend of unification of vision and language [35, 41] in the filed of Computer Vision (CV), text-to-CAD will be a very interesting problem in the intriguing CAD applications.In this work, we tackle the problem of leveraging text for parametric CAD generative modeling. An important difference from previous tasks of text-to-image or text-to-3D [1, 19, 25, 30, 38, 42, 50] is that the generated parametric CAD model can be further edited. This is very practical for CAD designers to quickly convert their ideas into coarse-grained CAD models based on text descriptions.Given parametric CAD models are editable, CAD designers can continue to utilize CAD designing tools to edit and modify these coarse-grained CAD models to obtain the final CAD model, without having to start editing command operations from scratch, accelerating the design process. From another perspective, the parametric sequence is also discrete just like a kind of sentence composed of texts, which builds the foundation for the task of text-to-CAD. As shown in Figure 2 , parametric CAD sequences consist of the type of command operations and their corresponding parameters [5, 53] . For each CAD model, parametric sequences can be seen as specific descriptions of its geometry. Executing these parametric sequences sequentially can construct its 3D shape. This is similar to describe features of the object with texts such as "square", "circle", "line" and so on. It motivates us to create the drive to establish a connection between texts and parametric CAD sequences, achieving the generation task of texts to parametric CAD sequences.Specifically, we design a CAD Translator based on an encoderdecoder framework, that effectively incorporates text awareness into parametric CAD sequences. As there is a large gap between the text description and parametric CAD sequences. To this end, we introduce a Cascading Contrastive Strategy (CCS) to make them aligned in the latent space. Inspired by mixup [56] , we further inject the awareness of parametric CAD sequences into texts via conducting CT-Mix to get a new fusion embedding after finishing alignment. Finally, we put all these fusion embeddings into the decoder to recover 3D parametric CAD sequences. By this design, only one-stage training is required to let CAD Translator know how to transfer text description into 3D parametric CAD sequences. When it is well trained to learn from texts, CAD Translator would have the ability to generate parametric CAD sequences using text as input alone, achieving text to 3D parametric CAD generative modeling easily. Given the text prompt for parametric CAD models is not available in relevant datasets, we first use PythonOCC to render a single image of each 3D CAD model within DeepCAD dataset [49] and then leverage the pretrained CoCa [54] to generate the text description for each parametric CAD model via feeding the rendering image. For more details about the dataset preparation please refer to Sec 5.1.In summary, our key contributions are as following: (i) We propose a Cascading Contrastive Strategy (CCS) controlled by learning steps to align texts and parametric CAD sequences. (ii) We design a CT-Mix to incorporate the awareness of parametric CAD sequences into texts and further consolidate CT-Mix and CCS into a novel multi-modal framework, namely CAD Translator, achieving the text to 3D parametric CAD generative modeling. (iii) Extensive experiments demonstrate the effectiveness of our framework on a new dataset with pairs of texts to parametric CAD sequences, namely Text2CAD, which is created on one benchmark dataset. 
related work: 
 Parametric CAD Modeling. We have witnessed significant progress in parametric CAD modeling based on deep learning recently [18, 21, 51] . The graph structure has been used as the representation in CAD for machining feature recognition [4] . BrepNet [20] achieves good performance on the segmentation task of CAD models with operating on B-rep models directly. More recent applications of deep learning to B-rep models have focused on reconstruction [11, 14] . Besides, several studies starts parsing reverse engineering CAD models. ExtrudeNet [39] designs an effective representation for "sketch-and-extrude" (a common and intuitive modeling process in CAD) to inverse this engineer processing of shape without supervision. SECAD-Net [23] achieves reverse engineering CAD models via learning the implicit sketches and differentiable extrusions from raw 3D shapes as supervision. Furthermore, auto CAD assembling as a practical application has attracted much attention [45, 47] . Unlike these interesting applications based on parametric CAD modeling, our CAD Translator is trying to face a challenge of the text to parametric CAD generative modeling, which has not been discussed well. Among of them, ReparamCAD [18] and DeepCAD [49] are most related to our work. DeepCAD essentially focuses on the downstream task of random generation of parametric CAD sequences and ReparamCAD [18] highlights on modifying the style of objects via feeding both the text description and parametric CAD model. Instead, CAD Translator is mainly focusing on the generation task of texts to parametric CAD sequences. Text-to-3D Shape. With the success of advanced technology in Large Multimodal Models (LMMs), many inspired applications on text-to-3D shape have raised a surge of interesting from community recently [6, 22, 25, 31, 34] . PointCLIP [58] conducts the alignment between point clouds and 3D category texts via CLIP encoding [35] . CLIP-Mesh [30] present a technique for zero-shot [7, 24] generation of a 3D model with the help of pretrained CLIP. DreamFields [13] optimizes the neural radiance fields (NeRF) [29] for diverse 3D models generation from zero-shot caption with CLIP as guidance. In summary, existing methods either focuses on pretrained LMMs together with distillation to generate 3D shape, or combines with NeRF, or trains a text-conditioned 3D generative model from scratch [6] . The common point of them is that the parametric CAD generation has not been considered yet. Besides, the text-conditioned for parametric CAD modeling has a wide range of applications in the industrial sector. Hence, it motivates us to dive in this interesting task. Large Multimodal Models. Motivated by the desire to boost the unification of language and vision. Large Multimodal Models (LMMs) have drawn significant attention recently. CLIP [35] demonstrates an effective ability to several few-shot tasks via training on a large dataset of image-text pairs. Following works start to combine or revise CLIP-based framework to pursue the better performance on multimodal tasks [3, 37] . The usual approach is to finetune LMMs on a specific task with designing appropriate projection heads or classifiers [2, 36] . Other previous works focus on designing a learnable adapter that can be pluged in LMMs to finetune on only small part of parameters [28, 33, 44] . More recent work 3DALL-E [27] integrates DALL-E [8] into 3D CAD software as a plugin to generate 2D concept maps of 3D objects in the design process. In our CAD Translator, we employ LMMs CoCa [54] to generate the description of CAD models for preparing dataset, and make it as a embedding tool for text descriptions. 
preliminary: 
 For easy to understand the rational design of CAD Translator, we first make a brief concept of the parametric CAD sequence. It is a special kind of text with command type and the corresponding parameters (𝑐 𝑡 , 𝑐 𝑝 ), as shown in Figure 2 , where it is always used in the computer-aided design. It allows a designer to modify these parameters to create the object by combining simpler, primitive shapes such as cubes, spheres, and cylinders. The parametric CAD sequence lends itself well to parametric design, where changes in parameters automatically update the model. In this paper, we focus on the generation of single object with command operations including "Line", "Circle", "Arc", and "Extrude". For more details about the definition of the parametric CAD sequence please refer to [49] . 
cad translator 4.1 overview: 
 The CAD Translator method is, essentially, a cross-modal autoencoderbased framework with one-stage training. Our method requires a collection of 3D parametric CAD models and associated text descriptions. The overall architecture is shown in Figure 2 . During training stage, we first adopt embedding for parametric CAD sequences and texts before feeding them into the encoder. Second, we design a Cascading Contrastive Strategy to bring parametric CAD sequences and texts closer after encoding, which is composed of both the constraints of single-modality and cross-modality. Third, we propose CT-Mix to achieve the fusion embedding to further inject the awareness of parametric CAD sequences into texts. Finally, the fusion embedding would be fed into the decoder to generate the parametric CAD sequence. Once the network is well trained, the text description as the sole input can go forward to generate the parametric CAD sequence in the inference stage. Following the rule of generated parametric CAD sequences, the 3D shape can be easily achieved and modified by related CAD tools (e.g., PythonOCC and AutoCAD). Please note that labelling the description for parametric CAD models is very time consuming and there is no ready-made text descriptions in existing public CAD datasets. It motivates us to create the text description for each parametric CAD model with the help of CoCa [54] . For more details about dataset preparation please refer to Sec 5.1. We will release it in the future.Embedding. As the usual settings of the transformer-based model, texts and parametric CAD sequences are first projected to an embedding space. For the parametric CAD sequence (𝑐 𝑡 , 𝑐 𝑝 ), we imitate the method in [49] to formulate it to an embedding 𝐸 𝐶 in three aspects:EQUATIONwhere 𝑒 is the same as in Transformer [46] , which is used to record the index of the command 𝑐 𝑡 in the complete parametric CAD sequence, In practice, the dimension of 𝑑 𝐸 is set to 768.For the text 𝑇 , we conduct the pretrained CoCa [54] to encode it to an embedding 𝐸 𝑇 with the dimension of 768, making it easy to match with 𝐸 𝐶 . Encoder and Decoder. In the training stage, we train an autoencoder with a CAD encoder, a text encoder, and a fusion decoder. For the CAD encoder and the text encoder, there are four layers of transformer blocks with eight attention heads per block and the feedforward dimension is 512. The output of encoders, latent vector dimension is fixed in 256. Based on the same configuration of the CAD encoder and the text encoder, parametric CAD sequences and texts are matched well in terms of dimensions, facilitating crossmodal alignment (Cascading Contrastive Strategy) and subsequent knowledge injection (CT-Mix). Note that the weights of these two encoders are independent. Specifically, the encoder 𝑓 𝑐𝑎𝑑 and the encoder 𝑓 𝑡𝑒𝑥𝑡 are used to encode 𝐸 𝐶 and 𝐸 𝑇 separately. This can be written as:EQUATIONEQUATIONThe fusion decoder 𝑓 𝑑 is identical to the encoder in all hyperparameter settings. One linear layer is connected to with the last block of the fusion decoder to predict parametric CAD sequences (𝑐 * 𝑡 , 𝑐 * 𝑝 ), as defined with 𝑓 𝑑 (𝑒 𝑐𝑎𝑑 , 𝑒 𝑡𝑒𝑥𝑡 ) = (𝑐 * 𝑡 , 𝑐 * 𝑝 ). When the network is well trained, the text as sole input can generate associated parametric CAD sequences:EQUATIONContrastive learning is widely used to learn representations via attracting positives and repelling negatives [9, 10, 35] . It must be acknowledged that contrastive learning does facilitate the alignment between different modalities for cross-modal learning. However, CAD Translator requires a clever design to fully leverage the potential of contrastive learning. We first denote the cross-modal dataset as 𝐷 = {(𝑐 𝑖 , 𝑡 𝑖 )}, where (𝑐 𝑖 , 𝑡 𝑖 ) denotes a pair of parametric CAD sequence and text description. Our goal is to bring 𝑐 𝑖 and 𝑡 𝑖 closer, reducing the gap between them. To go for this, we consider adding both contrastive constraints of the single-modality and the crossmodality simultaneously. For the single-modal contrastive learning on 𝑐 𝑖 , we let each 𝑐 𝑖 pass forward the CAD encoder 𝑓 𝑐𝑎𝑑 twice with dropout under the different rate to generate a pair of positives (𝑒 𝑐𝑎𝑑 , 𝑒 ′ 𝑐𝑎𝑑 ). For each 𝑒 𝑐𝑎𝑑 , the rest of embeddings 𝑒 * 𝑐𝑎𝑑 within one mini-batch are all negatives. Our single-modal contrastive learning aims to catch the similarity within the augmented variants of parametric CAD sequences, making the learned representation preserve the knowledge of 𝑐 𝑖 comprehensively. For the cross-modal contrastive learning on (𝑐 𝑖 , 𝑡 𝑖 ), each 𝑐 𝑖 is fed into the text encoder 𝑓 𝑡𝑒𝑥𝑡 to obtain 𝑒 𝑡𝑒𝑥𝑡 and paired with associated 𝑒 𝑐𝑎𝑑 . Similar to the single-modal contrastive learning, 𝑒 * 𝑐𝑎𝑑 are as negatives for each 𝑒 𝑡𝑒𝑥𝑡 . The cross-modal contrastive learning aims to utilize the knowledge of parametric CAD sequences for better textual feature learning and strike a well connection between them. Finally, the constraints of the single-modality L C-CAD and the cross-modality L C-CT can be defined with InfoNCE [32] as following:EQUATIONEQUATIONwhere 𝑓 𝑘 is defined with 𝑒 sim(h ′ 𝑖 ,h ′′ 𝑖 )/𝜏 . sim( * , * ) denotes the cosine similarity and 𝜏 is a temperature hyper-parameter with 0.05. 𝑋 denotes the size of one mini-batch during the training.Inspired by the multi-stage training strategies adopted frequently in machine learning. We propose a Cascading Contrastive Strategy (CCS) to split the participation of L C-CT and L C-CAD during training stage. The reason is that here lies the conflict: if L C-CAD and L C-CT are activated simultaneously at the beginning of the training, these two constraints are getting some overlaps, making it difficult for model to strike a well balance between them. This would finally hinder the ability of contrastive learning (Recall in Ablation Study 5.2). Hence, CCS activates L C-CT solely from the scratch and incorporates L C-CAD later, which is effective to alleviate this conflict and fully leverage the capabilities of both two constraints. We utilize the training step as a participating signal for L C-CAD . It can be defined with:EQUATIONwhere E denotes the current epoch and S is the hyper-parameter of training steps for the adaptive selection of L CCS . The advantage of CCS is to make 𝑒 𝑐𝑎𝑑 and 𝑒 𝑡𝑒𝑥𝑡 establish a solid connection in early training steps, and then L C-CAD starts to optimize the learned representation, making it maintain the knowledge of parametric CAD sequences comprehensively. This is quite important for delivering the knowledge of parametric cad sequences to texts.Mixup [57] conducts the linear interpolation between two different samples to obtain new augmented data. The detailed implementation can be defined as following:EQUATIONEQUATIONwhere 𝜆 is sampled from 𝐵𝑒𝑡𝑎(𝛼, 𝛽) distribution. (𝑥 1 , 𝑥 2 ) denotes the two samples randomly chosen from datasets, and (𝑦 1 , 𝑦 2 ) represents the labels of them. (𝑥 * , 𝑦 * ) is a new sample by linear interpolation. Given its flexibility and friendly implementation, it sparks numerous Mix-based adaptations [43, 52, 55] for operating data augmentation tailored to specific tasks.The way Mix-based methods construct new data inspires us to transfer it to mix parametric CAD sequences and texts, namely CT-Mix. Compared to previous methods, our CT-Mix differs in the following two aspects: (i) CT-Mix operates the mixing operation between two different modalities rather than focusing on the single modality. (ii) The goal of CT-Mix is to inject the awareness of parametric CAD sequences into texts instead of adopting the data augmentation. The advantage of CT-Mix is to achieve a fusion embedding 𝑒 𝑐𝑡 that preserves the knowledge of both texts and parametric CAD sequences, further reducing the gap between them. Practically, we conduct the mixing operation between 𝑒 𝑐𝑎𝑑 and 𝑒 𝑡𝑒𝑥𝑡 with random masking in the latent space to achieve the fusion embedding 𝑒 𝑐𝑡 . Based on Equation 7, This process can be defined as:EQUATION)where 𝛾 is a hyper-parameter to control the mixing ratio of 𝑒 𝑐𝑎𝑑 and 𝑒 𝑡𝑒𝑥𝑡 . In practice, we conduct 𝛾 as a random 0-1 vector with a setting threshold 𝑅 (Recall in Hyper-parameter Discussion 5.2) for controlling the ratio of 0 and 1. ⊙ represents the point-wise multiplication. This is, essentially, a mask operation to combine 𝑒 𝑐𝑎𝑑 and 𝑒 𝑡𝑒𝑥𝑡 , which can be considered as using 𝑒 𝑡𝑒𝑥𝑡 to fill the masked parts of 𝑒 𝑐𝑎𝑑 . Under this setting, the awareness of parametric CAD sequences is easily injected into texts. For the same 3D object, 𝑒 𝑐𝑎𝑑 and 𝑒 𝑡𝑒𝑥𝑡 are supposed to be its two different representations.Hence, the directivity of 𝑒 𝑐𝑡 would also aim to the same 3D object. Based on this assumption and Equation 8, the label of 𝑒 𝑐𝑡 can be obtained as:EQUATIONwhich means the labels of them are consistent.We simultaneously conduct Cascading Contrastive Strategy and MSE constraint L M to align 𝑒 𝑐𝑎𝑑 and 𝑒 𝑡𝑒𝑥𝑡 . Then we further adopt CT-Mix to achieve the fusion embedding 𝑒 𝑐𝑡 . Finally, 𝑒 𝑐𝑡 is fed into the decoder 𝑓 𝑑 to generate the parametric CAD sequence (𝑐 * 𝑡 , 𝑐 * 𝑝 ). To measure the distance from (𝑐 * 𝑡 , 𝑐 * 𝑝 ) to (𝑐 𝑡 , 𝑐 𝑝 ), we use a standard Cross-Entropy loss L CE . The whole constraint of our model L CT is defined as following:EQUATIONEQUATIONEQUATIONThe experiments are all trained on one NVIDIA RTX 3090 GPU with a batch size of 256 under 100 epochs. Initial learning rate is set to 0.001 with warm up [12] and gradient clipping of 1.0 is applied in back-propagation.Accuracy. As seen in Figure 2 , parametric CAD sequences are defined with the command 𝑐 𝑡 and its parameter 𝑐 𝑝 . [49] proposes to measure the accuracy of the recovered CAD sequence (𝑐 * 𝑡 , 𝑐 * 𝑝 ) by calculating 𝐴 𝐶 and 𝐴 𝑃 separately. However, The generated parametric CAD sequence with the high accuracy of 𝐴 𝐶 or 𝐴 𝑃 may still be failure to construct 3D shape. To make it more reasonable, we add Sucecssful Ratio (𝑆 𝑅 ) into Accuracy. It is a measurement of the ability to reconstruct the known CAD model, where the input is the existing CAD sequences of the test set. Finally, Accuracy (Acc) is defined:EQUATIONEQUATIONEQUATIONEQUATIONwhere ▽[ * ] is a boolean function with scalar 0 or 1. 𝑇 is the total number of parameters in all correctly predicted commands. We set 𝜂 = 3 in practice as the error threshold. 𝑇 𝑅 denotes the total number of predicted CAD sequences. 𝐹 𝑅 represents the total number of shapes constructed unsuccessfully by predicted CAD sequences. Shape Construction. When the parametric CAD sequence is constructed into 3D shape, we can convert it into point clouds by randomly sampling 𝐾 points on its surface. In practice, we set 𝐾 = 2000. To measure the differences between a real shape and the predicted shape, we calculate Median Chamfer Distance (MCD) of them. Furthermore, we also adopt Minimum Matching Distance (MMD) to measure the fidelity of generated shapes with calculating the Chamfer Distance from the 3D shapes in the test set to their nearest neighbors in the generated set. Finally, Jensen-Shanon Divergence (JSD) can be calculated with these converted point clouds in the test set and the generated set, measuring the difference of their data distributions. CT-Score. CT-Score is an important metric to evaluate the similarity of 𝑒 𝑐𝑎𝑑 and 𝑒 𝑡𝑒𝑥𝑡 . It shows the effectiveness of CCS and CT-Mix in reducing the gap between parametric CAD sequences and texts, which can guide to determine the appropriate hyper-parameter settings of CCS and CT-Mix (Recall in Hyper-parameter Discussion 5.2). To achieve this, we calculate the cosine similarity of 𝑒 𝑐𝑎𝑑 and 𝑒 𝑡𝑒𝑥𝑡 in the latent space.EQUATION) 
experiments 5.1 dataset preparation: 
 As labelling description for parametric CAD models is very time consuming and the text description is unavailable in existing datasets of CAD parametric models, we move to leverage a pretrained CoCa [54] to generate text for each parametric CAD model. Precisely, we first choose a benchmark dataset, called DeepCAD, which is consist of 178,238 CAD models with their parametric construction sequences [49] . With PythonOCC (OpenCASCADE technology in the Python version), these parametric CAD models can be easily visualized to capture perspective images of them. Specifically, we set position (150, -150, 150) and rotation radian (0.7854, 0.6155, 0.5236) with the setting of front x-axis, right y-axis, and up z-axis as a main viewport for rendering each 3D CAD model. Under this viewport, it ensures capturing as much semantic information of the object as possible in a single image. To complement the visual information missing under the main viewport, we further make the viewport to rotate around the z-axis in 𝜋/6 Text to Parametric CAD Sequence. Theoretically, the parametric CAD sequence is akin to the discrete language. These commands and their parameters can be seen as "vocabulary" to form "sentence", which documents the manufacturing process of the 3D object. This formalism gives us the opportunity to leverage language models such as Transformer [46] to achieve this new task. natural language, as shown in Figure 2 . Apparently, there is a noticeable gap in the representation of the same 3D object between parametric CAD sequences and texts. This is telling that, the "bruteforce" regression strategy to minimize the difference between them is difficult. To demonstrate this, we first maintain the same configuration of encoder and decoder and remove parametric CAD sequences from the input. Then we make texts go forward to directly approach the parametric CAD sequences, treating it as a "brute-force" seq2seq regression method, namely BFSR. Besides, we further adopt some useful learning tricks on BFSR to improve its performance (e.g., increasing the layers of network, longer training epochs, contrastive learning, data augmentation.), trying to figure out whether BFSR has the potential to catch up with CAD Translator. Specifically, we make the following adjustments to BFSR: (i) increasing two layers of encoder and decoder separately, (ii) twice training epochs as CAD Translator, (iii) conducting twice dropout in the latent space to generate positive pairs, the same way also used in Equation 6as a part of CCS, (iiii) randomly masking 20% of each embedding in every epoch with a certain probability as the data augmentation. The detail results can be found in Table 1 . Note that the shape design is the ultimate goal of CAD models, which means the rationality of parametric CAD sequence is quite important. Our first goal is to generate the valid parametric CAD sequence as much as possible, which can be finally reconstructed into 3D shape. CAD Translator outperforms BFSR and its variants in all shape construction related metrics and achieves more than about 6% improvement on the accuracy of parametric CAD sequence generation. It demonstrates that the awareness of parametric CAD sequences is injected into texts successfully. Compared to BFSR, this further brings texts and parametric sequences closer, making the CAD Translator has the strong connection between these two representations. As seen in Figure 3 , some key points of final 3D shape generated by BFSR is seriously shifted away from Ground Truth (GT) compared to CAD Translator. Again, it proves that the task of texts to parametric CAD sequences is challenging, and directly making texts regressed to parametric CAD sequences is difficult. Cross Dataset Generalization. To further validate the generalization capability of CAD Translator, we pick up another CAD dataset,"An illustration of an exhaust manifold gasket" "An image of an object that is in the shape of an arch" ∅ "An illustration of a rolling machine" "A 3d rendering of a metal arch with three holes" "An illustration of a piece of metal with two holes" "An isometric view of a bench seat by the parterre" "an illustration of a metal disc with four holes" Fusion 360 Gallery, which is composed of 2D and 3D parametric CAD models [48] . Then we choose the reconstruction subset of Fusion 360 Gallery and also leverage CoCa [54] to generate text descriptions of parametric CAD models. This is similar to how we construct Text2CAD dataset and finally 6,708 samples (Text360) are created. Next, we train BFSR and CAD Translator on the Text2CAD dataset and make them tested on the Text360 dataset directly. The comparable results are as shown in Table 2 . It can be found that CAD Translator still outperforms BFSR and its variants in 𝐴 𝐶𝐶 and all shape construction metrics (especially with a larger margin in 𝐴 𝐶𝐶 and JSD). Compared to BFSR, the shape generated by CAD Translator is closer to GT and more compatible with the associated text description (Figure 4 ). It again proves that CAD Translator injects the awareness of parametric CAD sequences into texts successfully, making it to learn a robust representation with the capability to adapt in another dataset without additional training. At the same time, it also demonstrates there indeed exists a significant gap between texts and parametric CAD sequences even though they are both some kind of discrete languages. Since the commands and parameters in parametric CAD sequences are fundamentally different entities, it is difficult to make texts approach them directly. Therefore, the "brute-force" method such as BFSR cannot effectively"An illustration of a stapler" "An illustration of a hexagonal object" "An isometric view of a metal handle" "A drawing of a round object with a hexagonal design" Conditioned on this analogy, the more similar 𝑒 𝑐𝑎𝑑 and 𝑒 𝑡𝑒𝑥𝑡 are, the more precise the accuracy of texts to parametric CAD sequences generation. Hence, we are trying to find a proper consolidation of CT-Mix and CCS via calculating the CT-Score (Equation 19) of texts and parametric CAD sequences after encoding. Let 𝑅 denotes the ratio of texts when conducting CT-Mix. For example, 𝑅 = 20% would result in 20% of 0-1 vector 𝛾 in Equation 10 is numerical value of 1.𝑆 is the hyper-parameter in L C C S (Equation 7). As shown in Figure 5 , it can be inferred that CAD Translator with (𝑅 = 40%, 𝑆 = 40) achieves the highest CT-Score and outperforms other consolidation strategies in all metrics. These comparable results validate this hypothesis, the better CT-Score achieved, the stronger capability to address this task. Besides, we discover that the performance of CAD Translator would decay significantly when 𝑆 and 𝑅 are too large or too small, as shown in the four corners of each image within Figure 5 . 𝑆 and 𝑅 essentially control the degree of participation for texts in the whole training process. Apparently, too much weight of texts makes the CAD Translator take a rough approach like the "brute-force" method in BFSR. However, too small weight of texts cannot fully absorb the awareness from parametric CAD sequences, which makes it challenging to bring them closer. Synonym Substitution. To break out of the prompt for each CAD model in the Text2CAD dataset, we conduct synonym replacement on text descriptions to generate similar statements. For example, if the original description of the object starts with "An illustration of... ", we substitute its with other descriptions such as "An isometric view of... " or "A description of... " and feed these new prompts into CAD Translator again to show what it can generate. From Figure 6 , it can be seen that changing the expression paradigm of describing an object would not create entirely different shapes, especially in only substituting the prefix. More interestingly, when substituting the key word to define the category of a object, it would make some reasonable alterations compared to the original shape (e.g., from "bench" to "long chair" in the second row of Figure 6 ). This demonstrates that CAD Translator is not limited to the fixed expression and has the ability to generate diverse shapes.Patching the parametric CAD sequence. Patching the parametric CAD sequence is very meaningful for the practical designing conditioned on reusing or recreating the designed entities with  
limitations: 
 Although CAD Translator shows the potential in the generation task of texts to parametric CAD sequences and can provide preliminary CAD modeling for designers, it is still unable to handle more complex CAD models in practical engineering applications. This is because complex CAD models often consist of multiple primitives, resulting in a longer parametric CAD sequence that would exceed the limit command length of 60 in CAD Translator. 
conclusion: 
 In this paper, we present CAD Translator for automatic text to parametric CAD generative modeling based on transformer network. CAD Translator effectively incorporates text awareness into parametric CAD sequences via conducting mixup operation in the latent space, making it possible to generate parametric CAD models with text description under one-stage training. The experimental results verify the effectiveness of our frameworks. Our approach opens up possibilities for leveraging text to parametric CAD generative modeling in the future. To be specific, we will aim to further explore text to parametric CAD modeling in two points: (i) novel ways to combine LMMs, (ii) parametric CAD sequences with text awareness in the latent space. 
cad translator: an effective drive for text to 3d parametric: 
 Computer-Aided Design Generative Modeling"A 3d rendering of a camera" "An isometric view of a whistle" "An illustration of a stapler""A 3d drawing of a circular object" "A 3d rendering of the letter t" "A 3d drawing of a pipe fitting" "A 3d image of a piece of metal""An illustration of a metal object with a hole" "An illustration of a piece of concrete with holes" "A 3d drawing of a knob""A 3d image of four rings""An isometric view of a concrete structure" Computer-Aided Design (CAD) generative modeling is widely applicable in the fields of industrial engineering. Recently, text-to-3D generation has shown rapid progress in point clouds, mesh, and other non-parametric representations. On the contrary, text to 3D parametric CAD generative modeling is a practical task that has not been explored well, where its shape can be defined with several editable parametric command sequences. To investigate this, we design an encoder-decoder framework, namely CAD Translator, for incorporating the awareness of parametric CAD sequences into texts appropriately with only one-stage training. We first align texts and parametric CAD sequences via a Cascading Contrastive Strategy in the latent space, and then we propose CT-Mix to conduct the random mask operation on their embeddings separately to further get a fusion embedding via the linear interpolation. This can strengthen the connection between texts and parametric CAD sequences effectively. To train CAD Translator, we create a Text2CAD dataset with the help of Large Multimodal Model (LMM) for this practical task and conduct thorough experiments to demonstrate the effectiveness of our method. 
 
 Appendix:  


Decision: This paper explores the interesting and practical task of text2CAD, presenting a well-organized and easy-to-follow study. It introduces a large-scale dataset with text and CAD pairs based on prior datasets and delves into the relatively under-explored topic of parametric 3D generation. Overall, the paper is well-structured and effectively addresses an interesting problem within the field.

All reviewers suggest accepting this paper. The AC also recommends acceptance. The authors should revise the paper to address reviewers' comments in the final version.

Reviews: review: paper_summary: paper_summary: The paper presents a CAD Translator designed on an encoder-decoder framework that can generate parametric CAD sequences solely from text descriptions to assist in the visualization and creation using CAD tools. Through the Cascading Contrastive Strategy (CCS), this framework aligns text descriptions with parametric CAD sequences in the latent space. Then, using the CT-Mix method, the awareness of parametric CAD sequences is infused into the texts to obtain a fused embedding.
strength_weakness: strength_weakness: * Detailed Experiments: The preparation of a detailed dataset, extensive testing, and clear benchmarking thoroughly demonstrate the effectiveness of the CAD Translator. Various metrics have been introduced, confirming that the model performs better than direct methods like BFSR.

* Innovation: The paper addresses the less explored area of text-to-3D parametric CAD generative modeling. By developing the CAD Translator, it achieves direct generation of CAD models from textual descriptions, which is rarely covered in previous research.

*  Clear Structure and Rigorous Logic: The structure of the paper is clear and the logic is rigorous. The methods are explained in detail and are easy to understand, allowing readers to clearly grasp the research results and methodological details.

* Broad Application Prospects: The research results have wide application prospects in the fields of industrial design and CAD software development. The automatic conversion from text to CAD models can significantly enhance design efficiency and lower the barriers to entry in design.
limitations: limitations: * Insufficient Conclusion Content: The current conclusion section is rather brief and does not adequately emphasize the significant findings of the study and its potential application impact. It is recommended that the authors provide a detailed summary of the main achievements of the research in the conclusion, highlight the practical engineering design prospects of this technology (such as the application of repairing parametric CAD sequences), and discuss possible directions for improvement or future research focuses. As mentioned in the limitations, future research could attempt to overcome the length restrictions in the CAD Translator to enable the generation of more complex CAD models.

 * Diversity and Bias: The paper mentions using CoCa to generate descriptions for CAD models, but does not discuss the diversity of the generated descriptions and potential biases. Discussing this issue would help understand how the model performs and its limitations when dealing with diverse data.

* Dataset Dependence: Currently, the training and evaluation of the model primarily rely on the specific Text2CAD dataset, which may limit the model's generalizability in different types of CAD applications.
**  Universality： To verify the universality of the model, future research could consider introducing a diverse range of datasets and testing the model's performance in different data environments.
 **  Generalization Experiments： Its generalizability has only been tested on the Text360 dataset, which is constructed similarly to Text2CAD. Perhaps future experiments with datasets constructed in new and different ways can provide more extensive generalization validation.

* Model Training Computational Costs: The paper does not mention the computational resources and time required for model training. In practical applications, these factors are important indicators for assessing the feasibility of the model. Providing detailed information on these aspects could help evaluate the practicality of the model.

* Lack of Technical Details: Although the paper provides conceptual explanations of the Cascading Contrastive Strategy and CT-Mix, it lacks sufficient implementation details, such as specific algorithm steps or pseudocode. Providing these details would aid in understanding and reproducing the research results.
paper_summary: paper_summary: This paper investigates a practical task i.e., text to CAD, that aims at generate 3D parametric CAD using textual descriptions. For this task, this paper presents a new framework namely CAD Translator. Specifically, a cascading contrastive strategy (CCS) is introduced, which aligns texts and parametric CAD sequences based on different training steps. In addition, this paper proposes a CT-Mix to incorporate the awareness of parametric CAD sequences into texts. To train and validate the proposed models, this paper repurposes existing datasets and introduce a new benchmark consisting of ~178K text and CAD pairs. Comprehensive experiments validate the effectiveness of the proposed method.
strength_weakness: strength_weakness: + This paper investigates an interesting and practical task (text2CAD).
+ This paper is well organized and easy to follow.
+ A large-scale dataset with text and CAD pairs is presented based on prior datasets.
limitations: limitations: - The proposed method heavily relies on prior methods [32, 57] with incremental improvements. 
- The proposed method seems to be able to only generate some toy CAD results. It would be better to provide some results with more detailed text prompt and showcase some results with fine-grained control capability. The generated CAD models could be useless, if we cannot accurately control or editing the shape of them using textual prompts. 
- It would be better to showcase the generated CAD models using videos. It is hard to evaluate the 3D assets using a few images.
paper_summary: paper_summary: The paper propose a framework for text to 3D parametric computer-aided design generative modeling.It first align texts and parametric CAD sequences via a cascading contrastive strategy and then use CT-Mix to  get a fusion embedding. The main contributions are the proposed original framework and its satisfactory effect.
strength_weakness: strength_weakness: 1.The paper is well-written, easy to follow, and technically sound. 
2.The core idea about  text to 3D parametric computer-aided design generation is original. Few researchers have explored it before. This paper can fill this gap.
3.The performance shows the effectiveness of the proposed method.
4.The qualitative results look impressive. 
5.The experiments in the paper are abundant.
limitations: limitations: 1.It is better to show some failure cases for simple sentence and some ones with complex description.

The following are some improvements to the paper polish that authors are advised to make in the next edition (especially if accepted)：
1.Ensure each line of text contains a minimum of one-third of its capacity to prevent excessively short lines.
2.Please unify all reference formats.
3.Fig.6 (a)  "for-way"
paper_summary: paper_summary: The paper introduces CAD Translator, an approach for text-to-3D parametric CAD generative modeling. It utilizes an encoder-decoder framework to align text descriptions with parametric CAD sequences, employing techniques like Cascading Contrastive Strategy and CT-Mix for enhanced embedding fusion.
strength_weakness: strength_weakness: 1. The topic is very interesting, parametric 3D generation is relatively under-explored in the ear of NeRF and Gaussian Splatting.
2. The paper is well-structured and easy to follow.
limitations: limitations: 1. Including visual comparisons in the ablation study might better illustrate the effectiveness of L_CSS, which currently appears to be less impactful.
2. The text descriptions generated may not accurately reflect the precise details such as lengths and angles specified in CAD commands, which are crucial for CAD models. It's worth investigating whether the current large language models can comprehend CAD commands as effectively as they do code.
3. The illustrations in the figures could be enhanced for clearer understanding.