Title: MeshCoder: LLM-Powered Structured Mesh Code Generation from Point Clouds
Venue: NeurIPS 2025
Accepted: yes
Keywords: 3D reconstruction

point cloud

LLM
Abstract: Reconstructing 3D objects into editable programs is pivotal for applications like reverse engineering and shape editing. However, existing methods often rely on limited domain-specific languages (DSLs) and small-scale datasets, restricting their ability to model complex geometries and structures. To address these challenges, we introduce MeshLLM, a novel framework that reconstructs complex 3D objects from point clouds into editable Blender Python scripts. We develop a comprehensive set of expressive Blender Python APIs capable of synthesizing intricate geometries. Leveraging these APIs, we construct a large-scale paired object-code dataset, where the code for each object is decomposed into distinct semantic parts. Subsequently, we train a multimodal large language model (LLM) that translates 3D point cloud into executable Blender Python scripts. Our approach not only achieves superior performance in shape-to-code reconstruction tasks but also facilitates intuitive geometric and topological editing through convenient code modifications. Furthermore, our code-based representation enhances the reasoning capabilities of LLMs in 3D shape understanding tasks. Together, these contributions establish MeshLLM as a powerful and flexible solution for programmatic 3D shape reconstruction and understanding.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces MeshLLM, a novel framework that leverages large language models (LLMs) to reconstruct 3D objects from point clouds into editable Blender Python scripts. The authors have designed a comprehensive set of APIs to model intricate geometries beyond simple primitives, enabling the synthesis of complex 3D structures. Then, a model trained to generate executable Blender scripts from point clouds, facilitating part-level reconstruction and intuitive editing through code modifications. MeshLLM outperforms existing shape-to-code methods in reconstruction tasks. The framework is evaluated properly, showcasing its ability to handle complex geometries and enable precise geometric and topological edits.\n\nSTRENGTHS AND WEAKNESSES:\nMeshLLM represents a notable advancement in 3D shape reconstruction and editing, offering a scalable and flexible solution for generating editable code from point clouds. Its strengths lie in its innovative use of LLMs, expressive APIs, and large-scale dataset construction. However, limitations such as restricted generalization to organic shapes and lack of open access to data/code may impact its broader applicability. A short summary is listed as follows:\n\n** Strength\n(1) It bridges the gap between 3D point clouds and editable code representations, offering a flexible solution for reverse engineering and shape editing.\n(2) The Blender Python APIs support advanced geometric operations (e.g., Boolean operations, sweeping trajectories), enabling the modeling of intricate real-world objects.\n(3) Quantitative results demonstrate significant improvements over baseline methods in reconstruction accuracy (e.g., higher IoU, lower Chamfer Distance).\n(4) The code-based representation allows for intuitive geometric and topological modifications, enhancing usability in downstream applications.\n\n** Weakness\n(1) The framework primarily targets human-made objects and may not generalize well to organic shapes (e.g., animals, humans), as noted in the limitations. Besides, it is tightly coupled with Blender, which may restrict its applicability to other 3D modeling software or environments.\n(2) While the paper mentions ablation studies, more detailed analysis of the impact of individual components (e.g., tokenizer design, LLM architecture) would strengthen the statement.\n\nQUESTIONS:\n(1) The paper acknowledges that MeshLLM is primarily designed for human-made objects and struggles with organic shapes (e.g., animals, humans). Given that many real-world applications (e.g., medical imaging, character modeling) require handling organic forms, how do the authors justify the framework’s broader applicability?\n(2) In the mentioned pipeline, if there are certain errors in the code predicted by the mesh after the object is split into parts in Section 3.3, then this would be considered an erroneous sample when constructing the dataset. It seems that such errors cannot be corrected in subsequent training. How do the authors address this issue? Additionally, under the current settings, are there potential failure cases? Furthermore, do common LLM hallucination issues, such as repeatedly generating similar code, also exist here? The authors are kindly requested to elaborate on these points.\n\nFrom the current submission format, it appears that the authors have placed greater emphasis on dataset construction, resulting in relatively limited theoretical innovation. Additionally, the ablation studies on various aspects of the model are somewhat insufficient. Therefore, I tentatively recommend rejection. However, if the authors can address the aforementioned issues, I am open to raising the score. Thanks.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a new dataset consisting of 1 million objects spanning 40 classes. Each object is composed of multiple parts with codes represented as a parameterized function implemented using the Blender API. The whole object is represented as a concatenation of the individual part codes.\n\nThe authors use a two stage regime to train a model that predicts object codes from an input point cloud. In the first stage a LLM model is trained to predict part codes from the corresponding point clouds. For each object in the dataset its part codes are predicted through this model and concatenated to form the object codes. The model is then initialized from the first stage weights and fine-tuned on the full object codes and point clouds.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\nThe paper writing is clear and well organized, covering most details regarding code representation and model training. The dataset provided is also reasonably large with a somewhat limited number of classes. However, it is significant considering the smaller number of available datasets available for program based object generation. The reconstruction results are convincing, with the LLM producing better shapes compared to prior program based methods. The method also has potential for applications in simulations where the part representation can yield cleaner meshes compared to other 3D representations like SDF.\n\n## Weaknesses\n\nWhile the paper has demonstrated impressive reconstruction results. It is lacking concrete examples that can better show how it can be applied in real world applications.  However, I believe this can be addressed with minor revision. I listed more specific questions regarding this below.\n\nQUESTIONS:\n1. I wonder how the model would perform if using point clouds from generative models as input such as Point-E [1]? This would demonstrate the model's generalization capabilities and extend it to generative applications using image or text.\n\n2. Additional details regarding the generated object's mesh quality can be useful. Comparing additional metrics such as face count against human made meshes or other recent 3D generation methods can help determine the model's usefulness in applications like simulations.\n\n3. For articulated objects in the dataset, I’m curious how the parts are defined and distributed and whether parts are semantically meaningful for articulation applications.\n\n4. I am curious about the scalability of such approaches to more object categories in the future. For example, how many more functions would need to be required in order to represent a dataset like Objaverse [2]? And whether it is actually feasible for such an approach given the effort needed to design these functions.\n\nI am willing to increase my score if these points are addressed in the rebuttal.\n\n[1] Nichol, Alex, et al. \"Point-e: A system for generating 3d point clouds from complex prompts.\" arXiv preprint arXiv:2212.08751 (2022).\n\n[2] Deitke, Matt, et al. \"Objaverse: A universe of annotated 3d objects.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nMeshLLM aims to train an object-to-code inference model. To accomplish this, MeshLLM first trains a part-to-code inference model that predicts code for a single part. Then, the difficulty is increased by training a model that predicts code for every part of an object. Finally, the codes for every part of the object are concatenated together to represent all parts of the object. To enable model training, MeshLLM constructs a synthesized dataset of paired parts and their corresponding codes. MeshLLM achieves shape-to-code reconstruction tasks by developing a comprehensive set of expressive Blender Python APIs capable of synthesizing intricate geometries.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n1. The paper defines a series of shapes and corresponding algorithmic rules, enabling the expression of complex shapes in a highly efficient manner. For example, the \"array\" representation allows for the concise construction of structures where certain geometric shapes are repeated in specific patterns, using array methods to build the entire structure at once.\n2. MeshLLM uses a triplane model to represent sparse point cloud structures as compact feature vectors, enabling an efficient tokenizer.\n3. MeshLLM achieves improved reconstruction performance compared to previous methods in terms of Chamfer Distance loss and IoU. In addition to quantitative evaluation, qualitative evaluation also shows improved geometric quality.\n\n**Weaknesses:**\n1. Regarding shape understanding, since GPT is a closed-source model, it cannot be fine-tuned to understand the series of shapes and algorithmic rules defined during creation. Does directly inputting these codes and their function names help with 3D understanding, without providing the specific algorithmic rules, as shown in Fig. 19? Furthermore, MeshLLM claims to enhance LLMs’ understanding of 3D objects. It would be better to provide comparisons, such as giving GPT multi-view images of objects or other methods, and comparing their 3D understanding capabilities with those when given the codes in MeshLLM.\n2. Minor weaknesses in visualization: Since MeshLLM is related to 3D objects, showing multi-view results of 3D objects or supplementary videos could better demonstrate the quality of the generated results.\n\nQUESTIONS:\nFor different objects, the number of points sampled by MeshLLM is fixed. For complex objects, such as those with many parts, will the tokenizer model's capability decrease? Thanks to the use of the triplane projection method, the model can project different numbers of sampled point clouds into a fixed length. Sampling different numbers of points may improve the model's capability, which could be a direction for future work.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an algorithm for reconstructing procedural 3D models (as Blender Python API scripts) from point cloud input. The primary contributions are an expressive Blender Python API library (embedded DSL) capable of some non-trivial CAD operations (primitive creation, beveling, patterning, swept and lofted shapes), a technique for generating large data sets of semantically commented and part segmented programs/models, and a large dataset of such models. The dataset is generated by first randomly synthesizing part-level programs and training a model to reconstruct parts in the embedded DSL from point clouds. The final dataset is constructed by sampling from Infinigen Indoors and using the part-level model to generate DSL code for each part of the sampled models, rejection filtering for reconstruction quality.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Dataset generation method is generalizable to larger procedural modeling contexts\n- Reconstruction in most categories appears robust and high quality\n- Output is semantically labeled procedural models, good for editability and understanding\n\nWeaknesses:\n- Since the test set is construction from the same base set of parametric models (Infigen Indoors) as the training set, it is possible that the model is primarily memorizing the structure of the infinigen procedural models rather than a generalizable pattern. It would be good to see reconstruction experiments performed on non-infinigen inputs.\n\nQUESTIONS:\n- Was the train/test split done over sampled models or over inifinigen parametric models?\n- why were \"translation\" and \"bridge loop\" used as names when these operations are traditionally called \"sweep\" and \"loft\"? Is there a difference between these and their normal CAD versions?\n- Are the part dataset probabilistic programs hand-crafted or somehow sampled from the grammar?\n- In Figure 3, the example code seems to show a syntax where the primitive creation operations can be passed arrays of names and parameter lists rather than single names and parameter lists; is this done in-practice in the data set, and if so why is this preferred over separating into multiple lines?\n\nLIMITATIONS:\nLimited number of object categories supported.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-03"}