Title: PhysX-3D: Physical-Grounded 3D Asset Generation
Venue: NeurIPS 2025
Accepted: yes
Keywords: 3D generation; Physical 3D assets
Abstract: 3D modeling is moving from virtual to physical. Existing 3D generation primarily emphasizes geometries and textures while neglecting physical-grounded modeling. Consequently, despite the rapid development of 3D generative models, the synthesized 3D assets often overlook rich and important physical properties, hampering their real-world application in physical domains like simulation and embodied AI. As an initial attempt to address this challenge, we propose \textbf{PhysX}, an end-to-end paradigm for physical-grounded 3D asset generation.
\textbf{1)} To bridge the critical gap in physics-annotated 3D datasets, we present \textbf{\ourname}\ - the first physics-grounded 3D dataset systematically annotated across five foundational dimensions:
\textbf{\textcolor{color2}{absolute scale}}, \textbf{\textcolor{color3}{material}}, \textbf{\textcolor{color1}{affordance}}, \textbf{\textcolor{color4}{kinematics}}, and \textbf{\textcolor{color5}{function description}}. In particular, we devise a scalable human-in-the-loop annotation pipeline based on vision-language models, which enables efficient creation of physics-first assets from raw 3D assets. \textbf{2)} Furthermore, we propose \textbf{PhysXGen}, a feed-forward framework for physics-grounded 3D asset generation, injecting physical knowledge into the pre-trained 3D structural space.
Specifically, PhysXGen employs a dual-branch architecture to explicitly model the latent correlations between 3D structures and physical properties, thereby producing 3D assets with plausible physical predictions while preserving the native geometry quality. Extensive experiments validate the superior performance and promising generalization capability of our framework. All the code, data, and models will be released to facilitate future research in generative physical AI.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a dataset of physical properties of 3D objects, building up on PartNet, including material, size, affordance, kinematics and functionality. The data was generated by prompting an LLM under human supervision. The paper further develops an approach to generate the physical properties within a 3D generation pipeline.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThe dataset is very valuable for research on 3D generation, as generating 3D objects for simulations or other applications requires such physical properties. The generative approach is compatible with existing 3D generation methods (e.g. Trellis) and the results demonstrate that the proposed joint approach improves over a separate physical property predictor.\n\nMajor weaknesses:\n1) The quality of the dataset is not validated in the paper. It depends a lot on the rigor of the human annotator (of 26k objects) whether the annotations are correct, since it is unlikely that GPT-4 has sufficient physical understanding to label these properties on a part level. How much correction was necessary? To verify the validity of the annotations, one could compare selected properties with the annotations from other datasets. For example, the labeled kinematics could be compared to the ones in the PartNetMobility dataset.\n2) The methodology is not explained well. Section 3.2 is very brief and not clear to me, e.g. how can k-means determine the location of the rotation axis? How is the type of kinematics (A-E) determined? What tasks in part 2 (kinematic parameter estimation) are done by the human annotator?\n3) The metric used (MAE of rendered images) is very unconventional. How are the properties encoded in these images? Why not use suitable per-property metrics from prior work? E.g., Euclidean distance for the absolute scale, text embedding cosine similarity for material and functionality, and suitable measures for kinematics such as instantiation distance [1]?\n\nMinor weaknesses:\n1) The dataset is presented as if it was a new 3D dataset, but the objects are entirely taken from PartNet. It should be clarified that this dataset is providing annotations for an existing dataset. \n2) Regarding “we merge the tiny parts whose vertices and area are smaller than a pre-defined threshold”: This sounds a bit dangerous with respect to kinematics. What if these merged parts have a non-rigid relationship?\n\nQUESTIONS:\nSee weaknesses above, summarized here:\n1) How to the labels compare to the ones from prior datasets, e.g. PartnetMobility?\n2) What parts of the annotation pipeline, especially the kinematics, is done by humans?\n3) How is the MAE between rendered images used to evaluate the property generation?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-13"}

comments: SUMMARY:
This paper introduces PhysXNet, a new dataset with rich part-level physical annotations, and a corresponding new physical generative model PhysXGen. PhysXNet is semi-automatically annotated with a human-in-the-loop review, ensuring its high quality. PhysGen further demonstrates the effectiveness of this dataset.

STRENGTHS AND WEAKNESSES:
### Strengths
- A new dataset with rich physical annotations. Happy to see a new PartNet-like dataset built with the help of VLM. This will advance physical-based generation and related fields such as robotics.
- A new method based on the dataset introduced, showing better performance than the combination of previous methods.

### Minor Weaknesses
- The design of PhysXGen is not very novel. It appears to be a new branch of adaptor based on pre-trained models. I suggest that authors show the computational cost of the adaptor and the whole model. It would be great if physical parameters could be predicted efficiently.

QUESTIONS:
- Are PhysXNet and PhysXPartNet the same thing?
- What is the metric for 'material'? I suggest authors add a brief introduction about metrics.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-29

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces PhysX, an end-to-end paradigm for generating 3D assets with physical properties, addressing a key gap in current 3D generative models. A central contribution is PhysXNet, \"the first comprehensive physics-grounded 3D dataset,  comprising over 26K richly annotated 3D objects with fine-grained physical attributes like scale, material, and kinematics. This dataset, built via a \"scalable human-in-the-loop annotation pipeline,\" enables PhysXGen, a novel framework that integrates physical knowledge into 3D asset generation, demonstrating strong performance in creating physically plausible models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n  \n- PhysXNet is the first dataset to systematically annotate 3D objects with a wide range of physical properties (scale, material, affordance, kinematics, function descriptions) at a fine-grained, part-level granularity. This goes beyond existing datasets that often provide only object-level or limited physical information.  \n- With over 26,000 objects, PhysXNet provides a substantial foundation for training and evaluating physics-aware 3D models.  \n- The proposed human-in-the-loop pipeline, integrating vision-language models (GPT-40) for initial labeling and human refinement, is an efficient and robust approach to creating such a detailed dataset.  \n- The paper proposes an end-to-end paradigm for physical-grounded 3D asset generation, addressing a critical and underexplored area in 3D content creation. This work has high potential impact for fields like robotics, simulation, and embodied AI. \n\nWeakness  \n\n- The paper's comparisons primarily focus on geometry-centric 3D generation or separate physical predictors. It would strengthen the evaluation to include comparisons with methods that directly learn material concepts from images/videos, such as those proposed by Zhai et al. (2024) or Zhong et al. (2024), to better contextualize PhysXGen's performance in specific physical property prediction.  \n\n- While the paper demonstrates performance on PhysXNet's test set, the generalization capability of PhysXGen to generating physical properties for 3D objects belonging to entirely novel categories or exhibiting physical behaviors (e.g., highly deformable objects, fluid interactions) not extensively represented in the PhysXNet dataset remains underexplored.\n\nQUESTIONS:\n1. What is the estimated accuracy of the VLM's initial annotations for each property type (e.g., material, affordance, kinematics), and what is the average human time/effort required for the \"expert refinement\" per object, especially for complex kinematic behaviors?\n\n2. How well would PhysXGen generalize to generating physical properties for 3D objects belonging to entirely novel categories or exhibiting physical behaviors not extensively represented in the PhysXNet dataset?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces PhysX, an end-to-end pipeline for physical-grounded 3D asset generation. The core contributions are the creation of PhysXNet, a large-scale, systematically annotated 3D dataset featuring part-level physical properties (such as absolute scale, material, affordance, kinematics, and rich function descriptions), and PhysXGen, a feed-forward dual-branch generative model that jointly models geometry and physical attributes for 3D asset synthesis. PhysXNet uses a human-in-the-loop annotation pipeline leveraging vision-language models for scalable property extraction, and PhysXGen combines geometry- and physics-aware components to inject physical realism in 3D synthesis. The paper demonstrates advantages over baselines through quantitative metrics and qualitative visualization, and benchmarks the impact of architectural choices via ablation studies.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1: The introduction of PhysXNet is a strong contribution, as summarized in Table 1 and discussed in Section 3. It offers over 26K 3D objects annotated with five key physical properties at the part level, which fills a significant gap in current 3D datasets. Its human-in-the-loop annotation pipeline (Fig. 2) with VLM assistance is thoughtfully developed and demonstrated to be practical and scalable.\n\n2: The taxonomy of physical properties (absolute scale, material and density, affordance, kinematics, and function description) is systematically discussed and collected, which is important for both simulation and embodied AI robotics tasks.\n\n3: The paper supports its claims with thorough empirical results. Experimental results show that the proposed dataset can facilitate 3D generation methods.\n\n4: The paper is well organized and written. Figures 4, 5, and 6 clearly show the architectural layout, generated outputs, and property-specific qualitative comparison, providing strong visual evidence of the method's efficacy. For example, Table 2 quantitatively demonstrates improvements in both physical property predictions and geometric fidelity, central to the paper's claim of joint modeling. Figure 6 visualizes similarity scores for physical attributes, supporting detailed comparative claims.\n\n5: The PhysXGen model is built on a dual-branch architecture that effectively utilizes pretrained structural priors and jointly learns correlations with physical properties. The architecture choices are clearly described and reasoned. The use of VAEs and conditional diffusion models for both geometry and physics is up to date, and training protocols appear sound.\n\nWeaknesses:\n\n1: **The experimental baselines are somewhat limited.** Although TRELLIS and TRELLIS+PhysPre are sensible, they are little compared to other recent strong generative models, especially those capable of handling physical aspects or integrating physical priors. The performance improvement is a somewhat narrow scope of empirical benchmarking (PSNR and CD).\n\n2: **Generalization and Diversity**. While the dataset is larger than some previous efforts, its generalization to other data domains (e.g., real-world scanned data, non-PartNet-derived assets) is not addressed. The claim of generalization is not fully substantiated with, for example, cross-dataset or out-of-distribution evaluations. \n\n3: **The applications are limited.** The authors only discuss that the proposed dataset can be used to generate physical 3D objects. From my understanding, the proposed dataset can also help 3D part-level perception models (like 3D parts segmentation) and part-based 3D generation models. It would be promising if the dataset could also improve the performance on these tasks (or this question could be simply left to the research community).\n\n4: **Physical Dimension Evaluation**. The evaluation of physical properties is reduced to a Euclidean comparison between rendered maps, which may not fully capture the semantics of errors in properties such as kinematics or affordances. This could be especially problematic for parts of the evaluation with a strong qualitative or functional component.\n\nAlthough I raise some minor weaknesses, it is noted that this paper makes a strong contribution to collecting data, along with many necessary physical attributes. Therefore, I vote for acceptance at this time and hope the authors could open-source the collected dataset.\n\nQUESTIONS:\n1: Where is the original data (such as paired part meshes and materials) from? Is it only from PartNet?\n\n2: Do authors try to curate existing datasets into PhysGen to scale up the dataset, although some of them may miss some kinds of annotations?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-03"}