Title: DIPO: Dual-State Images Controlled Articulated Object Generation Powered by Diverse Data
Venue: NeurIPS 2025
Accepted: yes
Keywords: articulated object

articulated object dataset
Abstract: We present **DIPO**, a novel framework for the controllable generation of articulated 3D objects from a pair of images: one depicting the object in a resting state and the other in an articulated state.
Compared to the single-image approach, our dual-image input imposes only a modest overhead for data collection, but at the same time provides important motion information, which is a reliable guide for predicting kinematic relationships between parts.
Specifically, we propose a dual-image diffusion model that captures relationships between the image pair to generate part layouts and joint parameters. In addition, we introduce a Chain-of-Thought (CoT) based **graph reasoner** that explicitly infers part connectivity relationships.
To further improve robustness and generalization on complex articulated objects, we develop a fully automated dataset expansion pipeline, name **LEGO-Art**, that enriches the diversity and complexity of PartNet-Mobility dataset. We propose **PM-X**, a large-scale dataset of complex articulated 3D objects, accompanied by rendered images, URDF annotations, and textual descriptions.
Extensive experiments demonstrate that DIPO significantly outperforms existing baselines in both the resting state and the articulated state, while the proposed PM-X dataset further enhances generalization to diverse and structurally complex articulated objects.
Our code and dataset are available at https://github.com/RQ-Wu/DIPO.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper propose a novel framework, called DIPO, which can controllablly generate 3D object.  Compared to one-image method, this paper utilizes a pair of image to provide more motion information which helps predicting kinematic relationships between parts. This paper also introduce a Chain-of-Thought (CoT)  to causally  infer the relationship of parts. A new dataset, PM-X, is proposed to make a contribution to community.  Moreover,  this paper conduct extensive experiments to demonstrate the outperformance of the proposed method compared to prior arts.\n\nSTRENGTHS AND WEAKNESSES:\nStrong:\n1. This paper propose  a 3D joint object generation framework based on image pairs of the resting state and the extended state. The dual-state image pairs offer important motion information. Compared with the single-image method, the dual-state input solves the fuzziness of component motion and spatial relationship. For multi-view methods, pair images are easy to obtain and cost less computation.\n2. To get more accurate relationships of parts, a Chain-of-Thought (CoT) tech is also introduced  to achieve better performance. \n3. This paper provides a new dataset, PM-X, which contains diverse and structurally complex joint objects, along with rendered images, URDF annotations, and language descriptions. I thinks the dataset has a great contribution to community.\n\nWeakness.\n1. The whole pipeline of this paper is complex, which relays several modules, such as diffusion model, COT and so on. And the implementation  details of this paper is not enough. So, I am afraid it is hard to reproduce the results of this proposed method.\n2. This paper just displays some demos about Cabinet.  So, I want to know how about this method perform on other objects. I think it is import to show the Generalization of this paper.\n\nQUESTIONS:\n1. This paper should give more implementation details to help readers easily follow.\n2. More objects demos should be given to demonstrate the generalization.\n\nLIMITATIONS:\nThis paper should discuss the limitations of the proposed method and the future work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work introduces DIPO, a novel approach to 3D articulated objects generation using dual-image inputs -- one depicting the object in a resting state and the other in an articulated state. Fundamentally, DIPO operates as a diffusion transformer model that infers part layouts and joint parameters from motion cues and spatial relationships derived from the image pair. Initially, the two images are processed through the **Graph Reasoner** module -- a LLM using Chain-of-Thought techniques -- to infer consistent part connection properties in the form of an adjacency matrix. The adjacency matrix, combined with the DINOv2 features extracted from the images, serves as a strong condition signal for the diffusion model using a dedicated cross-attention mechanism. The generated articulated abstraction obtained from the diffusion model is then transformed into an actual articulated asset via retrieval methods (similar to the CAGE approach).\n\nAdditionally, to address the scarcity of large-scale datasets featuring high-quality articulated objects, the authors developed an agentic data augmentation pipeline -- named **LEGO-Art** -- and used it to improve the diversity and complexity of PartNet-Mobility dataset -- the current reference dataset for articulated assets. The result is an enriched dataset, termed **PartNet-Mobility-Complex (PM-X)**, where each articulated asset comes with corresponding multi-state rendered images, URDF annotations and a comprehensive language description.\n\nThe results are compelling: DIPO outperforms existing methods -- in particular NAP and SINGAPO -- in generating 3D articulated objects and provides enhanced controllability.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths:\n- Strong motivation and related works section: I believe your analysis regarding input modality constraints and limitations in training data -- in the introduction section -- is spot on. \n- Additionally, the fact that DIPO uses easily available data (picture of articulated object in two different states) makes it quite handy from a final user perspective. Approaches such as NAP, although quite innovative, are much trickier to control. \n- LEGO-Art is definitely a relevant contribution to the field as the lack of large scale curated dataset for articulated objects is quite problematic. \n- Strong experiment part\n- Relevant ablation study\n\n### Weaknesses: \n\n- The graph reasoner module is insufficiently documented. \n- The remaining are only minor considerations detailed in the \"Questions\" paragraph.\n\n### Conclusion: \n\nThis paper is clear, well written and relevant to the field. It is backed by solid experimental evidence and a relevant ablation study. I believe it is up to the standards of NeurIPS: my opinion is that it should be accepted.\n\nQUESTIONS:\nAs mentioned earlier, I only have a few minor questions/suggestions: \n- Which model did you use as a graph reasoner ? How many billion parameters ?\n- As far as I understand, you only tested your approach on the Storage Furniture category. \n    - How would you generalize your approach to more complex categories such as, say, robots ? \n    - PartNet Mobility contains a bunch of complex, highly redundant assets (e.g. keyboard): how robust is your approach to these assets ? Note that I do not expect new experiment results here: just a few sentences would be enough in my mind to explain this. \n\n- PartNet Mobility contains several assets where one of the mobile parts might be occluded in one of the images (e.g. the tray of a dishwasher): how would DIPO be dealing with this type of assets ? Suggestion: state a few research tracks as future works in your paper. \n\n- What about the assets whose state is not noticeably changing on the two images (e.g. globe with a uniform texture) ? \n\n- In Figure 1, the range of the articulated states seems fully covered, from lower to upper limits. In Figure 5, on the third row, the drawers in the upper and middle assets are only partially opened in the articulated images. Does this affect the joint range of the generated asset ?\n\n- How long does the training take with your setup ? How long does inference takes ?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes DIPO, a framework for generating articulated objects conditioned on a pair of images representing different articulation states. DIPO comprises three main components: (1) a diffusion model that synthesizes articulated objects conditioned on paired images and connectivity graphs; (2) a graph reasoner that predicts connectivity graphs; and (3) an automated agentic data expansion pipeline, LEGO-Art, which leverages VLMs to generate articulated objects as training data. The authors collect a dataset using the proposed data expansion pipeline and conduct experiments to validate the performance of the overall framework.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. The proposed framework is well aligned with the problem and demonstrates potential for scaling up.\n2. The architecture and module design of the diffusion model are well-suited for image-conditioned articulated object generation.\n3. The paper presents comprehensive experiments to evaluate the proposed method's performance on articulated object reconstruction and analyze the contributions of individual components.\n4. The paper is well written and easy to follow.\n\nWeaknesses\n1. Although acknowledged in the limitations section, the experiments are restricted to cabinet-like object categories.\n2. The paper does not evaluate the success rate or robustness of the graph reasoner and the data expansion pipeline. Since these components rely heavily on VLMs, the impact of hallucination, biases and ambiguity inherent to VLMs should be assessed.\n3. LEGO-Art employs an occupancy grid to represent articulated object structure, which may be effective for boxy, cabinet-like objects but less suitable for those with irregular geometries. Additionally, VLMs are known to have limited spatial reasoning capabilities. Thus, further justification for this design choice is necessary.\n\nQUESTIONS:\n1. For the PM-X objects generated by LEGO-Art, is there any manual filtering involved after generation? What is the success rate of the VLM visual filtering?\n2. Please also see weakness\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a dual-state image model for controllable articulated 3D object generation, consisting of layout diffusion and CoT-based connectivity reasoning. It also introduces a pipeline to construct structurally diverse articulated objects. Thanks to the introduced pipeline, a new large-scale dataset, PM-X, is proposed with better diversity and complexity.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n1. The proposed framework for articulated object generation is technically sound to me. Combining resting-state image and articulated-state image dose provide enriched motion information for predicting kinematic relationships between parts.\n\n2. The proposed automated dataset expansion pipeline and the large-scale dataset of complex articulated 3D objects are helpful for enhancing the performance of articulated 3D object generation.\n\n3. Part of the code is provided.\n\n### Weaknesses\n\n1. Quality. The experimental settings require clarification. In Tables 2 and 3, the meanings of RS- and AS- metrics are ambiguous — do they correspond to models using only resting-state or articulated-state images? It is also unclear how the model performs when both types of images are used together. Additionally, the experiments are limited to StorageFurniture and Table categories, leaving the generalization ability to other articulated objects unverified.\n\n2. Clarity. The writing could be improved for better readability. For example, the AI Agent Designer introduced in Lines 179–182 does not seem to be a direct part of the synthesis pipeline. Also, the placement of Table 4 after Table 5 disrupts the logical flow. Furthermore, the descriptions of the synthesis pipeline and its modules are insufficiently detailed, which hampers understanding.\n\n3. Significance. The contribution of the PM-X dataset is unclear. In experiments, it is only used to enrich training data, without standalone evaluation. Moreover, PM-X includes only two object categories, which limits its broader applicability. A more detailed discussion on its role and value is needed.\n\n4. Originality. The framework for articulated 3D object generation and the synthesis pipeline relies on existing techniques such as LLM, VLM, and retrieval algorithms. This integration, while practical, may reduce the perceived novelty of the approach. Clarifying the unique contributions beyond combining known components would help better establish the work's originality.\n\nQUESTIONS:\n###  Questions\n\n1. Could the authors clarify whether the RS- and AS- metrics in Tables 2 and 3 correspond to using only resting-state or articulated-state images? Is there an experiment that evaluates performance when both image types are used together?\n\n2. Have the authors conducted experiments on other articulated object categories beyond StorageFurniture and Table? If not, is there a specific reason why the method is limited to these categories?\n\n3. Is there an evaluation of the model performance on PM-X alone to justify its value?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-05"}