Title: CoDA: Coordinated Diffusion Noise Optimization for Whole-Body Manipulation of Articulated Objects
Venue: NeurIPS 2025
Accepted: yes
Keywords: Whole-body motion generation

human-object interaction

articulated object manipulation
Abstract: Synthesizing whole-body manipulation of articulated objects, including body motion, hand motion, and object motion, is a critical yet challenging task with broad applications in virtual humans and robotics.
The core challenges are twofold.
First, achieving realistic whole-body motion requires tight coordination between the hands and the rest of the body, as their movements are interdependent during manipulation. 
Second, articulated object manipulation typically involves high degrees of freedom and demands higher precision, often requiring the fingers to be placed at specific regions to actuate movable parts.
To address these challenges, we propose a novel coordinated diffusion noise optimization framework.
Specifically, we perform noise-space optimization over three specialized diffusion models for the body, left hand, and right hand, each trained on its own motion dataset to improve generalization.
Coordination naturally emerges through gradient flow along the human kinematic chain, allowing the global body posture to adapt in response to hand motion objectives with high fidelity.
To further enhance precision in hand-object interaction, we adopt a unified representation based on basis point sets (BPS), where end-effector positions are encoded as distances to the same BPS used for object geometry.
This unified representation captures fine-grained spatial relationships between the hand and articulated object parts, and the resulting trajectories serve as targets to guide the optimization of diffusion noise, producing highly accurate interaction motion.
We conduct extensive experiments demonstrating that our method outperforms existing approaches in motion quality and physical plausibility, and enables various capabilities such as object pose control, simultaneous walking and manipulation, and whole-body generation from hand-only data.
The code will be released for reproducibility.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel framework for synthesizing whole-body manipulation of articulated objects by coordinating body, hand, and object motions. The approach leverages three separately trained diffusion models for the body, left hand, and right hand, and performs joint noise-space optimization to enable coordination through the human kinematic chain. A unified object-hand representation using basis point sets (BPS) enhances fine-grained control and precision in interaction. The method supports capabilities such as object pose control, walking during manipulation, and motion synthesis from hand-only data. Experiments show improved motion quality and physical plausibility over prior work.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. The paper tackles the problem of whole-body human-object interaction, which is more realistic than hand-only interaction. The proposed method decomposes the task into object trajectory generation, end-effector trajectory generation, and coordinated diffusion noise optimization via the kinematic chain. This modular design improves joint control and generalization. The use of forward kinematics enables automatic optimization and adaptability across different humanoid configurations.\n\n2. The paper is well-written and clearly presented, with high-quality visualizations that effectively convey the model’s capabilities.\n\n3. The authors provide comprehensive comparisons against relevant baselines and include meaningful ablation studies to validate key design choices.\n\n**Weaknesses**\n1. The evaluation is limited to static HOI datasets. The work would be strengthened by demonstrating performance in simulation environments (e.g., SAPIEN, Isaac Sim) or on real-world humanoid robots to assess applicability in dynamic, embodied settings.\n\n2. The articulated objects used (from the ARCTIC dataset) are relatively simple two-part structure, which may not capture the complexity of real-world articulated objects. Evaluation on more diverse datasets such as PartNet-Mobility[1], AKB-48[2], DexArt[3], or AdaManip[4] would improve the generality of the conclusions. This limitation is also acknowledged by the authors.\n\n3. Several components of the proposed framework (e.g., the BPS-based representation, RoPE-style encoding from CaPE, and DDIM sampling from DNO) are adaptations of existing techniques. While the integration is novel, the method may be viewed as an incremental combination rather than a fundamentally new paradigm.\n\n[1] Xiang, Fanbo, et al. \"Sapien: A simulated part-based interactive environment.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.\n\n[2] Liu, Liu, et al. \"Akb-48: A real-world articulated object knowledge base.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.\n\n[3] Bao, Chen, et al. \"Dexart: Benchmarking generalizable dexterous manipulation with articulated objects.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n\n[4] Wang, Yuanfei, et al. \"Adamanip: Adaptive articulated object manipulation environments and policy learning.\" arXiv preprint arXiv:2502.11124 (2025).\n\nQUESTIONS:\n1. Articulated object manipulation has been extensively studied in the robotics community. Could the authors elaborate on how their approach relates to prior work in robotics, particularly those involving interaction with complex mechanisms (e.g., PartNet-Mobility, AKB-48, DexArt, AdaManip)? A more thorough discussion would help clarify the contribution relative to existing manipulation pipelines.\n\n2. Given the limitations of the ARCTIC dataset and the use of static HOI datasets and perceptual metrics (e.g., FID, R-Precision), the current evaluation may not fully reflect the model’s effectiveness in physically grounded scenarios. Have the authors considered evaluating their method in interactive simulation environments such as SAPIEN or Isaac Sim? Benchmarking the success rate of generated motions in these settings would provide more robust validation. As noted in the limitation section of the paper, extending the framework to support sequential interactions remains an open challenge—one that datasets like AdaManip are designed to capture.\n\n3. Have the authors considered validating their method on a real-world humanoid robot? While simulation is a useful intermediary, demonstrating that the generated motions can transfer to physical platforms would significantly strengthen the practical relevance of this work.\n\nLIMITATIONS:\nSee the Questions section. The paper would benefit from additional evaluation on more diverse and realistic simulation benchmarks and real-world experiments to better demonstrate the method's generalizability and practical applicability.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper aims to address a new task that generates whole-body human-object interaction motions with articulated objects. Leveraging human-object interaction and human-body locomotion data, the paper proposes to split the interaction generation problem into loss function generation and whole-body diffusion noise optimization. Experiments show that the method outperforms several existing methods, and they demonstrate various method applications.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1) The paper is solving a new and challenging task. Existing interaction generation methods either focus on interacting with rigid objects or neglect human body movements.\n2) The proposed method is straightforward.\n3) The evaluation metrics comprehensively reflect motion naturalness, motion diversity, and physical plausibility.\n4) The ablation study demonstrates the value of individual method designs.\n5) The writing is clear and easy to follow.\n\nWeaknesses:\n\n1) The paper claims that the method allows the use of hand-only datasets to generate whole-body motion, whereas the body diffusion model requires motion priors from existing human-body datasets.\n2) Head orientation is often unnatural in the qualitative results of the method (e.g., Figure 1(d)). This could be due to the data used to train the body diffusion model is not fully interaction-aware. It can be challenging to generate coordinated and natural bodies and hands using only the gradient descent without the coordination of motion semantics.\n3) The time consumption of the method could be significantly large. This limits the real application of the method.\n4) Traditional diffusion-based methods (e.g., OMOMO) directly perform one forward diffusion process during inference time, which does not take good physical plausibilities into consideration. The paper's method defines physical loss functions and performs test-time optimization, which ought to achieve better physical performance. Comparing the method with physics-aware baselines (e.g., OMOMO + physics-based post optimization, OMOMO + diffusion noise optimization) is necessary.\n5) In the existing baseline methods, only IMoS aims to solve the whole-body interaction generation problem. The paper misses more advanced whole-body interaction generation baselines such as [1-2].\n6) Some method names (e.g., \"HOIFHLI\" in Line 115) are unofficial and could be misleading. It is better to use author names (e.g., \"Wu et.al.\").\n\nReferences:  \n[1] Wu Z, Li J, Xu P, et al. Human-object interaction from human-level instructions[J]. arXiv preprint arXiv:2406.17840, 2024.  \n[2] Tendulkar P, Surís D, Vondrick C. Flex: Full-body grasping without full-body grasps[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 21179-21189.\n\nQUESTIONS:\n1) Can the idea of the proposed method apply to the existing human-object interaction generation tasks (e.g., whole-body interaction with rigid objects, hand-object interaction with articulated objects)? If so, the contribution of the paper could increase.\n\n2) In Lines 291-292, the paper points out that the method allows control over keyframe object poses. The object trajectory generation is a simple conditional diffusion model. What is setting optimization targets?\n\n3) What happens if we set the object trajectory or the text as the condition of the hand diffusion models? Is it better than the existing method design?\n\n4) What is the time cost of the proposed method?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors introduce CoDA, a method for generating full-body motion of articulated object manipulation. The method can be used for text-to-motion generation, motion generation using object key frames, and motion generation from hand trajectories. The key insight is to model motion generation of the two hands and body separately, allowing to be supervised by each component's own data source. This is the first method for full-body articulated-object motion generation. Results show superior performance against baselines.\n\nSTRENGTHS AND WEAKNESSES:\n## Pros\n\n**The results are impressive**: They are both quantitatively and qualitatively significantly better than the baselines. \n\n**Comprehensive experiments**: The authors provide detailed comparison with many existing baselines (see Table 3) in terms of FID, diversity, and physical metrics like foot skating and interpenetration volume etc on both ARCTIC and GRAB. Detailed ablation is provided in Table 4 to ablate the effect of invididual components such as end-effector BPS and AMASS. \n\n**Good presentation**: The writing is easy to understand. The video is intuitive.\n\n## Cons\n\n**Comparison/clarification on interaction sensors**: Overall, I quite like the paper. The authors introduced \"end-effector BPS\", a distance-based features to represent the positions of joints in the object coordinate systems. In general, there are similar \"sensors\" or \"interaction encoder\" in the motion generation line of work. It would be better if the authors could clarify if the choice of such representations actually make a big difference in the results? For example, in ManipNet, they introduced  \"Proximity sensor\" and \"Signed Distance Sensor\", how do they differ in performance compared to \"end-effector BPS\"?\n\n**Quantitative evaluation on unseen object generalization**: Both ARCTIC and GRAB are a bit limited in terms of object diversity. This means, generalization to unseen object is important for future work. I understand that this is an extremely challenging task given the limited data. However, it is best for the authors to provide some quantitative measures for future work to compare against. \n\n## Justification\n\nOverall, I think the paper is well-written, easy to understand. The experiments are quite comprehensive. The only suggestions are the clarification on existing interaction representations (do they really make much difference?). Further, generalization to unseen objects is an important future direction, and it should be comparable.\n\nQUESTIONS:\nSee \"Cons\".\n\nLIMITATIONS:\nThe authors discussed the positive impacts but I think a discussion on the negatives could benefits in a long term. For example, when combined with human pose conditioned video generation methods, this proposed method can be used to generate videos of real individuals (e.g., public figures) performing actions that they did not do. This can raise concerns on misinformation.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-30"}

comments: SUMMARY:
This paper proposed a novel coordinated diffusion noise optimization framework for human-objection interactions generation. The specialized hands, body, object diffusion models are trained on its own datasets separately. The proposed unified representation captures fine-grained spatial relationships between the hand and articulated object parts, obtaining highly accurate interaction motion. Extensive experiments show the superiority of the proposed method in quality, physical plausibility, and capabilities.

STRENGTHS AND WEAKNESSES:
+ The proposed coordinated diffusion noise optimization framework is interesting. It allows to train specialized diffusion models on its own datasets and can merge the expertises together to generate whole-body interactions.
+ The proposed unified BPS-based representation for object and end-effectors is novel and seems effective. It facilitates geometric alignment between end-effectors and object parts.

- Computational Cost and Efficiency. The multi-stage pipeline, especially the optimization process with diffusion, is significantly more computationally intensive than standard generative approaches.

QUESTIONS:
1. Will the separate models limit the potential to learn inter-correlation between parts, such as how the object types/movements affect the body motion. For example, if a person will have a less probability to run if he is operating a larger items like microwave, compared with some small items like phone.
2. Is it pipeline limieted to hand manipulation? Is that possible to extend it to whole-body contact?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03