Title: Grasp2Grasp: Vision-Based Dexterous Grasp Translation via Schrödinger Bridges
Venue: NeurIPS 2025
Accepted: yes
Keywords: dexterous grasping

generative modeling

robotic manipulation

robot learning
Abstract: We propose a new approach to vision-based dexterous grasp translation, which aims to transfer grasp intent across robotic hands with differing morphologies. Given a visual observation of a source hand grasping an object, our goal is to synthesize a functionally equivalent grasp for a target hand without requiring paired demonstrations or hand-specific simulations. We frame this problem as a stochastic transport between grasp distributions using the Schrödinger Bridge formalism. Our method learns to map between source and target latent grasp spaces via score and flow matching, conditioned on visual observations. To guide this translation, we introduce physics-informed cost functions that encode alignment in base pose, contact maps, wrench space, and manipulability. Experiments across diverse hand-object pairs demonstrate that our approach generates stable, physically grounded grasps with strong generalization. This work enables semantic grasp transfer for heterogeneous manipulators and bridges vision-based grasping with probabilistic generative modeling. Additional details at https://grasp2grasp.github.io/.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of transferring successful grasps from one robot hand to another based on the well-known Simulation-Free Schrödinger Bridges. The method first encodes grasps from each hand into a shared latent space, then uses the Schrödinger Bridge to model the optimal transport between the two latent distributions. Extensive experiments show that the proposed approach significantly outperforms existing grasp transfer methods.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n- Overall, this is a well-written and insightful paper that addresses the long-standing challenge of transferring dexterous grasp.\n\n- The use of Simulation-Free Schrödinger Bridges for transferring grasp is both novel and well-motivated.\n\n- Experiments on the MultiGripperGrasp dataset demonstrate substantial improvements over state-of-the-art methods.\n\n- The ablation studies (Tables 1–3) further validate the effectiveness of the proposed optimal transport cost design.\n\n**Weaknesses:**\n\nI have no major concerns about the paper. However, here are some suggestions to improve clarity and completeness:\n-  *Lack of evaluation of the VAE*. Although the VAE is used primarily as a means to encode grasps across different robot hands, its performance should still be evaluated, for instance, reconstruction quality. Additionally, some visualization of the latent space trajectories during the Simulation-Free Schrödinger Bridge process would help further understand the proposed solution.\n- *Concerns over presentation.* While the paper is generally well-written, I find Sections 3.1 and 3.2 to be somewhat redundant, as they closely follow the original Simulation-Free Schrödinger Bridges paper [78]. These sections could be reduced or moved to the appendix. In contrast, the core technical contribution—modeling the Schrödinger Bridge between latent grasp spaces via $v_\\theta(t, z)$ and $s_\\theta(t, z)$—is essential to the paper but is currently under-explained. I recommend moving this part to the main text to better clarity.\n\nQUESTIONS:\n- Would it be possible to include the remaining three transfer settings (A→H, S→H, and A→S)? Including these would make the evaluation more completed and reduce potential bias.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces Grasp2Grasp, a method for grasp pose translation that aims to transfer grasping poses from one embodiment to another. Grasp2Grasp adopts a novel strategy by modeling valid grasps as an object-conditioned distribution and learning a distributional transformation between different embodiments. The translation process involves sampling from the latent space of the source embodiment, transforming the latent representation to align with the target embodiment's distribution, and decoding it into a corresponding grasp pose. To ensure the plausibility of the translated grasps, the method incorporates several cost terms that measure differences between grasp poses not only in appearance but also in terms of task effectiveness. Experimental results demonstrate the method’s ability to generate plausible and task-relevant grasping poses across embodiments.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n- The paper presents a well-formulated problem and an interesting approach. Framing grasp pose translation as a stochastic process that transforms the source distribution into the target distribution is both a strong and reasonable formulation. The use of SB to model the relationship between the two distributions, combined with OT to complete the pose transformation, is well-designed and conceptually sound.\n- The inclusion of algorithms in the method section enhances clarity and facilitates quick understanding of the approach.\n\nWeaknesses\n\n- **Experiments**: The experimental results raise several concerns regarding quality. As shown in Figure 3 and the supplementary figures, although the generated grasp poses do not exhibit significant penetrations into the object, they appear physically unstable. Furthermore, the qualitative results are limited and insufficient to convincingly demonstrate the method’s effectiveness, as only results on two objects—*sofa* and *cup*—are included. In particular, the generated poses for the *cup* differ significantly from the source poses, suggesting that the method sacrifices too much visual similarity in favor of physical validity.\n\n  Additionally, no real-world experiments are presented. This absence raises further doubts about whether the generated poses are truly physically plausible in practical settings.\n\n- **Presentation**: The paper's presentation requires substantial improvement. In Section 4, it is unclear why equations from Section 3 are reused without further clarification. In Section 4.2, the integration of the proposed cost terms into the OT process is not adequately explained. Simply stating \"following XXX\" is insufficient; the authors should not assume that readers are fully familiar with the referenced works. Clear and self-contained explanations are essential for understanding the methodology.\n\nQUESTIONS:\n- Does the encoding process take the grasping pose into account? According to the formulation $z_0\\sim q_0(z_{source}\\vert o_{obj})$, only the object observation is used. However, Figure 2 suggests that the observation also includes the hand pose. \n- The roles of the source hand pose gsg_sgs in Algorithms 2 and 3 are unclear. It appears that $g_s$ is not involved in either the encoding or the sampling process. \n- What metric is used to compute the distance between the two 6D rotation representations in Equation 9? \n- Are the generated grasping poses validated in a physics simulator?\n\nLIMITATIONS:\n- Quality. The quality of the results is not satisfactory. Moreover, the paper lacks a thorough discussion and analysis of the experimental outcomes, which weakens the overall impact and clarity of the contributions.\n- Method and the presentation. The methodological description and overall presentation are unclear. It is difficult to grasp the detailed process proposed in the paper, which raises doubts about whether the method can effectively address the grasping pose *translation* problem.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a vision-based dexterous grasp translation method framed as a Schrödinger Bridge problem. This method aims to translate grasp intentions from a source robotic hand to a target hand with different morphologies without relying on paired demonstrations. By employing physics-informed cost functions and learning distributional grasp translations, the method attempts to bridge morphological gaps between diverse robotic manipulators.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n- The proposed method leverages an innovative Schrödinger Bridge formulation, enabling a principled probabilistic approach to grasp translation.\n- Introducing physics-informed cost functions is a valuable contribution, effectively guiding the grasp translation process to maintain functional grasp similarity despite morphological differences.\n- The method demonstrates promising performance in simulation environments across multiple hand-object pairs, indicating strong potential for generalization.\n\n**Weaknesses:**\n\n- The paper appears to be slightly over-claiming, which needs further clarification:\n    - A central claim made by the authors is that *“While recent works in data-driven grasp generation have shown success in generating feasible grasps for specific hand object pairs, they often rely on large-scale datasets curated through computationally expensive simulation or real-world trials.”.* However, the proposed method itself is trained on MultiGripperGrasp, a substantial dataset containing approximately 30.4 million grasps across 11 robotic manipulators and 345 objects. Thus, the claim is misleading.\n    - More critically, another prominent claim is about overcoming the hand-specific limitations of existing models *“Moreover, such models typically operate in a hand-specific setting, which requires retraining or significant fine-tuning to adapt to new robotic hands.”*. The authors argue their approach generalizes to unseen dexterous hands without requiring retraining or significant fine-tuning. To validate this claim meaningfully, the authors need explicit evidence demonstrating robust generalization to unseen robotic hands not included in the training dataset. Otherwise, the claimed advantage over existing hand-specific methods becomes questionable.\n- Tables and figures are not self-contained or sufficiently explained. For instance, in Figure 3, terms like \"pose\" and \"contact\" lack clear definitions within the figure caption, causing confusion and reducing readability.\n- The paper lacks real-world experiments entirely. Without empirical validation in real-world scenarios, the practicality and applicability of the proposed method remain questionable.\n\nQUESTIONS:\n- Regarding the human hand scenario, the input to the VAE consists of rendered images of a skeletal hand model, making practical application to actual human hands infeasible. This effectively treats the human hand as merely another robotic hand model. Could the authors clarify how their approach can be realistically extended to accommodate real human hand data?\n- It is unclear whether the input to the VAE is RGB image or point cloud. Figure 2 indicates point cloud, while the text description indicates RGB. If it’s RGB, how to do the segmentation in real-world? What if the input image is a highly occluded one, which is quite common in dex mainpulation? If it’s point cloud, how to obtain it during inference?\n- Do we need contact anchor during inference? My current understanding is “yes”. But how can we get it during inference?\n\nLIMITATIONS:\nSee Weakness and Question", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents an approach to grasp translation across robot hands. It formulates the problem as a stochastic transport between grasp distributions of different hands and learns to map between two latent distributions via score and flow matching. The translation is guided using alignment cost functions, such as pose, contact maps, wrench space, and manipulability alignment. Experiments include comparisons with previous methods and a diffusion baseline in terms of quality, diversity, and alignment, as well as ablation studies for diffusion rate and discretization steps.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1. The formulation of stochastic transport for grasp translation and learning the mapping via score and flow matching is novel and interesting.  \n2. Compared to previous works, the four types of alignment (pose, contact maps, wrench space, and manipulability alignment) are comprehensive.  \n3. The proposed method outperforms the previous method RFP in all metrics in Table 1.  \n4. The presentation quality is high, and the figures are clear.  \n\n**Weaknesses:**\n\n1. It is unclear which alignment metric is most suitable for the task. The paper adopts 6D GWH IoU as the alignment metric, and the contact map alignment cost achieves better alignment. However, this metric may not be useful for teleoperation tasks, which are likely the main application of grasp translation. It would be better to incorporate pose alignment metrics for evaluation.  \n2. Since grasp translation is often used for real-time teleoperation, a detailed efficiency report is expected, especially for the diffusion-based methods.  \n3. While the main contribution is latent score and flow matching for grasp translation, it would be better to include more baselines such as Dex-Retargeting, which is widely used in teleoperation papers, or neural retargeting method like CrossDex. The reviewer would like to see comparisons in terms of success rate, pose alignment, and efficiency.  \n4. Real-world experiments would make the paper more convincing.\n\nDex-Retargeting: https://github.com/dexsuite/dex-retargeting  \nCrossDex: https://github.com/PKU-RL/CrossDex/\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}