Kati8/ICLR_2023_2025 huggingface dataset:
authors: Tai Hoang
authors: Huy Le
authors: Philipp Becker
authors: Vien Anh Ngo
authors: Gerhard Neumann
reviews: 
Comments: The paper trains policies for one or two end effectors to manipulate rigid and deformable objects (ropes and cloths). It models the actuator and object of interest with a heterogeneous graph and advocates for a heterogeneous equivariant policy (HEPi). The paper benchmarks its method on tasks including manipulating rigid and deformable objects in IsaacLab and reports its superior performance over typical baselines.
Strengths: In general, I find the story in this paper intuitive and convincing. While the two core building blocks (heterogeneous graphs and equivariant policies) have been explored in prior arts, this paper presents a proper combination of both in the setting of rigid/deformable object manipulation. I feel the proposed method has the potential to be extended to manipulate fluid and rigid/soft/fluid-coupled systems, and it would be a pleasant surprise if the paper could demonstrate a fluid scene similar to the “Pour Water” task in Lin et al 2020. Of course, I understand this is outside the scope of the current submission.
Weaknesses: I generally agree with the limitations listed in the paper. I think the paper can be improved in the following ways:

1. While the technical method and the story look promising, the benchmark scenes are still relatively simple because the setup only models a point-like end effector, ignoring kinematic and dynamic constraints in practical robotic hand/arm structures. Therefore, transferring the result to a real-world robot is not straightforward and probably requires more algorithmic development.

2. From a dynamic perspective, the 2D rigid-sliding task is too trivial to be a valuable benchmark for evaluating this paper and its baselines. This is already reflected in the result: “HEPi and Transformer policies perform comparably, suggesting that the limited task complexity does not fully leverage the benefits of equivariant constraints. ” I suggest this experiment remove the suction gripper and try pushing the rigid object to its target position and orientation with one/two end effectors and their contact with the object, similar to the setup in http://sain.csail.mit.edu/.

3. Deformable objects can be classified by their dimensions: codimension-2 (e.g., ropes), codimension-1 (e.g., cloths), and codimension-0 (e.g., rubber balls). The paper seems to consider the first two categories only. Handling full-dimension soft objects in the proposed method may be non-trivial because it requires observations of object coordinates and cannot be easily obtained by “integrating state-of-the-art computer vision techniques to extract key points from cameras.” I suggest the paper clarify this point by being more precise with the concept of deformable objects it deals with.
Recommendation: 8
Reviewer confidence: 2
Soundness/correctness: 3
Clarity: 3
reviews: 
Comments: This paper addresses the task of manipulating diverse rigid and deformable objects with geometry-aware rl policy.

A heterogeneous graph is proposed to represent rigid and deformable object manipulation tasks. To leverage geometric symmetry for better task performance and sample efficiency, a heterogeneous equivariant policy that utilizes SE(3) equivariant message passing networks is proposed. 

Evaluation is carried out on a self-curated rl benchmark, including rigid insertion of diverse objects, as well as rope and cloth manipulation with multiple end-effectors. The proposed method outperforms baseline methods in terms of average returns, sample efficiency, and generalizability.
Strengths: The proposed heterogeneous graph representation is a general representation for both rigid and deformable object manipulation tasks, and captures the geometric structure of the object. HEPi is a novel formulation of graph-based equivariant policy build on top of the heterogeneous graph representation that enables rl for manipulation of diverse shapes and deformable objects.

The paper has demonstrated thorough evaluations and ablations in simulation, with convincing results proving that the proposed method has better performance, sample-efficiency, and generalizability. Detailed discussions are provided for the results, providing interesting insights.
Weaknesses: The object model that contains vertices are required for the object node representation, which is the main observation for the policy. This might not be easy to get in the real world due to a lot of self-occlusions during deformable object manipulation.

The paper demonstrates thorough evaluations across various simulation benchmarks, but lacks real-world evaluations to make the method fully convincing in terms of it's usefulness to real-world robot applications.

typo at line 74: equivariacne->equivariance.
Recommendation: 8
Reviewer confidence: 3
Soundness/correctness: 3
Clarity: 3
reviews: 
Comments: The authors propose a novel SE(3) equivariant RL method called HEPi, and introduce a new benchmark for future geometry-aware RL evaluations.  The paper use EMPNs as algorithm backbone to allow model generalize between poses and its heterogeneity. The paper use graph as representation and do thorough mathmetical analyze.
Strengths: + Clever Design: This paper includes several ingenious designs. 
   + The use of EMPNs The use of SE(3)-equivariant networks ensures that the model naturally possesses SE(3) generalization, thereby reducing the search space and complexity. 
   + The use of TRPL the introduction of TRPL in place of traditional PPO makes hyperparameter tuning easier, addressing a significant challenge in RL training.

+ Detailed Appendix Experimental Description: One of the major issues in RL is poor reproducibility. However, in this paper, the authors provide a detailed appendix that lists experimental information, including the reward function and hyperparameters. This makes the paper highly reproducible.

+ Benchmark Design: This paper provides a detailed demonstration of the proposed benchmark tasks in the video and clearly defines these tasks in the appendix. As a result, these tasks and environments can be easily adopted by the research community.

I am inclined to accept this paper.
Weaknesses: + Real-World Application: As the authors note, this paper uses ground-truth model inputs and does not incorporate a physical robot, which means it cannot be directly applied to real-world scenarios. However, given the significant contributions of this paper in terms of benchmarking and methodology, I do not consider this a critical issue. Nonetheless, I still recommend that the authors include some specific experiments to evaluate this aspect. I suggest author can simply add visual capture pipeline. The details are as follows:
    + For task like rigid insertion and rigid sliding, you can simply add camera to take picture then use a common pose estimation module to estimate the position. The potential errors in pose estimation can be used to test the robustness of the proposed method to input errors.
    + For tasks involving fabrics or ropes, you can directly use a camera to capture point clouds and then construct a graph using the point cloud data.
+ Geometry aware: This paper does not seem to have any special designs focused on geometry, although graphs do play a role. However, the primary focus of the paper appears to be on SE(3)-equivariant designs.
Recommendation: 8
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 4