Paper: Language Conditioned Equivariant Grasp (ICLR-2024)

Reviewer comment (Experiments):
The paper claims that the proposed inductive bias leads to high sample efficiency, but there is no direct evidence in the experiments.

Author response:
We respectfully disagree with the point raised. As shown in Tables 1 and 2, our proposed LEG-UNET and LEG-Cliport models outperform four strong baselines in 25 out of 26 testing scenarios, trained with either 1 or 10 demonstrations. Notably, our models trained with just a single demonstration can surpass the baselines trained with 10 demonstrations in the 'pick-tool' and 'pick-toy' tasks. Furthermore, they achieve a success rate above 90% in 6 out of 13 tasks. It is important to note that we trained our models exclusively on 'pick-random-v1' or 'pick-random-v2' and tested them on other V-1 or V-2 tasks.

Reviewer comment (Novelty):
Neural Descriptor Field [1] is an SE(3) equivariant grasping method with a similar design to the proposed method. Please cite and analyze the similarities and differences between this paper and [1].

Author response:
While NDF is commendable work, we believe that our design is not similar to it. Firstly, our focus is on language-conditioned grasping, learning directly from pixel space, whereas NDF does not incorporate language instructions and learns from point clouds. Secondly, our approach does not assume objectness, segmentation, or categorization, in contrast to NDF, which requires a segmentation module and learns a category-level pick-and-place policy. Thirdly, our method does not require any pretraining process, while NDF necessitates pretraining to acquire point descriptors. Although both methodologies learn from demonstrations and leverage task symmetries, it is intriguing why the reviewer perceives them as similar.

Reviewer comment (Theory):
The experiment section mentioned designing and obtaining a reward, suggesting that the proposed method is RL-based. However, there is no mention of reinforcement learning or any related formulation in the method section, except naming \( p(a_t|o_t,l_t) \) as a policy once.

Author response:
As mentioned in the first sentence of Section 3.1, our method employs behavior cloning or imitation learning. The use of reward and policy from the Markov Decision Process (MDP)[1] is a common practice in robotic learning and learning from expert demonstrations, and it does not imply that our method is based on Reinforcement Learning (RL). Additionally, as detailed in **Section 4.1.1, Settings and Metrics**, the reward function is utilized to assess the performance of the policy during the evaluation phases.

Reviewer comment (Reproducibility):
The paper did not explain how the model is trained. If it is an RL-based method, what are the observations and rewards, the algorithm used, and what hyperparameters are used? If not, what is the loss function, and how is it trained?

Author response:
As stated in section 4.3 (training and testing details), We use the expert action to
generate one hot map as the ground truth label. Cross-entropy loss is used to train the model end
to end. Each model is trained with the number of 40k SGD steps and we test the model every 10k
steps. We report the best performance for each model.

Reviewer comment (Presentation):
The language part and equivariant part are not directly related to equivariance itself. I think what the paper tries to claim is finding a better way to condition and ground constrained networks here. However, starting with language conditioning can mislead the readerss to focus on the wrong thing.

Author response:
The primary focus of our work is to investigate efficient representations for grounding language in language-conditioned grasping tasks. As such, we place a strong emphasis on the language-conditioned steerable kernel, namely the language grounding aspect. Key points include:

1. **Language-Conditioned Policy**: Our policy is language-conditioned. This means that for a given image, the action distribution changes according to the language goal.
2. **SE(2) Equivariance and Dynamic Steerable Kernel**: The SE(2) equivariance, as stated in Equation 2, is achieved through a dynamic steerable kernel derived from the pre-trained language embedding. The process of generating this steerable kernel from the language embedding is a learnable aspect of our model.
3. **Steerable Kernel Generation**: To clarify, unlike previous approaches that may rely on E2CNN[1], our network does not constrain itself to achieve equivariance through such means. Instead, we generate the steerable kernel via lifting, a process detailed and validated in Appendix A.2.3.

Our approach demonstrates a novel method of integrating language grounding with action planning in robotic tasks, leveraging the unique capabilities of dynamic steerable kernels.

Reviewer comment (Experiments):
No equivariant baselines that simply takes in CLIP features by concatenation. If previous equivariant grasp models are not applicable here, please explain clearly why concatenation doesn't work.

Author response:
Excellent question. There are several key points to consider regarding the integration of language features with visual features:

1. **Local Symmetry**: Merely attaching the same language feature to each pixel is not sufficient to achieve local symmetry. This approach overlooks the complexity of how language features interact with spatially varying visual features.
2. **Inconsistency**: The direct concatenation of the language feature (trivial representation which is no change if there is a rotation) with equivariant visual feature (commonly, a regular representation which permutes if there is a rotation on the image) is not consistent.
3. **Data Augmentation**: Our baseline model, FCGQ, employs the UNet architecture. In this model, the hidden visual feature is concatenated with the language feature in the bottleneck layer. Moreover, the model's training involves extensive data augmentation, aligning with the variation suggested by the reviewer. The results in Table 1&2 show that our method with steerable kernels outperforms the concatenation baseline by a large margin.

Reviewer comment (Experiments):
Do you have results for baselines in real-world experiments?

Author response:
No. Given the comparison as shown in the pick-part task in Table 2, baselines are already underperforming with limited data (1/5 demonstrations) in simulation. Given the fact the performance will definitely drop with noisy observation in real world, we believe it is redundant to include baselines like CLIPort, FCGQ ,and ViT in real-world experiments. We chose the pick-part task for our real-world experiments because it represents a challenging yet intriguing aspect of robotic grasping. This task requires the model not only to pick the object but also to differentiate specific parts of it. And, we provide three metrics and experiments with different numbers of demonstrations to show the effectiveness of our method. We believe, given the difficulty of the task, various metrics, and the unseen-object evaluation, our physical experiments are sufficient to demonstrate the effectiveness of the real-world applicability of our method.

Reviewer comment (Experiments):
I am wondering how would the performance change according to the number of grasp bins.

Author response:
According to the theory of antipodal grasping [2], the grasp/friction cone could tolerate a 15-degree offset for the friction factor of 0.3. In our settings, we use 5-degree discretization. Although the language-conditioned grasp requires higher precision than the general grasp, 36 bins with 5 degrees are enough. On the other hand, our model generates the continuous orientation distribution and could be sampled with any resolutions. However, more exploration in this direction is interesting but is not the focus of this paper.

Reviewer comment (Theory):
It is rather strange to see that rotation is applied to the language embedding, but not the image space...Need an explanation on this.

Author response:
Excellent question. The direct way to apply rotation to image space is to feed a stack of rotated images to the network. It is computationally expensive compared with the use of a set of rotated kernels or steerable kernels, especially when the number of rotations is large. The language-conditioned steerable kernel is more manageable and efficient. Theoretically, we can derive from the **commutative** property of the convolution action $a*b=b*a$, where a, b denotes two feature maps and $*$ denotes convolution. Further, with group action $g \in SE(2)$, $(g\cdot a)*b = a*(g\cdot b)$.

Reviewer comment (Theory):
Also, CLIP language features may not the best representation of actions such as "pick", "move" etc. Since the policy is grasping only it may not impose a bottlneck as of now.

Author response:
Good observation. We used CLIP because its text encoder is trained to align with the image distribution, which makes it a motivation for some recent visual-text tasks in robotics [1][7]. However, as mentioned in section 3.2.2, our proposed framework is suitable to fit any pre-trained language encoder or VLM. Finding a better VLM model suitable for robotics tasks could be a potential future work. Moreover, since in this work the only robot action is “grasping”, if the language-conditioned kernel can learn to extract the object type from the CLIP text embedding, it is sufficient to perform correct grasping. Our experiments successfully demonstrate such learning ability of our language-conditioned kernel.

Reviewer comment (Theory):
Need some discussions on why the whole policy, not just the steerable kernel, is also equivariant under the SE2 symmetric group.

Author response:
Good question. The rotation symmetry is realized by the languaged-conditioned dynamic steerable kernel. The translational symmetry is realized by the cross-correlation action, which means that convolution actions are naturally equivariant with respect to translation before adding the additional SO(2)-equivariant property. Detailed explanations can be found in Appendix A.2.2.

Reviewer comment (Experiments):
In the paper it is not clear if there is a pre-training phase. It is hard to believe that the network weights (of \phi, and psi) are properly learned with just one trajectory if they are always randomly initialized from the beginning.

Author response:
We do not engage in any pre-training. Our approach solely utilizes the pre-trained CLIP model. Specifically, for LEG-UNET, we utilize only the CLIP text encoder. In the case of LEG-Cliport, both the CLIP image encoder and text encoder are employed. Details regarding our training methodology and network specifications can be found in Section 4.3 and Appendix A.3, respectively.

For few-shot learning, equivariant networks are noted for their fast convergence and enhanced sample efficiency. This has been substantiated by multiple studies, as referenced in [1], [2], [3], [4], [5], and [6].

In our analysis, we focused on the symmetry inherent in the tasks and proposed a comprehensive framework that incorporates a dynamic steerable kernel. This framework is designed to exploit task symmetry, thereby achieving improved sample efficiency. We consider this ability to leverage symmetry as a significant strength of our work, rather than a limitation.

Reviewer comment (Novelty):
The mainstream of grasping research is in SE(3), but this paper remains in SE(2). This paper seems to have conducted implicit pose estimation or simply learned the rotation angles of objects. However, it appears to lack novelty.

Author response:
We respectfully disagreed with the reviewer that this is the weakness of our method.

1. The mainstream of SE(3) grasping does not indicate the lack of novelty of all SE(2) grasping currently. Language-conditioned grasping is being actively researched but still under-explored. As the other reviewers point out, there exist very recent publications such as [1][5] on SE(2) language-conditioned grasping. Most methods rely on segmentation [1, 2], cross-attention [5], or direct concatenation of language features and visual features [3, 4, 6]. To the best of our knowledge, our work is the first method that grounds language using language-condition steerable kernels, and we show its effectiveness compared with a strong baseline [6]. We are happy to address further concerns if the reviewer can kindly provide some related state-of-the-art work on language-conditioned grasping.
2. Our method, i.e., language-conditioned dynamic kernel, is straightforward to be extended to 3D to solve SE(3) grasping using 3D kernels, which is based on irreps of SO(3), i.e., Wigner-D matrices. This change is secondary compared to our approach on integrating language-conditioning.
3. To clarify, grasping is inherently a harder task compared to pose estimation. Previous work has demonstrated the complexity of the grasping problem [7, 8]. For example, a 6-Dof pose of a bowl does not necessarily provide information about where to grasp. And, object-level pose estimation methods cannot do our pick-part task.

Reviewer comment (Experiments):
Real-world experiments are limited to a single task (pick-by-part) and a small set of objects. Some of the experimental objects are not sufficient to demonstrate effectiveness, as they are mostly cylindrical in shape. As long as the object is between the grippers, closing the grippers will definitely be able to grasp it.

Author response:
First, we chose the pick-part task for our real-world experiments because it represents a challenging yet intriguing aspect of robotic grasping. This task requires the model not only to pick the object but also to differentiate specific parts of it. And, we provide three metrics and experiments with different numbers of demonstrations to show the effectiveness of our method. We believe, given the difficulty of the task and various evaluations, our physical experiments are sufficient to demonstrate the effectiveness of the real-world applicability of our method.

Second, we respectfully disagree with the cylindrical grasping argument. We have many objects that are not cylindrical. Take the bowl as an example; it is not in a cylindrical shape. The model needs to find the rim and reason the right angle of picking at a certain point on the rim. Meanwhile, simply grasping the estimated pose of the object will lead to failed grasping. 

Furthermore, beyond grasping, the model also needs to select the right object given language instruction. The model not only needs to pick the object but also needs to differentiate specific parts of it.

Reviewer comment (Experiments):
Could you explain the reason for adding language embedding in the φ branch? It seems to be redundant and does not have significant impact on performance.

Author response:
The main motivation is that we deliberately design the $\phi$ network as a vision-language module so that it provides flexibility for our framework to utilize any state-of-the-art VLMs. We provided visualizations in Figure 2 that $\phi$ network can highlight the region in the dense feature map from the observation. Also, empirically, $\phi$ network with language does not harm performance as shown in Table 4, meanwhile providing extra flexibility.

Another motivation is that using language in both $\phi$ and $\kappa$ potentially enables multi-round reasoning between vision and language features, possibly allowing more complicated instructions.

Besides, we have the ablation study in Table 4 to compare $\phi$ trained from scratch (LEG) and $\phi$ with an off-the-shelf pretrained VLM (LEG-Cliport). It shows that our method gains better performance by having the flexibility of using pre-trained VLM on $\phi$ .

Reviewer comment (Experiments):
The author fails to compare their method against some most recent work in language conditioned grasping. For example, this work [1] also has the task of semantic grasping (grasping some object by some particular parts).

Author response:
- We appreciate the suggestion and have updated the related work section to include a comprehensive comparison with all relevant studies known to us.
- We clarify that KITE[1] is a 3D policy network, primarily focusing on aspects other than grasping. The key pixel identification mechanism in KITE aligns closely with the attention module of Cliport[3], which serves as our primary baseline. The major difference between KITE and our Cliport-UNet is KITE's use of depth images projected into point clouds for learning in 3D space, whereas our Cliport-UNet decodes pixel features into 2D grasp orientations. Additionally, we provide some key differences between our method and KITE[1] as shown in **Table** R1.
- We acknowledge the recent work OCID-VLG/CROG[2], which shares a similar focus. We've thoroughly analyzed the differences between our method and theirs in the updated section. We also note that this is a very recent publication lacking available code. Moreover, the 4D grasp they refer to involves top-down grasping with gripper width prediction, differing from 6 DoF grasping in 3D space.

Reviewer comment (Experiments):
This paper only focuses on 2D grasping with language instructions but some objects may not be graspable from just a 2D top-down grasp. Some work [2] have also shown their method on 4D grasping in cluttered scenes with language instructions.

Author response:
- The critique about the limited variety of objects and variations in our real-world experiments compared to the simulations has been noted. We chose the pick-part task for our real-world experiments because it represents a challenging yet intriguing aspect of robotic grasping. This task requires the model not only to pick the object but also to differentiate specific parts of it.
- We evaluated performance using three metrics: grasping success rate, accuracy in selecting the correct object, and precision in grasping the intended part. The model was tested with varying numbers of demonstrations (1,5) and on unseen objects. The consistency of our real-world experiment's performance with the simulated pick-part experiments was established.
- Additionally, our manuscript's Tables 1 and 2 compare our method's performance with baselines across 13 tasks involving hundreds of objects. Thus, we believe our physical experiment sufficiently demonstrates the real-world applicability of our method.

Reviewer comment (Experiments):
In the real world experiment, the tested objects and variations are very limited compared to the simulation. It is more convincing to show more objects in the real world experiments and potentially show the results of the different methods, not just LEG-Unet, in the real world.

Author response:
- The critique about the limited variety of objects and variations in our real-world experiments compared to the simulations has been noted. We chose the pick-part task for our real-world experiments because it represents a challenging yet intriguing aspect of robotic grasping. This task requires the model not only to pick the object but also to differentiate specific parts of it.
- We evaluated performance using three metrics: grasping success rate, accuracy in selecting the correct object, and precision in grasping the intended part. The model was tested with varying numbers of demonstrations (1,5) and on unseen objects. The consistency of our real-world experiment's performance with the simulated pick-part experiments was established.
- Additionally, our manuscript's Tables 1 and 2 compare our method's performance with baselines across 13 tasks involving hundreds of objects. Thus, we believe our physical experiment sufficiently demonstrates the real-world applicability of our method.

Reviewer comment (Experiments):
The current language instructions used in the experiments are provided in the appendix. It is unclear if this method can generalize to other language instructions that have the similar meaning but don't exist in the training dataset or language instructions that are in free-form.

Author response:
- **Generalization Capability:** Addressing this important aspect, we provided results of LEG-UNETL trained with a specific language instruction and tested with unseen, similar language instructions. The findings, presented in our manuscript's Table, demonstrate a slight decrease but affirm the model's capability to interpret and respond to flexible language instructions. Further, we here provide an extended experiment as shown in Table R3, where we train the model with one fixed language template and test the model with unseen instructions, given 1/10 demonstrations. The results show that our model can generalize to *free-form* language instructions with similar performance as reported in our manuscript.
    
    
    | Table R2 | pick-parts | pick-random-v1 | pick-random-v2 |
    | --- | --- | --- | --- |
    | 1 demo | 88.25% | 88.6% | 53.4% |
    | 10 demos  | 95.25% | 94.6% | 85.6%  |

Reviewer comment (Evaluation):
For the experiment section, the authors mention that LEG-Unet is better than LEG-CLIPort on most tasks. The reviewer requests more explanation for this result.

Author response:
- **Performance Analysis:** We clarify that LEG-UNet, especially when trained from scratch, shows superior performance over LEG-Cliport on tasks that primarily involve shape differences. This could be attributed to two factors:
    1. The data preprocessing in LEG-Cliport to reshape images for feeding into the CLIP image encoder, which might impact the CLIP geometric features of each pixel.
    2. Training from scratch likely retains more of the original geometric information, providing an edge in performance.