Discovering Robotic Interaction Modes with Discrete Representation Learning
Authors: Liquan Wang
Ankit Goyal
Haoping Xu
Animesh Garg
Venue: CoRL 2024 (2024)
Keywords: Discovering Robotic Interaction Modes with Discrete Representation Learning
TL;DR: This paper presents ActAIM2, a system that unsupervisedly learns discrete robot manipulation modes representation, improving manipulability and generalizability with articulated objects.

Abstract: Abstract: Human actions manipulating articulated objects, such as opening and closing a drawer, can be categorized into multiple modalities we define as interaction modes. Traditional robot learning approaches lack discrete representations of these modes, which are crucial for empirical sampling and grounding. In this paper, we present ActAIM2, which learns a discrete representation of robot manipulation interaction modes in a purely unsupervised fashion, without the use of expert labels or simulator-based privileged information. Utilizing novel data collection methods involving simulator rollouts, ActAIM2 consists of an interaction mode selector and a low-level action predictor. The selector generates discrete representations of potential interaction modes with self-supervision, while the predictor outputs
corresponding action trajectories. Our method is validated through its success rate in manipulating articulated objects and its robustness in sampling meaningful actions from the discrete representation. Extensive experiments demonstrate ActAIM2’s effectiveness in enhancing manipulability and generalizability over baselines and ablation studies. For videos and additional results, see our website: https://actaim2.github.io/.

Discovering Robotic Interaction Modes with Discrete Representation Learning
Liquan Wang
Ankit Goyal
The Georgia Institute of Technology 1wang831 (are33/g@@ttchhheee
Nvidia
petoyal@@vidia.com
Haoping Xu
Animesh Garg
University of Toronto
The Georgia Institute of Technology animesh wwwggaateg....eee.c..
haoping xu@mail. wuudmmil..oorootooc.. utoronto.ca
Abstract: Human actions manipulating articulated objects, such as opening and closing a drawer, can categorized into multiple modalities we define as interac- tion modes. Traditional robot learning approaches lack discrete representations of these modes, which crucial for empirical sampling and grounding. In this paper, present ActAIM2, which learns discrete representation of robot manipulation interaction modes in purely unsupervised fashion, without the use of expert labels or simulator-based privileged information. Utilizing novel data collection methods involving simulator rollouts, ActAIM2 consists of an interaction mode selector and low-level action predictor. The selector generates discrete representations of potential interaction modes with self seff-supervision, while the predictor outputs corresponding action trajectories. Our method is validated through its success rate in manipulating articulated objects and its robustness in sampling meaning- ful actions from the discrete representation. Extensive experiments demonstrate ActAIM2's effectiveness in enhancing manipulability and generalizability over baselines and ablation studies. For videos and additional results, see our website: https://actaim2.. github.io/.
Keywords: robot manipulation, discrete representation learning, interac- tion mode, sulf-supervised
Introduction
Humans exhibit an exceptional aptitude for manipulating articulated objects by utilizing prior knowl- edge and learning through imitation. Generally, the outcome after manipulating the articulated objects categorical such as opening or closing the door. Motivated by this cognitive process, our study seeks to develop discrete representation of object affordance by merging behavior cloning with interaction mode identification. We focus on objects with multiple moving parts to explore a variety of distinct and meaningful outcomes, which we define as interaction modes. These interaction modes can be represented as a discrete set of options from which the agent can sample during inference to determine the appropriate interaction mode for the object. Notably, we characterize interaction mode as an affordance property of the object itself which can be learned from observation data, To transfer such discrete interaction modes into robotic action, an action predictor is learned from play data containing these diverse modes. Therefore, argue that the manipulation policy is joint distribution over both interaction mode selector and action predictor.
The robotics field is replete with studies addressing such policy decomposed into modes (or skills) and action distribution. However, most hehavior cloning approaches [1, 3, 4] require expert data for task representation supervision, especially using language description as the task representation such RLBench [5], Calvin [6], SayCan [7], etc. This assumes that the distribution of the interaction mode is known and can be represented language prompting. Other works learn the interaction mode 8th Conference Robot Learning (CoRL. 2024), Munich, Germany.

---

Figure 1: ActATM2 identifies mean- ingful interaction modes such as open and close drawers from RGB-D FEB-DDRe im- ages of articulated objects and robots. It represents these modes as discrete clusters of embeddings and trains policy to generate control actions for each cluster-based interaction.
Action Predictor
Selactor
Visual Input
Discrete Interaction Mode
Trajectory Inference
(skill) distribution using unsupervised reinforcement learning [8. 9, 10, 11] with self-supervised intrinsic reward weak supervision for task representation. Moreover, there are other interaction mode learning approaches [12, 13, 14] which learn distinguishable interaction mode (skill) prior. However, the distribution of interaction modes (skills) learned in these works does not specifically map to correspondent outcomes, indicating that the agent cannot sample skills with reasonable and limited options. Therefore, these works fail to find structural and disentangled space skill for an agent sample the interaction mode discretely based on observation. To solve the issues above, we require that the policy 1) captures various interaction modes without necessitating expert labels privileged information and 2) allows the agent to make discrete choices within this space, reflecting the finite interaction modes available in given scene.
We introduce ActAIM2, which splits the policy into a discrete mode selector using a Gaussian Mixture Model for discrete sampling and an action predictor that processes sampled task embeddings predict corresponding action sequences, trained via behavior cloning. Both components utilize self- supervised play data from dataset collected through adaptive data collection and heuristic grasping, ensuring balanced representation of interaction modes. Building upon ActAIM's method [12], our data collection does not use privileged information like reward functions or part segmentation and enhances realism by employing a complete robot instead of a floating gripper. To summarize, our contributions are threefold: First, we introduce ActAlM2, a self-supervised learning approach that enables discrete sampling across different interaction modes. Second, We devise a novel data collection methodology and have constructed a dataset with diverse interaction modes for model training. Finally, thoroughly evaluate model against spectrum of generative models and behavior cloning agents, demonstrating ActAIM2's superior performance over existing baselines and the capability of performing the discrete sampling.
2 Related Work
Articulated object manipulation The manipulation of articulated objects is challenging due to their complex geometries and kinematics, often understood through partially observed data like images. Common methods deduce the object's kinematic structure passive observation [15, 16, 17] interactive perception [18, 19]. These techniques are crucial for modeling articulated objects in robotics and incorporating these models into planning. While imitation learning relies on expert demonstrations [20, 21], it is limited by the need for extensive and costly data collection. Conversely. recent research explores obtaining actionable visual priors through direct interaction with objects [22, 23, 24], focusing on their geometric and semantic properties. Additionally, learning- based methods utilizing simulation supervised visual learning, and visual affordance learning are emerging [23, 24, 25). ActAIM2 addresses these challenges by demonstrating the stability of our model the experiments section.
Transformer-based policies Many recent works have proposed Transformer based policies for robotic control. For behavior cloning, Transformers model conditional action distributions [26, 3] and encode observations into latent vectors [2]. They also process language commands to specify and generalize tasks, demonstrating flexibility and robustness [27, 28, 29]. Additionally, Transformers utilized to learn latent 3D representations from scenes, building their success in computer vision

---

[30, 31, 32], and to process voxelized scenes from RGB images and point clouds [2, 33]. However, challenges remain, such as high memory demands for high-resolution voxelizations that slow training, and susceptibility to noisy data when re-rendering views from point clouds [3]. Transformers also map language and visual inputs directly to robot actions, ranging from basic attention layers to complex pre-trained models [27. 34]. ActAlM2 builds on this foundation using the RVT model but innovates by integrating novel discrete interaction mode representation into the skill language prompting.
Unsupervised Skill Learning Unsupervised skill discovery enables agents learn distinct behaviors without a reward function but often struggles with inadequate state space exploration when using variational Mutual Information maximization [35, 36]. This limitation hinders its effectiveness for complex tasks. Some strategies counteract this by focusing the learning smaller, Euclidean spaces to diversify movements [37], although this often restricts the learning to navigation and simple coordinate- based tasks. Alternative approaches propose auxiliary exploration mechanisms and novel training methods to enhance state space exploration [36, 38]. ActAIM2 addresses these issues by introducing the term "interaction mode," defined by significant visual changes from changes in the degrees of freedom (DoF) of an articulated object. By maximizing the mutual information loss and using contrastive evaluation of visual changes, ActAlM2 encourages the model identify discrete interaction modes and develop disentangled latent space for effective sampling.
3 Problem Formulation
The problem we are solving is how to use a parallel-jaw gripper robot to manipulate various articulated objects and generalize such skills among all types of articulated objects. We adopt a -wo-phase methodology to enhance a robot's ability to manipulate articulated objects: data collection and model training, inspired by [12]. We employ a structured predefined action primitive, executing series of actions across four heuristic phases (initiation, reaching, grasping, manipulation) to gather comprehensive dataset of observations O;, including RGBD images and multi-view camera positions, without relying on predefined inputs. Within each action phrase, we define the action as the keypose (p, R. q) of the parallel-jaw gripper similar to [5]. The key pose is defined as the concatenation of the 3 terms, which are p E R as the gripper position, R SO(3) as the gripper rotation quaternion, and q E {0, }} as the binary parameter indicating whether the gripper is open close.
Our model training aims to uncover the policy's 's distribution P(aaoo), with representing the obser- vation and = (p, R, q) the action, through a decomposition strategy that recontigures the action distribution as:
P(aao) = f P(alo.e)
P(clo de
action predictor mode selector
The mode selector P(cjo), contrasting with ActAIM [12]'s Gaussian space, utilizes a mixture of Gaussian distributions define distinet interaction modes, facilitated by discrete, latent space Z for enhanced action prediction and mode selection.
ActAIM2: Robotic Interaction Mode Discovery
We aim to derive the policy P(alo) from Equation 1, employing the robot as the agent within an environment populated by various articulated objects. Motivated by [12], our preliminary step was to gather offline, self-supervised data via simulation, which served the foundation for training the policy (Pllo)). Guided by Equation 1, we dissected the target policy into the action predictor P(a e) and the mode selector Pec|o). For training efficacy, the action predictor Paaaoo e) and mode selector ((elo) were pre-trained individually before collectively fine-tuning the overarching pipeline P(alo).
3

---

4.1 Iterative Data Collection
We collect trajectory data 7 = {(as, 0;)li 0,1,2,3); where O2 are RGBD observations from configuration five cameras encireling the articulated object, and a, = (p, R. q), represents the key pose and state of the gripper. Inspired by ActAIM [12], we use a similar GMM adaptive method collect diverse interaction modes. To collect the data with self-supervision, for each trajectory, characterized by the initial observation Ojmir On, and final observation Oj Offmml O3js utilize pre-trained image encoder &o to transform the image observations into latent vector v. The task embedding 2j for each trajectory T is defined as follows:
2j ofuit o onfinal Eo(O)rit) (O
(2)
To determine the success failure of manipulation, introduce threshold ε, defining a trajectory T as successful if 2j > 2. Details of the dataset are presented in Appendix 7.4.
4.2 Learning Interaction Modes with Discrete Representations (P((())
Learning Generative Model using GMM Prior Based on Equation I, we aim identify effective mode selector to generate relevant task latent space E &. Inspired by the concept of visual affordances in [39], we learn such latent space encapsulates all conceivable future states using the conditional Gaussian Mixture Variational Autoencoder (GMVAE) generative model.
To learn the mode selector as prior P(c|o), we use the data from our collected trajectory and select data (Orii , Ofimal) for mode selector training. Knowing that from Equation 2 contains the complete information of final states given initial observation, We pre-compute the ground-truth label as learning target and regard the initial observation Oänät as the conditional variable to generate the task embeddings. We aim to learn a generative model which predicts all possible task embeddings from the conditional initial observation O inst Taking inspiration from [40], we construct the generative model as a Transformer-base
Conditional GMVAE, as shown in Figure 2a. We defined our inference process as Oreit -> (c, y) > x + e where e a categorical variable with p(c) = Multi( and y is Gaussian distribution with p(y) (N I) which jointly forms Gaussian Mixture distribution the prior. We expect the output learned would have similar distribution the ground- -tuth task embedding since can be represented as a Mixture of Gaussian distribution fixing the initial observation (fixing the initial object and object state). Formally, we write the distribution of the variable under the following process; for simplification, we denote Oénit as O',
Pt.a(e, x, y, c(0)
(3)
K
Pe(zly,c,00) = II Nue, Ev, 1=1
(4)
pa(c 0+) = =Nss(( (") Ez(z, O'))
(5)
where K represents the number of mixture components, hyper-parameter within the training regime, and Por Ee M4. Eg denote networks to be trained where & and 3 are the parameters of these networks. In our implementation, Me, Ex,, are instantiated as multi-layer ResNet [41] architectures, and μs, Eg as a transformer with 4 self-attention layers to enhance the stability of reconstruction. To improve the training stability, we model the categorical distribution p(c) by the Gumbel-Softme distribution [42].
Mode Selector Training Loss -The conditional-GMVAB generative model optimized using the variational inference objective, combining reconstruction loss with the log-evidence lower bound (ELBO) loss. The ELBO loss expressed
2,,
LELBO = Ey
(6)
q(z, y. clc, O1)
where the proxy posterior g(z, y, clc, O*) is approximated qiz, clc II go. (zlee O*) O')qv, (yle, Oz. (dz,, O¹), with Uae By representing the parameters of the

---

Contrestie Difirrnne
E
R,
GAMM clustere Interaction
space
Mutliview Transformi forner
takedking
Action
Mas Ie
embeddinng
Pesttle
Exod
category
Taak anobedding
Conditonn tmage
Decader
Pear Closd Anotoctian
(b)
Figure (a) GMM Model Selector The mode selector, a generative model, processes the differences between the initial and final image visual embeddings as generated data, using the initial image embeddings as the conditional variable. (b) Behavior Cloning Action Predictor Interaction mode is sampled from latent space embedding from model selector. 5 Multiview RGBD observations from circled cameras are backeprojected and fused into color point cloud render novel views. Rendered image tokens and interaction mode token contacted and fed through a multiview transformer to predict action a = (p, R. q).
ly.. and dv. networks. Based on the decomposition, We refer the terms in the lower bound as the reconstruction term, conditional prior term, y-prior term and r-prior term respectively. The prior term for variables c, y, x is computed as the Kullback-Leibler (KL) divergence, which penalizes the difference between the learned latent variable distribution and the prior distribution. We formalize the reconstruction term as the 1.2 loss between task embedding predictions and ground-truth 2, written Creconstruct ||e - =||² in practice. Mathematical details of the conditional GMVAE presented in Appendix 7.5.1.
4.3 Supervised Action Predictor Learning E))
Our final objective is to infer a sequence of low-level actions a = (p, R, q) from the current observation O given the predicted task representation e, ensuring the action sequence effectively accomplishes the articulated object manipulation task while aligning the specific given interaction mode c. As shown in Figure 2b, The model inputs RGB-D images from encompass multi-view cameras, the present state of the robot gripper, and the task latent embedding From the data collection phase, we collect successful manipulation trajectories Tj. We decomposed T to individual key-frame actions and observations (a;, Oi), E Tý E D as data points for training. To this end, we propose a multi-view transformer architecture as the basis for our behaviour cloning agent, aimed at learning the action distribution e).
Novel View Rendering and Multiview Transformer- Building upon the RVT [3] approach, we utilize novel view rendering from RGB-D multi-view cameras as our visual ohservation, strategically positioning five cameras around the robot and articulated objects to generate a merged RGB point cloud. This cloud is normalized the scene center and projected onto orthogonal image planes, creating novel views from the top, front, behind, left, and right, each incorporating RGB color, XYZ position, and depth channels as model inputs. These rendered views processed by multiview transformer model, where images are patchified and encoded with MLPs and positional encoding, similar to the ViT process. Additionally, the task embedding and the current state of the gripper are encoded and their features are concatenated with image token representations. The transformer then processes the merged tokens, outputting per-view 2D heatmaps and global features which predict the R)
current state action
Action Predictor Training Loss We optimize our action predictor model utilizing a behaviour cloning loss framework. For the position heatmap, cross cross-entropy loss is employed, with the ground- truth image being synthesized from the 3D ground truth point p through projection onto 2D orthogonal view, following a Gaussian distribution for spatial smoothing. Similarly, the rotation heatmap is refined using cross-entropy loss for each Euler angle axis, translating the ground-truth rotation R into an analogous one-hol hot vector representation. The binary classification loss, essentially
5

---

Top Predicted Hestmup
Execution Trajectory
Figure 3: Given different task em- bedding,
Opening
Closing
Opening
Closing
how action predie- tor produces actions representing dis- tinet interaction modes. Here, we vi- sualize the camera view and the pre- diction heatmap from the top for ob- ject instances. The first heatmaps for pushing and pulling handle, while the second row shows heatmaps for closing the left or right door. More qualitative results please the appendix 7.6
shows
Clasing Left
Clasing Right
Cosing Left
Closing Right
a cross-cntropy loss, updates the gripper jaw's open-close state q. Accordingly, the comprehensive training loss for the action predictor articulated as:
Laction
hp +
Ln La
CF(qq (.)
4.4 Training Procedure
In our training process, we jointly train the policy P(alo) as formulation:
P(alo) P(alo, sg(c)) P(clo)
(8)
In this context, sg[:] stands for the stop-gradient operator, which stops the flow partial derivatives to the next network layers. During the training phase, include the combined total loss from both the action predictor (behavior cloning loss) and the mode selector (ELBO loss), represented as Lioral =
Luction + LELBO.
5 Experiments
Our experimental setup is designed to assess how well the proposed method, ActAIM2, performs several important areas: 1) ActAIM2 proficiently handles various interaction modes, adapting to differences across object instances and categories. 2) Utilizing Gaussian Mixture Model (GMM) based priors facilitates more structured latent space, enabling targeted searches for specific samples within the interaction modes. To evaluate ActAIM2, report the average success rate, sample success rate, and the average reward achieved during interaction mode grounding iterations.
5.1 Experimental setup
Leveraging the success of previous studies [43, 2, 12], we use articulated objects from the SAPIEN dataset [44] for our experiments. Our training dataset includes six categories: faucets, tables, storage furniture, doors, refrigerators, and switches, while the testing dataset features three new categories: windows, boxes, and safes, with each category comprising unique objects. We employ IsaacGym our simulation platform [45], where have designed custom environment featuring Franka Emika robot adjacent to articulated object with 5 cameras circling the object. In evaluations, we define successful trajectory follows: the model takes initial observation and iteratively predicts a series of actions, executed across four steps. After these steps, we evaluate the object's 's state. the Degrees of Freedom (DoF) of any part of the object has changed by more than 30%, we deem the trajectory successful. This approach allows us to assess the efficacy the robot's manipulation capabilities in dynamically altering the object's state based on the model's 's iterative predictions.
Baselines And Ablation Study
-We compare our results with the following baselines and ablation study. Data Collection: Adaptive data collection using a GMM-based heuristic grasping method to sample action sequences. Where2Act [43]: Calculates priors for discretized action primitives, using the complete object point cloud instead of segmented movable points. ActAIM [12]: Combines Conditional Variational Autoencoder (CVAE) [46] with transformer-base action predictor [47] to sample and forecast manipulation trajectories, employing unrealistically simplified floating gripper.

---

Table I: Robotic Interaction Mode Discovery: We evaluate our design decisions through baseline comparisons and ablation studies using the sample-sueces rate (SSR) metric. We find that that ActAlM2 consistently surpasses competing models across various object types.
Test Set
Seen Objects
Unseen instances
Unseen Cats
SSR 7
F.
AVG
F.
AVG
[
B
AVG
Data Collection
12.8
8,9
9.4
16.9
10.4
8.9
11.2
11.4
9.5
14.0
13.2
11.9
12.9
12.2
20.4
17.3
15.0
17.6
Where2Act [43]
33.3
7.0
7.0
17.9
12.1
4.1
13.6
33.0
13.8
19.2
16.9
1399
15.4
18.7
15.0
16.8
15.2
15.7
ActAIM [12]
49.3
41.4
36.2
28.6
24.5
19.7
33.3
22.0
38
35.5
21.0
18.2
16.2
25.2
38.4
24.1
31.8
31.5
Goal-RVT
58.4 48.9
51.2
72.1
23.2
33.3
47.8
40.2
43.4
39.2
65.1
18.3
25.3
38.6
32.3
23.1
33.9
29.8
AcAAMM
65.3 43.2 52.1
69.2
25.3
36.2
48.6
44.9
41.2
41.5
60.2
20.1
24.4
38.7
34.3
28.9
34,1
32.4
Table 2: Mode Sampling Evaluation ActAIM2 executes actions sampled uniformly across 8 clusters, while Goal-ActAIM and Goal-RVT use manually selected goal images as expert labels for modc-specific sampling. The best performance is highlighted in bold, underscoring ActAlM2's 's consistent improvement across evaluations.
Seen Ohjects
Unseen instunces
Unseen
Algorithm
Mode SSR %
F:
AVG
F:
AVG
G
AVG
Aoal-A
common mode
25,3 37.5
19.3
62.9
24.3
61.
38.4
20.3
36.3
18.2
43.0
21.3
31.4
28.4
29.3
15.2
43.5
29.3
ture
16.2
11.3
28.4
13.1
10.8
37.5
20.4
Goal-RVT
1st mode goal
42,4
57.4
76.5
75.4
44.9
65.4
60.3
40.2
48.8
72.4
74.3
30.2
50,3
52.7
35.5
35.5
49.2
40.1
2nd mode goal
17.2 28.3
31.2
70,4
20.1
34.2
33.6
24.3
20.4
15.2
64.2
10.1
20.3
25.8
10.4
5.1
5.1
6.9
VOVAF-RVT
1st mode voctor
56.3
61.3
45.6
39.3
29.1
30.2
39.4
36.4
32.5
2nd mode vector 0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
AdATM2
1st mode cluster
95.4
45.3 83.4 80.4
58.8
65.4
71.5
70.4
45.4
72.4
78.5
40.3
58.9
61.4
43.8
34.9
65.8
48.2
2nd made cluster
15.4
24.5
20.4 60.4
10.2
31.2
27.0
10.1
15.8
15.2
58.3
5.0
20.3
20.9
10.4
5.1
5.1
6.9
Goal-RVT: A supervised version of ActAIM2 that uses directly provided goal images, bypassing GMM prior sampling, to feed into the action predictor, demonstrating comparable performance to other supervised methods. VQVAE-RVT: Inspired by Genie [48], replacing the interaction mode selector with a VQVAE codebook, using a codebook size equal to the GMM cluster number in ActAIM2, and sampling task embeddings discretely during tests.
Evaluation Metrics We conduct two types of evaluations to assess the ability of models for suc- cessful interaction and discrete sampling, testing across three object categories: Seen Objects (from the training set), Unseen Instances (same categories as Seen but different instances), and Unseen Categories (from outside the training set). Interaction Mode Discovery metric assesses the average sample success rate (SSR) defined as the ratio of successful trajectory samples total samples. For Goal-RVT, expert-labeled goal images are used to ensure viable trajectories, whereas ActAIM2 samples from all 8 clusters to calculate SSR. Moreover, we measure SSR for dominant interaction modes within the selected 2 clusters as Mode Sampling Evaluation. This indicates that the evalu- ated method is required to have at least 2 interaction modes discovered and distributed in separate sample options. For ActAIM2 and VQVAE-RVT, we sample and execute actions from 8 distinet clusters, documenting the success rate of the primary interaction mode in the first and second clusters. Goal-RVT and Goal-AetAIM follow a similar approach but use ground truth goal images instead of sampled clusters, adjusting for changes in DoF.
5.2 Discussion of Results
Interaction Mode Discovery Table I shows that ActAIM2 often outperforms baselines in interaction success rates for both trained and unseen instances. Where2Act and ActAIM use simplified scenario with heuristic floating gripper, whereas ActAIM2 utilizes a complete Franka Emika robot setup but still performs a higher success rate. This means ActAIM2 performs well even when it has to discard trajectories due to unreachable states or collisions. This highlights its ability to accurately predict gripper positions and efficiently sample from a learned latent space Z, surpassing the performance of Goal-RVT's broader goal image approach. We did not report the VQVAE-RVT results here since VQVAE-RVT outperforms around 5% in each test compared to ActAIM2 averagely. However, despite the average sample success rate, show that VQVAE-RVT does not meet our requirement for discrete sampling in the following two experiments.
7

---

Open
Close
Initial State
Figure 4: The figure illustrates the drawer manipulation task conducted the Kinova Gen2 robot arm. The task involves interacting with three-drawer shelf, starting from an initial half-open state (center). The robot executes two modes: opening (left) and closing (right) the drawers, with arrows showing the gripper's movement direction during each interaction.
Mode Sampling Evaluation ActAIM2 and Goal-RVTT were evaluated for their ability distinguish between interaction modes. ActAIM2 proved stable and successfully identified the primary interaction mode across various categories and conditions, matching Goal-RVT's performance even for secondary mode. Larger objects like doors and refrigerators consistently showed successful interaction trajectories, demonstrating ActAIM2's adaptability to different object sizes and complexities. In contrast, VQVAE-RVT struggled to offer a viable second dominant interaction mode. Our tests showed that VQVAE-RVT tends to produce similar action heatmaps across different code vectors. complicating the identification of specific interaction modes. These issues are further explored in our reinforcement learning experiments interaction mode grounding.
5.3 Real Robot Experiments
We evaluate ActAIM2 in real-world setup using visual data. A Kinova Gen2 robot arm interacts with a small shelf featuring three drawers, performing opening and closing actions on half-open drawers (see Figure 4). Azure Kinect RGB-D camera captures the scene, and after calibrating the camera extrinsic, we process point clouds with ActAIM2. The robot's gripper grasps the drawer edge pull (open) or push (close) the drawers. We set three initial states with the top, middle, or bottom drawer half-open, introducing task variation by moving the shelf's position and randomly selecting grasp points. ActAIM2 is pre-trained on simulated data and fine-tuned with 60 real-world trajectories collected via expert control using four action primitives. After fine-tuning, ActAIM2 achieved a 75% success rate for both pushing and pulling tasks, accurately classifying interaction modes. Sampling from the clusters representing interaction modes, specific clusters had success rates over 87.5%, mirroring the mode sampling evaluation and demonstrating the benefit of incorporating human demonstration data. Some failures occurred due to the gripper missing the drawer edge when the object's position shifted slightly, suggesting the need for more detailed tuning and increased variation in training to improve object positioning accuracy. For more detailed information the real robot experiments, please refer to Section 7.1 and our website.
6 Conclusion
ActAIM2 marks major advancement in self-supervised learning for robotic control, enabling robust discrete sampling across diverse interaction modes. Our specialized data collection and dataset lay a solid foundation for future enhancements. Extensive comparisons show ActAIM2's superior performance over existing models, enhancing discrete interaction mode learning and strengthening self-supervised discovery techniques for practical applications.
Limitations. ActAIM2 offers promising self- supervieedd method for learning interaction modes, but balancing success rates across modes remains challenging, especially for less common inter- actions. Iterative data collection targeting these modes could improve performance. Additionally, ActAIM2 assumes interactions are predefined simple action sequences, which limits its application to complex multi-stage manipulations. For a detailed discussion on assumptions and future work, see Appendix 7.3.

---

Acknowledgments
We would like express our sincere gratitude Professors Humphrey Shi and Sonia Chernova from the Georgia Institute of Technology for providing the resources that made this research possible. Their generous support, through access to computation resources and necessary hardware, was invaluable to the success of this project. We also extend our thanks to the Vector Institute for providing computation resources. Additionally, we thank the members of PairLab for their helpful discussions and feedback, which contributed to shaping the direction of this research.
References
[1] Z. J. Cui, Y. Wang, N. M. M. Shafiullah, and L. Pinto. From play to policy: Conditional behavior generation from uncurated robot data. In I Ith International Conference on Learning Representations (ICLR), 2023.
[2] M. Shridhar, L.. Manuelli, and D. Fox. Perceiver-actor: A multi-task transformer for robotic manipulation. In 6th Annual Conference on Robot Learning (CoRL), 2022.
[3] A. Goyal, J. Xu, Y. Guo, V. Blukis, Y.-W. Chao, and D. Fox. Rvt: Robotic view transformer for 3d object manipulation. CoRL, 2023.
[4] Gervet, Z. Xian, N. Gkanatsios, and K. Fragkiadaki. Act3d: feature field transformers for multi-task robotic manipulation, 2023.
[5] S. James, Z. Ma, D. R. Arrojo, and A. J. Davison. Rlbench: The robot learning benchmark and learning environment. In IEEE Robotics and Automation Letters (RA-L), 2019.
[6] O. Mees, L. Hermann, E. Rosete-Beat Beas, and W. Burgard. Calvin: A benchmark for language- conditioned policy learning for long-horizon robot manipulation tasks. IEEE Robotics and Automation Letlers (RA-L), 7(3):7327-7344, 2022.
[7] M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Fino, C. Fu, K. Gopalakr- ishnan, K. Hausman, A. Herzog. D. Ho, J. Hsu, J. Ibarz, B. Ichter, A. Irpan, E. Jang, R. J. Ruano, K. Jeffrey, S. Jesmonth, N. Joshi, R. Julian, D. Kalashnikov, Y. Kuang, K.-H. Lee, S. Levine, Lu, L. Luu, C. Parada, Pastor, Quiambao, Rao, J. Rettinghouse, D. Reyes, P. Sermanet, N. Sievers, C. Tan, A. Toshev, V. Vanhoucke, F. Xia, T. Xiao, P. Xu, S. Xu, M. Yan, and A. Zeng. Do can and not say: Grounding language in robotic affordances. In arXiv preprint MoXiv:200.0019// 2022.
[8] M. Laskin, D. Yarats, H. Liu, K. Lee, A. Zhan, K. Lu, C. Cang, L. Pinto, and P. Abbeel. URLB: Unsupervised reinforcement learning benchmark. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021. URL
[9] B. Eysenbach, A. Gupta, J. Ibarz, and S. Levine. Diversity is all you need: Learning skills without reward function, 2018.
[10] A. Srinivas, M. Laskin, and P. Abbeel. Curl: Contrastive unsupervised representations for reinforcement learning, 2020.
[11] M. Laskin, H. Liu, X. B. Peng, D. Yarats. A. Rajeswaran, and P. Abbeel. Cic: Contrastive intrinsic control for unsupervised skill discovery, 2022.
[12] L. Wang, N. Dvornik, R. Dubeau, M. Mittal, and A. Garg. Self-supervised learning of action affordances as interaction modes. arXiv preprint arXiv:2305. /7565, 2023.
[13] N. M. M. Shafiullah, J. Cui, A. Altanzaya, and L. Pinto. Behavior transformers: Cloning modes with stone, 2022.

---

[14] S. Lee. Y. Wang, H. Etukuru, H. J. Kim, N. M. M. Shafiullah, and L. Pinto. Behavior generation with latent actions, 2024.
[15] X. Li, H. Wang, L. Yi, L. J. Guibas, A. L. Abbott, and S. Song. Category-level articulated object pose estimation. In 2020 IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3703-3712, 2020. doi: 100::001100//CPP22000002222 20220000770
[16] B. Abbatematteo, S. Tellex, and G. Konidaris. Learning generalize kinematic models to novel objects. In L. P. Kaelbling, D. Kragic, and K. Sugiura, editors, Proceedings of the Conference on Robot Learning, volume 100 of Proceedings of Machine Learning Research, pages 1289-1299. PMLR, 30 Oct-Ol Nov 2020. URL v100/ abbatemattee22aa. html.
mlr .ress//
[17] A. Jain, R. Lioutikov, C. Chuck, and S. Niekum. Screwnet: Category-independee articulation model estimation from depth images using screw theory. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 13670-13677, 2021. doi: 109/1CR
95611 132.
[18) R. Martin-Martin S. Höfer, and O. Broek. An integrated approach to visual perception of articulated objects. In 2016 IEEE International Conference on Robotics and Automation (ICRA), pages 5091-5097, 2016.
[19] K. Hausman, S. Niekum, S. Osentoski, and G. S. SuKhhatme Active articulation model estimation through interactive perception. In 2015 IEEE International Conference on Robotics and Automation (ICRA), pages 3305-3312, 2015. 100.000009/CCAA211......1
[20] T. Welschehold, C. Dornhege, and W. Burgard. Learning mobile manipulation actions from human demonstrations. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3196-3201, 2017. doi: 10. (00//ROS.2017..800
[21] H. Xiong, Q. Li, Y.-C. Chen, H. Bharadhwaj, S. Sinha, and A. Garg. Learning by watching: Physical imitation of manipulation skills from human videos. 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7827-7834, 2021. URL https:
/ //api. www.stemmaniiii....m semanticscholarr cscholar
Peg/CorpussDD :t:pppus:::311355777 231632575
[22] R. Wu, Y. Zhao, K. Mo, Z. Guo, Y. Wang, T. Wu, Q. Fan, X. Chen, L. Guibas, and H. Dong. VAT-mart: Learning visual action trajectory proposals for manipulating 3d ARTiculated objects. In International Conference on Learning Representations, 2022. URL https: //pprr/vee.e........ htttfforumidd=ixxPPP
[23] K. Mo, L. Guibas, M. Mukadam, A. Gupta, and S. Tulsiani. Where2act: From pixels to actions for articulated 3d objects. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 7803, 2021. (bb:10.110//CCC4489222 2021.00674.
[24] Z. Xu, Z. He, and S. Song. Universal manipulation policy network for articulated objects. IEEE Robotics and Automation Letters, ((2):244722455) 2022. doi: 1 109/L19 2022.313222...11223999
[25] T. Mu, Z. Ling, F. Xiang, D. C. Yang. Li, S. Tao, Z. Huang, Z. Jia, and H. Su. Maniskill: Generalizable manipulation skill benchmark with large-scale demonstrations. In Thirty- Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). 2021.
[26] N. M. M. Shafiullah, J. Cui, A. Altanzaya, and L. Pinto. Behavior transformers: Cloning k modes with one stone. In Thirty-Sixth Conference Neural Information Processing Systems (NeurlPS), 2022. URL
[27] M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakr- ishnan, K. Hausman, et al. Do as can, not as say: Grounding language in robotic affordances. 2022.
arXiv preprint
10

---

[28] Nair, E. Mitchell, K. Chen, b. ichter, S. Savarese, and C. Finn. Learning language-conditione robot behavior from offline data and crowd-sourced annotation. In A. Faust, D. Hsu, and G. Neumann, editors, Proceedings of the 5th Conference on Robot Learning, volume 164 of Proceedings of Machine Learning Rexearch, pages 1303-1315. PMLR, 08-11 1 Nov 2022. URL mlr press//v66/nnai222 html.
[29] Y. Zhang and J. Chai. Hierarchical task learning from language instructions with unified transformers and self-monitoring. In C. Zong, F. Xia, W. Li, and R. Navigli, editors, Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4202-42114 Online, Aug. 2021. Association for Computational Linguistics. doi: 10.:10..18.0 18653/ 1/202 .iwdinngi-ncl.... URL https: / /a//accaannhooooo
2021. acldnggsacl.... 368.
[30] E. Jang, A. Irpan, M. Khansari, D. Kappler, F. Ebert, C. Lynch, S. Levine, and C. Finn. Bc-z: Zero-shot task generalization with robotic imitation learning. In Conference on Robot Learning. pages 991-1002. PMLR, 2022.
[31] A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, J. Ibarz, B. Ichter, A. Irpan, T. Jackson, Jesmonth, N. Joshi, R. C. Julian, D. Kalashnikov, Y. Kuang. I. Leal, K.-H. Lee, Levine, Y. Lu, U. Malla, D. Manjunath, I. Mordatch, O. Nachum, Parada, J. Peralta, Perez, Pertsch, J. Quiambao, K. Rao, M. Ryoo, G. Salazar, P. R. Sanketi, K. Sayed, J. Singh, S. A. Sontakke, A. Stone, C. Tan, H. Tran, V. Vanhoucke, S. Vega, Q. H. Vuong, F. Xia, T. Xiao, P. Xu, S. Xu, T. Yu, and B. Zitkovich. Rt-1: Robotics transformer for real-world control scale. ArXiv, abs/2212.068177 2022. URL semanticscholar lr owwoooog//oo/eeeoeooooo :0:rpusDD:2545912250
[32] Y. R. Wang, Y. Zhao, H. Xu, S. Eppel, A. Aspuru-Guzik, F. Shkurti, and A. Garg. Mvtrans: Multi-view perception of transparent objects. In International Conference on Robotics and Automation (ICRA) 2023, 2023.
[33] Y. 7e, G. Yan, Y.-H. Wu, A. Macaluso, Y. Ge, J. Ye, N. Hansen, L. E. Li, and X. Wang. Gnfactor: Multi-task real robot learning with generalizable neural feature fields. In CoRL2023 Oral, 2023.
[34] A. Brohan, N. Brown, J. Carbajal, Chebotar, X. Chen, K. Choromanski, Ding, D. Driess, A. Dubey, C. Finn, P. Florence, C. Fu, M. G. Arenas, K. Gopalakrishnan, K. Han, K. Hausman, A. Herzog, Hsu, B. Ichter, A. Irpan, N. Joshi, R. Julian, D. Kalashnikov, Y. Kuang, Leal, L. Lee, T.-W. E. Lee, Levine, Lu, H. Michalewski, Mordatch, K. Pertsch, K. Rao, K. Reymann, M. Ryoo, G. Salazar, P. Sanketi, P. Sermanet, J. Singh, A. Singh, R. Soricut, H. Tran, V. Vanhoucke, Q. Vuong, A. Wahid, S. Welker, P. Wohlhart, J. Wu, F. Xia, T. Xiao, P. Xu, S. Xu, T. Yu, and B. Zitkovich. Rt-2: Vision-language-action models transfer web knowledge to robotic control, 2023.
[35] M. Laskin, D. Yarats, H. Liu, K. Lee, A. Zhan, K. Lu, C. Cang, L. Pinto, and P. Abbeel. URLB: unsupervised reinforcement learning benchmark. CoRR, abs/2110.15199, 2021. URL org/ /2100. 15191.
[36] D. Strouse, K. Baumli, D. Warde-Farley, V. Mnih, and S. Hansen. Learning more skills through optimistic exploration. CORR, abs/2107. 14226, 2021. URL htttt://///rxii http://axxxvvoo/abb///// org/abs/217. 14226.
[37] Park, Choi, Kim, Lee, and G. Kim. Lipschitz-conssrraned unsupervised skill discovery. In International Conference on Learning Representations, 2022. URL https: // //openreview
[38] Z. Jiang, J. Gao, and Chen. Unsupervised skill discovery via recurrent skill training. In A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, editors, Advances in Neural Information Processing Systems, 2022. URL httpss//openreveee wtwp://wwww..tetttooooo//rd/aaaa DxNNjh.

---

[39] M. Hassanin, S. Khan, and M. Tahtali. Visual affordance and function understanding: A survey. ACM Comput. Surv., 54(3), apr 2021. ISSN 0360-0300. https://doo org/10. 10. 1145/3446370.
URL
[40] J. Bai, S. Kong, and C. P. Gomes. Gaussian mixture variational autoencoder with contrastive learning for multi-label classification. In International Conference on Machine Learning, pages 1388.1398. PMLR, 2022.
[41] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Puttern Recognition (CVPR), pages 770-778, 2016. doi: 10..119//VPR..206......
[42] I. A. M. Huijben, W. Kool, M. B. Paulus, and R. J. G. van Sloun. A review of the gumbel-mas trick and its extensions for discrete stochasticity in machine learning, 2022.
[43] K. Mo, L. J. Guibas, M. Mukadam, A. Gupta, and S. Tulsiani. Where2act: From pixels to actions for articulated 3d objects. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 681 1813--22 13-6823, October 2021.
[44] F. Xiang, Y. Qin, K. Mo, Y. Xia, H. Zhu, F. Liu, M. Liu, H. Jiang, Y. Yuan, H. Wang, L. Yi, A. X. Chang, L. J. Guibas, and H. Su. SAPIEN: A simulated part-based interactive environment. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
[45] J. Liang, V. Makoviychuk, A. Handa, N. Chentanez, M. Macklin, and D. Fox. Gpu-accelerated robotic simulation for distributed reinforcement learning, 2018.
[46] K. Sohn, H. Lee, and X. Yan. Learning structured output representation using deep condi- tional generative models. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015. URL 8i552966baa3667722755002221
neuriiis httpp..cc.ppoooo cemeer_files/paaaae htt.fille//pppr/21/////11. pdf.
[47] A. Jaegle, S. Borgeaud, J.-B. Alayrac, C. Doersch, C. Ionescu, D. Ding, S. Koppula, D. Zoran, A. Brock, F. Shelhamer, et al. Perceiver io: A general architecture for structured inputs & outputs. 2021.
[48] J. Bruce, M. Dennis, A. Edwards, J. Porker-Holder, Y. Shi, E. Hughes, M. Lai, A. Mavalankar, R. Steigerwald, C. Apps, Y. Aytar, S. Bechtle, F. Behbahani, S. Chan, N. Heess, L. Gonzalez, S. Osindero, S. Ozair, S. Reed, J. Zhang, K. Zolna, J. Clune. N. de Freitas, Singh, and Rocktäschel. Genie: Generative interactive environments, 2024.
[49] P. Auer, N. Cesa-Bianchi, and P. Fischer. Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47(2): 2002.
[50] R. Y. Rubinstein. The cross-entropy method for combinatorial and continuous optimization. Methodology and computing in applied probability, 122):127- 1999.
[51] S. James and A. J. Davison. Q-attention: Enabling efficient learning for uision-based robotic manipulation. In IEEE Robotics and Automation Letters (RA-L), 2022.
[52] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. pages 1-14. Computational and Biological Learning Society, 2015.
[53] M. Ester, H.-P. Kriegel, Sander, and X. Xu. A density- based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, KDD'96, 96, page 226-231. AAAI Press, 1996.
[54] H.-S. Fang, C. Wang, M. Gou, and C. Lu. Graspnet-lbiliin: A large-scale benchmark for general object grasping. In 2020 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 11441-11450 1000 100010999CVR422000200000000 146.
12

---

[55] M. Sundermeyer, A. Mousavian, R. Triebel, and D. Fox. Contact graspnet: Efficient 6-dof grasp generation in cluttered scenes. International Conference on Robotics and Automation (ICRA) 2021, 2021.
[56] M. Ahn, M. G. Arenas, M. Bennice, N. Brown, C. Chan, B. David, A. Francis, G. Gonzalez, R. Hessmer, T. Jackson, N. J. Joshi, D. Lam, T.-W. E. Lee, A. Luong, S. Maddineni, H. Patel,
J. Peralta, J. Quiambao, D. Reyes, R. M. J. Ruano, D. Sadigh, P. Sanketi, L. Takayama, Vodenski, and F. Xia. Vader: Visual affordance detection and error recovery for multi robot human collaboration, 2024. URL https: /ttp://aavvvvv.. org
16021.
[57] Y. Geng, B. An, H. Geng. Y. Chen, Y. Yang. and H. Dong. End-to-end affordance learning for robotic manipulation, 2022. URL https:///aaaiiiii wttp 2tt.b///2.999.4499
13

---

(b)
Figure 5: (a) Real World Setup: In the image, we demonstrate the real-world experiment setup using single RGB-D camera (Azure Kinect) to capture visual information and Kinova robot equipped with parallel jaw for interacting with an articulated shelf object. (b) Shelf Object Point Cloud Illustration: We present the point cloud of the shelf object with three movable drawers, extracted from the RGB-D D camera.
7
Supplementary
7.1 Real Robot experiments
In this section, we evaluate ActAIM2 using real visual sensory data by training and testing the model in real-world setup. For further insights into the model's performance, we have included attached videos, which are also available on our website.
Real World Setup- We employ a Kinova Gen2 robot arm with a spherical wrist, 7 degrees freedom, and parallel jaw gripper (Robot Type: j2s7s300), placed on a table alongside the articulated object shown in Figure 5a. The scene captured using statically mounted Azure Kinect (RGB-D) camera in third-person view. We first calibrate the robot-camera extrinsics and transform the perceived color point clouds to the robot's base frame before processing them with ActA1M2. During evaluation, based on the target pose predicted by ActAIM2, we utilize the Movelt motion planner to direct the robot to the target pose, incorporating trajectory generation and feedback control.
Tasks We focus manipulating a small shelf with three drawers showed in Figure 5b, defining two interaction modes for each half-open drawer: opening and closing. For the opening interaction, the gripper manually grasps the edge of the drawer and pulls it out. Conversely, for closing, the gripper grasps the edge and pushes the drawer in. We utilize a small shelf with three drawers as the articulated object and set three plausible initial states: the top, middle, and bottom drawers are each set to half-open. Task variation introduced by altering the position of the small shelf on the table.
Data Collection and Training Our evaluation focuses on a single task: opening and closing a half-open drawer on a small shelf. Initially, ActAIM2 is pre-trained using domain randomization on our simulated dataset. We currently employ ActAIM2 trained on the single-vier dataset referenced in Table 4. For expert demonstration, we collect real-world interaction data via expert control to either open close the drawer. The robot control strictly adheres the simulator setup, utilizing four predefined action primitives: initializing, reaching, grasping, and interacting. To introduce variation in the training trajectories, each expert-controlled interaction randomly selects different parts of the drawer for the robot to grasp. Throughout these interactions, the articulated object remains stationary. Simultancously, RGB-D images are captured as the robot moves to the target pose, creating a dataset of RGB-D frames paired with target pose annotations. For each initial state (top. middle, and bottom drawers half-open), we collect 10 expert data sets for each mode, totaling 60 trajectories (10 pushing and 10 pulling trajectories for each state).
Results - We initially trained ActAIM2 on simulated data and later fine-tuned it using real-world trajectories. This process yielded notable success rate of 75% for both the pushing and pulling tasks involved in drawer interactions. Importantly, ActAIM2 accurately classifies the interaction modes of pushing and pulling. demonstrating well-aligned discrete representation. To assess the performance,
14

---

Object Ground-iruth Datribution from ActAMM2
Reinforcement Learniig
Opening
0.33
State
Closing
(b)
Figure 6: (a) Visualization of Reinforcement Learning on ActAIM2: ActAIM2's grounding through reinforcement learning is demonstrated, highlighting successful openings (yellow). closings (blue), and failures (red) across eight interaction clusters. After 50 iterations, clusters accurately identified, enabling consistent trajectory generation.
(b) Reward Optimizing Plot:
This plot displays reward optimization over iterations (x-axis) and average reward (y-axis, indicative of success rate for the targeted mode). ActAlM2's structured latent space shows clear advantages in sampling and convergence efficiency compared to ActAIM [12] and VQVAE-RVT.
we sampled from each of the 8 clusters designed to represent distinct interaction modes either opening or closing the drawer. Clusters specific to pushing and pulling were found have success rates exceeding 87.5% mirroring the mode sampling evaluation. This improvement is largely due to incorporating human demonstration data, which helps correct any imbalance in interaction mode representation during training.
However, in our real robot experiments, we observed some failure cases where the gripper failed to grasp the edge of the drawer. These failures are linked task variations, specifically the slight movement of the object along the x-axis, which led to the gripper failing due the agent's incorrect determination of the object's position. To address this issue, suggest implementing more detailed coarse-to-fine fine tuning approach on the vision data and training with increased variation to improve accuracy in object positioning.
7.2 Interaction Mode Grounding with ActA1M2 using Reinforcement Learning
We further explore ActAIM2's capability prior in reinforcement learning to achieve specified interaction modes given goal, iterating through all plausible interaction modes effectively ground and distinguish each mode. In our setup, we used an unseen table (ID:20411) 1) as the articulated object with sparse reward function defined as r = if dd d, < d, else O, where d, is the selected ith degree of freedom (DoF), d is the target DoF, and d, the threshold. Reinforcement learning was employed using ActAIM2 as the prior, framing the task Multi-Arm Bandit problem [49], which narrows the sampling space and enhances learning efficiency. For comparison, we used ActAIM and RVT-VQVAE the prior in the baseline experiments, applying the same reward function but updating the sample task embedding via the cross-entropy method [50].
Qualitative and quantitative results are visualized in Figure 6a and Figure 6b. Figure 6a shows agents refining their understanding interaction modes through repeated sampling, leading to self- supervised re-clustering. Figure 6b illustrates that using ActAIM2, the reward- equivalent to the average sample success rate-smoothly increases, whereas ActAIM struggles ground the task embedding specific interaction mode representation, and VQVAE-RVT updates slowly due its inability to provide distinct interaction mode representations.
15

---

7.3 Assumptions and Future Research
7.3.1 Assumptions
During the data collection phase, ActA1M2 operates under the assumption that: 1) the manipulations straightforward enough to be captured using limited set of action primitives such grasping, pushing, or pulling; 2) an interaction mode is identified upon observing significant visual changes; 3) the interaction modes can be categorized into few distinct types. A more detailed discussion of these assumptions is provided below.
Simple Action Space We employ scripted, self-supervised method to collect actions that en- compass diverse interaction modes. The action space is sufficiently simple, focusing primarily on heuristic grasping and random actions. For more complex tasks, such as hammering, washing dishes, or cooking, our current method fails to collect adequate data. Addressing these more intricate tasks would require a more comprehensive and extensive dataset.
Significant Visual Change - Our data collection is entirely self-supervised, devoid of any expert data or privileged information. We define an interaction as successful if results in a significant visual alteration the targeted objects. This approach is effective for articulated objects in our studies, such doors, windows, tables, which typically remain stationary except for their movable components. However, challenges arise with objects like tools (e.g., hammers, cups, knives), where it is difficult to discern visual changes either in the tools themselves or the targeted objects (e.g., nails, cup holders, or deformable objects). Especially in tasks requiring repetitive actions, like continuously striking nail or repeatedly wiping dishes, a more nuanced and generalized method is necessary to determine meaningful interactions are occurring.
Discrete Interaction Modes - Articulated objects, by design, often have limited manipulation options. However, when dealing with other objects such as tools, the number of potential interaction modes significantly increases. The functionality of these objects can be diverse: for example, a hammer might used not only for hammering but also for hooking or reaching. Even the act of grasping these objects presents countless variations, complicating the task of clustering them into discrete modes.
7.3.2 Future Research
Based on the assumptions discussed earlier, we have identified two primary avenues for extending our current research: long-horizon planning tasks and enhancing tool manipulation strategies.
Long-horizon Planning Tasks Leveraging the discrete representation of interaction modes provided by ActAIM2, we propose its application to long-horizon planning tasks. Examples of such tasks include sequentially opening table drawer, locating and opening box within the drawer, and finally pressing button inside the box. These tasks illustrate the potential of ActAIM2 to serve as foundational prior, streamlining the process to discrete searches within complex sequences. To ensure the robustness of our approach, it is crucial that the model accurately predicts feasible interaction modes based on the given scenario.
Extension to Tool Manipulation Tasks Another direction for expansion involves applying our work to tool manipulation. Here, defining the interaction modes for various tools will be pivotal. robust dataset specifically tailored for tool manipulation is essential to support this endeavour. Additionally, a more sophisticated scene descriptor is required effectively determine which objects manipulate and which designate targets. This development would facilitate more nuanced and effective tool interactions in automated systems.
16

---

7.4 Dataset Generation
7.4.1 Iterative Data Collection Method
When collecting data, we employ a strategy of random sampling, subsequently filtering successful actions as determined by our vision model without resorting to any privileged information. Drawing inspiration from [43, 51], we delineate the task of manipulating articulated objects into four fundamen- tal poses: initiation, reaching, grasping, and manipulating. Throughout these stages, we capture the robot's key action poses 0; (p. R., q), and RGBD observations O2 from a configuration of five cam- eras encircling the articulated object. Upon collecting the trajectory T₃ = ( O,) 0, 1, 1,2, 2,2,3) 3- we also archive the initial and final observations, O and Ofimal respectively, captured from the multi-view cameras with the robot occluded, to facilitate manipulation success evaluation.
We introduced our method of identifying successful interacted trajectories, which can be purely from vision data, specifically the initial and final observation. For each trajectory, characterized by the initial observation Ojmii and final observation O j/inel we utilize a pre-trainee image encoder & to transform the image observations into a latent vector v. The task embedding 2j for each trajectory T₃ defined as follows:
Zj = vi - of fifiall -
In our implementation, we employ pre-trained VGG-19 network [52], without the final fully connected layers, to serve as our image encoder Eg. To determine the success failure of manipulation, we introduce threshold E, defining a trajectory T3j as successful if 2j >> Σ. It is important to note that this process does not rely on any privileged information. To illustrate the validity of our method, we define the trajectory's success as a 30% change in the ground-truth DoF value. The efficacy of this criterion is validated against the ground-truth DoF values, demonstrating a 97.4% accuracy rate across our training and testing dataset. The collected trajectories must exhibit the diversity of interaction modes of the articulated objects. Thus, we employ three distinct methods of action sampling, outlined below. The final dataset is a composite of these three methods.
Random Sampling- We generate play data for manipulation without prior interaction. First, we select an interaction point P1 E R3 on the articulated object, ensuring it lies within the robot's workspace. Subsequently, we sample uniformly random manipulation rotation Rg E SO(3) and manipulation position P2 within the valid area, applying filters to exclude any configurations that would result in collision. The robot's s initial position po is also determined through random sampling, which is specified distance from the interaction point Pl. ensuring a feasible starting position for the manipulation task. Based on the previous sampling, we define the randomly sampled action sequence as ((po. Ro, 0), (p1, Ro, 0), (pr, Ro: 1), (p2: Ro, 1)}.
2. Heuristic Grasping Sampling- Heuristic grasping sampling is employed to select interaction points on the articulated object to enhance the precision of grasping actions. Utilizing the RGB-I) observations, we crop the articulated object and transform it into an RGB point cloud, which under- goes preprocessing with DBSCAN clustering
[53], aimed at identifying segments with significant geometric features, such handles or buttons. After clustering, each segment is analyzed by pre-trained GraspNet model [54] to generate set of potential grasps. From this set, grasps with the highest scores are selected, with the grasp point designated as the interaction point and the grasp orientation as the gripper rotation for the trajectory. The initial and manipulation poses are determined using the previously described random sampling approach. This heuristic approach to grasping not only bolsters the stability of grasp actions but also enriches the dataset with a higher proportion of complex interaction modes, such as "grasp to open". enhancing the dataset's diversity and utility for training models manipulate articulated objects in 'hard' interaction scenarios.
3. GMM-based Adaptive Sampling To foster a wide array of interaction modes within our dataset, we implement GMM-based adaptive sampling inspired by the methodology outlined in [12]. Following the acquisition of M trajectory datasets {Tilj 1, 2,2..... ..., M) } through random and heuristic grasping sampling from previous interactions, we compute the task embeddings {zylj 1, ..., M)
based on Equation 9. A Gaussian Mixture Model (GMM) prior is constructed from these task
17

---

Close Moddle
Open Righ
No Semarti Change
Close Fight
Cloza
(b)
Figure 7: (a) GMM clustering adaptive sampling: his figure illustrates the visualization of using GMM to represent different interaction modes.
(b) Visualization of Heuristic Grasping: We illustrate proposed grasping using our predefined heuristic with ContactGraspNet [55].
embeddings, denoted as P(z10) Ex kk= kxp(al0s). where 02 represents the parameters of each Gaussian component within the mixture. The choice of K, the number of clusters, is hyper- parameter that reflects the presumed number of interaction modes inherent to the object.
Subsequently, we cluster the task embeddings 2j. assigning a unique cluster label to each correspond- ing trajectory. We found that the task embeddings from different trajectories grouped within the same cluster indicate similar interaction mode, they share proximate visual characteristics from initial and final observation. Upon clustering. new GMM is formulated for each cluster, based on the action sequences, represented as PA (alø) = E Bips (xp=(aló)) (aléı). We then aim to sample an equal number of actions from each cluster, ensuring that the representation of actions-and, by extension, interaction mitis the dataset are as diverse as possible, thus facilitating a comprehensive exploration of the articulated object's potential interactions.
Utilizing these sampling methodologies, concurrently collect data across all articulated objects within our dataset, culminating in dataset denoted as:
D = (T Fradoom U gTihgrapp U (T, JMM
(10)
= ((a,, Oi)g random U [(a,, O,) } U {(a,, Oi)
(11)
= {(O,, a;) randommigrrapUUGMMM
(12)
After data collection, we enrich each trajectory within our dataset by associating the respective task embedding with the data tuple (O, a), thereby forming atomic training data instances represented as (O, a, a).))
7.4.2
Data Collection Algorithm
The dataset we developed for training purposes is available official website. Our dataset constructed through combination of random sampling, heuristic grasp sampling. and Gaussian Mixture Model (GMM)-based adaptive sampling, featuring the Franka Emika robot engaging with various articulated objects across multiple interaction modes. It encompasses categories such as faucets, tables, storage furniture, doors, refrigerators, and switches, with 8 unique instances per category. For each instance, we collected 150 trajectories, ensuring comprehensive coverage of the objects' interaction modes. Objects were scaled to realistic size and initialized 'half-open' state, denoting median value for each degree of freedom (DoF). The data collection methodology is detailed in Algorithm 1.
7.5 Model Architecture and Implementation Details
This section outlines the detailed implementation of the model architecture, encompassing both the mode selector and the action predictor components.
18

---

Algorithm Data Collection Algorithm
Require:
Initial observation O Number of GMM component K, hyper-paramter M for GMM in
each cluster
Ensure: All sampled trajectories filtered successful by evaluating B> Set the initial dataset to be emply
D t 0
while D have enough data do
D, = {(a, o) ~ RandomSampling
b Random Sampling
G {gi} GraspNet((")
Sample Grasp using GraspNet Gnerate trajectory based on grasp
D3 9 = ((a ~ GenerateTraj(G)
D DUDUUD
D
D Compute task embedding in current D)
Cluster 6, with GMM, assign cluster label on each trajectory (Dyli= 1, 1,..., }} <- D
DGMM -
for j in range K do
Extract D in D based on cluster label
p(D,IT, n, Σ) = II
(Em=n TmN (Xn Mmm Emm)
b fit GMM
Dj + o(a, p p(D uu, E)
Sample action from GMM
DOMM & DGMM Dj
D - DU DGMM
7.5.1
Mode Selector Architecture and Implementation Detail
This section revisits the stochastic variables' definitions and distributions, as previously emphasized. The distributions of the model parameters are formalized as follows:
p(c) Multi(m)
(13)
P(v) = N(O.
(14)
Pe y, c0') pl)ppcheeealuccc))))p 0')
(15)
pelxly.c, 0+) II Niec (v, 0*) Lo, O'))
(16)
ps(e|z,0') = (us((x, 0+), Ez(z, 0'))
(17)
Here, Pec M3, Eg are the model parameters to be optimized. Furthermore, we delineate the generative model and compute the inference test time by defining the posterior follows:
q(>, y, clc, = II qo.. (xlee, O') (yle, O³ O') (clo, y, O')
(18)
This necessitates the computation of three additional network parameters: Iy.: Iv, Io.. We then elaborate on deriving the posterior (s (c|a,, y, O²) for categorical variables e, employing the Gumbel Softmax for the representation of categorical distributions.
Notice that is categorical parameter that Multi(=). We defined that c C {cr, ...... Ck and the each class probability described { T2,..., ]. We use the Gumbel Softax trick which provides simple and efficient way to draw samples from a categorical distribution with class probabilities {TI, TT2 ...,
..., ]. The following form represents the categorical as,
c == (arrmma [g: + log m:))
(19)
where (91, 92, ..., 9k are i.i.d samples drawn from Gumbel(0,1). Assuming that categorical samples are encoded as k-dimensional one-hot vectors lying the corners of the (k -)ddimensional simplex A k-1 We use the softmax function as continuous, differentiable approximation to arg max, and generate k-dimensional sample vectors w E k-1 We defined w as
exp(()og(T) (TT, + 9:)/T)
Wi * 1= exp((log(mm) g:)/T)
(20)
19

---

y
a
4
E
B
0
Generative
Variational
Figure 8: The graphical representations elucidate the Conditional Gaussian Mixture Variational Autoencoder (CGMVAE) framework, showcasing two distinct models: the generative model the left and the variational family on the right. These graphical models serve to visually communicate the structural and functional relation- ships between variables within the CGMVAE, illustrating the data generation process and the approximation strategy employed by the variational family to infer latent variable distributions.
Where is the temperature the hyperparameter. Therefore, we define the density of the Gumbel- Softmax distribution
p(c) = Pr. (wr, w/) = T(K)T=-1
n
(21)
Now, given the representation of the categorical distribution of e from Equation 21, derive how we compute the posterior qo: for We consider the posterior qv. (c cj|z, y, O³) given Cj.
1r,,,,,, pcc= mc=c))((lc==s 9, O*)
4v. (v = (c=
(22)
(c_p(c=a)p(((c- ,, O+)
(23)
Ei Tppl/c c. Ot)
Therefore, we derive the posterior qv. directly and leave posterior network Tol, Iv, to be trained. Based on the following discussion, draw the generative model and variational model view as graphical models in the Figure 8.
In the implementation detail, we write parameters ps = (As, Eg) and = (Jez Ex,) to generate a Gaussian distribution with each representing the mean and variance. We implement the network Pe, Day Vaz th with a multi-layer ResNet and implement the network ys,,. as multi-view transformer since both Oc and represent multi-view information with the same number on the channel the correspondent view number. We show model μ43; Eg architecture in Figure 9.
7.5.2 Mode Selector Training and Inference
We illustrate the functionality and application of mode selector through two distinct plots, highlighting both the training process and the inference mechanism for task embedding generation.
Figure Figure 1 la depicts the model's operation during training, where it processes the conditional variable O' along with the ground truth data e, to accurately reconstruct the task embedding.
Conversely, Figure Figure llb demonstrates the inference stage, where the model, requiring only the initial observation O' and discretely sampled cluster (employing 8-cluster configuration for implementation), successfully generates the corresponding task embedding e.
7.5.3 Action Predictor
We provide the architecture of the action predictor which is joint transformer that takes in task embedding and novel view input. The detailed implementation is shown at Figure 10.
20

---

P&
Σ
Joint
Attention X
Vec_ view_0
Vec view_1
Vec view 2
Vec view
Vec_
VGG-19
O
O O
O
O
O
Figure 9: Mode Selector Decoder Architecture: The depicted architecture highlights the functionality mode selector decoder, which is designed to process two primary inputs: multi-view RGBD images O' = (Os, Oi, O2, Os, Oi), and the Mixture of Gaussian (GMM) variable æ. is important to note that a can be represented as multi-view feature vector, with our encoding approach preserving the separation of multi-view channels. Initially, the multi-view RGBD images are passed through pre-trained VGG-19 image encoder to extract feature vectors for each view. Subsequently, these feature vectors, along with the GMM variable v, are inputted into joint transformer. This transformer, featuring four attention layers, tasked with producing the means and variances associated with the reconstructed task embedding e.
Rotation
Patchify
Heatmap
E 3
Task Embedding E
Attention X 4
Attention
Gripper Open or Close
(p. R. q)
Orthogonal Projection
Novel
Multiview
Figure 10: Action Predictor Architecture: This model integrates multi-view observations directly as input, sourced from predefined cameras within the scene. The process begins with the extraction of five RGBD images. which subsequently transformed into RGB point clouds. These then subject orthogonal projection generate novel view images. Subsequently, these novel views are partitioned into smaller patches and fed into a joint transformer. This transformer, characterized by four attention layers, integrates the sampled task embedding derived from Mixture of Gaussian distribution. The architecture of the joint transformer encompasses eight attention layers, culminating in the production of heatmap. This heatmap delineates the action's translation, the discretized rotation, and binary variable indicating the gripper's state state-open closed.
7.6 More Qualitative Results
We supplement our presentation with additional qualitative results, further elucidating the model's proficiency learning the disentanglement of interaction modes. Initially, we demonstrate the efficacy of the mode selector through a I-SNE plot. This choice of visualization is motivated by our methodology of training the mode selector and action predictor independently, allowing for focused examination of the mode selector's performance.
21

---

Contrastive Difference Training Purposes Only
Reconstruction Loss
O1
Ou
GMM clustered interaction
mode latent space
f(
Task embedding
p(c)
Multi (m)
moode
D mode
E
category
Task embedding Transformer Decoder
f(
Conditional Initial Image Inference
On
(a) Training Process of the Mode Selector: This figure illustrates the training procedure of mode selector, mirroring the approach of conditional generative model. It highlights the contrastive analysis between the initial and final observations- tte latter serving as the ground truth for task embedding to delineate generated data against the backdrop of encoded initial images the conditional variable. The process involves inputting both the generated encoder, which then predicts the categorical variable c. Following the Gaussian Mixture Variational Autoencoder (GMVAE) methodology. the Gaussian Mixture Model (GMM) variable is computed and introduced alongside the conditional variable to the task embedding transformer decoder. This model is tasked with predicting the reconstructed task embedding, sampled from the Gaussian distribution outlined in the architecture af the mode selector decoder, and calculating the reconstruction loss against the input ground truth data.
embedding data and conditional variable a 4-layer Residual network-based mode
GMM clustered interaction mode latent space
Task embedding Transformer Decoder
p(c)
Multi (T)
Task embedding
D mode
E
Discrete
Sampling
f(
Conditional Initial Image
Oo
Inference
(b) Inference Process: In the inference phase, agent discretely samples cluster from trained Gaussian Mixture Variational Autoencoder (GMVAE) model to calculate the Mixture of Gaussian variable r. This variable in conjunction with the conditional variable (initial image observation), is then inputted into the mode selector transformer decoder. The objective is to reconstruct the task embedding for inference, effectively translating the conditional information and sampled cluster actionable embeddings.
22

---

Subsequently, we extend our qualitative analysis with figures akin to those presented in the main paper, offering a comprehensive view the model's capabilities. These additional figures serve to reinforce the insights gained from the initial results, showcasing the model's 's nuanced understanding of interaction modes through the distinct visual representations of the data.
7.6.1 Mode selector TSNE plot Figure 16
Utilizing our pre-traine pre-trained Conditional Gaussian Mixture Variational Autoencoder (CGMVAE) mode selector, we conduct disentanglement learning visualization ically, we focus on the "single drawer" object (object ID: 20411), 1), employing the mode selector to delineate the generated clusters and compare them with the ground truth task embeddings. The data for this visualization is derived from our dataset, and we calculate the task embedding ej for each data point as the difference between the initial and final object states, represented by
comprehensive dataset. Specif-
inst
final Eo(O) - Eo(0) inal
Cj
Subsequently, we employ t-SNE plot to simultancously visualize the ground truth and generated task embeddings. In this visualization, distinct colors within the ground truth plot indicate data points originating from different interaction modes. Similarly, varied colors in the generated plot correspond to data points arising from disparate clusters within the Mixture of Gaussians model. Through this approach, we demonstrate that:
1. The ground truth task emheddings distinctly clustered based on the interaction modes.
2. The CGMVAE model effectively generates clusters that categorize data points by their respective categories
3. The reconstructed data closely aligns with the ground truth data points, with the majority of the clustered data encompassed within the respective ground truth clusters.
This visualization underscores the efficacy our generative model mode selector extracting task embeddings for further application in the action predictor, highlighting the model's capability to discern and categorize interaction modes accurately.
7.6.2 Action Predictor Qualitative Results
We present extensive qualitative results in Figure 17a, Figure 17b, Figure 18a, and Figure 18b, demonstrating the model's ability to predict distinct interaction modes through discrete sampling. For each object, we explore three different clusters, each representing a unique interaction mode. The initial state the robot and the articulated object is depicted from three perspectives: top-down, front, and side views. The heatmaps, derived from the top view during manipulation steps, highlight the variance in action space corresponding different sampled interaction modes. Subsequent imagery illustrates the robot's movement within the simulator and the outcome following interaction with the articulated objects. It is important to note that comprehensive video demonstrations accompany this document and are accessible on our website, https://actamm2,iiibbbb
7.6.3 Comparison of ActAIM2 and VQVAE-RVT
Inspired by the Genie [48] approach, we have compared our ActAIM2 with VQVAE-RVT assess the efficacy of these models in discerning discrete interaction modes robotic manipulation tasks. Our primary objective to evaluate the distinction between interaction modes using a simplified scenario, single-drawer table, which naturally exhibits two distinct interaction modes: opening and closing.
In our experiments, we visualized the latent spaces generated by both ActAIM2 and VQVAe-RVT. Particularly for VQVAE-RVT, the latent space visualization involved examining the distribution of eight code vectors. As depicted in Figure 12, these vectors clustered into two categories, which ideally should correspond to the two expected interaction modes of the drawer. This clustering pattern was anticipated and desired as it suggests clear demarcation between the distinct modes of interaction.
23

---

Figure 12: Visualization of Latent Space Clustering in VQVAE-RVIS This figure illustrates the distribution of eight code vectors within the latent space, categorized into two distinct clusters. These clusters are intended represent the discrete interaction modes of opening and closing drawer. The spatial arrangement highlights the expected separation code vectors, symbolizing the potential for mode-specific action mapping robotic manipulation tasks. Despite this apparent clustering, subsequent heatmaps (see Figure 13) reveal lack of diversity in the action predictions, undermining the practical utility of this model configuration.
However, subsequent visualizations raised concerns about the practical efficacy of the VQVAE-RVT model in our application context. When we explored the heatmaps generated by the VQVAE-RVT model, we observed a critical limitation: all 8 code vectors produced essentially the same heatmap, despite their differing positions in the latent space. This heatmap, illustrated in Figure [Y], consistently depicted all plausible interaction modes for the drawer, regardless the specific code vector used. This outcome was stark contrast to the results from ActAIM2, where distinct heatmaps clearly indicated specific interaction actions like pushing or pulling, depending on the sampled cluster within the latent space.
These findings led us to conclude that merely replacing the GMVAE component with VQVAE in the setup did not achieve the desired disentanglement of interaction modes. The VQVAE-RVT model failed to map the code vectors to unique, mode-specific interaction strategies, instead converging on a generalized representation that was not useful for distinguishing between the actionable options of opening and closing the drawer. Consequently, ActAIM2's ability to discriminate between distinct interaction modes via cluster-specific sampling proves superior in contexts demanding discrete and distinguishable action representations.
We also explain this from mathematics which provide an intuition of why GMVAE perform much better than VQVAE. Referring to Figure 8 and the ELBO loss computation in Equation 18 from the appendix, compared to the VQVAF, the most unusual term our ELBO the c-prior term. The c-posterior caleulates the probability assigning data point cluster by measuring the distance between x and each cluster position from y. The c-prior helps reduce the KL divergence between the c-posterior and a uniform prior by adjusting both the clusters' positions and the encoded point x. This term aims to merge clusters by increasing their overlap and bringing their means closer together. However, like other KL regularization terms, it can conflict with the reconstruction term and may become too dominant with more training data. This over over-regularization issuc, common in traditional VAEs, also affects GMVAE clusters, leading to large, degenerate clusters.
7.7 Generation of Demonstration Videos
To illustrate the practical applications and effectiveness of ActAIM2, we generated demonstration videos by employing its inference mechanism. The process involves several key steps:
Generative Mode Selection: Initially, observations are inputted into the generative mode selector of ActAIM2. This component responsible for reconstructing the task's 's latent space, which is modeled as Mixture of Gaussians. This structure enables discrete sampling of clusters, which represent distinct interaction modes that the robotic system exccute.

---

vqvae code vector
ActAIM2 cluster
ActAIM2 cluster
Top view
Figure 13: Comparative Visualization of Action Heatmaps and Observational Data From left to right: (1) VQVAE-RVT action heatmap synthesized using all eight code vectors, showing identical outcomes board. indicating failure to differentiate interaction modes. (2) Action heatmap generated by VActAIM2 when sampling from one cluster, demonstrating a specific interaction mode. (3) Action heatmap from VActAIM2 when sampling from different cluster, showcasing another distinct mode of interaction. (4) Top-view observation of the drawer, correlating with the spatial contexts of the heatmaps, providing visual reference for the interaction zones mapped by the heatmaps. This series highlights VctAlMM2's capability discern and represent distinct action strategies through targeted cluster sampling.
the
Open Drawer
Figure 14: Opening and Closing Drawer: This figure demonstrates the effective action sequence generated by ActAIM2 for drawer. The left part of the image shows the drawer being opened, showcasing the robot's approach and grip adjustment. The right part of the image captures the drawer in fully closed position, illustrating the final state after the action sequence execution.
2. Sampling and Action Prediction: From the reconstructed latent space, sample the task beddings by selecting cluster within the Gaussian Mixture Model (GMM) and its corresponding Gaussian distribution. This sampled task embedding is then forwarded to the action predictor. The action predictor generates the specific actions needed to interact with the environment effectively. Simulation and Recording: As depicted in Figure 14 and Figure 15, ActAIM2 reconstructs an object- GMM and samples different task embeddings. Depending on the sampled task embedding, different interactions are reconstructed and exccuted within a simulator. We recorded the manipulation processes, which are detailed in the video provided in the supplementary files. Each video showcases how ActAIM2 navigates through different interaction scenarios, reflecting the diverse capabilities of the model in real-time applications.
This comprehensive demonstration not only validates the functionality of ActAIM2 but also provides a visual understanding of its potential in diverse robotic manipulation tasks. The videos highlight the nuanced interactions achievable through targeted sampling within the model's structured latent space.
25

---

Cloce Right Door
Figure 15: Opening and Closing Door: This figure illustrates the ActAlM2's manipulation capability with
door. The left image displays the door heing opened, highlighting the robot's positioning and the initial interaction phase. The right image shows door completely closed, detailing end the manipulation sequence and the effectiveness of the action predictor.
26

---

Generated TSNE
Ground
Mixed TSNE
Figure 16: Disentangtement Visualization with CGMVAE: This figure illustrates the efficacy of the Condi- tional Gaussian Mixture Variational Autoencoder (CGMVAE) disentangling interaction modes for the "single drawer" object (ID: 2041 I), using t-SNE plot visualization. Task embeddings çj, defined by the variance hetween initial and final object states, are visualized distinct colors to denote various interaction modes and clusters. The sequence of figures demonstrates the CGMVAF's precision in clustering and aligning data points with their respective interaction modes: (1) Generated clusters from the CGMVAE mode selector reveal distinct groupings. (2) Ground
task embeddings confirm the model's capacity accurate interaction mode classification. (3) A combined visualization underscores the alignment between generated clusters and ground truth, showcasing the model's 's' ability to consistently categorize tasks within identical interaction modes.
27

---

Predicted Heatmap In Top View
Initial Novel View
Execution Trajectory
Final Outcome
Close left door
Front View
Open right door
Top View
Close right door
Side View
(a) Door, Object ID: 8961
Predicted Heatmap In Top View
Initial Novel View
Execution Trajectory
Final Outcome
Turn On right faucet switch
Front View
alr
Turn On left faucet switch
Top View
Turn Off
Side View
left faucet switch
(b) Faucet, Object
28

---

Predicted Heatmap In Top View
Initial Novel View
Execution Trajectory
Final Outcome
Open bottom-right drawer
Front
View
Open top-feft drawer
Top View
Close
Side View
top-left
drawer
(a) Table, Object ID: 19898
Predicted Heatmap
Execution Trajectory
Final Outcome
Initial Novel View
Top View
Close
Front View
top-left drawer
Top View
top-right drawer
Side View
Open
top-right drawer
(b) Storage, Object ID: 41083
29

---

Table 3:
Object Mode Sampling Evaluation with More Interaction Modes: We selected objects with more than two interaction modes and evaluated the mode sample success rates using ActAIM2. In practice, ActAlM2 executes actions sampled uniformly across clusters and then reports the corresponding interaction mode success rates. The reported interaction modes are illustrated Figure 19.
Object Category
Object ID
1st Mode f
2nd Mode% t
3rd Mode %
4th Mode %
F. Faucet
154
63.4
16.3
6.2
0
Table
19898
40.1
16.2
13.2
6.3
Door
8961
79.3
43.2
14.2
10.2
Storage
41083
55.4
26.1
17.3
0
7.8
More Experimental Results
In this section, have added two additional experimental results: object-specific mode sampling evaluation over ActAIM2, where the objects exhibit more than two interaction modes, and average sample success rate evaluation over ActAlM2 with limited perception information. These results aim to demonstrate the stability and robustness ActAIM2 when predicting outcomes for more complex objects. Furthermore, we demonstrate that ActAIM2 can still accurately predict general outcomes with minimal visual information. This suggests the potential for simplifying the scene setup adapt ActAIM2 for real-world experiments.
7.8.1 Mode Sampling Evaluation on objects with more Interaction Modes
In this section, we assess the mode sampling effectiveness on an object-wise basis as detailed in the experiment section our paper (see Table 2). We focus on the first and second mode clusters identified by ActAIM2, presenting only two modes because we are examining the average mode sample success rate
categories, typically involving around eight different instances. For simpler articulated objects like table with single drawer faucet with single switch, there are typically only two valid interaction modes. This is because these objects are initialized at the mean joint value Juit = (Juax + m/imm/// 2. However, most articulated objects feature more than two joints, leading to multiple interaction modes.
To illustrate this, we select more complex objects from various categories, including faucets, tables, doors, and storage units. We first showcase plots depicting the interaction modes for each object from Figure 19. Following this, we present how ActA1M2 predicts the various interaction modes. These objects and their interaction demonstrations are also featured on our website through illustrative videos.
From Table 3, we find that ActAIM2 capable of predicting multiple interaction modes for more complex objects and achieves reasonable success rate among the demonstrated interaction modes.
7.8.2 ActAIM2 evaluation with less visual information
In the main paper, we utilize five cameras positioned around the articulated object to gather visual information. To assess the model's 's applicability in real-world scenarios, we evaluate its performance when trained with reduced visual information. Specifically, we conduct two additional experiments: one training with only depth information and another using single view RGB-D camera. We employ the same dataset as in the experimental section but mask out extra information. For qualitative results, please refer to. Our results demonstrate that ActAIM2 can effectively train with less visual data, with only acceptable level of performance decrease.
From Table 4, we find that training with only depth information does not significantly affect perfor- mance, with less than 5% decrease in success rate when color omitted. However, using only single camera view impacts the success rate more substantially, particularly when the model is applied to larger objects. Scalability also significantly decreases with single view camera. Nonetheless, performance on previously seen objects remains similar level.
30

---

Table 4: Robotic Interaction Mode Discovery with Less Vision We evaluate ActA1M2 under conditions of reduced visual information demonstrate its stability and robustness. Specifically, we list experiments where ActAlM2 trained using only depth data and single-view camera observations. Despite these constraints, maintains a success rate drop within an acceptable range.
Test Set
Seen Objects
Unseen instances
Unseen Cats
SSR %
F:
1
AVG
F:
AVG
18
AVG
AcAIM2
65.3
43.2 52.1 69.2 25.3 36.2
48.6
44,9
41.2 41.5 60.2
20.1
24.4
38.7
34.3
28.9
34.1
32.4
Depth Only ActA1M2
61.5
41.9 49.3
61.5 26.3
35.1 45.9
44.3
42,6
34.2
43.7
214
20.4
34.4
30.2
21.2
29.5
27.0
Single View ActATM2
55.3
32.1
35.2 45.3 214 27.9
36.2
32,5
26.1
29.7
32.1
15.3
13.2
24.8
15.3
13.2
12.3
13.6
7.9
Comparison between Prior Works
7.9.1
Comparison between ACtAIM [12]
We state that the main differences between our work and [12] lie in major aspects which are data collection (environmental setup) and method (discrete representation).
Data Collection For ActAlM, we utilized a floating parallel-jaw gripper for data collection, eliminating concerns related to invalid collisions or reachability issues. The action primitives learned by ActAIM include identifying the point to graspor interact and determining the direction to move post-grasp akin to the approach used in Where2Act [43].
For ActAIM2, employed full Franka Emika robot, necessitating consideration of workspace constraints and potential collisions with the robot body. Directly applying the GMM adaptive data collection method from ActAIM proved challenging: hence, we incorporated heuristic grasping strategy to aid in identifying graspable parts of the object. We developed an analytical technique to determine which segments of the point cloud are likely to correspond to handles on articulated objects. This was followed by using DBSCAN to identify and filter potential heuristic handles. Subsequently, we employed ContactGraspNet [55] to predict possible grasping points, as illustrated in Figure 5 of the Appendix. We combined these heuristic grasping actions with randomly sampled data execute GMM adaptive sampling, similar to ActAIM, ultimately integrating these components into our final dataset.
Method Both ActAIM and ActAIM2 share the approach of decomposing the policy into mode selector and an action predictor. The key difference lies in how they represent the interaction mode. In ActAIM, simple Conditional Variational Autoencoder (CVAE) is used as the mode selector to represent the interaction mode for manipulating articulated objects. However, the latent space in this approach forms a Gaussian distribution, which cannot be easily sampled or clustered.
In Section 5.3, we experimented on interaction mode grounding, demonstrating that ActAIM struggles to associate task embedding with specific interaction mode. The purpose of this experiment to show that if the structure of the latent space is well-organized (meaning that all interaction modes are distinctly learned), the grounding should converge after a few interactions. The results revealed that ActAIM fails to provide a clear and discrete representation of interaction modes.
Furthermore, we visualized the interaction mode representation from ActAIM, as shown in Figure 14 of the Appendix. The visualization indicates that the interaction modes learned by ActAIM noisy and disorganized, with different modes scattered throughout the space. This explains why the experiment in Section 5.3 did not successfully ground specific interaction modes.
7.9.2 Comparison between works with affordance prior
Interaction modes are characterized by significant visual changes in the observation space. In contrast to methods such as SayCan [7], VADER [56], or RLAfford [57]. our approach focuses more on visual affordance without incorporating action data. The interaction mode seeks to predict and categorize the limited future states of articulated objects, such opening or closing partially open drawer. Ideally, the mode selector p(clo) could function as the encoder in video generative model. In the context of articulated object manipulation, where visual information is readily available,
31

---

our mode selector model is trained using pre-trained visual encoder (VGG-19 in our case). This mode selector could be substituted with any world model or video prediction model that forecasts possible future states. Ultimately, the interaction mode infers a distribution over plausible future states achievable through interaction, based on an input image of a scene. For articulated objects, any visual change in the object, especially if it occludes the robot, significant. We aim to extract and cluster such information to facilitate learning in the action decoder.
7.10 Limitations
In this section, we discuss 3 major limitations of the current ActAIM2 which are imbalanced success rate in different interaction modes, lack of general scalability, and limited to articulated object manipulation. Moreover, we provide potential solution to all these limitations.
7.10.1 Imbalanced Success Rate
The experimental results demonstrate that while ActAlM2 can represent different interaction modes using discrete clusters, the success rates among these modes are imbalanced. The Mode Sampling Evaluation table shows that the average success rate in the second mode cluster is much lower than in the first mode cluster for ActAIM2. Additionally, Figure 6b illustrates this imbalance using half-open drawer demonstration: pushing the drawer in has an over 80% success rate, whereas opening the drawer only achieves about a 20% success rate.
This imbalance in success rates could impact the application of ActAIM2, particularly interaction mode grounding and long-temn planning. As shown in section 5.3, the interaction mode grounding experiment using RL required nearly 50 iterations distinguish between closing and opening modes, with the average success rate for opening still not being high enough.
In more complex scenarios with multiple articulated objects, this imbalance might affect the planning process if ActAlM2 used as a prior. It can difficult determine whether the sampled interaction mode will lead "nothing happens" outcome rare interaction mode. Therefore, multiple sampling attempts are needed, reducing the efficiency of data use.
Based on our experience with model training, we believe that an imbalanced dataset is the cause of this issue. Although we use the GMM adaptive data collection method to increase interaction mode diversity, the challenge remains that modes requiring precise grasping need more accurate action labels. To address the problem of imbalanced success rates, we propose collecting more stable data. Specifically, suggest incorporating more expert tele-operation control data into the dataset to achieve more reliable grasping, rather than relying solely completely self- ooff-generated data.
7.10.2 Lack of General Scalability
In the experimental table, we observed a significant decrease in success rates from seen objects to unseen objects, and further to unseen categories. The success rate for generalizing across different categories dropped by nearly 50%. While ActAIM2 shows some level of scalability, performs poorly when scaling across different categories.
To address this scalability issue, suggest training model more data. Currently, ActA1M2 is trained on only categories with 8 unique instances, and we have collected 150 trajectories for each articulated object. We believe that incorporating greater variety of object categories is necessary to better demonstrate scalability across different types of articulated objects.
7.10.3 Limited to Articulated Object Manipulation
ActAIM2 designed by decomposing the policy into a mode selector and an action predictor, where the mode selector defines distinct interaction modes. In this work, define an interaction mode as significant visual change in the articulated object, assuming that any visual change is meaningful if it pertains the object. However, this definition doesn't 't hold true for tool manipulation tasks. For
32

---

example, in hammering task, simply moving the hammer around on the table without grasping it is meaningless, yet would still qualify as an interaction mode under the current definition.
This misalignment leads inefficiencies during training when applying ActAIM2 tasks like hammering. Additionally, data collection becomes significant challenge due the longer task duration and the need for more precise actions.
To address these issues, we suggest first improving the data collection process and redefining the concept of interaction modes for tool manipulation tasks. Specifically, we should introduce stable grasping as an additional interaction mode and utilize large video-language models assess whether the resulting visual changes are meaningful.
33

---

1st Mode Cluster
2nd Mode Cluster
3rd Mode Cluster
11
Faucet ID: 154
Initial State
Turning Off the switch on the Left
Turning On the switch on the Right
Turning Off the switch on the Right
Mode Cluster
2nd Mode Cluster
3rd Mode Cluster
Mode Cluster
Table ID: 19898
Initial State
Closing the top drawers the Left
Opening the door on Left
Opening the top drawer the Left
Opening the bottom drawer the Right
1st Mode Cluster
2nd Mode Cluster
3rd Mode Cluster
4th Mode Cluster
Door ID: 8961
Initial State
Closing the Left door
Closing Right door
Opening the Right door
Opening the Left door
1st Mode Cluster
2nd Mode Cluster
3rd Mode Cluster
Storage ID: 41083
Initial State
Closing the drawer on the Right
Opening the drawer on the Right
Closing the drawer on the Left
Figure 19: We illustrate objects from various categories that exhibit more complex interaction modes. These objects qualitatively evaluated based on their sample success for each presented interaction mode. Video demonstrations manipulating objects are also available on our website.
34


Reviews:
[{"review_id": "rwB1pvcW2O", "reviewer": "Reviewer_BZdG", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "Clustering-based approach to manipulating articulated objects with multiple articulation mode", "summary_of_recommendation": "I do not feel strongly about this work, and I wonder how it can be translated into reality. I am willing to change my mind based on a solid recommendation for real-world deployment.", "technical_quality": 3, "clarity_of_presentation": 2, "potential_impact": 2, "robotics_focus": 3, "quality_of_limitations": 2, "questions_for_rebuttal": "1. What are the blockers to applying this in a real world settings? Could the sampling based data collection be replaced by human demo collection in the real world?\n2. What are the authors' opinion on why VQ-VAE sampling fails to get a second mode?\n3. Why stop sampling modes at two? Shouldn't there be objects with more than two modes of interaction?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gCOzYWB4jA", "reviewer": "Reviewer_ituT", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 2, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "In this paper, the authors proposed a method to discover action modes for articulated objects and manipulate these objects using these modes. The authors trained a transformer-based Conditional Gaussian Mixture Variational Autoencoder (CGMVAE) to learn the action modes after collecting data by defining actions as moving the end effector between 4 poses: initiation, reaching, grasping, and manipulating. During data collection, grasping is handled by a pre-trained model, and a Gaussian Mixture Model(GMM)-based adaptive sampling is used to ensure the diversity of the data to discover modes. The success is defined by enough visual change between initial and end images. Afterward, a transformer-based action selector predicts actions using the interaction modes. The authors conducted simulator experiments to verify the model’s capability in comparison with recent methods [3, 43, 12] in the literature. The video only includes robot executions in simulation.", "summary_of_recommendation": "My questions are mostly answered. I updated my score.", "technical_quality": 4, "clarity_of_presentation": 3, "potential_impact": 2, "robotics_focus": 3, "quality_of_limitations": 2, "questions_for_rebuttal": "What are the differences between this work and [12] in the method and the data collection?\n\nHow would color change in the environment affect the model? Can different visuals of objects be in extrapolation regions of the models?\n\nIs the pre-trained encoder used in CGMVAE?\n\nWhat is a different instance of the object? What changes?\n\nWhat are the second modes for one joint object, such as a faucet, drawer, and door?\n\nWhat is the percentage of failures during random sampling in data collection?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "abkm2bew9Q", "reviewer": "Reviewer_DsEP", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 2, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "ActAIM2 is a manipulation method learns a discrete representation of robot manipulation interaction modes in an unsupervised manner, without expert labels or simulator-based privileged information. The method consists of an interaction mode selector and a low-level action predictor. The selector generates discrete representations of potential interaction modes, while the predictor outputs corresponding action trajectories.", "summary_of_recommendation": "I like the method presented but it is explained poorly. The current limitations section could also be improved.", "technical_quality": 3, "clarity_of_presentation": 3, "potential_impact": 2, "robotics_focus": 4, "quality_of_limitations": 1, "questions_for_rebuttal": "- Would the authors have any intuition to why the VQ-VAE learnt representations much worse than the GMM vae repesentations? Could this be attributed to codebook initialization?\n- It is unclear if the method is self supervised, when all interaction modes are provided. Making the distinction in the paper would improve clarity.\n- How are interaction modes significantly different from using affordances as priors? (Additional citations could help.)\n- Figure 2: typos - multiview transformer(figure); mode selector(caption)", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This work introduces a variant of ActAIM that uses a CGMVAE to represent action modes and that can handle multiple views. It appears to be conceptually simple, quite robust and effective in simulation. However, the reviewers perceive this work as rather incremental. It does not clearly state how exactly ActAIM2 differs from ActAIM, and it is unclear how it would fare in the real world. All reviewers ask questions about evaluation. A systematic ablation study would help. Also, Reviewer BZdG asks an important question about the relation to VQVAE-RVT.

As part of the rebuttal, the authors added an appendix describing real-world experiments, additional results, comparisons and limitations. It also satisfactorily answers many questions of the reviewers. The most important points should be integrated into the final version of the paper, if accepted.
Decision: Accept