Learning View-invariant World Models for Visual Robotic Manipulation
Authors: Jing-Cheng Pang
Nan Tang
Kaiyuan Li
Yuting Tang
Xin-Qiang Cai
Zhen-Yu Zhang
Gang Niu
Masashi Sugiyama
Yang Yu
Venue: ICLR 2025 Poster (2025)
Keywords: Robotic manipulation
reinforcement learning
world model
Primary area: reinforcement learning
TL;DR: We study robust robotic manipulation under viewpoint disturbance by learning view-invariant representation and world model.

Abstract: Robotic manipulation tasks often rely on visual inputs from cameras to perceive the environment. However, previous approaches still suffer from performance degradation when the camera’s viewpoint changes during manipulation. In this paper, we propose ReViWo (Representation learning for View-invariant World model), leveraging multi-view data to learn robust representations for control under viewpoint disturbance. ReViWo utilizes an autoencoder framework to reconstruct target images by an architecture that combines view-invariant representation (VIR) and view-dependent representation. To train ReViWo, we collect multi-view data in simulators with known view labels, meanwhile, ReViWo is simutaneously trained on Open X-Embodiment datasets without view labels. The VIR is then used to train a world model on pre-collected manipulation data and a policy through interaction with the world model. We evaluate the effectiveness of ReViWo in various viewpoint disturbance scenarios, including control under novel camera positions and frequent camera shaking, using the Meta-world & PandaGym environments. Besides, we also conduct experiments on real world ALOHA robot. The results demonstrate that ReViWo maintains robust performance under viewpoint disturbance, while baseline methods suffer from significant performance degradation. Furthermore, we show that the VIR captures task-relevant state information and remains stable for observations from novel viewpoints, validating the efficacy of the ReViWo approach.

Published as a conference paper at ICLR 2025
LEARNING VIEW-INVARIANT WORLD MODELS FOR VISUAL ROBOTIC MANIPULATION
Jing-Chen Pang1.2.3. Nan Tangl..., Kaiyuan Li Yuting Tang 1.2, Xin-Qiang Cai?,
Zhen-Yo Yu Zhang', Gang Niu?, Masashi Sugiyama & Yang Y,..3,0
National Key Laboratory for Novel Software Technology, Nanjing University, China
School of Artificial Intelligence, Nanjing University, China;
2 RIKEN Center for Advanced Intelligence Project, Japan:
3 Polixinai, China; 4 The University of Tokyo, Japan
ABSTRACT
Robotic manipulation tasks often rely on visual inputs from cameras to perceive the environment. However, previous approaches still suffer from performance degradation when the camera's viewpoint changes during manipulation. In this paper, we propose ReViWo (Representation learning for View-invariant World model), leveraging multi-view data to learn robust representations for control un- der viewpoint disturbance. ReViWo utilizes an autoencoder framework to recon- struct target images by architecture that combines view-invariant represen- tation (VIR) and view-dependent representation. To train ViWo, we collect multi-view data in simulators with known view labels. Meanwhile, ReviWo is simutaneously trained on Open X-Embodiment datasets without view labels. The VIR is then used train world model on pre-collected manipulation data and a policy through interaction with the world model. We evaluate the effectiveness of ViWo in various viewpoint disturbance scenarios, including control under novel camera positions and frequent camera shaking, using the Meta-world & PandaGym environments. Besides, we also conduct experiments on real world ALOHA robot. The results demonstrate that Re maintains robust perfor- mance under viewpoint disturbance, while baseline methods suffer from signifi- cant performance degradation. Furthermore, we show that the VIR captures task- relevant state information and remains stable for observations from novel view- points, validating the efficacy of the Re VoWo approach.
INTRODUCTION
Developing robots capable of completing various manipulation tasks is a hallmark of machine intel- ligence. Previous approaches in this domain often rely on visual inputs from cameras to perceive the environment, with a policy trained on these visual inputs using reinforcement learning (RL) algo- rithms and pre-collected robotic manipulation data (Kalashnikov et al., 2018b; Quillen et al., 2018; Agarwal et al., 2020; Levine al., 2020; Pang et al., 2024). However, the policy learned in this way often suffers from performance degradation when merely changing the camera position during deployment (Liu et al., 2024). This challenge, referred to as the viewpoint disturbance, arises from the policy's 's learned representations, which fail to separate the vivw-invariant information from the observation: they encode the entirety the observations, including both the view-invariant task state and the view-dependent information like light. As result, the changes in the viewpoint can lead to a substantial transformation in the learned representations, which can compromise the policy's s effectiveness when exposed to novel viewpoints (Liu et al., 2024).
There are existing works focusing on training robust robotic manipulation policies under viewpoint disturbances. For example, MVWM (Seo et al., 2023) tries to train robust image representation using a multi-view masked autoencoder, by encoding a masked image and reconstructing target im- ages in different viewpoints. The learned representation is then utilized for policy maakin..-making..
* Equul contribution. Full authorship contrihution statements appear after the main text. Corresponding:
yuy @ uuy@nju.edu.cn.

---

Published as a conference paper at ICLR 2025
However, the learned representation can still be sensitive variations in viewpoint because it re- lies on a single encoder model to extract information, and then reconstruct complex, viewpoint- dependent visual information which can be highly challenging. RT-X series works (Brohan et al., 2023; Zitkovich et al., 2023; O'Neill et al., 2024) attempt to address viewpoint disturbance by train- ing policies on extensive data. RoboUniView (Liu al., 2024) learns unified representation from 3D multi-view images using an autoencoder (Bank et al., 2023), but it requires all viewpoints to contain similar information, and relies on 3D data with camera calibration. Although these ap- proaches have demonstrated degree of effectiveness in addressing viewpoint disturbances, they have yet achieve the complete and effective decoupling of view-invariant state information from the viewpoint, which is important for robust robotic control.
In this paper, we investigate robust robotic ma- nipulation under viewpoint disturbance. Moti- vated by human's 's ability adapt to changes viewpoint, a skill that is supported by cognitive science research in human psychology (Souto Kerzel, 2021), we propose to decompose the vi- sual observation into a view-invariant represen- lation (VIR) for robot control, as shown in Fig. 1. More specifically, we propose Representa- tion learning for the View-invariant World model (ReViWo), which employs two encoders to sepa- rately extract view-invariant and view-dependent representations from the image, and a decoder to reconstruct target image that combines the view- invariant and viewpoint information from two encoders' inputs, respectively. This training process requires the view labels, which can be obtained when collecting multi-view data from fixed-position cameras. Leveraging the learned VIR, ReViWo trains a world model to learn the environment dy- namics from the offline manipulation data. Finally, ViWo trains policy with RL algorithm and using the world model.
View-invariant representation
Policy
decomposition
action
View new-ddepnndent
Observation
representation
Robot
Figure I:
An illustration of the robotic manipu- lation with view-invariant representation.
Our contributions are as follows: First, we introduce observation decomposition idea for enhanc- ing visual robotic manipulation under viewpoint disturbances, which enables the robot to focus on invariant information across viewpoints. This decomposition distinguishes this work from previous works that process the observation into a single representation. Second, we implement this idea by proposing Re ViWo, comprising a world model and autoencoder framework that learns the view- invariant representation from pre-collected multi-view data with view labels. Besides, we also train ReViWo with Open X- X-Embodiment (O Neill et al., 2024), to incorporate the knowledge from more diverse robotic manipulation data. Lastly, we conduct extensive experiments to verify the efficacy of the proposed method: Meta-worid (Yu et al., 2019) and PandaGym (Gallouédec et al., 2021) sim- ulation environments and real world ALOHA robotics arm. The experiment results demonstrate the robustness of ViWo in the face of two types of viewpoint disturbance: a novel camera installation that results in 10 + 90 degrees of viewpoint offset from the training setting, and scenario with continuous camera shaking.
2 BACKGROUND
RL. and Offline RL. We consider an RI. problem aiming at learning a policy that maximizes the expected cumulative discounted reward in a Markov Decision Process (MDP) (Puterman, 1994; Sutton & Barto, 1998; Yuan et al., 2023; Cai et al., 2023b), which is represented by the tuple M = (S, A, P, R, 7). In this tuple, S denotes the state space, the action space, P S x A + S the transition function of the environment, R. S A + RR the reward function that evaluates the quality of the agent's ' action, and 7 E (0, the discount factor which balances the immediate and future rewards. A policy
S + A defines the agent's 's strategy, mapping states to a distribution over possible actions. The RI. agent interacts with the enviromment as follows: at each timestep 4, the agent observes a state 8; from the environment. It then selects action a, based on the policy π (-1sy) and executes it in the environment. Next, the agent receives a reward Ft and the environment transitions
new state s according to the transition function P( (-1ss, at). The objective of
RL is to find a policy that maximizes the expected sum of rewards over time: E, [Et20 7*rz]. In
offline RI, we only have access to pre-collected interaction data: ((5 at, rt, 144)); }):} and no further
2

---

Published as a conference paper at ICLR 2025
(A) View-invariant representation learning
(C) Policy learning
World
model
Policy
Sp. V1
View-invariant encoder
(B) World modelling
Decoder
Reconstruct
2
World model
Sp-1
Image combining S and Va information
St.
View-dspendent encoder
Gp Spa
$2, 22
control
Figure 2: Overall training framework of ReViWo method. (A) View-invariant representation learn- ing process trains two encoders to decompose view-invariant and view-dependent information within images, and a decoder to reconstruct the target image. (B) The World modeling process builds world model upon the learned VIR. (C) Policy learning by interacting with the world model.
environment interacting allowed. The objective of offline RL is to find a policy ((ar)st) that, when deployed, maximizes the cumulative reward within the confines of the MDP as characterized by the dataset.
VAE and VQ-VAE. Variational Autoencoder (VAE) (Kingma & Welling, 2014) is effective tool to extract information from the image, which learns to encode an image into a continuous latent space and subsequently reconstruct the image from this space. VAE represents an image with latent vector that captures the essential information required to reconstruct the image. We use E #xCC to denote the image, where H, W, and C represent the height, the width, and the number of channels of image o. The encoder network (z(o) maps the input image to distribution over the latent space, and the latent vector is sampled from this probability density: 2 (2lo). The decoder network po (o] aims to reconstruct an image from the latent vector. The objective of VAE minimize the negative logelikelihood as below:
(0, x) (=)0(= log (o]=)) (xi(as(= (=)o)==))))))
where the Buulback-Leibbee divergence term regularizes the latent space by enforcing similarity to prior distribution p(2), typically chosen be standard normal distribution.
Vector Quantized VAE (VQ-VAB (van den Oord et al., 2017) extends VAE by introducing a discrete latent space, which can be more suitable for information extraction. VQ- VAE introduces two main modifications: (a) The continuous latent vector is discretized by mapping it to the nearest vector This mapping results in a discrete latent variable ag. which is used for reconstruction; (b) The training objective includes codebook loss. The codebook loss consists of two terms: a commitment loss that encourages encoder outputs to commit to codebook vector, and quantization loss that moves the codebook vectors towards the encoder outputs. The overall loss function is:
learnable codebook e {eL, 62, ... (K)
CVQ-VAB(O, Φ; o) [ log (ol=)) [210]) ell2 (1aa(2)0) sg[e]12.
(2)
where sgi denotes the stop-gradient operator.
METHOD
This section presents the proposed ReViWo method. The main idea of ReViWo is decompose visual observation into view-invariant representation (VIR) and view-dependent representation (VDR) separately. In the next, give formal problem definition and then claborate on the two key steps of Ve method: (1) view-invariant representation learning and (2) world modeling and behavior learning.
3.1 PROBLEM FORMULATION
This study focuses on training a policy for visual robotic control, which is robust to viewpoint disturbances. Consider have the following datasets:
3

---

Published as a conference paper at ICLR 2025
1. A multi-view dataset O {(o"). .'....... .", ):). the set of viewpoints V. o denotes a visual observation, N the number of viewpoints, and 8, the the state presented in the image. Here we have multiple trajectories from N fixed viewpoints. In practical, we can collect this dataset by recording robotic manipulation with multiple cameras. 2. A robotic manipulation dataset D = {(s), dy ri, ss where the state s, is visual observa- tion from fixed position camera, 0g is robot behaviors and ri denotes the observed reward.
where V2 E V is specific viewpoint within
The primary objective is to derive policy that maximizes the rewards on visual observations of novel viewpoints that are not present in the manipulation dataset, thus ensuring the policy's 's decision- making capability is not limited to the viewpoints within the training data. We will provide detailed discussion in Sec. 4.1 on how to construct viewpoint disturbance in the experiments.
3.2
VIEW-INVARIANT REPRESENTATION LEARNING
The first step of ViWo is to learn VIR using multi-view images. The main idea is to separate the view-invariant and view-dependent information within the image and find a training objective to enable both two representations to be meaningful. Then the policy can make decisions based on the view-invariant information, which is robust to the viewpoint changes. ReviWo learns the representation using an autoencoder architecture, as shown in Fig. 2 (A). The framework comprises two encoders and decoder, where each encoder accounts for capturing specific information within an image, and the decoder reconstructs the images from the encoders' output.
Encoders for information extraction. ReviWo utilizes two encoders extract view-invariant and view-dependent from visual images separately. Specifically, we use ()) to represent the view-invariant encoder (VIE) and q% (o) for view-dependent encoder (VDE), where 6 denotes the parameters of the autoencoder model. The encoders implemented upon the vision transformer (ViT) architecture (Dosovitskiy et al., 2021), which processes an image by partitioning it into a sequence of patches. Each patch converted into token embedding via learnable convolutional neural network (CNN). These token embeddings are then concatenated with their corresponding positional embeddings, which are computed using the function (1 - t iimestep), to form the inputs for the transformer at each timestep. We present our implementation of ViT architecture in Fig. 9. For the VIE, the transformation can be formalized as 2, 4 (o"), where o" IR Mx
denotes the input image patches and 2, E RRMxC represents the outputted VIR. Here, the dimensions W x И denote the size of each image patch, M the patch number and C denotes the dimensions of the output feature space. Thus the output can be regarded as a sequence of features corresponding to each input patch. The VDE employs the same architecture the VIE and yields an output (o'), capturing the view-dependent features.
2. 2/ (o"),
Decoder for image reconstruction. The decoder is also a transformer-based architecture, which inputs both view-invariant and view-dependent features and outputs a reconstructed image: o ky)) The features z, and 2u could be extracted from different images within the same do- main, and the goal for the decoder is generate novel image that integrates the features of the two inputs. The decoder's input is the concatenated outputs from the two encoders. Instead of generating the image patch by patch an auto-regressive manner, we follow the Genie approach (Bruce et al., 2024) to output all patches simultaneously to enhance efficiency. Each output feature then processed by a CNN to form an image patch, with the collective patches constituting the entire image.
Training objective. The autoencoder trained to reconstruct pixels following VAE-like objective (Kingma& & Welling, 2014), which involves optimizing the parameters ø by minimizing the negative log-likelihood as in Eq. (3):
LAB(O)= =
E 02. 105m o
[- log Po
q% (o.", + x Lvv(() + & (d)). (0).
(3)
The first term, the image reconstruction loss, measures the model's ability to predict an image of given state from a novel viewpoint, by using state information from one image and viewpoint infor- mation from different image. Minimizing this term enables the model to disentangle state and view information, as it must accurately predict the target image by fusing these two distinct information types. The reconstruction of the image, 0t* conditioned on the state-related features from " trains
4

---

Published as a conference paper at ICLR 2025
the model to concentrate on certain aspects in the image that invariant to viewpoint changes, such as the state of the object. The inclusion of &, in the reconstruction encourages the model to learn viewpoint-specific features, such lighting, angle, background, and perspective, which are inde- pendent of the object's state. The second term, Lvo, is from VQ-VAE (van den Qord et al., 2017), which is known for its efficacy in image reconstruction. It comprises commitment loss that ensures the encoder's output vectors are proximate to their nearest vectors in the quantization dictionary and quantization Ioss that ensures the quantized vectors accurately represent the encoder's outputs. We refer readers to Eq. (2) for VQ-VAE loss. The term Coontastive is designed to optimize the variance of the latent representations 2g and 2n for multi-view images under variety of conditions. This term encourages 2, remain consistent across identical states and varies across different states. Similarly, ensures that 2u is consistent for identical viewpoints and varies for different viewpoints, thereby enhancing the model's s ability to capture the nuances of multi-view image representations. Refer to Appendix A for the mathematical justification that Eq. (3) can effectively separate view-invariant and fiew-indepenndet representations.
Integration of Open X-Embodiment data without view labels. In addition to the data with view labels, we also involve multi-view data without view labels from the Open X-Embodiment dataset (O'Neill et al., 2024), which are readily available on the internet. This dataset encompasses a wide arry of robotic manipulation tasks performed by various robots, thereby enriching the training of the autoencoder with broader spectrum of information and diversity. Note that in the absence of labels, the inputs to the VIE and the VDE correspond identical states observed from different viewpoints. To prevent the VDE from encoding the entirety of the image information for image reconstruction, we introduce a weighting factor the loss calculation for these unlabeled data.
3.3 WORLD MODELLING AND POLICY LEARNING FROM OFFLINE MANIPULATION DATA
View-invariant representation as the state for visual control. The previous section introduces how ReV learns the VIRs, which capture the invariant features of the environment that are essen- tial for decision-nen maaiing and are not affected by viewpoint changes. This is similar to how humans focus on the relevant aspects of task, regardless of how they are viewing the scene. By focusing on these invariant features, the policy can make decisions based on the true state of the environment rather than on extraneous visual details that may change with the camera angle. Therefore, we nat- urally regard the VIR as the task state for control. Building on this foundation, ReViWo proceeds to train a world model and policy via offline reinforcement learning.
World modeling and behavior learning. We employ the COMBO algorithm (Yu et al., 2021) for world modeling and behavior learning, which is an offline model-based reinforcement learn- ing methodology. COMBO algorithm first trains an ensemble of world models, utilizing ensemble methods (Ganaie et al., 2022) to improve the prediction accuracy and robustness of the world model. Then, the policy interacts with the world model, from an initial state that is sampled from the offline dataset, for a rollout. The policy is optimized by utilizing a combination of the real offline dataset and the simulated rollouts generated by the world model, The world model Ma (st), at), pa- rameterized by 0, is trained by minimizing the difference between the predicted next state with the target state, as shown in Eq. (4):
CwM(
(=.4..9:11 |- log (e((((()))))))))(
(4)
In addition to the world model, we also train a reward model by supervised learning on the offline control data, which is capable of predicting the rewards for given state-action pairs. This reward model provides the necessary reward signals for the policy's learning process.
EXPERIMENT
In this section, we conduct extensive experiments to evaluate our proposed method for robust control under viewpoint disturbance. The goal of the experiments is to answer the following key questions: (1) How does ReViWo perform compared to existing baseline methods under viewpoint disturbance (Sec. 4.2)? (2) How is the algorithm's 's robustness on viewpoints variations, and how much data is required to train Re ViWoSSee 4.2)? (3) Does Re ViWo learn meaningful view- niw-iivariiant and view- dependent representation (Sec. 4.3)? (4) What is the impact of each component in Re on the overall performance of the algorithm (Sec. 4.4)? We first introduce the experiment setup.
5

---

Published as a conference paper at ICLR 2025
(A) Meta-world
(B) PandaGym
(C) ALOHA
Figure 3: A visualization of the environments in our experiments. (A) In the Meta-world, the agent controls a Sawyer robot to manipulate various objects such as door, drawer and window. (B) In the PandaGym environment, the agent controls a Franka robot with 7-DoF. (C) ALOHA robot that manipulates a bottle to the plate.
4.1 EXPERIMENT SETUP
Evaluation environments. We conduct experiments on two robotics manipulation environments: Meta-world (Yu et al., 2019) and PandaGym (Gallouédec et al., 2021), shown in Fig. 3. (1) Meta- World: This environment requires the agent to control a Sawyer robotics arm with 7 degrees of freedom (DoF) and a parallel finger gripper. The action space is 2-tull. The first element of the tuple is the displacement in 3D space of the end-effector, and the second is normalized torque to be applied by the gripper fingers. Meta- World offers suite of 50 distinet manipulation tasks, covering wide array of scenarios, such as interactions with doors, windows, drawers, and balls. For our experiments, we assess the performance our methods on a subset of tasks: door opening, drawer opening, and window closing. (2) PandaGym: This environment involves a Panda robotic arm by Franka Emika, which has DoF and parallel finger gripper. PandaGym is primarily focused on block manipulation tasks, which are designed to test the robot's foundational skills, such reaching a goal point. The action space is defined by the gripper's movement command, which includes three coordinates for spatial movement, and the finger movement, which is single coordinate reflecting the gripper's aperture. For certain tasks, the gripper is fixed in a closed position, which reduces the action space only the gripper's spatial movement command. (3) ALOHA: We include a real world ALOHA robot to manipulate bottle to a plate. The task involves three stages: reaching for bottle (stage 1), grasping it (stage 2). and then placing on a plate (stage 3).
We mainly use Door Open, Drawer Open, and Window Close from Meta-world, and Reach from PandaGym for evaluation, and we also conduct experiments on Coffee Button, Faucet Open, Dial Turn (Appendix F.I). The observations on all tasks are images with 128 x 128 pixels, which are cap- tured by the fixeitposition third-person camera. To introduce viewpoint disturbance, we can modify camera parameters, such as its azimuth, pitch, and height, to alter the viewpoint. For our experi- ments, we specifically alter the azimuth value, as this adjustment alone can introduce considerable visual perturbations and is sufficient to evaluate the robustness of the proposed methods.
Dataset for training. Multi-view dataset for training autoencoder includes three parts: (1) Meta- world data. We collect 51 trajectories of 17 tasks from Meta-worke and record them utilize 20 cam- eras from different viewpoints, resulting in 1020 observation sequences (1 12040 observations) in to- tal. The trajectories sampled by pre-trained expert policies with Gaussian noise. (2) PandaGym. In this environment, we collect 30 trajectories of tasks with 20 viewpoints, resulting in 600 obser- vation sequences (30k observations) in total. (3) Open X-Embodiment (Routing Primitive Dataset (Luo et al., 2023)), including viewpoints for manipulating multi-stage cable routing tasks. This dataset does not provide viewpoint label. We utilize 101 trajectories in this dataset keep data balance, with 10064 observations in total. We refer readers to Appendix C.I for the detailed setup of camera parameters for collecting data. For offline control data, we collect 400 trajectories on Meta-world, with single viewpoint that doesn't 't exist in the multi-view data and 100 trajectories on PandaGym. Then, we train the world model and RL policy on two tasks separately. We investigate the algorithm's 's performance when collecting multi-view images on narrower range of available viewpoints, and present the result in Sec. 4.4. For real world data collection, we collect a dataset of 128 trajectories using three cameras (two third- cameras and one gripper camera). The dataset includes 10 different types of bottles and plates, which are placed randomly within the operational area during data collection. We use this real-world dataset to train both VIE and policy.

---

Published as a conference paper at ICLR 2025
Success Rate
Sircess Rate
12 2 E
Success
Success
Tranting
CIF
CSit
Traniing
CSH
Toican
CStt
Traniing
CIP
CSU
(a) Door Open
(b) Drawer Open
(c) Window Close
(d) Reach
Figure 4: Performance of different methods under various viewpoint disturbances. The x-axis de- notes the disturbance type, and the y-axis denotes the average success rate of the last two check- points, by evaluation for 30 episodes. The error bars stand for the half standard deviation over three random seeds. Results on more tasks are shown in Appendix F.I.
Viewpoint disturbance setup. In our experiments, we investigate the policy robustness under two distinct types of viewpoint disturbances: (1) Camera installation position (CIP): This category of viewpoint disturbance is designed to simulate the situation where the camera's installation position deviates from that of the policy training dataset. To this end, we adjust the camera's 's azimuth angle. Specifically, for the Meta-world experiments, we set the azimuth to a 10-degree offset, while for the PandaGym experiments, we apply 90-degree offset. This alteration is intended to evaluate the bustness policies to variations in the camera's 's installation angle. We also conduct experiments to investigate Viois performance This disturbance is relevant for scenarios with a hand-held camera, where the viewpoint can be con- tinuously changing. To mimic this, we introduce dynamic disturbance where the camera's azimuth angle is continuously adjusted during the robot's 's manipulation task. It is challenging because the model learns with fixed viewpoint during training while being evaluated on a variable one. We present the visualization of the viewpoint disturbance in Appendix C.I.
various azimuth offsets (Fig. 5). (2) Camera shaking (CSH):
Implementation Details. For autoencoder training, For world modelling and behavior learning for both ViWo and baseline methods, we utilize OfflineRL-kit (Sun, 2023), well-verified offline RL codebase. For all methods, the model is trained with an offline RL algorithm for 25000 gradient steps, and evaluated for 40 episodes. We conduct all experiments with four random seeds, and the shaded area or the error bars the figures represent the standard deviation across four trials. We use 64 CPU cores (AMD EPYC 9654 @ 2.4GHz) and 4 GPUs (NVIDIA GeForee RTX 4090) for our experiments. More implementation details can be found in Appendix C.
4.2 PERFORMANCE UNDER VIEWPOINT DISTURBANCE
Baselines for comparison. We choose the following representative approaches in the domain of multi-view robotics manipulation offline RL for comparison. (1) MVWM (Seo et al., 2023) han- dles the view disturbance by learning a multi-view masked autoencoder which reconstructs pixels of randomly masked viewpoints. It then learns world model operating the representations from the autoencoder. In MVWM's original implementation, the world model is learned in an online setting. We align it with our setting by training only with offline data and utilizing COMBO train both world model and policy. (2) COMBO (Yu et al., 2021) is a model-based offline RI algorithm that trains a value function using both the offline dataset and the world model generated data while also additionally regularizing the value function on out-of-support state-action tuples generated via model rollouts. In the experiments, the COMBO method first pre-trains the vision representation on the same multi-view data as ViWo using standard VAE and then trains a policy that makes decisions based on the learned vision representation. (3) Behavior Cloning (BC) adopts supervised learning approach to mimic the actions within the offline dataset. Similar to COMBO, we also use VAE to process the visual observation when implementing the BC baseline. All these baseline meth- ods use the same multi-view dataset to pre-train vision representation with VAE. (4) Conservative Q-Learning (CQL) (Kumar et al., 2020) is an offline RL method. In our experiment Sec. 4.4, we utilize CQL to train policy with the leamned VIR, to verify the effect of the world model in Re Vo..
Main results. Fig. 4 presents the success rate of different methods under various levels of viewpoint disturbance. In general, proposed ViWo method outperforms the baseline methods across two types of viewpoint disturbances. While the baselines demonstrate considerable performance from the training viewpoint, they suffer from clear decline when subjected viewpoint variations.

---

Published as a conference paper at ICLR 2025
This conclusion is supported by COMBO performance (23.3 + 0 on Door Open, 94.4 0 on Drawer Open, and 98.9 - 22.8 on Window Close), which demonstrates the influence of viewpoint disturbance on these visual-based policies. BC shows promising performance viewpoints, yet its success rate substantially decreases under viewpoint disturbances. This perfor- mance degradation is due to the limitations of supervised learning on offline control data, which fails to preserve its robust performance when encountering novel observations. In contrast, VeViWo not only performs comparably to the training viewpoints and the CIP on tasks such as Door Open, but also demonstrates its robustness to viewpoint disturbances. This robustness is further evidenced by Re s ability to handle tasks with a camera shaking, which is a significant challenge for policies to perform robustly (Seo et al., 2023). Interestingly. ReViWo achieves the highest performance on the training viewpoint for certain tasks. We hypothesize this is due to that VIR focuses on task- related domains within the observation and masks some unrelated information, thereby enhancing the policy learning for specific tasks.
certain training
Impact of training data variations & Performance under various viewpoint offsets. In this experiment, we study two important questions: the quantity of data necessary for training the au- toencoder using Re ViW, and the degree of its robustness changes in viewpoint. We train the autoencoder using images collected from Meta-world, employing two configurations: one with 10 random viewpoints across a 90-degree azimuth range (10V+90D), and another with 20 random view- points over a 180-degree azimuth range 20V+180D) For evaluation, we evaluate the algorithm's performance across various degrees of azimuth offset, ranging from 0 to 15 degrees. Fig. 5 shows the comparative performance of ViWi trained with different data against various levels of camera viewpoint offset. Key findings include: (1) 10V+90D setting is enough for ReVi to achieve a good performance, while additional data can further improve the algorithm's 's efficacy. For instance, the performance of Re Vo with 10V+90D is comparable to that with 20V+180D when the azimuth offset is < 10, and only marginally inferior when the offset >> 10. In contrast, the COMBO method has noticeable decline under even slight disturbance (e.g.. 98.9 + 55.6 when the offset is 2.5); (2) ViWo can effectively maintain the policy performance even when offset reaches 15 degrees, which is a considerable range during the practical robotic deployment. These insights serve as a guideline for applying Re ViWo, and highlight its adaptability viewpoint variations.
Real world evaluation. Tab. presents the real world experiment results. We implement Re VoW- BC as follow: training VIE the real world images and conducting behavior cloning on the control data with VIE representation. In this experiments, the policy inputs with all three camera images. To simulate viewpoint disturbances, we adjusted the azimuth by +15 degrees and the elevation by -15 degrees on two third-person cameras. We evaluate the method on both training and CIP viewpoint for 10 trajectories with random initialization. We also consider a baseline, ACT (Action Chunk- ing Transformer) (Zhao et al., 2023), for comparison. ACT performs end-to-end imitation learning directly from real demonstrations, with pre-trained ResNet-50 as the vision encoder. The results indicate that the proposed method maintains robust control when subjected to novel viewpoints in a real-world setting, while imitation learning-basse method suffers from a performance degrada- tion Cai et al. (2023c;a). This result serves preliminary evidence of the applicability of Re- ViWo method. In contrast, ACT performs comparably to ReViWo-BC on training viewpoint, while exhibiting significant decline in performance when viewpoint changes. These real world results further substantiate the efficacy of the ViWo in handling viewpoint disturbances.
Success Rate
ReViWo-BC (Training)
ReViWo-BC (CIP)
ACT (Training)
ACT (CIP)
Stage 1
100%
100%
100%
100%
Stage 2
80%
60%
60%
0%
Stage 3
60%
50%
60%
0%
Table 1: Results on real world ALOHA robot.
4.3 ANALYSIS ON VIEW-INVARIANT REPRESENTATION LEARNING
Previous results demonstrate ReViWo can effectively handle common viewpoint disturbance com- pared to baseline methods. In this section, we investigate the source of the robustness and analyze the quality of the leamed view-invariant representation.
Analysis on the generated VIR. We analyze the effectiveness of the learned view-invariant repre- sentation by projecting the resulting representations onto a two-dimensional plane using t-SNE (der
8

---

Published as a conference paper at ICLR 2025
100
Success Rate (%)
Method
w/ Open-X
w/o Open-X
Task
Door Open
CIP
42.2
22.8
(20V-1880)
CSH
19.4
25.0
Ve ((0)))90))
Drawer Open
CIP
55.0
30.6
15
CSH
37.8
25.6
Azimuth Offsct (Degree "
Figure 5: levels of viewpoint offsets.
Performance across differ-
Table 2: Ablation study of Re ViWo that trains w/ and w/o Open X-Embodiment dataset.
bey.
(a) ReViWo
(b) MVWM
(C) VAE
Figure 6: The t-SNE projections of representations generated by various methods. Points with the same color encode six images for the same state but with six different novel viewpoints that do not exist in the multi-view datasets. The presented Re ViWo representation is generated by VIE.
Maaten al., 2008). We first sample the state from the Merl-World environment, from which we render a series of images from multiple viewpoints. Subsequently, we encode these images using the VIE, well as several baseline models for comparison. The t-SNE projections of these encoded images are illustrated in Fig. 6, where each color of points correspond to the representations of the same state from various viewpoints. We observe a notable clustering VIR points, which are in closer proximity each other as compared to those of the baseline models. In contrast, the MVWM and VAE representations exhibit greater diversity compared ReViWo. These results suggest their representations significantly different across viewpoints, potentially distracting the robust pol- icy execution. This finding indicates the learned VIR is more adept at capturing the essential state information of task, even in the presence of novel viewpoints.
Examples of the decoder output. To investigate whether the trained autoencoder learns meaningful representation, present the decoded images when VIE and VDE from different images, as shown in Fig. 7. We have several conclusions from the results. First, VIR correctly focuses on the task state, which consistent despite changes in viewpoint. For example in each row in Fig- 7, the decoder can generate the images for the same state from two viewpoints. Second, the view- dependent representation indeed captures the property of viewpoint, thus facilitating the generation of the image with the same viewpoint as the viewpoint reference image. We present more examples of decoder output in Appendix F.3. Besides, the trained autoencoder can applied in different domains (i.e., Meta-word world and PandaGym).
4.4 ABLATION STUDY AND APPLICABILITY ANALYSIS
Effect of the Open X-Embodiment data. To evaluate the impact of Open X-Embodiment data on the efficacy our proposed method, ReViWo, we conducted comparative analysis of the model's performance with and without the integration of this data. The results of this analysis are detailed Tab. The integration of Open X-Embodiment data into ReViWo significantly enhances the model's robustness to variations in viewpoint, as shown in the experiment results. This could be attributed to that the diverse visual representations in the Open X-Fmbodiment data contribute to more comprehensive understanding of the world by the model, as well better understand- ing of different viewpoints. These results imply a potential to scale up the Ve training for more challenging robotic manipulation. Note that the performance decline Door Open (CSH) can be attributed to the diverse and unstructured nature of the Open X- X-Embodiment dataset, which introduces variability that the model struggles generalize under dynamic conditions.
Effect of the world model. We verify the effect the world model by training a policy with the learned VIR and offline RI. method, COL (Kumar et al., 2020), the offline RL dataset. Tab.

---

Published as a conference paper at ICLR 2025
VIE
VDF.
Decoder
Ground Truth
VIF
VDE
Decoder Output
Ground
Input
Input
Output
Input
Input
Truth
Figure 7: Examples of the decoder output the corresponding ground truth image. The decoder can generate the images that combine the task state in VIE inputs, and the viewpoint in VDE inputs.
3 presents the performance of ReViWo method with and without world model (WM). The results present a consistent performance enhancement for ReViWo w/ WM, suggesting that the imaginary rollouts generated by the WM contribute to the enhancement of both the robustness and the overall learning efficiency the ViWi framework. The reduced performance on Drawer Open (CSH) is likely due to the world model's difficulty in accurately predicting the next state in highly variable and unstable visual conditions, leading to less reliable policy execution. Notably, the Vi w/o WM still surpasses the performance of baselines like COMBO (result in Sec. 4). This finding reveals the efficacy of the VIR in facilitating robust control, independent of the world model.
Task
Drawer Open
Window Close
Method
Training 92.8
CIP
CSH
Training
CIP
CSH
w/ WM w/o WM
30.6
25.6
98.9
71.4
23.3
60.0
28.3
27.2
97.2
68.3
17.8
Table Performance Re ViWo w/ and w/o world model (WM). We implement ViW w/o WM by training policy with CQL and using the learned iie-invvariant representation.
CONCLUSION AND LIMITATION
This study explores robust robotic manipulation under viewpoint disturbances. We propose novel approach, ReViWo, which learns viewpoint-invariant representations that are subsequently lever- aged for robotic control. We conduct extensive experiments and demonstrate that Re ReViWo is capa- ble being applied to a variety of robotic platforms, enabling robust control under various types of viewpoint disturbance. Despite the promising results, there are still limitations. One limitation of our work is the reliance on the labeled viewpoints. In a practical setting, these labels can be gener- ated by sampling multiple trajectories using fixed-position cameras. However, it is worth exploring to eliminate this dependency and to leverage a broader spectrum of multi-view data from the real world. A potential solution train with label-free multi-view data and constrain the information contained within VDR, by information bottleneck (Tishby & Slonim, 2000) technique or by masking portions of the image input the VDR, thereby preventing the VDR contains too much informa- tion. Besides, the experiment scale is limited, in terms of the dataset scale and model size. In future works, we hope to scale up the framework solve more challenging tasks. For instance, employing pre-trained model such as CLIP (Radford et al., 2021), and training with more data from Open X-Embodiment and other open multi-view data (Yu et al., 2023). Lastly, currently we implement the world model based ulation tasks with more complex dynamics, it would be beneficial to use more powerful structures to build the world model, e.g., recurrent state-space model (Hafner et al., 2019). We believe these interesting directions are worth further exploration for developing smarter and more robust robots with the support of world models and reinforcement learning.
simple multi-layer perceptron architecture. To deal with robot manip-
10

---

Published as a conference paper at ICLR 2025
AUTHORSHIP AND CREDIT ATTRIBUTION
The contributions of the authors are as follows:
Y. Y.: led the direction, oversaw the project;
J.-C. P:: designed the method;
N. proved the theory;
N. T.: implemented the codes;
J.-C. P.: designed the experiments;
K. L.:: processed the data;
N. T., K. L.: conducted experiments;
J.-C. P., K. L., T.: analyzed the results;
J.-C. P., X. Q. C., Y. Z.-Y. Z.: wrote the paper:
Y. Y., M. S., G. N.: revised the paper;
J.-C. P., G. N.:
discussed and wrote the response;
. Y. Y., M. S., G. N.: provided guidance.
All authors discussed and contributed to the final manuscript.
ACKNOWLEDGMENT
This work was supported by the National Science Foundation of China (62495093) and Jiangsu Science Foundation (BK20243039). The authors would like thank Zhilong Zhang, Junyin Ye and Haoxiang Ren for their assistance in real world deployment, and anonymous reviewers & chairs for their valuable comments.
Masashi Sugiyama was supported by JST ASPIRE Grant Number JPMJAP2405 and by grant from Apple, Inc. Any views, opinions, findings, and conclusions or recommendations expressed this material are those of the authors and should not be interpreted as reflecting the views, policies or position, either expressed or implied, of Apple Inc.
ETHICS STATEMENT
In the development and evaluation of ReViWo for robotic manipulation, we have carefully con- sidered the ethical implications of this research, particularly they pertain the of robotic manipulation tasks and artificial intelligence. The proposed involves the collection and use multi- view data in simulators and Open X-Embodiment datasets and is designed to respect privacy and ensure the security of data. The datasets used do not contain any personal or sensitive information, and data collection processes comply with relevant legal standards and best practices research ethics. The potential for bias and discrimination has been addressed by ensuring that the VIR does inadvertently encode any biased representations the environment. This is particularly impor- tant in maintaining fairness and avoiding any form of discrimination that could arise from biased training data. The research been conducted with a commitment to research integrity, including thorough documentation and adherence to IRB guidelines where applicable. We recognize that the insights and methods presented in this paper must be applied responsibly, avoiding any potentially harmful applications. The technology developed is intended for beneficial purposes and should not be used in ways that could cause harm or diminish the safety of individuals. The experiment re- sults are reported with the transparency and accuracy, reflecting our commitment advancing knowledge in the field of robotic manipulation while upholding the highest ethical standards.
11

---

Published as a conference paper at ICLR 2025
REFERENCES
Rishabh Agarwal, Dale Schuurmans, and Mohammad Norouzi. An optimistic perspective on offline reinforcement learning. ICML, 2020.
Ron Alterovitz, Sven Koenig, and Maxim Likhachev. Robot planning in the real world: Research challenges and opportunities. Al Magazine, 37(2):76-8,, 2016.
Dor Bank, Noam Koenigstein, and Raja Giryes. Autoencoders. Machine learning for data science handbook: data mining and knowledge discovery handbook, pp. 253-374, 2023.
Anthony Brohan, Noah Brown, Justice Carbajal, and al Yevgen Chebotar. RT-1: robotics trans- former for real-world control at scale. In RSS, 2023.
Jake Bruce, Michael D. Dennis, Ashley Edwards, Jack Parker-Holdes Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, Yusuf Aytar, Sarah Bechtle, Feryal M. Behbahani, Stephanie C. Chan, Nicolas Heess, Lucy Gonzalez, Simon Osindero, Sherjil Ozair, Scott E. Reed, Jingwei Zhang, Konrad Zolna, Jeff Clune, Nando de Freitas, Satinder Singh, and Tim Rocktäschel. Genie: Generative interactive environments. In ICML, 2024.
Xin-Qiang Cai, Yao-Xiang Ding, Zi-Xuan Chen, Yuan Jiang, Masashi Sugiyama, and Zhi-Hur Zhou. Seeing differently, acting similarly: Heterogeneously observable imitation learning. In ICIR, 2023a.
Xin-Qiang Cai, Pushi Zhang. Li Zhao, Jiang Bian, Masashi Sugiyama, and Ashley Llorens. Distri- butional pareto-optimal multi-objective reinforcement learning. In NeurIPS, 2023b.
Xin-Qiang Cai, Yu-Jie Zhang, Chao-Kai Chiang, and Masashi Sugiyama. Imitation learning from vague feedback. In NeurlPS, 2023c.
Van der Maaten, Laurens, and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9:2579-2605, 2008.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image worth 16x 16 words: Transformers for image recognition scale. In ICLR, 2021.
Frederik Ebert, Chelsea Finn, Sudeep Dasari, Annie Xie, Alex X. Lee, and Sergey Levine. Vi- sual foresight: hodel-based deep reinforcement learning for rision-based robotic control. arXiv preprint, abs/1812.00058, 2018.
Quentin Gallouédec, Nicolas Cazin, Emmanuel Dellandréa, and Liming Chen. panda-gym: Open- source goal-conditioned environments for robotic learning. arXiv preprint, abs/2106. 13687, 2021.
M. A. Ganaie, Minghui Hu, Ashwani Kumar Malik, Muhammad Tanveer, and Ponnuthurai N. Sug- anthan. Ensemble deep learning: A review. Engineering Applications of Artificial Intelligence, 115:105151, 2022.
David Ha and Jürgen Schmidhuber. World models. In NeurlPS, 2018.
Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha. Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. In ICML, 2019.
Danijar Hafner, Timothy P. Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learn- ing behaviors by latent imagination. In ICLR, 2020.
Danijar Hafner, Timothy P. Lillicrap, Mohammad Norouzi, and Jimmy Ba. Mastering atari with discrete world models. In ICLR, 2021.
Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy P. Lillicrap. Mastering diverse domains through world models. arXiv preprint, abs/2301.04 .04104, 2023.
12

---

Published as a conference paper at ICLR 2025
Dmitry Kalashnikov, Alex Irpan, Peter Pastor, Julian Ibarz, Alexander Herzog, Eric Jang, Deirdre Quillen, Ethan Holly, Mrinal Kalakrishman, Vincent Vanhoucke, and Sergey Levine. Qt-opt: Scalable deep reinforcement learning for vision-based robotic manipulation. arXiv preprint, abs/1 0as/1806.11 10293, 2018a.
Dmitry Kalashnikov, Alex Irpan, Peter Pastor, Julian Ibarz, Alexander Herzog, Eric Jang, Deirdre Quillen, Ethan Holly, Mrinal Kalakrishnan, Vincent Vanhoucke, and Sergey Levine. Scalable deep reinforcement learning for vision-based robotic manipulation. In CoRI., 2018b.
Yue Kang. Explorations in Stochastic Bandit Problems: Theory and Applications. ProQuest Dis- sertations & Theses. University of California, Davis, 2024.
Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In ICLR, 2014.
Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. Conservative q-learning for offline reinforcement learning. In NeurIPS, 2020.
Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tuto- rial, review, and perspectives on open problems. arXiv preprint, abs/2005.00 0146,,, 2020.
Fanfan Liu, Feng Yan, Liming Zheng, Chengjian Feng, Yiyang Huang, and Lin Ma. Robouniview: Visual-language model with unified view representation for robotic manipulaiton. arXiv preprint, abs/2406. 01172440..8977, 2024.
Jianlan Luo, Charles Xu, Xinyang Geng, Gilbert Feng, Kuan Fang, Liam Tan, Stefan Schaal, and Sergey Levine. Multi-stage cable routing through hierarchical imitation learning. arXiv preprint, abs/2307.08927, 2023.
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Belle- mare, Alex Graves, Martin A. Riedmiller, Andreas Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wier- stra, Shane Legg, and Demis Hassabis. Human-level control through deep reinforcement learning. 2015.
Nature, 518(7540):559-555
Abby O' O'Neill, Abdul Rehman, Abhiram Maddukuri, Abhishek Gupta, and et al Abhishek Padalkar. Open x-embodiment: Robotic learning datasets and RT-X models: Open x-embodiment collabo- ration. In /CRA, 2024.
Jing-Cheng Pang, Tian Xu, Shengyi Jiang, Yu-Ren Liu, and Yang Yu. Reinforcement learning with sparse-executing actions via sparsity regularization. arXiv preprint, 0bs/2105.08666, 2021.
Jing-Cheng Pang, Si-H Si-Hang Yang, Xiong-Hui Chen, Xinyu Yang, Yang Yu, Mas Ma. Ziqi Guo, Howard Yang. and Bill Huang. Object-oriented option framework for robotics manipulation in clutter. In IROS, 2023a.
Jing-Cheng Pang, Xinyu Yang, Si-Hang Yang. Kiong-Hul Chen, and Yang Yu. Natural language instruction-folllwiing with task-related language development and translation. In NeurIPS, 2023b.
Ling-Cheng Pang, Si-Hang Yang, Kaiyuan Li, Jiaji Zhang, Xiong Hui Chen, Nan Tang, and Yang Yu. Knowledgeable agents by offline reinforcement learning from large language model rollouts. arXiv preprint, abs/2404.09248, 2024.
Martin L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wi- ley Series in Probability and Statistics. Wiley, 1994.
Deirdre Quillen, Eric Jang, Ofir Nachum, Chelsea Finn, Julian Ibarz, and Sergey Levine. Deep reinforcement learning for vision-based robotic grasping: A simulated comparative evaluation of off-policy methods. ICRA. 2018.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning transferable visual models from natural language supervision. ICML, 2021.
13

---

Published as a conference paper at ICLR 2025
Nikunj Saunshi, Orestis Plevrakis, Sanjeev Arora, Mikhail Khodak, and Hrishikesh Khandeparkar. A theoretical analysis of contrastive unsupervised representation learning. In ICML, 2019.
Edward Schmerling, Karen Leung, Wolf Vollprecht, and Marco Pavone. Multimodal probabilistic model-based planning for human-robot interaction. In ICRA, 2018.
Younggyo Seo, Junsu Kim, Stephen James, Kimin Lee, Jinwoo Shin, and Pieter Abbeel. Multi-view masked world models for visual robotic manipulation. In ICML, 2023.
Pierre Sermanet, Corey Lynch, Yevgen Chebotar, Jasmine Hsu, Eric Jang, Stefan Schaal, and Sergey Levine. Time-contrastive networks: Self-supervised learning from video. In ICRA, 2018.
David Souto and Dirk Kerzel. Visual selective attention and the control of tracking eye movements: critical review. Journal of Neurophysiology, 125(5):1522-1576, 2021.
Yihao Sun. Offlinerl-kit: An elegant pytorch offline reinforcement learning library, 2023.
Richard Sutton and Andrew G. Barto. Reinforcement learning: introduction. IEEE Transac- fions Neural Networks, 95):1055-1004,, 1998.
Naftali Tishby and Noam Slonim. Data clustering by markovian relaxation and the information bottleneck method. In NeurlPS, 2000.
Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojeiech Zaremba, and Pieter Abbeel. Do- main randomization for transferring deep neural networks from simulation to the real world. In IROS, 2017.
Aäron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learn- ing. In NeurIPS, 2017.
Zhao-Heng Yin, Lingfeng Sun, Hengbo Ma, Masayoshi Tomizuka, and Wu-Jun Li. Cross domain robot imitation with invariant representation. In ICRA, 2022.
Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In CoRL, 2019.
Tianhe Yu, Aviral Kumar, Rafael Rafailov, Aravind Rajeswaran, Sergey Levine, and Chelsea Finn. COMBO: conservative offline model-based policy optimization. In NeurIPS, 2021.
Xianggang Yu, Mutian Xu, Yidan Zhang, Haolin Liu, Chongjie Ye, Yushuang Wu, Zizheng Yan, Chenming Zhu, Zhangyang Xiong, Tianyou Liang, Guanying Chen, Shuguang Cui, and Xi- aoguang Han. Mvimgnet: A large-scale dataset of multi-view images. In CVPR, 2023.
Lei Yuan, Ziqian Zhang, Lihe Li, Cong Guan, and Yang Yu. A survey of progress on cooperative multi-agent reinforcement learning in open environment. arXiv preprint, abs/2312.01058. 2023.
Andy Zeng, Shuran Song, Stefan Welker, Johnny Lee, Alberto Rodriguez, and Thomas A. Funkhouser. Learning synergies between pushing and grasping with self-supervised deep re- inforcement learning. In IROS, 2018.
Tony Z. Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-graine grained bimanual manipulation with low-cost hardware. In RSS, 2023.
Brianna Zitkovich, Tianhe Yu, Sichun Xu, and et al Peng Xu. RT-2: vision-language-action models transfer web knowledge robotic control. In CoRL, 2023.
14

---

Published as a conference paper at ICLR 2025
Appendix
Table of Contents
A Mathematical Justification for Representation Learning
16
B Related Work
17
More Implementation Details & Experiment Setup
Viewpoints for Autoencoder and RI. Training
C.2 Architecture and Optimization Details
................................
18
C.3 Computation. ................................ ..a.eaaaiii.....................
18
C.4 Hyperparameters.
19
C.5 Detailed Descriptions of Baselines
19
D Algorithm Description
21
E Discussion on the Usage of Dataset with View Labels
21
F Additional Results
22
E1 Results on More Tasks
22
F2 Imitation Learning with View invariatt Representation
22
F3 More Examples Decoder Output
24
15

---

Published as a conference paper at ICLR 2025
A MATHEMATICAL JUSTIFICATION FOR REPRESENTATION LEARNING
Here we provide a mathematical explanation to justify why ReViWo could separately learn the view- invariant and view-dependent representations from the learning objective in Eq. (3), The intuition is that the disentanglement can be achieved by minimizing the InfoNCE losses, which equals to the contrastive term in Eq. (3).
Denote V. L as the random variable of the viewpoint and the scene occupancy, Zu- Z as the VDE and VIE. I(V; L) = O, where I denotes the mutual information. Each RGB image X corresponds to a unique pair (V, L). i.e. = X(V, L.)
obvious that the viewpoint is independent of the occupancy, which means that
During training, we sample a training image batch with size (N, N.) at each step. In this sample matrix, the samples of the same row are sampled from the same state and the samples of the same column are sampled from the same viewpoint.
We use .t) as the similarity metric, and for certain VDE Eps denote zy its positive sample and *k its negative samples, the InfoNCE loss for VDE is:
Fu
Ni
N
log exp(f(25) ))
LVDDFVDD
N No
k== (fp(f(e)" 2())
log exp(s(Ev: 2p))
By Pa Pravaa
Σ expls (cpp((Zes 211)
exp(f(2n:24 2(²))
2 P.arn
log N. [exp(] ( [1 =())]l
2 ByNP 1- I (lar (1((,))))))) [-](zu =0']] + Pe log( (1 (9)
(1=()119))
eex( (Zor z 2
log N
f 2y) +
f(err 20) + log N.
(5)
The last inequality in Eq. (5) is derived from Jenson inequality. We set f(zus 2/0) log Pizer x::)
log P(z.) + log P(z (z0), i.e. the energy function, we have:
Lnce(V (E) 2 Ey- su
|- [f =p]]
B ( (1r((,
.())
f(zv 2/)
+ +og N
E P
log P(zx) log P(z [u)]] +
En I-P Pe.PP Pr
log P(z.) + log P(z:) + log
H(Zzlu) (Z,) log
= =-((>,; V) + log N.
We justify that f(2e, 2 can be represented log P(zy, 2,),, which allows relate the similarity metric to probabilistic terms. This relationship crucial for interpreting the InfoNCE loss in terms of entropy and mutual information. Similarly, we obtain:
Luce(VIE) 2 - -1 L) + N,
We then prove that the distance metric d(X, Y), defined in terms of mutual information, satisfies the triangle inequality. This metric is used to measure the disentanglement between Z, and Z₁. We denote
I(X:Y)
d( =1 max (H)) HY))
16

---

Published as a conference paper at ICLR 2025
and X2,", ,,,) - mar(H(X)); .. .H()))).
According to triangle inequality, we have:
d(V,L) - d(Z V) + d(Z,, L) + d(Zu, 21)
(8)
which can be simplified as:
2 I(V:
I(L; ZL)
I(Zo; Zi) S A(Z Zi)
A(V, 2n)
A(L, Zr.)
I.)
I(L; ZL)
< A(z,, Zr V, 2 A(V, Zu, L, Zr)
A(L, Zt, V, (..
= 2 oZ 2,, L) - I(V; Zo) I(L; Zi)
2 A(Z., Z, L) + Lue((VDE) + Leco(VIE) log N.
From Eq. (9), imizing the upper bound of the mutual information between the view-invariant and view-dependent representations.
that minimizing A(Z,,, Z,,, L), Ieee(VEE) and (Ve))VD)) equals min-
Based on the Data Processing Inequality, we show that A(Z.: Zı, L) is a constant C, which represents the maximum entropy among the variables. This conclusion based on the fact that processing data cannot increase mutual information:
A(Z,, Zı, L) mar
H())
mar HL))
(10)
C.
Finally, we establish a clear connection between the InfoNCE loss and mutual information. Mini- mizing the InfoNCE losses for Z,, and Z₁ corresponds minimizing the mutual information between these representations, thus facilitating their disentanglement.
B RELATED WORK
Robotic manipulation with visual inputs has become prominent field of study, leveraging cam- eras to capture environmental data and employing carning-based methods, such as reinforcement learning, to train control policies (Zeng et al., 2018; Kalashnikov et al., 201 1b; Quillen et al., 2018; Ebert al., 2018; Pang al., 2023a;b; 2021). For instance, the work of VPG (Zeng et al., 2018) utilizes the Deep Q-Network algorithm (Mnih et al., 2015) to train policy for block grasping task. Similarly, QT-Opt (Kalashnikov et al., 2018a) has demonstrated the feasibility of real-world grasping by introducing sulfrvuuerei vision-based reinforcement learning framework.
critical challenge in training policies with visual inputs the issue viewpoint disturbance, the camera position changes during deployment. Prior research has investigated a variety of meth- ods to mitigate the effects of viewpoint changes. A prevalent technique is domain randomization (Tobin et al., 2017), which augments the training data with synthetic variations to emulate poten- tial environmental disturbances. Another approach is the adoption of unified view representations (Liu et al., 2024; Sermanet et al., 2018; Yin et al., 2022). For instance, RoboUni View (Liu al., 2024) and TCN (Sermanet et al., 2018) aim to learn a unified representation for images captured from disparate viewpoints. However. the enforcement of viewpoint invariance, which presumes the same information across different viewpoints, necessitates the careful selection of positive and neg- ative data pairs. This is akin to the challenges faced by contrastive learning methods, which often rely on complicated choices with regard to the sampling of such pairs (Saunshi et al., 2019; Seo et al., 2023). Imitation learning methods, such as RT-1 (Brohan et al., 2023), RT-2 (Zitkovich et al., 2023), and RT-X (O "eileil et al., 2024), seek to learn robust control by leveraging extensive manip- ulation datasets, but this approach inevitably increases data processing and computational demands. While these methods have shown certain level of success in handling viewpoint disturbances, they fail to isolate view-invarian information, which is essential for robust robotic control Kang (2024), from the viewpoint information. In contrast, this work proposes novel approach to this challenge.
17

---

Published as a conference paper at ICLR 2025
We introduce an autoencoder framework with dual encoders, designed to decompose wow-innvri information, which is shown to be effective in robust robotic control.
World Models for Robotics Manipulation. Model-based RL trains world model that learns the environment dynamics through supervised learning. In robotics, world models are crucial for enabling robots predict the outcomes of their actions. There are common ways utilize the world model in robotics: planning (Alterovitz et al., 2016) and policy learning (Ha & Schmidhuber, 2018). For example, Schmerling et al. (2018) proposes model-based planning for numan-machinn interaction. Ha & Schmidhuber (2018) introduces the concept of world models in the context of reinforcement learning, demonstrating that compact and efficient representations of the environment can be learned, which in turn can be used to train agents with fewer interactions with the real world.
Due to the challenge in modeling the environment with image as input/output, recent works pro- pose to separate the representation learning and world modeling (Hafner et al., 2020; 2021; 2023). However, these world models seldom consider the viewpoint disturbance. In this work, we follow the representation learning world modeling framework but attempt to learn view-invariant world model that is robust to the viewpoint disturbances.
C MORE IMPLEMENTATION DETAILS & EXPERIMENT SETUP
C. VIEWPOINTS FOR AUTOENCODER AND RL TRAINING
To construct the dataset to train the autoencoder, we use a camera configuration that encom- passes a diverse set of azimuth angles. Specifically, the azimuth angles employed in Meta- world are {8, 11, 17, 20, 28, 35, 136, 144, 148, 152, 159, 180, 194, 195, 198, 219, 324, 333, 339, 351}. Note that these angles
selected to span
180-degree range, encompassing two distinct 90-degree intervals: [-45, 45] and [135, 225] degrees, respectively. For PandaGym, the azimuth angles are {6, 21, 39, 60, 69, 85, 114, 125, 153, 159, 162, 170, 179, 205, 209, 213, 217, 224, 238, 244] se- lected from interval of [0, 270] degrees.
For RL training, we select a distinct azimuth angle of 22.5 degrees for the viewpoint of MetaWorld and 45 degrees for the viewpoint of PandaGym. This angle is set be different from any of the azimuth angles used for training the autoencoder to ensure diverse and comprehensive learning. This choice of viewpoint designed to evaluate the generalization capabilities of the RL algorithm by exposing it to a novel viewpoint that is not encountered during the autoencoder training phase. We present the visualization of different viewpoints in Fig. 8.
C.2 ARCHITECTURE AND OPTIMIZATION DETAILS
The autoencoder architecture is based on the ViT (Dosovitskiy et al., 2021). The architecture of the encoder and decoder are presented in Fig. We first utilize a one-layer CNN to project the input image with size 128 * 128 into patch embeddings where the patch size is 16 16. We employ bidirectional transformer structure for both encoder and decoder block, consisting of 8-head self- attention mechanisms and feed-forward networks with layer normalization and residual connections included. The transformer's 's intermediate embedding dimension 256 and the dropout rate is set 0.1. For VIE, we utilize vector quantizer with a codebook size of 512 and an embedding dimension of 64 to obtain the view-invariant encoding. To prevent codebook collapse, we utilized k-means initialization for our vector quantizer. As for VDE, we introduce simple linear layer to get the view- dependent encoding. The view-invariant encoding and view vie-ddepnndent encoding are concatenated and fed into the decoder block, after which the decoder output is projected into the image space through a lne-layer deconvolution.
C.3 COMPUTATION
We use 64 CPU cores (AMD EPYC 9654 @ 2.4GHz) and 4 GPUs (NVIDIA GeForce RTX 4090) for our experiments. The software stack employed for our experiments includes Python 3.11 and PyTorch 2.1.0. On average, the training of the autoencoder to the point of convergence our dataset
18

---

Published as a conference paper at ICLR 2025
aa) Meta-world (training) (h) Meta-world (offset=10) (c) Meta-world (offset=20)
PandaGym (training)
PandaGym (offset=45) (f) PandaGym (offset=90)
Figure 8: Visualization of the training viewpoint and novel viewpoint for robotics control. In experiments, Meta-world (offiet=10) and PandaGym (offset=90) are utilized for evaluating policy performance under CIP.
Encoder
Patch image
Decoder
Catenate encoders'
output
Encoder
(2) Decoder
Figure 9; Network architecture of encoder and decoder of ViWo.
requires approximately 12 hours. Subsequently, the training of the world model and the policy completed in an average of 2 hours.
C.4 HYPERPARAMETERS
The hyper-parameters for implementing ReViWo are presented in Table 4. When implementing baseline methods, we use the same hyper-parameters of offline RL for training the world model and policy.
C.5 DETAILED DESCRIPTIONS OF BASELINES
MVWM (Seo et al., 2023): MVWM is reinforcement learning framework that trains a multi-view masked autoencoder for representation learning and a world model to solve visual manipulation tasks. The autoencoder consists of synergistic combination of view-masking: which masks view- points at random, and video autoencoding: which reconstructs video frames both masked and unmasked viewpoints. In Seo et al. (2023), the authors find that the autoencoder effectively leams representations that capture useful information from the current viewpoint but also the cross- cross-view view information from different viewpoints. For behavior learning, MVWM learns world model
19

---

Published as a conference paper at ICLR 2025
Table 4: Hyper-parameters for training ViWo and basclines.
Hyper-parameters
Value
Patch Num.
8 x 8 64
Embedding dimension for each patch
256
Encoder output dimension
64
Enocoder block Num.
8
Decoder block Num.
8
Coef. for VQ loss
0.25
Coef. for VDE contrastive loss
0.1
Coef. for VIE contrastive loss
1
Learning rate of authencoder Optimizer for autoencoder Learning rate of world model Optimizer for world model Learning rate for policy Optimizer for policy World model structure World model ensemble Num. Policy structure
3e-5
Adam
le-3
Adam
le-4
Adam
[imput dim, 2048, 512, 256, 256, output dim]
7
[imput dim, 2048, 512, 256, 256, output dim]
frozen representations from either single-view or multi-view data, which is particularly feasible as the autoencoder consists of vision transformer layers that take inputs of variable sizes. Then the actor and critic are trained with imaginary trajectories from the world model. Although MVWM is originally implemented for online RI., it be easily adapted to offline settings by combining its representation learning technique with offline model-based RI. (Agarwal et al., 2020).
COMBO (Yu et al., 2021): COMBO is novel approach in the field of reinforcement learning, designed to effectively learn policies from offline data without further interaction with the environ- ment. It addresses the challenge of distributional shift, where the policy encounters states that not well-represented in the offline dataset, which can lead to poor performance or even catastrophic failures when deployed. COMBO integrates conservative policy evaluation with model-based plan- ning, using learned dynamics model to simulate future states and rewards. By combining conser- vative value estimation and model-based rollouts, COMBO aims to safely optimize policies while mitigating the risks associated with out-of-distribution states. This approach allows for more ro- bust policy learning from static datasets, expanding the applicability of reinforcement learning to scenarios where active data collection is impractical impossible.
CQL (Kumar et al., 2020): CQL is an algorithm developed for offline reinforcement learning, where the goal is to learn effective policies from fixed dataset without further interaction with the envi- ronment. The key challenge in offline reinforcement learning is to avoid overestimation of the action values (Q-values) for state-action pairs not well-represented in the dataset, which lead sub- optimal or dangerous policies when executed in the real environment. CQL addresses this issue by introducing a conservative estimation Q-values during training. It does so by penalizing the Q- values of actions that are not supported by the data and by ensuring that the learned policy does not deviate too much from the behavior policy that generated the dataset. This conservative approach reduces the likelihood of overestimating the Q-values of unseen actions, leading to more reliable policy performance. CQL particularly useful scenarios where data collection is expensive or risky, such robotics, healthcare, or finance, where it is critical to learn from limited data without the opportunity for trial-and-error learning in the actual environment. By focusing on the safe and robust optimization of policies from offline data, CQL represents significant step forward in the practical application of reinforcement learning.
BC: Behavior cloning (BC) is straightforward method in the realm of robotic learning, where the goal is to mimic expert behavior. BC trains policy (typically represented by a neural network) to replicate the actions taken by expert in various states. This is achieved by collecting dataset of state-action pairs from an expert's demonstrations and then using this dataset to train the policy via supervised learning, treating the problem simple function approximation task where the input
20

---

Published as a conference paper at ICLR 2025
the state and the output is the action. The appeal of BC lies in its simplicity and efficiency, does not require reinforcement signals or interaction with the environment during training. Thus, BC has emerged policy by online interaction is challenging and expensive. However, it also has limitations, such the tendency accumulate emrrs due covariate shift, where the policy encounters states that are not well-represented in the training data, leading to actions that deviate from the experts', and thus to states that are even less well-represented, its limitations, BC can be quite effective for tasks where the expert's policy is easy to capture with supervised learning, and it serves as foundation for more complex imitation learning algorithms that seek to address its shortcomings.
popular method employed in the realm of robots, where learning capable
potentially compounding fashion. Despite
D ALGORITHM DESCRIPTION
The practical implementation of VeViWo method is presented in the form of pseudo-code in Algo- rithm 1.
Algorithm Representation learning for View-invariant World model (ReViWo)
Required: a multi-view image dataset O, offline control data D and a empty replay buffer for world model rollout DD
Output: the optimized robotic control policy æ.
I: Initialize the autoencoder Pa: 9a- world model Mg. reward model Re and policy Tæ, where the subscript denotes their parameters.
2: // Training autoencoder
3: while training not converge do
4: Sample (o%; o0 o"; ) data from
5: Update ø based AE training objective in Eq. (3).
6: end while
7: Pre-process visual observation in 'D from 8t 2s,-
8: Update the offline control dataset with (Ze,) 2o.41: rt).
9: // Training world model
10: while AE training not converge do
Sample (sy, ag rt) from D.
12: Update world model parameters 9 with world model training objective in Eq. (4).
Update reward model parameters θ with supervised learning on reward data.
14: end while
15: // Training policy
16: while policy training not converge do
17: Sample in the world model with Fe and Re to collect (E.,, Ot, FF.)
18: Update replay buffer: D D U (Ex, dt; Fe)
Sample from both and D update & with COMBO algorithm.
20: end while
21: return the optimized policy म.
E DISCUSSION ON THE USAGE OF DATASET WITH VIEW LABELS
In this research, we present method for robust robotic manipulation that relies on multi-view dataset that has implicit view labels to train the autoencoder. We would like to discuss the rational- ity of employing such labels for robust robotic control. The view information is important visual robotics control. The ability to discriminate between features that are relevant under different view- ing conditions is paramount. View labels facilitate this process by guiding the model associate specific features with the viewpoints they are most relevant to, thereby enhancing the model's ability to generalize these features across a range similar viewpoints. Some previous works (Liu et al., 2024) also attempt train with the view label. Different from Liu et al. (2024), do not explicitly use the view labels like camera parameters, but the multi-view images that are naturally aligned in states viewpoints.
21

---

Published as a conference paper at ICLR 2025
Besides. in practice, it is not expensive to obtain the viewpoint labels, ple manipulation trajectories from some exed-position cameras. Such view information is not just facilitator for quicker learning, but also critical component that imbues the model with deeper un- derstanding of its operational context. It empowers the model to develop a robust, interpretable, and adaptable framework for robotic manipulation that is essential for real-world applications. More- over, we demonstrate that VeViWo can be integrated with multi-view data with that is without view label. Our experiments in Tab. 2 demonstrate the efficacy such integration.
record multi-
F ADDITIONAL RESULTS
F. RESULTS MORE TASKS
We conduct experiments on more tasks in the Meta-world environment, including Coffee Button, Faucet Open, and Dial Turn. The experiment results are presented Fig. F.I.
Success Rate
-00
40
Success
Success
40
20-
Training
CIP
CSH
Training
CIP
CSH
Training
CIP
CSH
(a) Coffee Button Open
(b) Faucet Open
(c) Dial Turn
Figure 10: Performance of Imitation Learning with View-invariant Representation. The x-axis de- notes the disturbance type, and the y-axis denotes the average success rate of the last check- points, by evaluation for 30 episodes.
F.2 IMITATION LEARNING WITH VIEW-INVARIANT REPRESENTATION
Imitation learning is widely-considered setup in the field of robotics. We further investigate whether the learned VIR can utilized for imitation learning. To this end, we apply BC with the leamee VIR using the offline control data. ReViWo's capability to learn view-invariant representa- tions enables the application of Imitation Learning as an alternative training world model for robotic control policy acquisition. Fig. 11 illustrates the success rate of Behavior Cloning utilizing view-invariant representations (RiVi-BC), While RiVi-BC does not surpass ReViWo, it outperforms conventional Behavior Cloning with VAE encoder. The performance drop of BC is primarily at- tributed to distributional shift and the accumulation of errors during sequential decision madisionnmaking... BC is inherently susceptible to these issues, it relies imitating actions from fixed dataset. However, is noteworthy that when BC is paired with VIR, outperforms BC with VAE represen- tations. This outcome underscores the robustness of VIR in novel viewpoints and improving overall performance.
22

---

Published as a conference paper at ICLR 2025
10c
100
ReVi-BC
Success Rate
Training
CIP
CSH
Training
CIP
CSH
Traming
CIP
CSH
(a) Door Open
(b) Drawer Open
(c) Window Close
Figure 11: Performance of Imitation Learning with View-invariant Representation. The x-axis de- notes the disturbance type, and the y-axis denotes the average success rate of the last check- points, by evaluation for 30 episodes.
23

---

Published as a conference paper at ICLR 2025
F.3 MORE EXAMPLES DECODER OUTPUT
Fig. 12 presents additional examples of the decoder output.
VIE Input
VDE Input
Decoder Output
Ground Truth
VIE Input
VDE Input
Decoder Output
Ground Truth
Figure 12: Additional examples of the decoder output the corresponding ground truth image. The decoder can generate the images that combine the task state VIE inputs, and the viewpoint in VDE inputs.
24


Reviews:
[{"review_id": "rg8kvCkd9M", "reviewer": "Reviewer_YacQ", "summary": "The goal of this paper is to train world models for robot manipulation\nthat are robust to changes in the position of the camera observing the\nmanipulator and the environment. This is a very relevant problem,\nsince cameras frequently move between training sessions and\ndeployment. To achieve this, the authors introduce a training setup in\nwhich a combination of two VQ-VAE encoders is trained on a multi-view\ndataset; one of the encoders, the VIE, learns to encode the\nview-independent setting and arrangement of the scene, while the other\nencoder, VDE, learns to encode the view-dependent aspects of the\nscene. Taking the two encodings together, a decoder can reconstruct\nthe input scene as if it was observed from the viewpoint represented\nin the other scene.\n\nThe VIE is then used to train a world model that can be applied for\npolicy learning or behavior cloning. The results indicate that the\napproach is able to disentangle the different aspects of the view\ninputs, leading to improved generalization to viewing changes between\ntraining and testing.", "questions": "Additional questions beyond those already mentioned in the weakness sections. \n\nI like the idea of training the encoder / decoder using multiple viewpoints. Using a different scene along with a different camera pose to training view dependent aspects seems elegant. However, I'm not sure that this is really necessary. What is there to be encoded beyond the pose of the camera relative to the manipulator? If there's nothing else, is training the VDE overkill to extract that information from the scene? It'd be interesting to validate what specifically is learned by the VDE (e.g. could you train a readout network to extract relative camera pose?). Also, does this work equally well for more translation and rotation of the camera?\n\nWhy does the model performance drop so significantly in the BC results shown in E.2? I can imagine that this might be due to the reference framework used for the robot actions.\n\nA discussion in the context of 3D representations for keyframe-based manipulation would be helpful. Several works, such as [1,2], train a model that is independent of the camera viewpoint by generating actions in the camera frame of reference and then translating these into robot actions using a calibrated camera pose. Could the same idea be applied here?\n\n[1] Perceiver Actor: https://peract.github.io/\n[2] 3D Diffusor Actor: https://3d-diffuser-actor.github.io/", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The training setup to enforce a separation of view-independent and view-dependent encoders is clever and seems novel.\n\n- The view generation results indicate that the encoders and the decoder learn the kind of representation intended by the authors. \n\n- The evaluation of the learned policies indicates strong improvements over prior approaches to world model learning with respect to\ngeneralization to novel viewpoints.", "weaknesses": "The main weakness of the paper lies in the experimental evaluation. As I understand, the model needs to learn (at least) two key aspects for a scenario: Where is the camera relative to the manipulator base (VDE) and what is the task-relevant layout of the scene (VIR). The decoder then needs to use that information to generate a novel viewpoint of the input scene. While this seems to work for the test cases, it is not clear whether the success is due to overfitting to a rather small number of settings and tasks, or whether the approach would scale to more complex scenarios, including real world data and camera pose changes beyond azimuth. The current evaluation does not provide sufficient evidence regarding real world significance.\n\nIn light of changing camera poses, the action space of the controller is very important. There are at least three I could imagine: (delta) joint space, (delta) end effector pose in camera frame of reference, (delta) end effector pose in manipulator frame of reference. The specific choice is extremely important for how a policy might transfer to different camera viewpoints. Which specific one was used in the experiments? Could the approach work for all of these? What if the camera calibration of a test scene is known?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fQW0pVkeCV", "reviewer": "Reviewer_juRH", "summary": "This study investigates robust robotic manipulation in the presence of camera viewpoints disturbances. It develops viewpoint-invariant representations learning methods with a VAE-like objective. The learned viewpoint-invariant representations are subsequently utilized for robotic control. The experimental results on two simulation environments demonstrate the enhanced robustness across two types of viewpoint disturbances.", "questions": "1. Is the proposed algorithm's performance under viewpoint disturbances stable across different tasks, and does it remain consistent outside the simulation tasks involved in this paper?\n2. Why does the integration of Open X-Embodiment data into ReViWo lead to a decline in model performance under Camera Shaking (CSH) for the Door Open task? Additionally, why does the integration of the world model into ReViWo result in reduced performance under Camera Shaking (CSH) for the Drawer Open task?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The approach for learning viewpoint-invariant representations is quite novel. This paper employs a view-invariant encoder and a view-dependent encoder, which take two images from different viewpoints as input. The features encoded by these two branches are then processed through a decoder, utilizing a VAE-like learning objective to decompose view-invariant and view-dependent information.\n2. Both the comparative experiments and the ablation studies are thorough.", "weaknesses": "1. The experiments are conducted on two simulation environments only, and the effectiveness of ReViWo on real-world robots remains unvalidated.\n2. From Figure 4, even when applying the proposed algorithm ReViWo in this paper, the success rate still significantly declines under disturbances caused by Camera Installation Position (CIP) and Camera Shaking (CSH). This indicates that the enhancement in robustness to viewpoint variations achieved by this method is limited.\n3. The representation of Figure 4 is quite misleading. It is recommended to revise it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "m8mgbbjGuA", "reviewer": "Reviewer_Czra", "summary": "The draft proposes and evaluates a new approach for learning a view-invariant encoding and world model, in the context of learning vision-based robotic manipulation.", "questions": "The values of the \\lambda_1 and \\lambda_2 coefficients for respectively the VQ-term and contrastive-term in the training objective (equation 3) are listed in Table 3 of the annex, but authors provide absolutely no clue on how these values were chosen, and do not analyze how critical these values might be for the result (my intuition is that they should have a significant impact). Could authors clarify that, and ideally provide at least a minimal quantitative analysis/assesment of how important it could be ?\n\nIn second paragraph of §4.2, authors write that their proposed method \"consistently surpasses the baselines\", but on figure 4 it can be seen that for task Window-Close, MVWM has much higher success rate (~85%) than ReWiVo on the CIP case, and slighltly higher succes rate on the CSH case. Authors should NOT \"over-claim\" in their text compared to figure, and should comment on this lesser performance of their method on this task.\nFurthermore, the presentation of figure 4 with \"skipping\" the 40%-80% part of the y axis is somewhat misleading: this must be corrected, possibly by using higher plots that do NOT skip the 40%-80% range.\n\nRegarding the use of Open X-embodiment data, ablation study reported table 1 shows it has a quite significant impact ; however, authors mention on line 227 that they \"introduce a weigthing factor in the loss calculation for these unlabeled data\", but it seems they provide no information whatsoever about what value is this weighting and how critical this value could be for the outcome. \nAlso, it appears in table 1 that inclusion of this extra data actually *degrade* result in the CSH case for Door-Open task ; author should comment on that.\n\nExamples of the decoder output shown in figure 7 are impressively similar to the ground truth, but are those examples on *test* data or on some of the training data ??\nFurthermode, since the success rate falls from over 90% on training data down to below 40% on test data of the 3 first tasks (a) (b) and (c) on figure 4, there must be a significant number of cases for which the view-invariance does not work so well --> authors should show and comment some of these failure cases, to allow readers to qualitatively evaluate what happens when results are not as good as in current figure 7...\n\nIn last part of §4.4, authors write that inclusion of their World Model (WM) \"present a consistent performance enhancement\", while actually in table 2 providing some ablation on the impact of their world model component, WM appears to slightly *degrade* result for CSH case of Drawer-Open task ; author should comment on that.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "The problem addressed by the draft, view-invariance or at least viewpoint-robustness of learnt robotic manipulation policies, and more generally learning of view-invariant world models, is extremely important and challenging.\nThe solution proposed by authors for learning a view-invariant representation, which consists in learning a distangling of view-invariant encoding and view-dependant encoding, is appealingly elegant and seems rather original. It also has the interest as a by-product of enabling generation of an arbitrary (?) new view-viewpoint for a given system state, and conversely.\nThe experiments conducted on Meta-world and Panda-gym reported in the paper are rather convincing regarding the ability of the proposed approach to bring significant viewpoint-robustness (figures 4 and 5), and to learn a relatively view-invariant embedding (figure 6).", "weaknesses": "The main weaknesses of the paper are the following:\n - while the \"baselines for comparison\" include MVWM, no comparison is conducted with other important very-related works mentioned: RT-X series works and RoboUniView\n- the experiments are conducted on a quite small number of tasks (3 out of 50 on Meta-World), and only one (!) in Panda-Gym ; this raises some doubt on possible \"cherry-picking\" approach for choosing these tasks", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JlXM0DQaDi", "reviewer": "Reviewer_Hd4G", "summary": "This paper presents ReViWo (Representation learning for View-invariant World model), a novel approach addressing the challenge of viewpoint changes in robotic manipulation tasks. Traditional methods struggle with performance degradation under varying camera angles; ReViWo overcomes this by leveraging multi-view data to learn robust representations for control under viewpoint disturbances. Using an autoencoder framework, ReViWo combines view-invariant and view-dependent representations, trained on both labeled multi-view simulator data and the Open X-Embodiment dataset (without view labels). Tested in Meta-world and PandaGym environments, ReViWo outperforms baseline methods, maintaining stable performance in scenarios with novel camera angles and frequent camera shaking. These results validate ReViWo’s effectiveness in providing robust, task-relevant representations across diverse viewpoints.", "questions": "1. To my knowledge, the output of VAE is blurred. The multi-step world model will enlarge the blurring problem. Could you please how this problem affects this performance? Also, I will appreciate if the author could release the structure of VAE for reproductibilty.\n\n2. Also, will the vae have some errors? For example, will the generated object have different geometries?\n\n2. See weakness.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "View variation is a very practical problem in robotic manipulation. This paper provides good experiments, and leverages Open X-Embodiedment, leveraging this dataset is a promising direction is robotics. Also, the manipulation video in the link is cool.", "weaknesses": "1. The paper shows application in simulation. However, in other specific real world scenarios, there is not enough multi-view data for training. So, I recommend authors to demonstrate how the study can work in the real world, and show real-world evaluation results.\n\n2. This paper only shows results in some simple tasks. Will this method work in more diverse tasks? For example, manipulating deformable objects like folding cloth. These experiments will improve the significance of this paper in robotics field.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UxQ50MsRj2", "reviewer": "Reviewer_pubk", "summary": "This paper presents a framework named as ReViWo to learn view-invariant representation (VIR), then uses the VIR as the low-dimensional state representation to train a policy with model-based offline RL (COMBO). \n\n\n1. ReViWo includes a view-invariant encoder and view-dependent encoder to reconstruct images at different viewpoints by combining VIR with view-dependent information. \n2. ReViWo was trained from multi-view simulation data and open-x datasets, and then evaluated on selected tasks on MetaWorld and PandaGym environments.\n3. The authors show that ReViWo is robust to viewpoint changes in evaluation.", "questions": "(see weakness)", "limitations": "", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. It is beneficial to improve the robustness of a robotic policy to camera shaking or viewpoint changes. \n2. Finding low-dimensional state representation for high-dimensional visual signals (such as images), and applying existing offline RL methods on that state representation, is an interesting policy structure.   \n3. The writing is easy to follow.", "weaknesses": "**1. (Fundamental Limitation)**\n\nRobotic tasks heavily depends on the understanding of multi-view camera data. For example, it is very common to have a static top camera and a moving on-gripper camera. The focus is on how to leverage information from different views in order to get better performance instead of only using the very limited invariant information. Therefore, by latching on view-invariant information, the proposed ReViWo is limited to only single-view RGB observation without depth, a highly constrained and often impractical setting in robotics  (Note that if we have RGBD, we could reproject the point cloud to multiple views, so RGBD can be considered as multi-view). \n\nMoreover, suppose there is a desk in a single RGBD image, the camera movement can be viewed as the relative movement of the desk as well. This means that a view-invariant representation will lose some ability to sense object layout changes. This is really undesirable. \n\n**2. (Lack of Technical Contribution)**\n\nThe method section looks hand-wavy. The authors propose a view-invariant encoder and view-dependent encoder to reconstruct images and learn view-invariant representation. However, the authors did not provide any mathematical guarantee or at least intuition on why the representation can be disentangled that way. It is very likely that the two encoders just work in parallel without having the expected property. \n\nMoreover, the authors mentioned the training of a world model and training a reward model, but in fact, they used the COMBO method to do all these [1]. I am not sure whether COMBO, an offline model-based RL method, can be called a world model because it only predicts the next low-dimensional state. The COMBO framework includes the training of a reward model, so it is not the contribution of this method. \n\n**3. (Lack of Proper Evaluation)** \n\nThe authors only evaluate on MetaWorld and PandaGym, two very simple task suites in terms of manipulation diversity, precision, horizon length. Even on these simple task suites, the authors only select 3 tasks from MetaWorld and 1 task from PandaGym, while the MetaWorld has roughly 50 tasks. This evaluation is very insufficient. \n\n\n[1] COMBO: Conservative Offline Model-Based Policy Optimization. NeurIPs 2021", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)