Learning to Manipulate Anywhere: A Visual Generalizable Framework For Reinforcement Learning

Authors: zhecheng  yuan
tianming  wei
shuiqi  cheng
gu  zhang
yuanpei  chen
huazhe  xu
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-05-14 04:41:27

Abstract: Can we endow visuomotor robots with generalization capabilities to operate in diverse open-world scenarios? In this paper, we propose Maniwhere, a generalizable framework tailored for visual reinforcement learning, enabling the trained robot policies to generalize across a combination of multiple visual disturbance types. Specifically, we introduce a multi-view representation learning approach fused with Spatial Transformer Network (STN) module to capture shared semantic information and correspondences among different viewpoints. In addition, we employ a curriculum-based randomization and augmentation approach to stabilize the RL training process and strengthen the visual generalization ability.  To exhibit the effectiveness of Maniwhere,  we meticulously design **8** tasks encompassing articulate objects, bi-manual, and dexterous hand manipulation tasks, demonstrating Maniwhere's strong visual generalization and sim2real transfer abilities across **3** hardware platforms. Our experiments show that Maniwhere significantly outperforms existing state-of-the-art methods. Videos are provided at https://maniwhere.github.io.

Main paper: 
 introduction: 
 Visuomotor control tasks present roboticists with a vexing issue -the hardware setup can severely influence the performance of the robot policies. A prime example arises from the issue of immovable cameras -envision a carefully calibrated visual sensor, painstakingly positioned to enable seamless real-world deployment, only to have it disturbed by a lab mate. This single, seemingly innocuous incident can grind progress to a halt, forcing tedious recalibration or the collection of new demonstration data. Furthermore, changes in the background or the presence of extraneous objects within the captured views may undermine the effectiveness of a trained policy. Such obstacles have long plagued the field of robotics, representing critical barriers to realizing the full potential of advanced visuomotor systems.Acknowledging these obstacles, when attempting to achieve sim2real visual policy transfer, it is common to instantiate a digital twin that closely resembles the actual real-world environment [1, 2, 3, 4, 5, 6, 7, 8] . Otherwise, the significant discrepancy between the digital twin and the real setting would render the trained models wholly ineffective. Hence, robots that adeptly handle in-the-wild scenarios should possess generalizability against various visual changes such as camera views, visual appearances, lighting conditions, etc.While prior works have sought to tackle the challenges against visual scene variations [9, 10, 11, 12, 13, 14, 15, 16] , these studies primarily focus on resolving a single aspect and are unable to handle multiple visual generalization types simultaneously. Meanwhile, it is non-trivial to incorporate various inductive biases into the training process. Naively applying domain randomization or data augmentation methods can destabilize the entire RL training, ultimately leading to divergence for the learned policy [4, 9, 12, 17] . More importantly, the generalization abilities of these methods have yet to be thoroughly evaluated on real robots.In this paper, we are dedicated to enabling robots to acquire strong visual generalization ability so that they can step out of simulations and apply their learned skills to complex real-world scenarios without camera calibration. We introduce Maniwhere: A Visual Generalizable Framework for Reinforcement Learning. As shown in Figure 1 , Maniwhere employs a multi-view representation objective to capture implicitly shared semantic information and correspondences across different viewpoints. In addition, we fuse the STN module [18] within the visual encoder to further enhance the robot's robustness to view changes. Subsequently, to achieve sim2real transfer, we utilize a curriculum-based domain randomization approach to stabilize RL training and prevent divergence. The resulting trained policy can be transferred to real-world environments in a zero-shot manner.To conduct the evaluation, we develop 3 types of robotic arms and 2 types of robotic hands to design a total of 8 diverse tasks, alongside 3 corresponding hardware setups to validate the efficacy of our algorithm. Our comprehensive experiments demonstrate that, in both simulation and real-world scenarios, Maniwhere significantly outperforms existing state-of-the-art baselines by a large margin. 
method: 
 In this section, we present Maniwhere, a generalizable framework for visual reinforcement learning. We propose a multi-view representation learning objective aimed at empowering the training agent with the ability to extract invariant features and generalize across different viewpoints. To further augment the model's spatial awareness, we incorporate an STN module into the visual encoder by actively spatially transforming feature maps. Additionally, we employ a curriculum of domain randomization to stabilize reinforcement learning (RL) training and facilitate sim2real. Next, having established the blueprint for Maniwhere, we proceed to elaborate it with details.To endow the agents' ability to adapt to different viewpoints, we propose a multi-view representation learning objective to achieve this property. At each timestep t, the simulation returns the RGBD observations from two cameras with different views: one is fixed, and the other one will randomly appear at different viewpoints. The range of camera randomization are listed in Appendix B.1. We denote the observation from the fixed viewpoint as o f ixed , the observation from the randomized one as o move , and the visual encoder as f θ , which is parameterized by θ. With respect to the first term, we adopt InfoNCE [19] to formulate our contrastive loss function J con (θ):EQUATIONHere o move+ is the positive example of o f ixed , which is rendered at the same timestep, while o move- is the negative example at different timesteps from the same batch samples. Inspired by Moco-v3 [20] , we utilize a symmetrized loss form to gain better performance.Moreover, recent works [21, 22] find that feature maps can be utilized to indicate correspondences of the images that share similar semantics. Hence, to endow agents with the ability to learn correspondences between different views, we also introduce an alignment loss function applied to feature maps across various layers:EQUATION)where B is the sampled batch, F is the flattened feature map embedding from a certain layer. The overall objective of Maniwhere is formulated as follows:EQUATIONwhere λ is the coefficient to weigh the scale between two terms. Through the guidance of L M aniwhere (θ), the agent enables to gain a better understanding of the semantics, correspondence, and view-invariant information within the whole visual scenarios via multi-view images extraction.Due to the high sensitivity of RL training towards different types of randomization, introducing additional noise can potentially lead to divergence in the entire training process. However, domain randomization and augmentation are indispensable for the sim2real transferability. Therefore, we propose a curriculum randomization approach in which the magnitude of randomization parameters is incrementally increased as training progresses. We employ an exponential scheduler to adjust the incremental change of the parameters. Additionally, we also establish a curriculum for the objective of stabilizing Q-value training [9] :EQUATION)where aug is the augmentation method for the image observations, Q tgt θ ′ is the target Q network. The augmented data incorporates increasing amounts of noise along with the training procedure.Here we choose SRM [11] with random overlay [23] , a frequency-based data augmentation as our augmentation method.Spatial Transformer Network (STN [18] ) enables the spatial transformation of data within the network, empowering the agent with enhanced abilities to perceive spatial information. Furthermore, to expand the model's capability for transformations beyond the 2D plane, we modify the affine transformations in the original STN to perspective transformations:x s i y s i 1 = θ 11 θ 12 θ 13 θ 21 θ 22 θ 23 θ 31 θ 32 θ 33   x t i y t i 1   (5)where θ ij is the learnable transformation parameters, (x t i , y t i ) denotes the target coordinates on the output feature map's regular grid while the (x s i , y s i ) is the counterpart from the source image. Additionally, we leverage the first two layers of ResNet18 [24] as the backbone of visual encoder [13] and integrate the STN within it. 
experiments: 
 In this section, we conduct numerous experiments in both simulated and real-world settings to showcase the effectiveness of Maniwhere in terms of generalizing to diverse visual scenarios with a combination of visual disturbance types.Tasks: We have developed 8 tasks based on MuJoCo engine [25] with joint position control, including a variety of embodiments and objects such as single arm, bi-manual arms, dexterous hands, and articulated objects. We also establish the real-world counterparts for these tasks. In both simulation and real-world experiments, the observations are 128 × 128 RGB-D images with 3 frame stacks.Sim2real: First, we train the agents in each simulated environment, where images from two different cameras will be observed: one offering a fixed viewpoint and the other moving throughout the given randomized range. Then, Maniwhere will integrate the knowledge from both viewpoints into the visual encoder via the approach mentioned in Section 2. Once finishing training in simulation, the trained model will be directly transferred to the real world in a zero-shot manner. It should be noted that during both simulation and real-world evaluation, the trained agents receive images solely from a single camera for inference. The visual scenes will be modified from various aspects, including appearance, camera view, lighting conditions, and even cross embodiments at evaluation time.Real Robot Setup: For gripper-based tasks, we utilize a UR5 arm equipped with a Robotiq gripper.Regarding tasks involving dexterous hand manipulation, we employ an Allegro Hand coupled with a Franka arm, and a Leap Hand [26] paired with an XArm mounted on a Ranger Mini 2 robot base from AgileX [27] . We use Realsense L515 camera to obtain visual inputs [28] .We compare Maniwhere with the following visual RL leading algorithms: SRM [11] : implement a frequency-based augmentation method to achieve better generalization ability for visual appearances; SGQN [14] : SGQN leverages saliency maps to enhance the agent's attention on task-relevant information, and as suggested by Yuan et al. [23] , it reveals better visual generalization capability across different camera views. MoVie [29] : utilizes domain adaptation to refine visual representations at new viewpoint through the dynamics model. MV-MWM [5] : applies MAE [30] to distill multiview information into the visual encoder. It is worth noting that, unlike MV-MWM, Maniwhere does not require additional expert demonstrations, nor does it necessitate the acquisition of new data to adapt to environments as Movie does. Maniwhere can seamlessly transition to the real world in a zero-shot manner. We evaluate each algorithm over 5 seeds.Generalize to different viewpoints. In this section, we evaluate Maniwhere and the baseline methods across 8 challenging tasks. For each evaluation, 50 episodes from different viewpoints are tested. As shown in Table 1 , compared to the existing baselines, Maniwhere achieves superior performance across all tasks with a large margin. The experiments indicate that the previous visual generalization algorithms struggle to manage visual changes in camera views. Regarding MoVie, while it adapts to the specific viewpoint change through domain adaptation, our setting involves different viewpoints among episodes. We find that MoVie cannot generalize to the visual scenarios where the viewpoint continuously changes. Hence, we argue that single-view image inputs are insufficient for fully perceiving spatial information. As for MV-MWM, it also utilizes multi-view images to enable the model to learn view-invariant features. Nevertheless, the experiment results exhibit that Maniwhere owns stronger multi-view generalization abilities than MV-MWM with a +68.6% boost on average.Generalize to different visual appearances. In addition to changes in camera views, we further alter visual appearances by perturbing the colors of the table and background. As shown in Figure 3 , despite Generalize to different embodiments. Then, we seek to validate the agent's generalization capability across different embodiments by replacing the UR5e robot arm with a Franka arm. As shown in Table 2 , we surprisingly find that our trained model can directly perform zero-shot transfer to a different embodiment while maintaining the camera-view generalization ability. The qualitative analysis can be found in Appendix C.2. Regarding real-world experiments, as shown in Figure 4 , we deploy our models trained in simulation on real-world scenarios across 3 hardware setups in a zero-shot manner. For gripper-based tasks, we implement multiprocessing alongside a shared memory queue to synchronize the execution of network inference and the controller [31] , thereby ensuring a smooth movement process. As for dexterous-hand tasks, we introduce a moving average factor to reduce the jittering motions during execution [32, 33] . We select 5 challenging tasks in simulation to verify the effectiveness of agent's sim2real tranferability. For each task, we choose 5 different viewpoints that cover the workspace, and the visual appearances of the scenario will be altered under each viewpoint as well. Each algorithm is evaluated 5 trials under every visual condition. In each trial, yaw and pitch angles of the camera will Figure 5 : Real-world snapshots. Real-world experiments under different visual conditions. be randomized. Figure 5 exhibits snapshots of real-world settings. As shown in Table 3 , consistent with the simulation results, Maniwhere outperforms MV-MWM on all tasks. The experiment indicates that Maniwhere not only narrows the sim2real gap but also enables the trained robots to achieve the real-world generalization ability. More details and results can be found in Appendix and webpage. To investigate the necessity of each component in Maniwhere, we ablate two main design choices in Maniwhere, including the multi-view contrastive representation learning objective and STN module. Our ablations are conducted on two lifting tasks and one pickplace task. As shown in Table 4 , we observe that the multi-view objective contributed significantly to the improvement; without it, the model would be deprived of its ability to generalize across different camera views. Meanwhile, the integration of the STN enhances the model's capacity to understand and adapt to spatial view changes. Furthermore, we adopt TCN loss [34] , which also applies multi-view contrastive learning, to replace our multi-view objective. The results reveal that there remains a significant generalization performance gap compared with Maniwhere, highlighting the advantages of our approach.To delve deeper into the reasons behind Maniwhere's superior performance, we examine it from the aspects of visual representations and Q-value functions of RL. Q-value distribution. Conceptually, if an RL agent can produce the Q-value distribution from noisy visual inputs that closely approximates that obtained from the original images, the trained agent can be regarded as a more robust and generalizable learner. [12, 9] We visualize the representation of the penultimate layer of the critic using t-SNE to examine how the Q-distribution differs under various viewpoints with our proposed multi-view representation learning method. As shown in Figure 6 , our method maintains a distribution similar to that of the original fixed viewpoint, whereas relying solely on the objective in Eq 4 fails to adapt to different camera views. Consequently, Maniwhere not only closes the distance between visual embeddings to obtain more robust visual representations, but also narrows the gap between Qdistributions, further stabilizing training and enhancing agent's visual generalization ability.Trajectory embedding. For the visual representation side, we visualize the feature maps of images rendered from different viewpoints along the same execution trajectory, and then apply t-SNE to Figure 7 : Trajectory embedding visualization. We capture images from 3 viewpoints at the same moment while executing an identical trajectory. As the timestep progresses, the color of the embedding becomes increasingly darker. We find that they exhibit similar visual representations.embed the feature maps. Figure 7 shows that Maniwhere is capable of mapping the images from different viewpoints into the similar regions as well as maintain consistency throughout the entire execution trajectory. 
related work: 
 Generalization in visual RL. In recent years, multiple works have resorted to addressing the critical issue of generalization [6, 35, 36, 37, 38, 39, 40, 41, 42] . Based on the strong data augmentation, several studies integrate advanced methods such as pre-trained visual encoders [13] , saliency maps [14] , and normalization techniques [41] to enhance the visual generalization capabilities of agents. Despite these advancements, current methods primarily address only variations in visual appearances and fall short when confronted with other types of visual changes. Another line of works devote to solving the camera view changes. For instance, MoVie [29] utilizes an inverse dynamics model to facilitate the model adapt to a novel view pattern. However, it is limited to a singular type of view and cannot accommodate multiple different view patterns. Meanwhile, MV-MWM [5] leverages model-based RL to train a multi-view masked encoder. However, its dependency on demonstrations for task completion remains a significant limitation. Moreover, these two approaches are unable to adapt to the changes of visual appearances. On the contrary, Maniwhere offers a versatile visual RL approach that is compatible with multiple visual generalization types and does not require any demonstrations.Representation learning for visuomotor control. Representation learning plays a critical role in visuomotor control tasks [43, 44, 45, 46, 47, 48, 49, 50, 51] . Recent works [13, 52, 53] have verified that leveraging the pre-trained visual encoders via representation learning approaches can facilitate the execution of numerous downstream control tasks. Furthermore, SODA [10] utilizes a BYOL-style objective to decouple augmentation from policy learning. RL3D [3] pretrains a deep voxel-based 3D autoencoder and continually finetunes the representation with in-domain data. H-index [54] applies the keypoint detection and pose estimation method to derive a customized representation for the hand.In contrast to these works, Maniwhere not only strives to obtain generalizable visual representations but also seeks to enable these representations to bridge the sim2real gap. 
conclusion and limitations: 
 In this paper, we present Maniwhere, a visual generalizable framework for reinforcement learning. Maniwhere leverages multi-view representation learning to acquire the view consistency information, and utilize curriculum randomization and augmentation approach to train generalizable visual RL agents. Our experiments demonstrate that Maniwhere can adapt to diverse visual scenarios and achieve sim2real transfer in a zero-shot manner. In the future, we plan to enhance Maniwhere's generalization ability across broader camera ranges and more diverse visual scenarios. Beyond visual generalization, we intend to incorporate spatial generalization methods to handle more complex object spatial relationships, with the ultimate goal of developing a robust sim2real framework. one PC is regarded as the client, while the other two serve as servers. The client PC receives visual input and performs network inference, subsequently transmitting the inferred actions via socket connections to the two server PCs. The server PCs are responsible for controlling the Franka arms and executing the received actions. This process is iterative, with the servers sending new visual input back to the client for continuous processing. Given that MV-MWM has a large model size and requires substantial memory for loading, we deployed it on a desktop equipped with an RTX 3090 GPU. In contrast, the deployment of Maniwhere demands significantly less hardware, allowing it to perform inference even on CPU desktops. Regarding the camera setup, we establish the evaluation viewpoints at three yaw angular ranges: [0, 5°], [10, 25°] , and [40, 55°] , on both the left and right sides. Additionally, across the five trials conducted at each viewpoint, the camera height will be varied within a range of -3 to 3 cm.Instance generalization. Thanks to the general grasping capabilities of the dexterous hand, Figure 10 shows that Maniwhere is not limited to a single object when executing the lifting behaviours and can generalize across different instances with various shapes and sizes. Figure 11 illustrates that when we first select a pixel point on the UR5 original image (marked with a red pentagram) and extract its feature (enclosed in the orange square) after passing through the convolutional layer, we compute its normalized cosine similarity with the image feature of Franka arm to obtain a similarity map. The point with the highest value in this map is identified as the most similar point between two images (marked with a red pentagram). As shown in Figure 11 , Maniwhere can effectively recognize semantically consistent positions between the two different embodiments.With respect to randomization, to enable the agent to capture the correspondence information through the multi-view representation objective, we do not augment the moving view image in Eq 4. 
 
 Appendix: We further investigate how Maniwhere's performance varies across different camera view ranges. We divide the randomized camera view range into three parts, within each of which the camera's pitch and field of view are randomly altered as well. The value for each range is calculated as the average of both the left and right sides. Due to the excessive angular range in handover task potentially obscuring the other arm, we confined the range for this task to 0-30 degrees. Table 7 illustrates that, although Maniwhere's performance exhibits a slight decline as the angle increases, it still retains the capability to handle these scenarios effectively. To ensure the depth images closely resemble real-world conditions, we first pre-process the depth image. We introduce Gaussian noise N (0, 0.01) and depth-dependent noise N (0, depth scale), where the depth scale equals np.abs(depth image) * 0.05. Then, we apply GaussianBlur to smooth the noise. Additionally, the depth values are clipped to within 2 meters and normalized to the range [0, 255]. During sim2real, we find that depth image can largely help to alleviate the ambiguity situation. Figure 12 shows that when encountering large camera viewpoints, the agent cannot accurately determine the grasping position since RGB information alone does not provide the necessary front-to-back positional relationship between the object and the gripper in the 3D world. However, by incorporating depth images, we observe a significant improvement in real-world scenarios.C. We also apply the data augmentation method on MV-MWM. As shown in Table 8 , MV-MWM suffers a significant performance drop while facing data augmentation. These results are consistent with the recent works [9, 12] . Naively applying data augmentation can cause instability and large variance during training. In turn, the results also prove that simultaneously handling multiple types of generalization is non-trivial and highlights the superiority of Maniwhere.Although we found that the agent demonstrates strong generalization capabilities when the visual scene is altered, including changes to the table, background, and the introduction of colorful distractors, it fails the task when the color of the target object is changed. Figure 13 exhibits that during executing a trajectory, the agent focuses more attention on the target object while ignoring task-irrelevant information, making it more sensitive to changes in the color of the target object. We use the Grad-CAM [55] to visualize the agent's attention. 


Decision: The reviewers were all impressed by this paper.

Reviews: review: title: title: Initial Review of the Manipulate Anywhere Paper
paper_summary: paper_summary: The authors propose a framework to train vision-based RL models for robotic manipulation in simulation, where the model is robust to visual disturbances and enables zero-shot sim2real transfer. The approach described combines three components for generalizable manipulation: (1) moving view to canonical view mapping; (2) spatial transformer component and (3) curriculum-staged domain randomization to learn an effective policy in simulation and achieve robust sim2real transfer. The authors train single-task models using this framework for different robotic tasks and different robotic arms and grippers. Several baselines are compared to the proposed method and the evaluation is done both in simulation and on the real systems and the results show that the proposed framework significantly outperforms the baselines.
main_review: main_review: The paper is generally well-written, and the main text or supplementary material clearly explains the details. The proposed framework shows promising results and would potentially be useful to the robotics community. 

Strengths:

-	Combines different components into one framework for robust vision-based RL and sim2real transfer
-	Extensive experiments on different robotic tasks using different robotic platforms both in simulation and on the real system
-	Ablation studies that show the necessity of the framework components
-	Significantly better results than baseline methods, which show the potential usefulness of the framework in the robotics community


Weaknesses:

-	Limited novelty as the different components of the framework already exist, but the combination of them in one framework and their application in robotic tasks is new
-	Evaluation only on own tasks and not on known/available benchmarks (especially in simulation), which would lead to more objective comparison with other methods and enhance reproducibility
-	Potentially limited applicability of the STM module to 3D transformations due to transforming projected views of 3D data (see questions below)
questions: questions: -	The STN model is extended to include a perspective transformation of the input data. However, the input data is in the form of a projected view in 2D (depth is an additional channel, as explained in the supplementary material). Therefore, it is questionable what kind of 3D transformation is achieved given that the projected view under significantly different camera angles w.r.t. the fixed view would show a completely different part of the robot/environment. Can you elaborate on whether this type of input data is a limitation, and wouldn’t a 3D representation as point clouds (e.g., [1]) allow better spatial transformation and robustness?

-	You mention that MV-MWM requires additional expert demonstrations to train. – Did you actually do this during your training process for the tasks you defined, or did you use a pre-trained MV-MWM model? It is a little unclear to me how the training procedure for MV-MWM was, and additional elaboration on this in the supplementary material will be useful.

-	From the task descriptions and example videos, the tasks look rather simplified w.r.t. spatial relations - does a robotic task always involve very similar spatial relation between objects (e.g., box always placed on the same side of the bowl in approx. the same distance), or you randomize the spatial relations and placements between the objects and object-robot?

-	The viewpoint randomization is described using yaw and pitch angles, which is a little confusing, as they usually are defined w.r.t. the camera coordinate system. I would suggest using azimuth and elevation angles to describe the viewpoint randomization, as they more appropriately refer to positioning the camera around the robot and are defined w.r.t its coordinate system.  

-	The limitations section is very general. You only test the robustness of the model to visual disturbances (there are no occlusions or depth sensor noise); it seems that the tasks always involve the same spatial relations between objects or object-robot, and the viewpoint randomization is within a specific range. Please consider improving the limitations section based on the answers to the questions above and these comments. 

References:
[1] Wang, Jiayun, Rudrasis Chakraborty, and X. Yu Stella. "Transformer for 3D point clouds." IEEE Transactions on Pattern Analysis and Machine Intelligence 44.8 (2021): 4419-4431.
limitations: limitations: 2
review_summary: review_summary: The work tackles relevant topics in robotics (vision-based RL from dynamic views, robust sim2rela transfer) and based on the results as well as the detailed presentation it will be relevant to the robotics community.
title: title: Maniwhere: A Reinforcement learning method that can generalize to both changes in camera views and visual appearances
paper_summary: paper_summary: The main contributions of the paper include a multi-view representation learning objective to learn invariant features and generalize across time steps, added stn module into the visual encoder (for spatially transforming feature maps), a curriculum of domain randomization to stabilize rl, and extensively evaluate the method on 8 different simulation tasks and their real world clones
main_review: main_review: ### **Quality**

The paper tackles the two most widely faced problems in both reinforcement learning and imitation learning, i.e., changes in visual appearances and camera views. It performs extensive experiments in both simulation and real-world environments. The method translates zero-shot from sim2real.

### **Clarity**

The paper is clearly written, and the contributions are clearly stated and easy to understand.

### **Originality**

The paper proposes a novel approach to learn multi-view representation utilizing one fixed view while varying another view, aiming to bring the learned representations for both views together.

### **Significance of this Work**

The paper makes a significant contribution towards handling variations in camera views and visual perturbations, outperforming all the baselines by 68.5% on average.

### **Strengths**

1. Extensive evaluations on both simulation and real-world tasks.
2. Zero-shot sim2real results: The direct sim2real performance (with no real-world demos) claimed in the paper could be a game-changer for how we deploy RL policies in the real world.

### **Weaknesses**

1. The paper needs to substantiate what components contribute to the handling of visual appearance change. The current presentation addresses and motivates handling multiple camera views, but the paper needs to provide more motivation and qualitative analysis on handling visual diversity. Currently, it's hard to decipher what components contribute to this.
2. In the experiments section, there is no discussion about how much the positions of the object in each task were varied. The policy could just be memorizing to predict the same sequence of actions regardless of the camera views. It is important to stress the generalization of the method to varying object locations.
3. The zero-shot sim2real transfer performance needs to be analyzed both qualitatively and quantitatively. Is the zero-shot transfer a consequence of simple and articulated objects used in the tasks? What happens with more complex tasks with more dynamic contacts where simulation is hard to match the real world?
questions: questions: Please refer to the weaknesses section for a list of questions to address for rebuttal.
limitations: limitations: 2
review_summary: review_summary: Maniwhere addresses two crucial issues in robot learning i.e. changes in camera views and visual observations. It could have a significant impact on the training and iteration speed of robot learning algorithms by removing the need for tele-operated demonstrations.
title: title: Exciting work on sim-to-real manipulation while being entirely background and viewpoint invariant
paper_summary: paper_summary: The authors use sim-to-real with extreme background and scene randomization to train a policy that can be invariant to background and camera pose in practice.
main_review: main_review: ## Summary
In this work, the authors present a class of general policies that can solve manipulation tasks in a background and viewpoint invariant fashion. The core of the work is using simulation environments to randomize the background and camera viewpoints, while also using a STN module on the visual encoder to encourage viewpoint independence.

The training process has two objectives, one is for multi-view agreement in the representation space, and the other is the standard RL objective. Using this, the authors present trained policies that can be deployed using different camera poses in the real world. The authors train on 5 tasks, and show success in scene variations on all tasks. They also ablate over different design choices and show that each of them has an important role to play.

## Strength
1. This paper pushes our robot capabilities in the right direction; calibrating camera is an incredible pain point for real robot, so I (and many roboticists) would want this work to succeed.
2. The method is generally sound and the design choices are well motivated and ablated.
3. Experiments over different tasks and many different real scenes show that it is not a fluke or trick but real progress.
4. The qualitative analysis is also interesting and adds a new layer to the understanding.

## Weaknesses
1. **No training curves!** This is an important point to understand if this result scales to for example imitation learning. How many samples does an encoder have to see before it can be completely viewpoint invariant?
2. I am having a hard time imagining how this work can scale to multi-task, language conditioned objectives. The invariance is largely a trick of the encoder, how do we condition our encoder to be multi-task like this?
questions: questions: See the weakness section above. The sample complexity of learning such an encoder is of particular interest.
limitations: limitations: 3
review_summary: review_summary: This work unlocks new capabilities in robotics and thus should be read by the audience of CoRL with haste.