Multi-View Masked Autoencoders for Visual Control
Authors: Younggyo Seo
Junsu Kim
Stephen James
Kimin Lee
Jinwoo Shin
Pieter Abbeel
Venue: Submitted to ICLR 2023 (2023)
Keywords: visual control
masked autoencoder
representation learning
world model

Abstract: This paper investigates how to leverage data from multiple cameras to learn representations beneficial for visual control. To this end, we present the Multi-View Masked Autoencoder (MV-MAE), a simple and scalable framework for multi-view representation learning. Our main idea is to mask multiple viewpoints from video frames at random and train a video autoencoder to reconstruct pixels of both masked and unmasked viewpoints. This allows the model to learn representations that capture useful information of the current viewpoint but also the cross-view information from different viewpoints. We evaluate MV-MAE on challenging RLBench visual manipulation tasks by training a reinforcement learning agent on top of frozen representations. Our experiments demonstrate that MV-MAE significantly outperforms other multi-view representation learning approaches. Moreover, we show that the number of cameras can differ between the representation learning phase and the behavior learning phase. By training a single-view control agent on top of multi-view representations from MV-MAE, we achieve 62.3% success rate while the single-view representation learning baseline achieves 42.3%.

Under review as a conference paper at ICLR 2023
MULTI- VIEW MASKED AUTOENCODERS FOR VISUAL CONTROL
Anonymous authors Paper under double-blind review
ABSTRACT
This paper investigates how to leverage data from multiple cameras to learn rep- resentations beneficial for visual control. To this end, we present the Multi- View Masked Autoencoder (MV-MAE), a simple and scalable framework for multi- view representation learning. Our main idea is to mask multiple viewpoints from video frames at random and train video autoencoder to reconstruct pixels of both masked and unmasked viewpoints. This allows the model to learn representations that capture useful information of the current viewpoint but also the cross-view information from different viewpoints. We evaluate MV-MAE on challenging RLBench visual manipulation tasks by training reinforcement learning agent top of frozen representations. Our experiments demonstrate that MV-MAE signifi- cantly outperforms other multi-view representation learning approaches. Moreover, we show that the number of cameras can differ between the representation learning phase and the behavior learning phase. By training a single-view control agent on top of multi-view representations from MV-MAE, we achieve 62.3% success rate while the single-view representation learning baseline achieves 42.3%.
INTRODUCTION
Recent supersupe learning approaches have been successful learning useful representations from multiple views of the data, including different channels (Zhang et al., 2017) or patches (Oord al., 2018) of an image, vision-sound modalities (Owens et al., 2016), vision-language modali- ties (Radford et al., 2021: Alayrac et al., 2022), and frames of a video (Wang & Gupta, 2015). The main underlying idea of these approaches is to utilize information about the same data from different perspectives as supervision for representation learning. Notably, Zhang et al. (2017) trained an au- toencoder that predicts a subset the image channels from another subset, and Radford et al. (2021) trained a vision- languae model that matches image-text pairs with contrastive learning. Promising results from these approaches suggest that data diversity can play a key role in representation learning.
In the context of visual control, the camera is easily accessible instrument that increase data diversity by providing information about the same scene from different viewpoints. For instance, it has been widely-used technique for roboticists to utilize multiple cameras for solving complex manipulation tasks (Akkaya et al., 2019; Akinola et al., 2020; Hsu et al., 2022; James et al., 2022; Jangir al., 2022). Yet these works mostly focus on the improved performance from utilizing multi-view observations as inputs, not investigating the effectiveness of representation learning with diverse data from multiple cameras. A notable exception is the work from Sermanet et al. (2018), which learns niew-invariant representations via contrastive learning. However, enforcing viewpoint invariance assumes that all viewpoints share similar information and thus requires careful curation of positive and negative pairs, similar to other contrastive approaches that often depend on complex design choices about sampling such pairs (Arora et al., 2019).
We present Multi-View Masked Autoencoders (MV-MAE), simple and scalable framework for visual representation learning with diverse data from multiple cameras. Our main idea is to mask randomly selected viewpoints and train autoencoder that reconstructs pixels of both masked and unmasked viewpoints. This allows the model to learn representations of each viewpoint that captures visual information of the current viewpoint but also cross-view information of other viewpoints. To further encourage cross-view representation learning, we propose to train a video autoencoder by masking multiple viewpoints from video frames random. Because the model can utilize information

---

Under review as a conference paper at ICLR 2023
CNN
Masking
Reconstruction
ViT Encoder
ViT Decoder
Front Camno
Framo
Wrist Camera
Front
Camera
Frame
Wrist Camera
Masked Features
Mask Tokens for Each Camera
Figure 1: Illustration of our representation learning scheme. We extract features from each viewpoint with convolutional networks (CNN) and mask all features from randomly selected viewpoints of video frames. We also mask randomly selected features from remaining viewpoints to encourage the autoencoder to learn information of unmasked frames. A vision transformer (ViT; Dosovitskiy et al. 2021) encoder processes visible features to fuse information from multiple views and frames. Then ViT decoder concatenates mask tokens for each view and processes inputs reconstruct frames. We note that the autoencoder reconstructs all frames at the same time.
from the current frame but also information from unmasked frames of the target view, find our approach helps the model to focus on predicting important details, egg., gripper poses. Then we utilize learned representations for visual control by training reinforcement learning agent that learns world model on top of frozen representations and utilizes it for behavior learning (Seo et al., 2022a).
Contributions. We highlight the contributions of our paper below:
We present MV-MAE, simple and scalable framework that leverage diverse data from multiple cameras for visual representation learning. The main idea of MV-MAE is training a video masked autoencoder (Tong et al., 2022; Feichtenhofer al., 2022) with view-masking strategy that encourages the model to learn spatial dependency between viewpoints.
We provide empirical evaluation of MV-MAE on challenging visual manipulation tasks from RLBench (James et al., 2020). Unlike other multi-view representation learning baselines that enforce invariance between multiple viewpoints (Sermanet et al., 2018; Assran et al., 2022), MV- MAE consistently outperforms a single-view representation learning baseline under challenging experimental setup with multiple cameras diverse types.
We demonstrate that data diversity from multiple viewpoints can play a crucial role in representa- tion learning for visual control. In particular, by training a single-view control agent on top of multi-view representations from MV-MAE, we find that our approach significantly outperforms sinlleview representation learning baseline, e.g., we achieve 62.3% success rate on six visual manipulation tasks while the baseline achieves 42.3%.
2 RELATED WORK
Unsupervised visual representation learning. Self-supervised learning on large-scale unlabeled datasets has been actively studied in the domain of computer vision (Noroozi & Favaro, 2016; Chen et al., 2020; Grill et al., 2020; He et al., 2021). 1). A line of research that has been successful is contrastive learning, which learns representations by maximizing the mutual information between
2

---

Under review as a conference paper at ICLR 2023
Masked view reconstruction on Take Lid Off Saucepan (left) and Put Rubbish in Bin (right) tasks from RLBench (James et al., 2020). We visualize ground-truth frames with masked viewpoints (upper two rows) and ground-truth frames with reconstructed frames (lower two rows). We find that the model can successfully reconstruct masked viewpoints.
Figure 2:
different views of images (Chen et al., 2020; He et al., 2020; Tian et al., 2020a:b). The work closest ours is Sermanet et al. (2018), which proposed to learn view-invariant representations with videos collected with multiple cameras. But contrastive approaches often require complex design choices with regard to sampling positive-negative pairs (Arora et al., 2019) and can fail when the assumption for induced invariance does not hold for downstream tasks (Xiao et al., 2021). We instead take simple approach that learns representations via masked reconstruction.
Our work is closely related to denoising autoencoding approaches (Vincent et al., 2010; Pathak et al., 2016) that train an autoencoder to reconstruct masked inputs. Notably, He et al. (2021) proposed a scalable masked autoencoder architecture that masks large portion of an image and only uses visible patches as inputs for solving reconstruction tasks. Our model built on top of its spatiotemporal extension that introduces video autoencoder (Tong et al., 2022; Feichtenhofer et al., 2022). This work follows this line of research by introducing the autoencoder that reconstructs masked viewpoints. Recently, Geng al. (2022) trained unified autoencoder for vision-language modalities with masked token prediction and found that data diversity allows for learning transferable representations. We support this finding with investigation that shows data diversity from multiple cameras can be helpful for representation learning in the context of visual control.
Unsupervised representation learning for visual control. Representation learning from visual observations has also been actively studied for learning to solve control tasks from easily accessible cameras (Watter al., 2015; Oord al., 2018; Gelada al., 2019; Hafner et al., 2019; Yarats et al., 2021b; Srinivas et al., 2020; Castro, 2020: Schwarzer et al., 2021a; Yarats et al., 202 Seo et al., 2022b). Recent works have demonstrated that self--upervised learning can enable agents to solve visual control tasks with frozen representations (Stooke et al., 2021; Schwarzer et al., 2021 bb; Nair al., 2022; Parisi et al., 2022; Xiao al., 2022; Seo et al., 2022a; Radosavovic et al., 2022). Our work further demonstrates that frozen representations learned with diverse data from multiple camera viewpoints can be effective for solving challenging visual manipulation tasks.
Visual control with multiple cameras. Leveraging multiple cameras has long been considered practical and feasible technique in robotics, as the camera usually an affordable and ubiquitous device (Sola al., 2008; Carrera al., 2011; Yang al., 2021). Based recent advances in computer vision and robot learning, there have been several approaches that utilize multi-view data from multiple cameras for visual control (Sermanet al., 2018; Akinola al., 2020; Zhan al., 2020; Chen et al., 202 Hsu et al., 2022: Jangir et al., 2022; Shridhar et al., 2022; Guhur et al., 2022). While most approaches utilize multi-view data directly inputs for robots, recent works have demonstrated that self-supervised learning that learns view-invariant representations (Sermanet al., 2018) 3D keypoints (Chen et al., 2021a) can be useful for downstream tasks. This work also demonstrates that multi-view representation learning can be beneficial for visual control.
3 MULTI-VIEW MASKED AUTOENCODERS FOR VISUAL CONTROL
To fully exploit the multi-view data for representation learning, it is important to encourage the repre- sentations to learn crews-view information between viewpoints. In this section, first introduce the
3

---

Under review as a conference paper at ICLR 2023
Representation Learning
Multi-view Control
Single-view Control
Sample Minibatch
Replay Buffer
Freeze
Store
RL Agegtt
Environment Interaction
Figure 3: Overall pipeline of our framework. the representation learning phase, we learn visual representation from multi-view data sampled from replay buffer, as we have illustrated in Figure I. In the control phase, we train an RL agent upon the frozen representations and store the samples collected from online interaction to the buffer. Thanks to our architecture design that can take inputs of varying lengths, MV-MAE can take inputs of either single-view or multi-view. This allows us to utilize different number of cameras for the representation learning and behavior learning phases. We provide pseudocode of our framework in Appendix B.
Multi-View Masked Autoencoder (MV-MAE) that reconstructs missing pixels of masked viewpoints for multi-view representation learning (see Section 3.1). We then describe how we utilize the visual representations for training reinforcement learning (RL) agent (see Section 3.2).
3.1 MULTI-VIEW REPRESENTATION LEARNING
Our main idea for multi-view representation learning is to learn cross-view information from multiple viewpoints by reconstructing missing pixels of randomly masked viewpoints. However, such masked view reconstruction might be too challenging for the autoencoder without any access to information from missing viewpoints. For instance, reconstructing all details of front camera observation, which contains a broader view of a robot workspace, with only having access to a wrist camera could be extremely challenging (see Figure 4 for examples of front and wrist cameras).
To address this issue, we propose to train a video masked autoencoder (Feichtenhofer et al., 2022; Tong al., 2022) with view-masking that reconstructs missing pixels of randomly masked viewpoints from video frames. Because the autoencoder attends to unmasked neighbor frames from the same view, the model focus modeling important information such as target object positions and the movement of robot arms, while ignoring redundant information such as background for reconstructing masked viewpoints. We provide the overview of representation learning scheme in Figure 1.
Convolutional feature embedding. Unlike prior work that masks random pixel patches (He et al., 2021; Feichtenhofer et al., 2022; Tong al., 2022; Geng al., 2022), embed camera observations into convolutional feature maps following the design of Seo et al. (2022a). Specifically, we downsample 96 66x96x3 input images to convolutional feature maps with the spatial size of 6x by introducing convolutional layers. Note that we separately process observations from each viewpoint with convolutional layers that share parameters. For each viewpoint, we add fixed 2D sin-cos position embeddings (Chen et al., 2021b) to the features. Then we add learnable ID parameters representing
'Seo et al. (2022a) observed that masked image modeling with pixel patch masking (He et al., 2021) can make it difficult for the model to learn fine-grained details within patches, egg., object positions and boundaries. which could be crucial for tasks that require precise control such as visual manipulation tasks.
4

---

Under review as a conference paper at ICLR 2023
each camera and timestep to features of each video frame from different viewpoints. This is based on the idea of Geng et al. (2022) that introduces learnable parameters for vision and language inputs. Then we flatten the features and concatenate them into single sequence.
View masking. We introduce a new view-masking scheme that masks all the features from a randomly selected camera viewpoint to encourage the model to learn cross-view information. Specifi- cally, we mask randomly selected viewpoints from video frames by randomly sampling one viewpoint for each frame. We also mask randomly selected features from remaining viewpoints (see Figure I) because we want the autoencoder to learn not only cross-view information but also the information within each viewpoint by reconstructing find that the proposed view-masking can be more effective than uniform-masking scheme by explicitly encouraging multi-view representation learning (see Figure 7(a) for supporting experiments).
visual observations with masked features. We empirically
Video autoencoding. For autoencoder architecture, we largely follow the design of masked autoen- coders (MAE; He et al. 2021; Feichtenhofer et al. 2022: Tong et al. 2022) that utilize the encoder and decoder consisting of vision transformers (ViT; Dosovitskiy et al. 2021). Specifically, the encoder processes sequence of unmasked features from all viewpoints and video frames through series of ViT layers. Then we concatenate a set of mask tokens with encoded features and add learnable parameters for each viewpoint and each time corresponding features and mask tokens. The decoder processes them through ViT layers and linearly projects them into pixel patch predictions (see Fig- ure 2 for examples of masked view reconstruction). The training loss for the autoencoder is the mean squared eтror between patch predictions and ground-truth pixel patches with spatial size of 16x 16.
3.2 VISUAL. CONTROI.
Once we learn multi-view representations, we train an RL agent on top of the representations for visual control. Because MV-MAE consists of ViTs based on transformer architecture that can take inputs of variable lengths (Vaswani et al., 2017), the encoder can extract both single-view and multi-view representations though utilize multi-view data for representation learning. This can be beneficial in practical scenarios where we can utilize additional third-person cameras during training, but the robot should operate on a fewer number of cameras at deployment time for faster inference.
Reinforcement learning. We consider masked world models (MWM; Seo et al. 2022a) as our base RL algorithm, which learns world model on top of frozen MAE representations and utilizes it for behavior learning (Hafner et al., 2021). Because MWM decouples visual representation and dynamics learning, it naturally aligns with our approach of separately training the autoencoder to learn visual information from multiple cameras. We provide more details in Appendix C.
Training. Throughout the training, we iterate the processes of (i) updating MV-MAE with multi- view data, (ii) training the MWM agent with single-view or multi-view data, and (iii) collecting samples with environment interaction. As inputs MWM agent, we concatenate the convolutional features with proprioceptive states and process them through the MV-MAE encoder. Because MWM model-based approach with a recurrent architecture, we extract representations from a single video frame for behavior learning. This is based on the observation of Feichtenhofer et al. (2022), where image representations from the video autoencoder are found be useful for image recognition.
4 EXPERIMENTS
We design our experiments to investigate the following questions:
Can MV-MAE effectively leverage multi-view data for representation learning?
Can multi-view representation learning be useful for single-view visual control?
Can MV-MAE work with more than two cameras?
What is the effect of each component in MV-MAE?
5

---

Under review as a conference paper at ICLR 2023
(a) Pick Up Cup
(b) Pick and Lift
(c) Phone On Base
Figure 4: Examples of multi-view data consisting front and wrist camera observations used in our RLBench (James et al., 2020) experiments. Front camera observations provide a broad look robot workspace, and wrist camera observations provide closer look at target objects. We provide the examples of multi-view data from all six tasks in Appendix D.
4.1 SETUP
Environments. We investigate the effectiveness of representation learning with data from multiple viewpoints on challenging visual manipulation tasks from RLBench (James et al., 2020) - a standard benchmark for vision-based robotics which has been shown to serve experiments (James & Davison, 2022). While RLBench is originally designed to evaluate the performance in sparse reward setup, we design dense rewards for six manipulation tasks. Moreover, to ease the difficulty of exploration in large action space, we enforce a robot gripper to be in an upright position without rotation. Following James Davison (2022). we also fill a replay buffer with 100 expert demonstrations. We find these schemes allow to focus effect of visual representation learning while avoiding the challenge from sparse rewards and hard exploration. Unlike prior approaches that utilize path planner with the policy to output next best gripper pose (James Davison, 2022; James et al., 2022), our RL agent outputs relative change in gripper position. We provide further details and source code for the environments in Appendix C.
proxy for real-robot
investigating the
Implementation. For all experiments, we use only 96x96 RGB observations from each camera. We downsample images by 16x obtain convolutional feature maps with spatial size of 6x6. Unlike prior approaches that train video masked autoencoders (Feichtenhofer et al., 2022; Tong et al., 2022), do not utilize temporal downsampling because the agent operates each image. For view-masking, we apply random siew-masking independently to each frame in video consisting of 4 frames. We compute the masking ratio in our experiments by including the number of features from masked viewpoints; hence 50% masking ratio means no features are masked from remaining viewpoints in a two-camera setup. Our autoencoder consists of the 8-layer ViT encoder and the 6-layer VIT decoder, where the embedding dimension is set to 256. We do not utilize a learning rate schedule as replay buffer keeps receiving new samples. Following the design of Seo al. (2022a), we also introduce a reward prediction objective for the autoencoder to encode task-relevant information into visual representations. We provide more details and source codes for reproducing our results in Appendix C.
Baselines and our framework. We consider following basclines for our experiments. For a fair comparison, ensure that all methods the same amount of data for representation learning. Specifically, for a single-view representation learning baseline, we train a single-view encoder on camera observations from both viewpoints. We also the same architecture for all methods. More details on baselines are available in Appendix A.
Masked world models (MWM; Seo al. 2022a): MWM learns visual representations of single viewpoint by training an image autoencoder and trains a world model on top of frozen representations. Because MWM does not encode cross-view information into representations, comparison with MWM evaluates the benefit of multi-view representation learning.
MWM Masked Siamese Network (MSN; Assran et al. 2022): MSN learns visual representations by matching the representations of masked images and the augmented images. We modify MSN to learn cross-view information by making it match the representation of the masked viewpoint with the unmasked viewpoint. We apply MSN MWM and use it as baseline. Comparison with MWM + MSN evaluates the benefit of our approach against representation learning that enforces view-invariance via positive pair matching (Grill et al., 2020; Assran al., 2022).

---

Under review as a conference paper at ICLR 2023
MAM MVVMMEE
MWM MSN
Success Rate (1%)
60
40
30
10
Mult-VVew Control (Front Wris)
SingleVView Control (Frent)
Single-View Control (Wrist)
Figure 5: Aggregate success rate of multi-view visual control and single-view visual control agents on six visual manipulation tasks from RLBench (James et al., 2020). We find that MV-MAE consistently outperforms both single-vies ninw and multi-vier baselines across all setups. Importantly, single-view control agents trained upon multi-vier representations from MV-MAE achieve superior performance to other approaches, demonstrating the importance of data diversity. The result shows the mean and standard deviation averaged over 24 runs. We provide the learning curve for all tasks Appendix F.
Time Contrastive Network (TCN; Sermanet al. 2018): TCN learns visual representations through contrastive learning that utilizes observations from simultaneous viewpoints as positive and frames from the same view but taken from different times as negative. We consider TCN as baseline to evaluate our approach against the approach that enforces view-invariance via contrastive loss requiring complex scheme for sampling pnsitivennegativ pairs.
MWM + MV-MAE (Ours): We build our approach upon MWM by learning a world model on top of frozen representations from MV-MAE. This differs from MWM in that train the video autoencoder with view-masking to learn cross-view information from multi-view data.
4.2 QUANTITATIVE RESULTS
Multi-view control with front and wrist cameras. We first evaluate the effectiveness of our multi-view representation learning scheme in a multi-view visual control setup, where the agent operates both front and wrist cameras. For all baselines, and use concatenated features as inputs to the agent. For our approach, we use generic representations from single multi-view encoder inputs. In Figure 5, we first observe that MWM MV-MAE and MWM + MSN outperform MWM, which shows the importance of multi-view representation learning. However, we find that TCN significantly fails to solve most of the tasks. This shows the critical drawback of TCN, which suffers from mode collapse when negative samples are too similar (e.g., wrist camera observations look similar after grasping the objects our case). We also find that MWM + MV-MAE outperforms both multi-view baselines, demonstrating that masked view reconstruction can effectively leverage multi-view data for representation learning compared to other approaches that enforce new-invariane despite its simplicity.
extract features from each viewpoint
Single-view control. We also report the performance of single-view visual control agents in Figure 5. For single-view control experiments, we learn visual representations using multi-view data consisting of front and wrist camera observations and train the RL agent that operates either front wrist camera. Interestingly, we find MWM + MV-MAE significantly outperforms MWM in both setups, which shows that multi view representation learning to single-view data for behavior learning. This result suggests that our approach can be useful for practical scenarios where we utilize additional third- third-person cameras during training, but the robot should operate on a single wrist camera at deployment time.
also be effective when only have access
Three camera experiments. One limitation of multi-view baselines is that is difficult to scale up them into setup with more than two cameras because they learn representations by matching the representation of two views. Meanwhile, our approach is naturally applicable to a setup with more than cameras because approach simply reconstructs multi-view data from all cameras. To
7

---

Under review as a conference paper at ICLR 2023
100
Takn
Fix Cap
92
Thin Unmoni Shand
80-
-NV-NAE (Front Wrat
80
-NV-NAE (Front Let Right
Suacess Ratt (55)
Success Rate (55)
60
Rate
MWM
-NN-WAE
MN--AAE WikI
00
(Furt 20000 90000 75000 102000 125000 150003
20000
40000
6C003
SE000
100003
25300
52000
75030
132000 125000 190002
Environment Stops
Envfronmant Stips
Envirooment Stips
25
Pickans Ltt
Phoro On Buse
40
P. RLeeRe in
MAK
-MVNNE (Foot WisS)
(Froit- -war -MV-NAE Lett
Supcese Rate (5)
20
-NN-MAE
-MUAAE (Froet
Succoss Ratte (%)
Rato
32
20
20000
43000
c0030
m9502 100000 100000
0
25000
50500
75000
100000 125000 150000
20000
43200
63000
10000 10000
Environment Steps
Endfronment Stips
Environment Shaps
Figure 6: Learning curves of single-view view visual control agents that operate on the front camera for solving six visual manipulation tasks from RLBench (James et al., 2020) as measured on the success rate. Front Wrist and Front Left Right in legends denote the combination of cameras used for multi-view representation learning. The solid line and shaded regions represent the mean and standard deviations, respectively, across four runs.
empirically demonstrate this, report the performance of MV-MAE with three cameras, i.e., front camera, left shoulder camera, and right shoulder camera (see Appendix E for examples of camera observations). We consider this setup motivated by the usage of multiple third-persod cameras in prior works (Akkaya et al., 2019; Akinola et al., 2020). Figure 6 shows the performance of representation learning with the three cameras, where find that MV-MAR with the three cameras outperforms MV-MAE with front and wrist cameras. This shows that the model can capture fine- fine-grained details from three third-person cameras without using the wrist camera, which provides closer look at the objects. However, we find that several RLBench tasks are still very challenging to solve only using the front camera for behavior learning, egg., success rates of single-view control agents on Pick and Lift are very low, which necessitates follow-up works representation learning for visual control. Given these results, large-scale investigation into the effect of camera configurations on representation learning for visual control could be an interesting future direction.
4.3 ABLATION STUDY AND ANALYSIS
Effect of view-masking. To investigate the effect of the proposed view-masking scheme, we report the performance with and without view-masking in Figure 7(a). find that utilizing the view-masking strategy for training the video autoencoder achieves significantly better performance than uniform-masking which supports that new-masking encourages the autoencoder to learn representations capturing useful cross-view information. On the other hand, we observe that view- masking can be harmful when training the image autoencoder unlike training the video autoencoder. This is because masked view reconstruction could be challenging task for the image autoencoder and thus prevent the model from learning useful representations, as we mentioned in Section 1.
Effect of video autoencoding. In Figure 7(a), we also investigate the effect of video autoencoding by reporting the performance of MV-MAE with and without video autoencoding. We find that performance significantly degrades without video autoencoding, which demonstrates that enabling the model to have access to unmasked frames of the same view is indeed crucial for representation learning with masked view reconstruction. However, we find that training the video autoencoder not effective without the view-maae maw-mns scheme, achieving the image autoencoder. This shows that the model can learn to exploit additional information from multiple frames for solving reconstruction tasks, making it difficult to learn useful representations.
worse performance than training
8

---

Under review as a conference paper at ICLR 2023
30-
MYNAEC Mass
MY-AAE Vie Mlaskingg
MV-MAEED of View Nisainn
Rato
02-
Success Rane
40
Succoss Fatt
NUUMGEE
Masking
NUMAA (ilso Vinn Mashing XI
-M-MAE Plac: 59%1
12
NUMAAE (1lko N/w Mapking
10
-M-MAE Porr 79%4
MUMAEE (19500 X Vie Sariing X0
-M-NAA Blace 50%5
Envronment 50000 100000
0C00O
20003
40280
B0000 100000
10000
20000
42000
50000
Envtronment Stops
Envronment Stops
Ablation study
(b) Masking ratio
(c) Number of masked views
Figure 7: Learning curves of single-view visual control agents operating on the front camera for solving three manipulation tasks from RLBench (James et al., 2020), investigating the effect of (a) view masking and video autoencoding and (b) masking ratio. (c) We report the performance of MV-MAE trained with four cameras (i.e., front, wrist, left shoulder, and right shoulder) and varying number of masked views (i.e., 0, 1, 2, and 3). The solid line and shaded regions represent the mean and stratified bootstrap confidence interval across 12 runs
Masking ratio. In Figure 7(b), we find that performance of MV-MAE keeps increasing with
higher masking ratio, achieving the best performance with an extremely high masking ratio of 95%. We hypothesize this is because spatial information redundancy (He et al., 2021) is more significant in camera observations from visual manipulation tasks than natural images. This also aligns with the observation of prior works (Feichtenhofer al., 2022: Tong al., 2022) where the high masking ratio of 90% has shown to be effective for videos with more information redundancy.
Number of masked views. In Figure 7(c), we further investigate how the proposed view-maskin scheme works with the different number of masked views. Specifically, we report the performance of MV-MAE with four cameras (i.e., front, wrist, left shoulder, right shoulder) and varying number of masked views. We observe that masking more views leads to better performance, outperforming a uniform-masken baseline. This shows that the proposed wew-masking can prevent the model from exploiting information redundancy and effectively encourage multi view representation learning.
5 DISCUSSION
We have presented Multi-View Masked Autoencoder (MV-MAE), a simple and scalable framework for multi-view representation learning with multiple cameras. Our experimental results demonstrate that multi-view representation learning can significantly outperform both single-view and multi-view representation learning baselines by a large margin on challenging visual manipulation tasks from RLBench (James et al., 2020). These results suggest that data diversity from multiple viewpoints can play key role representation learning for visual control. We believe that our work can be useful for various robot learning scenarios with multiple cameras, by providing a framework that can work with different number of cameras for representation learning and behavior learning phases. We hope our work facilitates future research on leveraging multiple cameras for representation learning.
Limitation and future directions. One limitation of our approach is that the compute cost of MV-MAE increases quadratically as the number of viewpoints increases. While we find it possible to train relatively large model thanks to the extremely high masking ratio, it would be interesting to incorporate more scalable architecture (Jaegle et al., 2021). As we mentioned in Section 4.2, would also be interesting to consider large-scale investigation into the effect of diverse camera configurations on representation learning, by conducting pre-training large-scale robotics datasets consisting of multi-view data (Dasari et al., 2019). Moreover, by combining the idea of MV-MAE and multimodal MAE (Geng al., 2022), training RL agent that can operate any modality or viewpoints is a direction we would like to pursue in future works. Finally, we are keen to evaluate the effectiveness of nie representation learning with MV-MAE real robots.

---

Under review as a conference paper at ICLR 2023
REPRODUCIBILITY STATEMENT
We provide the implementation details of our approach and baselines in Section 4, Appendix A, and Appendix C. We also provide our source code to reproduce main results Appendix C.
ETHICS STATEMENT
Learning visual control agents operating on easily accessible cameras can be helpful for various applications, such factory automation, autonomous driving, and training collaborative robots. However, there could be a scenario where the robot is misused by malicious users, e-g,, attackers might inject rewards that teach undesirable behaviors which could be harmful to society. To prevent such abuses, it is important to develop algorithms with an eye towards safety as well as performance.
REFERENCES
Iretiayo Akinola, Jacob Varley, and Dmitry Kalashnikov. Learning precise 3d manipulation from multiple uncalibrated cameras. In 2020 IEEE International Conference on Robotics and Automation (ICRA), 2020.
Ilge Akkaya, Marcin Andrychowicz, Maciek Chociej, Mateusz Litwin, Bob McGrew, Arthur Petron, Alex Paino, Matthias Plappert, Glenn Powell, Raphael Ribas, et al. Solving rubik's 's cube with robot hand. arXiv preprint 11Xiv:1910.071113 2019.
Jean-Baptiste Alayrac Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lene, Arthur Mensch, Katie Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. arXiv preprint arXiv:2204. 14198, 2022.
Sanjeev Arora, Hrishikesh Khandeparkar, Mikhail Khodak, Orestis Plevrakis, and Nikunj Saun- shi. A theoretical analysis of contrastive unsupervised representation learning. arXiv preprint 1902.099020222 2019.
Mahmoud Assran, Mathilde Caron, Ishan Misra, Piotr Bojanowski, Florian Bordes, Pascal Vincent, Armand Joulin, Michael Rabbat, and Nicolas Ballas. Masked siamese networks for label-efficient learning. arXiv preprint arXiv:2204.07144. 2022.
Gerardo Carrera, Adrien Angeli, and Andrew J Davison. of a multi-camere rig. In 2011 IEEE International Conference on Robotics and Automation (ICRA), 2011.
automatic extrinsic calibration
Pablo Samuel Castro. Scalable methods for computing state similarity in deterministic markov decision processes. In Proceedings of the AAAI Conference on Artificial Intelligence, 2020.
Boyuan Chen, Pieter Abbeel, and Deepak Pathak. Unsupervised learning of visual 3d keypoints for control. In International Conference on Machine Learning. 2021a.
Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever. Generative pretraining from pixels. In International Conference Machine Learning, 2020.
Xinlei Chen, Saining Xie, and Kaiming He. An empirical study training self-supervised vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021b.
Sudeep Dasari, Frederik Ebert, Stephen Tian, Suraj Nair, Bernadette Bucher, Karl Schmeckpeper, Siddharth Singh, Sergey Levine, and Chelsea Finn. Robonet: Large-scale multi-robot learning. In Conference on Robot Learning, 2019.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. International Conference Learning Representaitons, 2021.
10

---

Under review as a conference paper at ICLR 2023
Christoph Feichtenhofer, Haoqi Fan, Yanghao Li, and Kaiming He. Masked autoencoders as spatiotemporal learners. arXiv preprint arXiv:2205.0911 220.00913, 2022.
Carles Gelada, Saurabh Kumar, Jacob Buckman, Ofir Nachum, and Mare G Bellemare. Deepmdp: Learning continuous latent space models for representation learning. In International Conference on Machine Learning, 2019.
Xinyang Geng, Hao Liu, Lisa Lee, Dale Schuurams, Sergey Levine, and Picter Abbeel. Multimodal masked autoencoders learn transferable representations. arXiv preprint arXiv:2205. 2tto::2255.222..... 2022.
Jean-Base Cean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, al. Bootstrap your latent: A new approach self-supervised learning. In Advances in Neural Information Processing Systems, 2020.
Pierre-Louis Guhur, Shizhe Chen, Ricardo Garcia, Makarand Tapaswi, Ivan Laptev, and Cordelia Schmid. Instruction-driven history-aware policies for robotic manipulations. arXiv preprini arXiv:2209.04899 2022.
Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. In International Conference on Machine Learning, 2019.
Danijar Hafner, Timothy Lillicrap, Mohammad Norouzi, and Jimmy Ba. Mastering atari with discrete world models. In International Conference on Learning Representations, 2021.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. arXiv preprint arXiv:2111.06377 2021.
Kyle Hsu, Moo Jin Kim, Rafael Rafailov, Jiajun Wu, and Chelsea Finn. Vision- manipulators need also see from their hands. arXiv preprint arXiv:2203.12667, 2022.
Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and Joao Carreira. Perceiver: General perception with iterative attention. In International Conference on Machine Learning, 2021.
Stephen James and Pieter Abbeel. Coarse-to-Fine Q-attention with Learned Path Ranking. arXiv preprint arXiv:2204.01577/ 2022a.
Stephen James and Pieter Abbeel. Coarse-to-Finn Q-attention with Tree Expansion. arXiv preprint arXiv:2204. 12471, 2022b.
Stephen James and Andrew Davison. Q-attention: Enabling efficient learning for vision-based robotic manipulation. IEEE Robotics and Automation Letters, 2022.
Stephen James, Zicong Ma, David Rovick Arrojo, and Andrew Davison. RLBench: The robot learning benchmark & learning environment. IEEE Robotics and Automation Letters, 2020.
Stephen James, Kentaro Wada, Tristan Laidlow, and Andrew Davison. Coarse-to-Fine FFnee Qaattention: Efficient learning for visual robotic manipulation via discretisation. Proceedings of the IEEE/CVF Conference Computer Vision and Patlern Recognition, 2022.
Rishabh Jangir, Nicklas Hansen, Sambaran Ghosal, Mohit Jain, and Xiaolong Wang. Look closer: Bridging egocentric and third-person views with transformers for robotic manipulation. IEEE Robotics and Automation Letters, 2022.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. In International Conference Learning Representations, 2014.
11

---

Under review as a conference paper at ICLR 2023
Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta. R3m: A universal visual representation for robot manipulation. arXiv preprint arXiv:2203. 2022.
Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. European conference on computer vision, 2016.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. In Advances in Neural Information Processing Systems, 2018
Andrew Owens, Phillip Isola, Josh McDermott, Antonio Torralba, Edward H Adelson, and William Freeman. Visually indicated sounds. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.
Simone Parisi, Aravind Rajeswaran, Senthil Purushwalkam, and Abhinav Gupta. The unsurprising effectiveness pre-trained vision models for control. arviv preprint arXiv:2203.03580, 2022.
Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, 2021.
Ilija Radosavovic, Tete Xiao, Stephen James, Pieter Abbeel, Jitendra Malik, and Trevor Darrell. Real world robot learning with masked visual pre-training. In Conference on Robot Learning, 2022.
Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2015.
Max Schwarzer, Ankesh Anand, Rishab Goel, R Devon Hjelm, Aaron Courville, and Philip Bach- man. Data-efficient reinforcement learning with self-predictive representations, In International Conference on Learning Representations, 2021a.
Max Schwarzer, Nitarshan Rajkumar, Michael Noukhovitch, Ankesh Anand, Laurent Charlin, Devon Hjelm, Philip Bachman, and Aaron Courville. Pretraining representations for data-efficient reinforcement learning. In Advances in Neural Information Processing Systems, 2021b.
Younggyo Seo, Danijar Hafner, Hao Liu, Fangchen Liu, Stephen James, Kimin Lee, and Pieter Abbeel. Masked world models for visual control. In Conference on Robot Learning, 2022a.
Younggyo Seo, Kimin Lee, Stephen James, and Pieter Abbeel. Reinforcement learning with action- free pre-training from videos. In International Conference on Machine Learning, 2022b.
Pierre Sermanet, Corey Lynch, Yevgen Chebotar, Jasmine Hsu, Eric Jang, Stefan Schaal, and Sergey Levine. Time-contrastive networks: Self-supervised learning from video. In 2018 IEEE international conference on robotics and automation (ICRA), 2018.
Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Perceiver-actor: multi-task transformer for robotic manipulation. arXiv preprint arXiv:2209.05451. 2022.
Joan Sola, André Monin, Michel Devy, and Teresa Vidal-Calleja. Fusing monocular information in multicamera slam. IEEE transactions robotics, 2008
Aravind Srinivas, Michael Laskin, and Pieter Abbeel. Curl: Contrastive unsupervised representations for reinforcement learning. In Internatial Conference on Machine Learning. 2020.
Adam Stooke, Kimin Lee, Pieter Abbeel, and Michael Laskin. Decoupling representation learning from reinforcement learning. In International Conference on Machine Learning, 2021.
Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT Press, 2018.
12

---

Under review as a conference paper at ICLR 2023
Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive multiview coding. In European conference on computer vision, 2020a.
Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. What makes for good views for contrastive learning? Advances in Neural Information Processing Systems, 2020b.
Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. Videomae: Masked autoencoders are data-efficient learners for sulf--uuer video pre-training. In Advances in Neural Information Processing Systems, 2022.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 2008
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, 2017.
Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol, and Léon Bottou. Stacked denoising autoencoders: Learning useful representations in a deep network with local denoising criterion. Journal of machine learning research, 2010.
Xiaolong Wang and Abhinav Gupta. Unsupervised learning of visual representations using videos. In Proceedings of the IEEE international conference on computer vision, 2015.
Manuel Watter, Jost Springenberg, Joschka Boedecker, and Martin Riedmiller. Embed to control: A locally linear latent dynamics model for control from raw images. In Advances neural information processing systems, 2015.
Tete Xiao, Xiaolong Wang, Alexei A Efros, and Trevor Darrell. What should not be contrastive in contrastive learning. In International Conference on Learning Representations, 2021.
Tete Xiao, Ilija Radosavovic, Trevor Darrell, and Jitendra Malik. Masked visual pre-training for motor control. arXiv preprint 2001v:2203.06173 2022.
Anqi Joyce Yang, Can Cui. Ioan Andrei Bårsan, Raquel Urtasun, and Shenlong Wang. Asynchronous multi-view slam. In 2021 IEEE International Conference on Robotics and Automation (ICRA), 2021.
Denis Yarats, Rob Fergus, Alessandro Lazaric, and Lerrel Pinto. Reinforcement learning with prototypical representations. In International Conference on Machine Learning, 2021a.
Denis Yarats, Amy Zhang, Ilya Kostrikov, Brandon Amos, Joelle Pineau, and Rob Fergus. Improving sample efficiency in model-free reinforcement learning from images. In Proceedings of the AAAI Conference on Artificial Intelligence, 2021b.
Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta-world; world: A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference on Robot Learning, 2020.
Albert Zhan, Philip Zhao, Lerrel Pinto, Pieter Abbeel, and Michael Laskin. A framework for efficient robotic manipulation. arXiv preprint arXiv:2012.07975, 2020.
Richard Zhang, Phillip Isola, and Alexei A Efros. Split-brain autoencoders: Unsupervised learning by cross-channel prediction. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2017.
13

---

Under review as a conference paper at ICLR 2023
A BASELINES
A.I MASKED WORLD MODELS
Masked world models (MWM; Seo al. 2022a) is visual model-based RL method that iterates the representation learning phase and the behavior learning phase. In the representation learning phase, MWM learns visual representation by reconstructing pixels given masked convolutional features. In the behavior learning phase, policy, along with the world model, is learned on top of frozen representation. We explain the details of each phase as follows.
Representation learning. For representation learning, MWM trains autoencoder by (i) recon- structing an image with masked convolutional features and (ii) predicting a reward. MWM processes raw input image using convolution stem, which is convolutional layers followed by flatten layer. Then, MWM randomly masks convolutional features, and feeds visible features into the ViT encoder. The outputs from the ViT encoder, along with (learnable) mask tokens, are fed into the VIT decoder, which reconstructs the raw image. For the auxiliary reward prediction, one additional learnable mask token is concatenated into the inputs of the ViT decoder, and the corresponding output representation followed by linear output head is used to predict the reward.
Behavior learning. Once visual representation is learned, MWM trains an RL agent by actor-critic learning scheme using the imaginary latent states from the world model. For the world model, MWM uses a variant Recurrent State Space Model (RSSM; Hafner et al. 2019). Formally, for a visual task formulated as partially observable Markov decision process (Sutton & Barto, 2018), which is defined as a tuple (,,A,, P, v, 7), O is the observation space, A is the action space, p(o act) is the transition dynamics, is the reward function that maps previous observations and actions to a reward r; = r(Oge, asz) and E [0. 1) is the discount factor. Let 24 be an output from the ViT encoder for current observation o,- The RSSM consists of the following learnable components:
Representation model:
8t ~ 90 (8el (t-11811, 11,
Transition model:
8. po (1]
Visual representation decoder:
& ~ pa & (sc)
Reward predictor:
A. pe (fz(s:).
The representation model extracts model state from previous model state St-1, previous action 04-1- and 24. The transition model predicts future state & without access to 2,. The visual representation model reconstructs 2, to provide learning signal. All model parameters 0 are jointly trained to minimize the negative variational lower bound (Kingma & Welling, 2014). For behavior learning, MWM predicts latent future states and train stochastic actor and deterministic critic to maximize the imagined returns (Hafner et al., 2021). We refer to Seo et al. (2022a) for more details.
A.2 MWM + MASKED SIAMESE NETWORK
MWM + masked siamese network (MSN; Assran et al. 2022) is a baseline that learns visual informa- tion via reconstruction with masked convolutional features, but also cross-view information from different viewpoints. Specifically, we extract the representations of one viewpoint from the encoder, and also the representations of another viewpoint from the target momentum encoder. We first use the representations of the encoder as inputs to the ViT decoder that reconstructs raw pixels. And then compute the representation matching loss that minimizes the distance between two representations. We maintain a target encoder that is updated via an exponential moving average of the (anchor) encoder parameters with a target decay rate of 0.999. For a given (anchor) frame, we regard the corresponding frame from another viewpoint as a target frame. For obtaining representations from each viewpoint, we conduct average pooling on token embeddings from the last layer the encoder. While original MSN utilizes learnable prototypes for obtaining representations, we find it extremely unstable in our setup. So we adopt the simple approach of Grill et al. (2020) that minimizes the distance between L2 normalized representations. We note that this is still called MSN because the main idea of matching the representation of masked viewpoint representations with unmasked viewpoint target representations.
14

---

Under review as a conference paper at ICLR 2023
A.3 TIME CONTRASTIVE NETWORK
Time contrastive network (TCN; Sermanet et al. 2018) a contrastive approach that learns view- invariant representations by attracting the representations of simultaneous viewpoints but making the representations from the same viewpoints be far located, For a given (anchor) frame in one viewpoint, we sample a positive and negative frame as follows. For the positive frame, we use the frame that has the same timestep as the given anchor frame but from another viewpoint. For the negative frame, we choose frame that is temporally faraway frame from the same viewpoint. Specifically, we sample random frame among frames that are at least 30 timesteps away from the anchor frame. After building triplet of anchor, positive, and negative frame, we train the encoder model with triplet loss (Schroff et al., 2015), which is formulated as follows:
Lre = ((x") - f(o")1l
IIf(o") - f(o") + a,
where o", of, and d" are anchor, positive, and negative frames, respectively, f(-) refers class embed- dings from the last layer of ViT encoder and a is margin, which set to 2. In the control phase, we freeze the encoder and use average pooled token embeddings an input for the RL agent.
B PSEUDOCODE FOR MULTI-VIEW MASKED AUTOENCODER
We consider a camera observation of where i E {1, T) denotes video timestep and {1, ... V} denotes a camera viewpoint. For clarity, we let T {I,... ...,, T} and V = {l,,, VJ. We let Castor and Ceritic be the actor and critic loss of MWM (Seo et al., 2022a).
Algorithm Multi View Masked Autoencoders for Visual Control
1: Initialize replay buffer B <- and parameters of autoencoder é, actor tù, critic g
2: for each environment step i do
3: // MULLII-IEW REPRESENTANE
LEARNING
4: Sample minibatch {ot} B
5: for each video timestep / do
6: Get convolutional features 2, 2,connv = =fcone (o))) for
7: Sample {z] ".covv Jven, ~ ViewMask( (2t 1.,,on Joev), where Q2 is set of unmasked views
8:
end for
9: Collect unmasked viewpoints . - S
10: Get reconstructions Jevv f0 fdacader (fancodar (f2) (2..ccar)))s )vEa)) for
11:
Update Φ by minimizing Σt=I" - 110g of
12:
I/ ACTOR CRITIC LEARNING
Sample minibatch {(ot..., of r)r B
14:
Get convolutional features {2]icobr JUEV
if single-vies control with Us then
16:
Get single-view representation h f rencoder ((=1
17:
else
18:
Get multi-vier view representation h fo
fancodar (fz) ccnny hvev)
19:
end if
20:
Update and by minimizing Lactor and feritic with ((h,ar))
21:
I/ COLLECT TRANSITIONS
22:
Get convolutional features {2],, v.conv JUEv from current observation {o", Yeev
if single-view control with v, then
24:
Get single-view representation h; pencoder ({=",; ([=])..oorr)
25:
else
26:
Get multi-view representation h; fencoder ((z)),00v )vev)
27:
end if
28:
Sample action a; ~ facter (h:) and collect reward r; from environment
29:
Store sample to replay buffer B <-
+-BU BU ((o))) ..., oY, ar r:)
30:
end for
15

---

Under review as a conference paper at ICLR 2023
C
IMPLEMENTATION DETAILS
Source code. Source code for reproducing our experimental results is available at:
https:/ pttps://anenymous /aaanyoooomusuu.
4open. wwtt/cc/ee//ccc112233111/3 mmmaa
RLBench details. For RLBench experiments, we designed dense rewards for six manipulation tasks used in our experiments. For simple tasks (i.e., Take Lid Off Saucepan, Pick Up Cup, Take Umbrella Out of Umbrella Stand), we design the reward be the sum of the distance between gripper and target object. For more complex tasks that require more long term behavior (i.e., Pick and Lift, Phone On Base, Put Rubbish in Bin), we first construct some checkpoints where the robot should reach. Then we define the reward to be the sum of the distance between the gripper and the nearest next checkpoint. For disabling rotation, we use path planner with identity quaternion to force the robot to be in upright position. Unlike prior approaches that use the path planner (James Davison, 2022; James et al., 2022; James & Abbeel, 2022a;b) to specify absolute (x, y. 2) position, we train the RL agent to output relative change in (x, y, z) position. This is available by using the action mode EndEf fect pecttore rp PePPoeeiii aP Peeeaanniii in (absolut wollutemmodee eattemmdd=Faall For more details, we refer readers to the source code we have attached.
in the RLBench.
Architecture and optimization details. Our architecture is based on the publicly available source code of Seo et al. (2022a), which is implemented with tfimm² library. For our method and all baselines, we use the same architecture consisting of 8-layer VIT encoder and 6-layer VIT decoder. For each view and time step, we introduce additional ID learnable parameters that have the same embedding size transformer blocks. We add these parameters to 2D fixed sin-cos embeddings and add them to features. We note that these parameters are shared across the same times and the same views. For optimization,
use Adam optimizer (Kingma & Ba. 2015) with the learning rate of 3e - 4, the weight decay le- - 6, and the batch size of 1024. For training MV-MAE, we apply warm-up learning rate scheduling over initial 2500 gradient steps from learning rate of O. We take 1 gradient step per every 2 environment steps. We follow the training schemes and details of Seo et al. (2022a) regarding the architecture, unless otherwise specified.
Computation. We use 8 CPU cores (Intel Xeon Gold 6226R @ 3.9GHZ) and I GPU (NVIDIA GeForce RTX 3090) for our experiments. We find that there no significant difference between all methods with regard to wall time for experiments because there is severe bottleneck from the rendering speed of the RLB ReBench simulator. Running experiments over 150k environment steps takes approximately 18 hours for all methods.
modell
16

---

Under review as a conference paper at ICLR 2023
Hyperparameters. We report the hyperparameters used in experiments in Table 1.
Table 1: Hyperparameters used in experiments. Unless otherwise specified, the same hyperparameters used in MWM (Seo et al., 2022a).
Hyperparameter
Value
Representation learning
Image observation
96 96
Image normalization
Mean: (0.485, 0.456, 0.406), Std: (0.229, 0.224, 0.225)
Autoencoder batch size
1024
Autocncoder initialization steps
10000
Autoencoder warm-up steps
2500
Autoencoder learning rate
3-10-1
Autoencoder masking ratio
0.95 (multi-view), 0.9 (single-view) layers, 4 heads. 256 units
Autoencoder ViT encoder size
Autoencoder ViT decoder size
layers, 4 heads. 256 units
Behavior learning (MWM)
Action repeat
Max episode length
150
Early episode termination
True (when path planner fails)
Reward normalization
True
World model batch size
50
World model sequence length
50
World model tradeoff (3) World model tradeoff free-bits
1.0
0.1
World model ViT encoder size
layers, 4 heads, 128 units
World model VIT decoder size
layers, 4 heads, 128 units
17

---

Under review as a conference paper at ICLR 2023
D TWO-CAMERA EXAMPLES
(a) Take Lid Off Saucepan
Pick Up Cup
(c) Take Umbrella Out Stand
(d) Pick and Lift
(c) Phone On Base
(f) Put Rubbish in Bin
Figure 8: Examples of multi-view data consisting front and wrist camera observations used in our RLBench (James et al., 2020) experiments. Front camera observations provide broad look robot workspace, and wrist camera observations provide a closer look at target objects.
E THREE-CAMERA EXAMPLES
(a) Take Lid Off Saucepan
(b) Pick Up Cup
Take Umbrella Out of Stand
(d) Pick and Lift
(e) Phone on Base
(l) Put Rubbish in Bin
Figure 9: Examples multi-view data consisting left shoulder, front, and right shoulder observations used in RLBench (James al., 2020) experiments.
18

---

Under review as a conference paper at ICLR 2023
F FULL MULTI-VIES EXPERIMENTAL RESULTS
F.1
SINGLE-VIEW VISUAL CONTROL WITH FRONT CAMERA
100
Up
DNI
56387
70
-VSN
70
50387 NSTN
00
INV -NV-WARE MARE TCN
5W00 - MARE
Success Rate (%6
TCN
Success Rate
30
-20
MOM
MSN
20000
40000
60003
3E00O
100000
25300
53000 76000
100000 125000 160003
25300
53000
76030
132000 126000 160003
Environment Steps
Environment Steps
Environment Steps
Phone Dien
25
MVVN
NWA
40-
MVN MBN MVNN NIAMAE
NWA NNN
35-
MMVR MBN
Succees Rate 1%6
20
TCN
HWAN -NV-WAE
Success Rate 186)
MWV MV MANE
TCN
16
Rate
42
25003
50500
73000 100000 125000 150002
20000
S0000
75000
102000 125000 150002
20000
S2000
105000 125000 150002
Environment Steps
Envtronment Stops
Environment Stops
Figure 10: Learning curves of RL agents that operate front camera observations for solving six visual manipulation tasks from RLBench as measured on the success rate. The solid line and shaded regions represent the mean and standard deviations, respectively, across four runs.
F.2 SINGLE- VIEW VISUAL CONTROL WITH WRIST CAMERA
Toko Of Bazzoser
PIK Cup
Teko Unbroal Out of
Succoss Rale (%)
Succasa Hate (5)
Succoss Rate
60
40
MWN
MNNI TCN
20
20
20-
MAN
Steps
Steps
Stepe
Pickand
Phore Bise
Put
80
70
50-
Success Paate
80
90
40
40
10
78260 100000 Steps
75500
50000
102020 Steps
Steps
Figure 11: Learning curves of RL agents that operate on wrist camera observations solving visual manipulation tasks from RLBench as measured on the success rate. The solid line and shaded regions represent the mean and standard deviations, respectively, across four runs.
19

---

Under review as a conference paper at ICLR 2023
F.3 VIL VISUAL CONTROL WITH FRONT AND WRIST CAMERA
100
K/GRK of Concel
NWN Wi Conce KWN INSN 53-
Success Rale (%)
00
TCN
Success Rato (16
KWVM NVV-WAE TCN
Success (16)
20-
40
Concal
M800 MEN
MM--AAE TCN
5000
10000 15000 20000 25000 50200
5000
10000 15000
20000
23000
30000
0
25000
50033 75000
1000C0 123000 150005 Envirorment Stapa
Endironmnnt Steps
Environment Staps
Pick
Phore Samn
54
NWA of Cencul
00
NWW Cencal
100
NWWMW Cencat
00
KWM -MBN
NWN -NN
MWWM- M4N
KNM -NVNNNE
NWN NYV-WN
80
HWMM- Mi WAE
Succers Rate 1963
TON
TON
(66)
TCN
20
20
10
25000
53000
76/00 100000 125000 150000
25000 50000
76000
100000
125000 1E0003
0
20000 E0030
100000 123000 160002 Environment Stops
75000
Environment Staps
Environment Steps
Figure 12: Learning curves of RI. agents that operate on front and wrist camera observations for solving six visual manipulation tasks from RLBench as measured on the success rate. The solid line and shaded regions represent the mean and standard deviations, respectively, across four runs.
G ADDITIONAL EXPERIMENTS
70
100
60
MWWM MV-MAE
60
TON
40
60
Suidees
60-
20
10
Encoder & Decodar:
Decode::
20000
40000
60000
80000
100000
100000 200000
300000 400000
500000
Envtranment
Environment Steps
Model size analysis
Meta-world experiments
Figure 13: Learning curves of single-vies visual control agents that operate the front camera for solving visual manipulation tasks from RLBench (James et al., 2020), investigating the effect of model size. (b) Learning curves of single-view visual control agents that operate on the wrist camera for solving visual manipulation tasks from Meta-world (Yu et al., 2020). The solid line and shaded regions represent the mean and stratified bootstrap confidence interval across 12 and (b) 20 runs.
Model size. In Figure 13(a), investigate the effect of scaling up MV-MAE and find that training larger model for visual representation learning with multi-view data improves sample-efficiency. Further scaling up the model and pre-training MV-MAE on large real-world robotics dataset (Dasari et al., 2019) containing multi-view data would be an interesting future work.
Meta-world experiments. In order to verify that our approach can also work for different bench- mark tasks other than RLBench, we provide additional experimental results on five visual manipu- lation t from Meta-world (Yu et al., 2020) benchmark. Specifically, we train single-view
We report the experimental results Baskethall, Shelf Place, Pick Out Of Hole, Pick Place, and Soccer.
20

---

Under review as a conference paper at ICLR 2023
control agent operating the wrist camera with multi-view representations learned using the front and wrist cameras. For the front camera, we use the use the wrist camera introduced in Hsu et al. (2022). For
camera viewpoint used in Seo
et al. (2022a). For the wrist camera, training the MWM agent, we use the same hyperparameters used in Seo et al. (2022a), egg. mask ratio of 75%, image resolution of 64,64, and taking one gradient step per every five environment steps. For training MV-MAE, we use 95% masking ratio which is the same ratio as used in RLBench experiments. In Figure 13(b), we find that MWM + MV-MAE exhibits better sample-efficiency than MWM, which again shows the benefit of multi-view representation learning. Moreover, we find that baselines struggle to outperform the single- view baseline MWM, and TCN completely fails with the wrist camera similar to the observation in RLBench experiments (see Figure 11), This again demonstrates the wide applicability of our approach to diverse tasks with diverse type of cameras.
Embedding analysis. We also visually demonstrate how representations learned differently for Figure 14. Specifically, we train multi-view agents using MV-MAE and TCN for solving Pick Up Cup task and extract the representations from the front and wrist camera using the visual encoders. Then we visualize the representations using 1-SNE (Van der Maaten & Hinton, 2008). We observe that MV-MAE representations of the front and wrist cameras are disentangled from each other, but one can see that representations from both cameras have similar representation structure. This is because cross-view information while also learn the information within each viewpoint simultaneously. On the other hand, front and wrist camera representations from TCN are entangled, which shows how qualitatively MV-MAE and TCN learn different representations.
MV-MAE and TCN that enforces
encourage the model learn
Front
Front Witst
(a) MV-MAE representations
(b) TCN representations
Figure 14: t-SNE (Van der Maaten Hinton, 2008) visualization of representations extracted from front and wrist camera observations from (a) MV-MAE and (b) TCN.
21


Reviews:
[{"review_id": "sR5raVGvYVY", "reviewer": "Reviewer_HaAG", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper explores multi-view representation learning and introduces the Multi-View Masked Autoencoder (MV-MAE) framework. A video autoencoder is updated to function with multi-view input video streams. The learned representations are shown to be useful for the downstream task of visual control by training reinforcement learning agents. Evaluation is conducted on visual manipulation tasks from RLBench. Results show improvement over baselines. ", "summary_of_the_review": "The topic area, multi-view representation learning, is interesting.  Empirically, the proposed method (MV-MAE) shows improvement over baselines on the downstream task of visual control. However in terms of technical novelty, the representation learning component is a slightly tweaked masked autoencoder. The visual control portion exists just for evaluating the learned representations, and uses an off-the-shelf RL algorithm from a recent paper. The manuscript is of high quality but very little space is allocated to describing the proposed method (essentially just Section 3.1). For me, the paper reduces to applying masked autoencoders to a pre-existing visual control task. The results are compelling, but novelty is lacking, which makes this a tough paper to review. \n\nUltimately, I feel this paper falls on the \"marginally below the acceptance threshold\" side of things. For me, I would want the paper to allocate more space to method description: highlighting important details that make it work (e.g., is it the specific choice of view masking strategy?) and adding experiments that reflect why this is the right way to structure things in the multi-view case. ", "strength_and_weaknesses": "Strengths\n\n- Interesting problem area.\n\n- Well written and easy to follow.\n\n- Evaluated versus baselines, including an ablation study.\n\nWeaknesses\n\n- Lack of technical novelty\n\nThe proposed MV-MAE is an agglomeration of recent techniques. The underlying method is MAE [1], but using input in the style of [2] (convolutional features, instead of raw pixels). A learnable parameter reflecting the camera viewpoint / frame timestep is added to each token in the style of [3]. The rest follows a standard video masked autoencoder [4]. Though a view masking strategy is proposed, it is ultimately not compared against any alternative strategies. The visual control portion of the paper uses representations from MV-MAE as input to an off-the-shelf RL algorithm from [2] with evaluation from benchmark suite RLBench.\n\n[1] He, Kaiming, et al. \"Masked autoencoders are scalable vision learners.\" IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.\n\n[2] Younggyo Seo, Danijar Hafner, Hao Liu, Fangchen Liu, Stephen James, Kimin Lee, and Pieter Abbeel. \"Masked world models for visual control.\" Conference on Robot Learning, 2022a.\n\n[3] Geng, Xinyang, et al. \"Multimodal Masked Autoencoders Learn Transferable Representations.\" International Conference on Machine Learning: Workshop on Pre-Training, 2022.\n\n[4] Feichtenhofer, Christoph, et al. \"Masked Autoencoders As Spatiotemporal Learners.\" arXiv preprint arXiv:2205.09113 (2022).\n\n- Lack of detail\n\nMost of the pieces of the proposed approach are from pre-existing works, therefore no detail is given in the manuscript outside of the reference. For example, what's the format of the learnable parameter representing camera ID / frame ID and how are they integrated? Additionally, what are the optimization details and how does the iterative training process work?\n\n- Limited evaluation\n\nThough the evaluation is fairly extensive, the experiments don't give insight into specific design choices that are crucial for MV-MAE to \"work\". For example, what is the impact of the view masking strategy? ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "p10xA_VajM", "reviewer": "Reviewer_Zds9", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents Multi-View Masked Autoencoder (MV-MAE) for multi-view representation learning and show its benefits in visual manipulation tasks. Experiments are conducted on RLBench demonstrating that MV-MAE (with MWM) significantly outperforms baselines (MWM, MWM+MSN, TCN) consistently across various setups. Ablation experiments provide enough details to understand the effects of different design choices. Written presentation is clear and easy to read & understand. ", "summary_of_the_review": "As mentioned above, the proposed approach is novel and experimental results solid, I would recommend an acceptance for this paper.", "strength_and_weaknesses": "# Strength\n- The idea of multi-view representation learning with MAE and applied to visual control tasks is novel and interesting.\n- Experimental results are strong and superior to baselines. The set of selected baselines fully cover most cases of comparisons(to understand the differences). Ablation experiments also fully cover most of design choices.\n- Written presentation is great (high clarity and easy to follow).\n\n# Weakness\n- Experiments are done on only one benchmarks.\n- On the downstream application of visual control tasks, only one model, aka MWM, is used with MV-MAE.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "f0qHn8eLgLy", "reviewer": "Reviewer_ntdx", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Motivated by the fact that data diversity plays a key role in representation learning, this paper proposes \"Multi-View Masked Autoencoder\" to leverage the cross-view and cross-frame information to learn the visual representation for control tasks. Experiments validate that the proposed multi-view representation learning framework outperforms both single-view and other multi-view representation learning baselines. ", "summary_of_the_review": "The paper is well written and it has its significance in experimental results as the improvement over baselines is noticable and the ablation study is well executed. But the technical novelty is limited as it directly applies existing Visual Control MAE model into the cross-view scenario. The message about the importance of using multi-view information is delivered by many other vision work already so it's not new as well. Given the great results and the marginal novelty, the paper straddles right on the acceptance threshold, leaning a bit towards rejection due to the lack of new messages. ", "strength_and_weaknesses": "__Strengths:__\n- The paper writing is in general clear and easy to follow. \n- The performance gain is significant compared with single-view and other multi-view baselines. Ablation experiments are also on point and validate various design choices\n-  Detailed appendix about reproducibility.\n\n\n__Major Weaknesses:__\n- \"Method\" part is argurably the weakest part of the paper, as this work pretty much follows the _Masked World Models for Visual Control_ paper and extend it into a cross-view setting without major changes. It applies MAE into the multi-view scenario and validates its performance in this setting.\n- The message this work delivers is well-known already. It is obvious that learning from cross-view cameras can boost the single-view results.   \n- Some minor issues regarding the paper writing, typo and citations. (See below)\n\n__Minor Issues:__\n- First paragraph of introduction. starting off the paper with multi-modal learning is inappropriate. The paper is using camera only and it's not comparing itself with nor improving upon multi-modal approaches. Quoting multi-view camera papers makes more sense in backing up your argument in diversity, as opposed to quoting multi-modal papers.\n- Do not include Google Brain in the citation of \"Time-contrastive networks: Self-supervised learning from video\" \n- Typo. Figure 1 caption, the autoencoder reconstruct\"s\" all frames at the same time", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "-p1MyOGalf", "reviewer": "Reviewer_mrqQ", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper designs a multi-view masked autoencoder for learning features for visual control. The motivation is that better visual representation leads to better control, which can be obtained from multi-view pre-training using the proposed framework. Results on RLBench have validated the effectiveness of the proposed framework and each introduced components. ", "summary_of_the_review": "This paper presents a framework for using multi-view visual pre-training for robot control. Even though sound and promising, the technical contribution and the experimental evaluations are both limited. ", "strength_and_weaknesses": "**Strength**\n\n- This paper first verified the effectiveness of using multi-view data for pre-training. Moreover, the multi-view pre-training also helps the single-view control tasks. \n- The proposed framework MV-MAE is conceptually simple yet works great. It out-performs the single-view baseline and alternative self-supervised methods baseline. Each sub-modules also contribute to the final performance. \n- Experiments are solid and the results are promising.\n- Writing is easy to follow. Related work is properly discussed.\n\n**Weakness**\n\n- To me, the problem setting isn't challenging enough to test transfer learning. The pre-training and the testing scene are all RLBench and almost the same (same background, robot, etc). Compared to the normal setting in CV tasks, the training-testing gap isn't significant enough. It's encouraging that the visual pre-training still help in this scenario, but it could be more convincing if such representations can be tested under different setting. \n- Even though the masked auto-encoder (MAE) models outperfoms the Masked Siamese Network (MSN) and Time Contrastive Network (TCN), I don't get the motivation behind it. What's the issue that multi-view MAE addressed? What's the scenario that MSN/TCN fail to handle but MAR is able to? What's the difference between the learned features? Can we somehow visualize it and see the difference clearly? \n- The technical contribution is somewhat limited as 1) video MAE is proposed in Feichtenhofer et al., 2022, and 2) the benefit of using multi-view data for control has been studied before (related work last paragraph). This work is a bit incremental considering prior work MWM (Seo et al. 2022a). \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject