Enhancing Visual Domain Robustness in Behaviour Cloning via Saliency-Guided Augmentation

Authors: zheyu  zhuang
ruiyu  wang
nils  ingelhag
ville  kyrki
danica  kragic
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-06-07 06:19:23

Abstract: In vision-based behaviour cloning (BC), traditional image-level augmentation methods such as pixel shifting enhance in-domain performance but often struggle with visual domain shifts, including distractors, occlusion, and changes in lighting and backgrounds. Conversely, superimposition-based augmentation, proven effective in computer vision, improves model generalisability by blending training images and out-of-domain images. Despite its potential, the applicability of these methods to vision-based BC remains unclear due to the unique challenges posed by BC demonstrations; specifically, preserving task-critical scene semantics, spatial-temporal relationships, and agent-target interactions is crucial. To address this, we introduce RoboSaGA, a context-aware approach that dynamically adjusts augmentation intensity per pixel based on input saliency derived from the policy. This method ensures aggressive augmentation within task-trivial areas without compromising task-critical information. Furthermore, RoboSaGA seamlessly integrates into existing network architectures without the need for structural changes or additional learning objectives. Our empirical evaluations across both simulated and real-world settings demonstrate that RoboSaGA not only maintains in-domain performance but significantly improves resilience to distractors and background variations.

Main paper: 
 introduction: 
 Vision-based behaviour cloning (BC) has made significant strides in transferring behaviour from expert demonstrations to robot skills [1, 2] . These demonstrations involve intricate spatial-temporal interactions between the agent and its environment and often require integration among multiple sensory modalities. However, given the high costs associated with collecting demonstration data, especially in real-world settings [3, 4] , datasets often prioritise task-related variability over visual diversity [5, 6] . This prioritisation presents considerable challenges for generalisation under visual domain shifts, such as variations in lighting, shadows, distractors, or backgrounds.While image-level augmentation techniques such as Random Crop and Colour Jitter preserve the intricacy of BC demonstrations and enhance in-domain performance [7, 8] , they fall short in addressing challenges across the broader visual domain (Sec. 4.2). Superimposition-based augmentation techniques, such as selectively removing parts of an image [9, 10] or overlaying images [11] , have proven effective in the computer vision community. However, their application in vision-based behaviour cloning remains limited due to the need to preserve task-critical semantics, spatial-temporal relationships, and agent-target interactions. To this end, we introduce RoboSaGA, a Saliency-Guided Augmentation designed for vision-based behaviour cloning. It utilises the input saliency back-propagated from visual features as per-pixel blending factors, allowing the preservation of task-critical regions while superimposing the in-domain images onto out-of-domain (OOD) images. Crucially, RoboSaGA does not require additional learnable modules or specific learning objectives, making it readily applicable to multi-view inputs and compatible with various BC policies.8th Conference on Robot Learning (CoRL 2024), Munich, Germany.< l a t e x i t s h a 1 _ b a s e 6 4 = " i R d L 4 V e m / q a w V I R N V 2 8 b q e 8 z i c M = " > A A A C B 3 i c b V D L S s N A F J 3 U V 6 2 v q E t B g k W o m 5 K I r 2 X R j c s K 9 g F N K Z P J p B 0 6 m Q k z N 0 I J 3 b n x V 9 y 4 U M S t v + D O v 3 H a Z q G t B 4 Y 5 n H M v 9 9 4 T J J x p c N 1 v q 7 C 0 v L K 6 V l w v b W x u b e / Y u 3 t N L V N F a I N I L l U 7 w J p y J m g D G H D a T h T F c c B p K x j e T P z W A 1 W a S X E P o 4 R 2 Y 9 w X L G I E g 5 F 6 9 m H U y / x A 8 l C P Y v N l P g w o 4 P G 4 4 p N Q w k n P L r t V d w p n k X g 5 K a M c 9 Z 7 9 5 Y e S p D E V Q D j W u u O 5 C X Q z r I A R T s c l P 9 U 0 w W S I + 7 R j q M A x 1 d 1 s e s f Y O T Z K 6 E R S m S f A m a q / O z I c 6 8 m e p j L G M N D z 3 k T 8 z + u k E F 1 1 M y a S F K g g s 0 F R y h 2 Q z i Q U J 2 S K E u A j Q z B R z O z q k A F W m I C J r m R C 8 O Z P X i T N 0 6 p 3 U T 2 / O y v X r v M 4 i u g A H a E K 8 t A l q q F b V E c N R N A j e k a v 6 M 1 6 s l 6 s d + t j V l q w 8 p 5 9 9 A f W 5 w / d J J n y < / l a t e x i t > f ✓ (•) < l a t e x i t s h a 1 _ b a s e 6 4 = " i q J y B E e y E G b l T a t 2 i x D E g A Y J R q o = " > A A A B 9 X i c b V C 7 T s M w F L 3 h W c q r w M h i U S E x V Q n i N V a w M B a J P q Q 2 V I = " > A A A B 9 X i c b V C 7 T s M w F L 0 p r 1 J e B U Y W i w q J q U o Q r 7 G C h b F I 9 C G 1 o X I c p 7 X q 2 J H t A F X U / 2 B h A C F W / o W N v 8 F p O 0 D L k S w f n X O v f H y C h D N t X P f b K S w t r 6 y u F d d L G 5 t b 2 z v l 3 b 2 m l q k i t E E k l 6 o d Y E 0 5 E 7 R h m O G 0 n S i K 4 4 D T V j C 8 z v 3 W A 1 W a S X F n R g n 1 Y 9 w X L G I E G y v d d w P J Q z 2 K 7 Z U 9 j X v l i l t 1 J 0 C L x J u R C s x Q 7 5 W / u q E k a U y F I R x r 3 f H c x P g Z V o Y R T s e l b q p p g s k Q 9 2 n H U o F j q v 1 s k n q M j q w S o k g q e 4 R B E / X 3 R o Z j n U e z k z E 2 A z 3 v 5 e J / X i c 1 0 a W f M Z G k h g o y f S h K O T I S 5 R W g k C l K D B 9 Z g o l i N i s i A 6 w w M b a o k i 3 B m / / y I m m e V L 3 z 6 t n t a a V 2 N a u j C A d w C M f g w Q X U 4 A b q 0 A A C C p 7 h F d 6 c R + f F e X c + p q M F Z 7 a z D 3 / g f P 4 A V o y T E w = = < / l a t e x i t >x < l a t e x i t s h a 1 _ b a s e 6 4 = " 1 Y 8 B P K N M r 6 L u g N y e y P 9 U m z 9 r 2 A A = " > A A A B 9 X i c b V C 7 T s M w F L 0 p r 1 J e B U Y W i w q J q U o Q r 7 G C h b F I 9 C G 1 o X I c p 7 X q 2 J H t A F X U / 2 B h A C F W / o W N v 8 F p O 0 D L k S w f n X O v f H y C h D N t X P f b K S w t r 6 y u F d d L G 5 t b 2 z v l 3 b 2 m l q k i t E E k l 6 o d Y E 0 5 E 7 R h m O G 0 n S i K 4 4 D T V j C 8 z v 3 W A 1 W a S X F n R g n 1 Y 9 w X L G I E G y v d d w P J Q z 2 K 7 Z U 9 j X v l i l t 1 J 0 C L x J u R C s x Q 7 5 W / u q E k a U y F I R x r 3 f H c x P g Z V o Y R T s e l b q p p g s k Q 9 2 n H U o F j q v 1 s k n q M j q w S o k g q e 4 R B E / X 3 R o Z j n U e z k z E 2 A z 3 v 5 e J / X i c 1 0 a W f M Z G k h g o y f S h K O T I S 5 R W g k C l K D B 9 Z g o l i N i s i A 6 w w M b a o k i 3 B m / / y I m m e V L 3 z 6 t n t a a V 2 N a u j C A d w C M f g w Q X U 4 A b q 0 A A C C p 7 h F d 6 c R + f F e X c + p q M F Z 7 a z D 3 / g f P 4 A V o y T E w = = < / l a t e x i t > x < l a t e x i t s h a 1 _ b a s e 6 4 = " w H r 6 U E Y l 7 s E j 8 P G 4 O 6 l tt / D + 1 c Y = " > A A A B + X i c b V C 7 T s M w F L 3 h W c o r w M h i U S E h h i p B v M Y K F s Y i 0 Y f U h s p x n N a q E 0 e 2 U 1 F F / R M W B h B i 5 U / Y + B u c N g O 0 H M n y 0 T n 3 y s f H T z h T 2 n G + r a X l l d W 1 9 d J G e X N r e 2 f X 3 t t v K p F K Q h t E c C H b P l a U s 5 g 2 N N O c t h N J c e R z 2 v K H t 7 n f G l G p m I g f 9 D i h X o T 7 M Q s Z w d p I P d v u + o I H a h y Z K 3 u a P J 7 2 7 I p T d a Z A i 8 Q t S A U K 1 H v 2 V z c Q J I 1 o r A n H S n V c J 9 F e h q V m h N N J u Z s q m m A y x H 3 a M T T G E V V e N k 0 + Q c d G C V A o p D m x R l P 1 9 0 a G I 5 W H M 5 M R 1 g M 1 7 + X i f 1 4 n 1 e G 1 l 7 E 4 S T W N y e y h M O V I C 5 T X g A I m K d F 8 b A g m k p m s i A y w x E S b s s q m B H f + y 4 u k e V Z 1 L 6 s X 9 + e V 2 k 1 R R w k O 4 Q h O w I U r q M E d 1 K E B B E b w D K / w Z m X W i / V u f c x G l 6 x i 5 w D + w P r 8 A f L x k + A = < / l a t e x i t > x ⇤ < l a t e x i t s h a 1 _ b a s e 6 4 = " 8 y / 0 U G x j o U g F R K O K 9 L e + T / a Y y R A = " > A A A C E n i c b V C 7 T s M w F H X K q 5 R X g Z E l o k J q G a o E 8 R o r W B i L R B 9 S W y L H c V q r j h P Z N 4 g q y j e w 8 C s s D C D E y s T G 3 + C 0 H U r L k S w f n X O v 7 r 3 H j T h T Y F k / R m 5 p e W V 1 L b 9 e 2 N j c 2 t 4 p 7 u 4 1 V R h L Q h s k 5 K F s u 1 h R z g R t A A N O 2 5 G k O H A 5 b b n D 6 8 x v P V C p W C j u Y B T R X o D 7 g v m M Y N C S U 6 z 4 T t J 1 Q + 6 pU a C / p A s D C j h N y 7 P i Y 3 p / X H G K J a t q j W E u E n t K S m i K u l P 8 7 n o h i Q M q g H C s V M e 2 I u g l W A I j n K a F b q x o h M k Q 9 2 l H U 4 E D q n r J + K T U P N K K Z / q h 1 E + A O V Z n O x I c q G w 7 X R l g G K h 5 L x P / 8 z o x + J e 9 h I k o B i r I Z J A f c x N C M 8 v H 9 J i k B P h I E 0 w k 0 7 u a Z I A l J q B T L O g Q 7 P m T F 0 n z p G q f V 8 9 u T 0 u 1 q 2 k c e X S A D l E Z 2 e g C 1 d A N q q M G I u g J v a A 3 9 G 4 8 G 6 / G h / E 5 K c 0 Z 0 5 5 9 9 A f G 1 y / w r Z 7 l < / l a t e x i t > To highlight RoboSaGA's strengths in handling visual domain shifts (VDS), we compare its performance against traditional augmentation methods and existing superimposition-based techniques from the computer vision field within the BC context. Specifically, we evaluate their performance in addressing variations in lighting, shadows, distractors, and backgrounds (with the texture of the operational surface considered part of the background). Performance is measured by the performance gap (Eq. ( 3)), defined as the difference in mean success rate between the in-domain baseline and the model trained with the augmentation method under visual domain shifts. Experiments span both simulation and real-world settings, focusing on key policy variants: BC-MLP (multi-layer perceptron), BC-RNN (recurrent neural network) [7] , and diffusion policy [2] . We summarise our insights:Q S z U i 8 C Y q L 8 7 M h z K Y k t V G W I Y y F m v E P / z e i k E l 0 7 G o i Q F G p H p o C D l B s R G k Y f h M 0 E J 8 L E i m A i m d j X I A A t M Q K V W V S F Y s y f P k / Z J 3 T q v n 9 2 d 1 h p X Z R w V d I A O 0 T G y 0 A V q o B v U R C 1 E 0 C N 6 R q / o T X v S X r R 3 7 W N a u q C V P X v• Simulated experiments show that both Random Crop and Colour Jitter struggle with distractors and background changes. While Colour Jitter improves performance under light intensity changes, it remains less effective under shadows (gap of 0.33). In contrast, all superimposition-based methods keep gaps below 0.1 under lighting and shadow variations.• In simulated experiments, the erase-based method effectively handles distractors by replacing task-trivial regions with OOD images but struggles with background variations.• Random Overlay, applying a constant blending factor for OOD image overlay, reduces the performance gap from Random Crop's 0.60 to 0.24 in simulations and from 0.71 to 0.18 in real-world tests, demonstrating its effectiveness as an augmentation method against VDS.• RoboSaGA dynamically adjusts augmentation intensity based on saliency maps and further improves performance by reducing the gap to 0.14 in simulations and 0.05 in real-world scenarios, consistently outperforming Random Overlay. 
related work: 
 Vision-based Behaviour Cloning. A typical vision-based BC policy π ξ {z j } t j=t-T , parameterised by ξ, derives actions from a temporal sequence of observation embeddings z over a window of length T [7] . At time step t, the observation embedding z t is defined as:z t := g ψ ({f θi (x vi t )} n i=0 , h ϕ (p t )), where x vi t and p t represent the visual input from the i th visual modality or camera view, and the proprioceptive state. The visual encoders f θi , parameterised by θ i , process the visual inputs, while the proprioceptive encoder h ϕ , parameterised by ϕ, handles the proprioceptive data. The function g ψ , parameterised by ψ, integrates these inputs into the latent variable z t . BC policies π ξ (•) can be categorised into explicit and implicit types. Explicit policies form the simplest version of BC, directly regressing actions from observation embeddings [12, 13] , but they struggle to handle the multi-modal nature of human demonstrations. To address this, Florence et al. [1] reformulate imitation learning as a conditional energy-based modelling problem. More recently, Chi et al. [2] introduce a diffusion-based policy that manages multi-modality by leveraging stochasticity in both the initialisation and sampling processes of diffusion models [14] .Data Augmentation in Computer Vision. Data augmentation techniques, traditionally involving randomisations in colour space and image geometry, such as colour jittering, cropping, and rotation, have been expanded with modern superimposition augmentation techniques. Formally, the augmented image x * is a weighted addition of the in-domain image x and out-of-domain (OOD) images x O , both of the same size h × w:EQUATIONwhere M ∈ R h×w is the blending matrix and ⊙ denotes element-wise multiplication. M can be uniformly filled with a constant value between zero and one [11] , binary with randomly located and sized patches [9, 10] , or binary patches guided by input saliency maps [15, 16] .x O can be all-zero matrices [9] , random noise [10] , or inter-class samples [11, 10] . KeepAugment [15] , conceptually closest to RoboSaGA, uses class activation maps to apply a fixed-size rectangular binary mask that covers the most salient areas, preventing the corruption of task-critical regions during augmentation.Data Augmentations for Robotic Manipulation. With collected datasets, mild augmentations like Random Crop are essential for improving in-domain generalisability of models [2, 7] , though they do not expand the semantic range to cover broader visual domains. Additionally, incorporating language descriptions, Abolghasemi et al. [17] learns language-conditioned masks to exclude taskirrelevant areas. Yu et al. [18] utilise off-the-shelf foundational modules to alter scene attributes, thereby facilitating skill acquisition and enhancing robustness. Superimposition augmentation techniques without task prior have received little attention in BC settings, in contrast to the image-based reinforcement learning (RL) community, where such techniques are more commonly used to enhance policy generalisability [19, 20, 21] . 
saliency-guided augmentation for robotic behaviour cloning: 
 RoboSaGA distinguishes itself from other saliency-based superimposition augmentation methods [16, 15] by employing adaptive per-pixel blending factors and extracting saliency directly at the visual encoder level. Unlike KeepAugment [15] , which relies on classification logits and fixedsize rectangular crops, RoboSaGA is specifically designed for vision-based manipulation tasks. Its encoder-level saliency extraction ensures modularity and scalability across multiple camera views and policy types, including BC-MLP, BC-RNN [7] , and Diffusion Policy [2] . The per-pixel blending approach more effectively handles spatially separated, variable-sized objects and provides greater robustness to visual domain shifts, such as background and lighting changes (Sec. 4.1).Saliency Extraction. RoboSaGA leverages FullGrad [22] , which back-propagates the full gradient from the output feature vector back to the full-resolution input, rather than being restricted to gradients from classification logits to specific layers, as in methods like Grad-CAM [23] or Smooth-Grad [24] . FullGrad also incorporates learned biases, in addition to network weights, during saliency computation, resulting in sharper and more accurate saliency maps.Saliency Clipping. RoboSaGA implements a clipping mechanism that caps the normalised saliency scores to a pre-defined threshold λ ∈ (0, 1). By applying a slightly clipped saliency score, Ro-boSaGA enhances the augmentation of task-critical regions through a subtle overlay, crucial for improving robustness against background changes. See Section 4.1 for details. Formally, given an input image x vi and its corresponding encoder f θi (•), we define the normalised saliency derived from Fullgrad as g vi := Fullgrad(f θi (x vi )). The saliency map s vi is then formulated as:EQUATIONData Augmentation. Within a training batch containing m trajectories (considering a single image as a trajectory of length one), RoboSaGA selectively augments α out of m trajectories, where α serves as a hyper-parameter. This selective augmentation strategy is designed to mitigate potential negative impacts on convergence that can arise from overly aggressive augmentation across the entire batch. Each sample within a trajectory is augmented independently, as no performance gap has been observed when using trajectory-wise augmentation, where all samples in a trajectory share the same augmentation parameters as described in [25] .Algorithm 1 RoboSaGA Saliency-Guided Augmentation Process get ood images() # get random out-of-domain images 8: Initialize RoboSaGA instance S with all-one saliency maps. 9: for each batch M with m trajectories do 10:for each visual encoder v in the policy network do Augment images:M v [I α ] ← S α ⊙ M v [I α ] + (1 -S α ) ⊙ S.get ood images() 20:end forOptimise the policy with augmented batch M * 22: end for Global Saliency Buffer. While Fullgrad produces high-quality saliency maps [22] , it incurs a higher computational cost due to the inclusion of both network weights and biases. RoboSaGA mitigates this by employing a global buffer that stores and monitors past saliency maps for each image, significantly reducing computational demands. The buffer is initially filled with ones and updates the saliency for the least recently updated β trajectories in each training batch, where β is a hyper-parameter. During the augmentation phase, α saliency maps are retrieved from the buffer. To conserve memory, these maps are stored as 8-bit, single-channel images at a reduced resolution. For example, storing maps for 10,000 samples at a resolution of 84×84 consumes approximately 67 MB of graphic memory. Through experiments, we found this delayed saliency update does not cause observable performance degradation. Analysis of the saliency buffer's performance-computation trade-off can be found in Appendix A. The comprehensive algorithm is detailed in Algorithm 1. 
experiments: 
 Manipulation Tasks and Out-of-Domain Settings. As illustrated in Fig. 2 , in simulations, we use expert human demonstrations from the Robomimic environment [7] to evaluate four tasks of varying complexity and visual diversity: Lift (pick), Square (pick and insert), Can (pick and place), and Transport (dual-arm pick, handover, place). Real-world experiments focus on Toy (pick and place). Most tasks use a second-person and an eye-in-hand camera, except Transport with four cameras, all at 84 × 84 resolution. In simulations, lighting is varied by adjusting intensity and colour, shadows are toggled on or off, and distractors are randomised by changing category, quantity, and location. Surface textures and backgrounds are shuffled using materials such as textiles, wood, and ceramics. In real-world settings, distractors are randomly selected from a set of items, and surfaces are altered using textile combinations to change backgrounds. Scenes are randomised per trajectory.Tested Policies and Evaluation Protocol. We evaluated three seminal policies: BC-MLP (Multi-Layer Perceptron), BC-RNN (Recurrent Neural Network), and the state-of-the-art Diffusion Policy [2] . All policies incorporate robot proprioceptive states. BC-RNN and Diffusion Policy also use past observations, while BC-MLP relies on the current state only. Following [7] , BC-MLP and BC-RNN employ a Gaussian Mixture Model to capture the multi-modal distribution in human demonstrations. Policies are trained from scratch by default. In simulation, the average task success rate is based on the top three checkpoints from 600 training epochs, saved every 50 epochs, and tested over 50 simulation rollouts per checkpoint. In the real world, the final checkpoint after 500 epochs is used, with task success rates averaged over 20 trials per data point.Implementation Details. For RoboSaGA, the clipping threshold (Eq. 2) is set to 0.8 across all experiments. The saliency buffer updates 10% of the training batch, and RoboSaGA augments half of the training batch. For OOD images, we use 5000 images from a subset of MSCOCO [26] combined with 1000 synthetic images including plain colours, grids and gradients. The BC-MLP and BC-RNN implementations are identical to the default configurations found in [7] . The Diffusion Policy aligns with the hybrid-CNN in [2] . See Appendix B for experimental setup and implementation details.Random Crop as the Default Augmentation. Random Crop significantly improves both in-domain and out-of-domain generalisation in robotic manipulation [7, 8] , making it the baseline augmentation in all experiments. Methods prefixed with "+" denote a combination with Random Crop. All networks use a crop size of 90% of the input, resized back to the original resolution.Colour Jitter. Colour Jitter baselines are obtained by fine-tuning the Random Crop baselines for 50 additional epochs, evaluating checkpoints every 10 epochs, and selecting the top three for comparison. Colour Jitter is implemented with PyTorch's built-in ColorJitter function with brightness, contrast, saturation, and hue set to 0.2, as in [8] .Evaluation Metric. We assess performance degradation using the performance gap ∆P +Aug , defined as the difference between the in-domain mean success rate of the Random Crop baseline and the mean success rate under visual domain shift scenarios with the applied augmentation method. Specifically, the performance gap is expressed as:EQUATIONThis formulation allows for a direct comparison of two augmentation methods by subtraction, effectively cancelling out the baseline performance. A smaller gap indicates greater robustness against the tested visual domain shifts (VDS). Metrics such as mean success rate, performance gap, and standard error are computed using pooled observations across checkpoints, tasks, and policies.RoboSaGA can be conceptualised as a hybrid method that synergistically integrates two distinct augmentation techniques: overlay and erase. Before expanding the scope of our experiments, we undertake an ablation study to dissect the intertwined roles of these techniques within RoboSaGA. The erasing and overlaying variants of RoboSaGA are:• Guided Erase: RoboSaGA binarises saliency maps at a pre-defined threshold for superimposition. This method, designed as a variation of KeepAugment [15] , uses variable-sized regions instead of fixed-sized patches and derives saliency from visual features rather than classification logits, enabling more adaptive augmentation.• Random Overlay: The context-aware per-pixel blending matrix from RoboSaGA is replaced by a global constant. This method also appears as a baseline in [20] . Both the binarisation threshold for Guided Erase and the blending factor for Random Overlay are set to 0.5. These implementations differ from RoboSaGA only in their approach to superimposition, while the overarching augmentation strategy remains consistent. Specifically, we randomly sample and augment 50% of the trajectories from each training batch for all experiments. Fig. 3 illustrates the augmented images from the above methods. We evaluated the efficacy of RoboSaGA and its two variants across three simulation tasks (Lift, Can, Square) with BC-MLP.As shown in Tab.1, Guided Erase fully replaces task-trivial regions, which enhances resilience to distractors, but it leaves task-critical areas untouched, limiting its performance against background variations. On the other hand, Random Overlay uses a universal blending factor, maintaining consistent performance across visual domain shifts but being less effective against distractors due to its milder modification of task-trivial areas. RoboSaGA combines the strengths of both methods, reducing Random Overlay's performance gap under distractors from 0.31 to 0.06, and improving Guided-Erase's performance gap under background variations from 0.55 to 0.13.In this section, we evaluate each selected augmentation method under visual domain shift scenarios across various behaviour cloning policies in both simulated and real-world settings. Full tables with task success rates and the standard error of the mean are provided in Appendix C. This evaluation aims to answer the following questions:Q1: How well do traditional augmentation methods handle visual domain shifts (VDS)?We evaluated two conventional augmentation methods, Random Crop and Random Crop + Colour Jitter, under four VDS scenarios. Tab. 2 presents the performance gap averaged across four tasks and three policies. The results show that lighting significantly degrades baseline performance. While Colour Jitter effectively handles plain lighting changes (intensity and colour), reducing the average gap to 0.1, its improvement is limited under shadows, with a gap of 0.33, compared to less than 0.1 for overlay methods. For distractors and background variation, colour jitter reduces the gap from 0.6 to 0.55, offering only an 8% relative improvement. Given the strong performance of tested superimposition methods in addressing lighting changes, the remainder of this section and the realworld experiments focus on VDS scenarios involving distractor and background variations.Q2: Is representation learning more effective than direct training with augmented images?In vision-based reinforcement learning, aggressive data augmentation is frequently implemented with representation learning techniques for improved policy stability [19, 20] . Within this frame- work, we aim to determine the necessity and effectiveness of representation learning in behaviour cloning (BC), where action labels are readily available. To this end, we adopted SODA [20] as our baseline, which utilises a student-teacher architecture with a consistency loss to encourage Euclidean similarity between features derived from images augmented by Random Overlay and their original counterparts. We assessed the performance of RoboSaGA and Random Overlay in comparison to SODA using the three policies across four simulation tasks. Experimental results in Tab. 2 show that direct training with augmented samples delivers comparable yet marginally improved performance, reducing the performance gap by an average of 0.04 under distractor and background variations, and by 0.02 under lighting and shadows.Q3: Does RoboSaGA's effectiveness against VDS extend to policies beyond BC-MLP?Tab. 2 and Tab. 3 tabulate the mean performance gap under visual domain shift (VDS). The results suggest that while performance varies across different policies, all three tested augmentation methods (Random Overlay, RoboSaGA, and SODA) consistently achieve translatable improvements. Notably, the Diffusion Policy exhibits the least performance degradation. These findings are confirmed by real-world experiments, where both Random Overlay and RoboSaGA significantly enhance performance against VDS compared to their Random Crop counterparts (Tab. 4).Q4: Does RoboSaGA maintain its performance lead against the computation-free Random Overlay?Random Overlay is an efficient and effective augmentation method against VDS, reducing the performance gap of Random Crop + Jitter from 0.55 to 0.24 for distractor and background variations, and from 0.21 to 0.06 for lighting and shadow variations. RoboSaGA further improves performance, lowering Random Overlay's gaps to 0.14 and 0.03, with relative improvements of 41.6% and 50.0%, respectively. In real-world experiments (Tab. 4), Random Overlay reduces the gap (averaged over three policies) from 0.71 to 0.18 for distractor and background variations. RoboSaGA further reduces it to 0.05, yielding a 72% improvement over Random Overlay.Q5: How effective RoboSaGA is in the real world?Instead of exploring a wide range of task variants, we focused on a relatively simple 3 degrees-offreedom pick-and-place task to rigorously evaluate the real-world performance of Random Overlay and RoboSaGA. These methods were tested using BC-MLP, BC-RNN, and Diffusion Policy in challenging real-world visual domain shift scenarios, where distractors appeared as clutter. In addition to the quantitative results reported in Q4 and Tab. 4, we observed highly desirable behaviours from RoboSaGA during testing under severe visual domain shifts across three policies. These behaviours included successful task completion despite aggressive lighting variations, which introduced strong shadows from distractors, and maintaining task execution even with significant occlusions in one view and background changes with added clutter (Fig. 4 ). Q6: How closely do saliency maps correspond to human expectations of task relevance? Saliency maps do not always align with the robot parts or targets that humans consider critical. As shown in Fig. 5a , while reaching for the nut, saliency highlights the general area around the gripper and nut, but the eye-in-hand view focuses only on the handle. When lifting the nut (Fig. 5b ), the saliency shifts away from the object. During insertion (Fig. 5c ), the front camera overlooks the target peg and nut, while the eye-in-hand view focuses on the peg and hole. We hypothesise that this salient misalignment with human intuition occurs because vision-based BC integrates multi-sensory inputs (e.g., proprioception, observation history, multi-views) and accounts for task progression. Notably, history-independent BC-MLP produces more human-interpretable saliency maps than its history-dependent counterparts. Examples are provided in Appendix D. 
limitations: 
 Computation Overhead. The implementation of a saliency buffer improves efficiency, but the computational demands remain considerable. For example, saliency computation with a ResNet18 encoder on 84 × 84 inputs takes about 1.5 times longer than training. To improve efficiency, approximating saliency at mid-feature layers or exploring alternative saliency extractors could help. For policies with larger encoders, knowledge distillation may offer a solution, where a smaller encoder is trained without augmentation, and saliency is computed in later epochs to create an offline buffer.Performance Degradation of BC-MLP and BC-RNN in Transport. As shown in Tab. 10, 11, and 12 in Appendix C, while the Diffusion Policy exhibits the expected improvements under the presence of distractors and changes in backgrounds, BC-MLP and BC-RNN demonstrate comparatively lower gains across all tested methods. This discrepancy may arise from the policies' differing abilities to distil task-relevant information from images with high task-related pixel density. As depicted in Fig. 3 , the second-person view in the Transport task is largely dominated by two manipulators, offering significantly less information about the targets.Focused Task Exploration. Our real-world experiment showcased our approach through a concentrated evaluation of a 3-DoF pick-and-place task, offering valuable insights under specific conditions. Future work could extend these findings to a broader range of tasks. 
conclusion: 
 In this work, we aimed to enhance the robustness of vision-based behaviour cloning against visual domain shifts, particularly with background changes and object distractors, using superimpositionbased image augmentation. Our empirical results from both simulated and real-world experiments demonstrate that overlaying random images onto training data is a cost-effective method that significantly improves performance under visual domain shifts, reducing the performance gap from 0.60 to 0.24 in simulations and from 0.71 to 0.18 in real-world tasks. We introduced RoboSaGA, which uses input saliency maps to adjust augmentation intensity dynamically. This method further reduces the performance gap to 0.14 in simulations and 0.05 in real-world settings, delivering relative improvements of 41.6% and 72%, respectively, compared to Random Overlay. RoboSaGA has revealed the extent of task-relevant semantics hidden within visual features, and the application of it for improving robustness against visual domain shifts. We are excited to explore how these task-specific insights can further advance vision-based behaviour cloning.We conducted two sets of simulated experiments on various policies, including BC-MLP, BC-RNN, and diffusion policies, across the four simulated tasks, examining the impact of lighting changes (varying intensity and colour by adjusting the diffuse parameter between 0.2 and 0.8) with shadow rendering both enabled and disabled (shadows turned off in the training data). The experimental setup, tasks, and evaluation protocol are identical to those in the submitted version, with random cropping applied by default. In the simulation, as illustrated in Fig. 8 , table textures are derived from images of textiles and patterns. Floor textures utilise common materials such as tiles and wooden flooring, while wall textures are selected from both indoor and outdoor scenes. Distractors, including items like a Coke can, bottle, cereal, bread, and lemon, form the default set. Some distractors may be removed to prevent duplicating task-specific objects. For example, the Coke can is excluded from the Can task. Distractors are strategically placed to minimise collisions with the manipulator and targets, although collisions are still possible. Each selected distractor has a 50% chance of appearing. 
 
 Appendix: Floor Texture Table Texture Wall Texture DistractorsIn the real-world setting, as shown in Fig. 9 , background shuffling is achieved by varying the combinations of textiles drawn from the default textile set. These textiles sufficiently cover the field of view for both the second-person camera and the eye-in-hand camera, with examples of the respective fields of view illustrated in Fig. 4 . Distractors are also drawn from the default distractor set, forming cluttered arrangements.All data points in the tables are presented as mean ± the standard error of the mean.  


Decision: AC summarizes the strengths and weaknesses of the paper based on the reviewers' comments as follows:

**Strength:**

- The paper is overall well-written
- The idea of using saliency for data augmentation is reasonable
- The results reported in the paper are promising.

**Weakness:**

- The idea is similar to that of KeepAugment
- There are some unclear points regarding experiments

=== comments after the rebuttal ===

In the rebuttal, the authors provided additional experiments on lighting variations, which demonstrate the advantage of the proposed method. The reviewers appreciate the authors' response and agree that the paper meets the criteria of CoRL.

Reviews: review: title: title: Review of paper 464
paper_summary: paper_summary: The author propose an image augmentation scheme called RoboSaGA for policy learning. The augmentation is $I = M \odot x + (1-M) \odot x_{\text{ood}}$ where the mask, $M$ comes from a saliency map. This augmentation decreases performance degradation in the presence of background changes and the presence of distractors. They test their augmentation with BC-MLP, BC-RNN, and Diffusion Policy across 3 Robomimic tasks and one real-world task.
main_review: main_review: The authors executed a really cool idea and I want to start by commending them! The idea to use saliency maps to ``force'' the policy to attend to action relevant information is clever and they show meaningful improvements over SODA in the presence of new distractors and backgrounds. 

However, to fully verify the experimental claims, the authors should address the following concerns.

1. There is not enough information about the data augmentations used in the baseline policy. The paper points out that the invariance assumptions of standard image augmentation techniques from computer vision do not hold (e.g., I can't horizontally flip an image), but there are many that do and these provide good robustness guarantees. I would expect the baseline policy to use something like RandomCrop, ColorJitter, and RandomRotation. [1, 2] show that these augmentations are important in getting good OOD performance. 

2. Why was lighting excluded as a test-time distribution shift? Prior work studying the problem of OOD policy performance tests lighting and it causes a non-trivial drop in performance [1, 2]. Lighting is also the only shift that affects the manipulation relevant artifacts (e.g., the robot arm and the target object) so it would be especially interesting to see how well this augmentation works in lighting shifts.

Also, I'm not sure if the number of gradient steps between the baseline and RoBoSaGA are fair. RoBoSaGA _sort of_ gets extra gradient steps  in the computation of the saliency map. I'm curious to hear the other reviewers thoughts on this. 

Finally, I think it's worth considering the computational expense of RoBoSaGA. The saliency maps are expensive to compute, which makes the asymptotic of there approach a little prohibitive. 

Here are more suggestions for improvement. They did not factor into my recommendation: 
- epsilon is weird notation in (2) because I'm used to seeing it refer to a small value, not an ood image.
- in line 62, does "visual modality" refer to camera viewpoint? This wording confused me.
- the first paragraph of the related work kind of feels like a preliminaries section. Lines 68-71 are also too broad for the paper. A better RW section would be a section on robustness of BC policies. 
- It would be great to see the time complexity of this method vs baselines in a table somewhere, maybe in the appendix. 
- I think it's cool that the saliency maps make the augmentation super policy relevant! I wish the authors discussed and analyzed this more.
- line 255: should distil be distill?

[1] Decomposing the generalization gap in imitation learning for visual robotic manipulation. Xie et. al. 2023. https://arxiv.org/pdf/2307.03659. 
[2] What Makes Pre-Trained Visual Representations Successful for Robust Manipulation? Burns et. al. 2023. https://arxiv.org/abs/2312.12444.
questions: questions: 1. What data augmentations are used in the baseline model and with what parameters? How well does a policy with RandomCrop, ColorJitter, and RandomRotation perform?

2. How does RoboSaGA perform under lighting distribution shifts?
limitations: limitations: 3
review_summary: review_summary: Leaning reject unless more answers can be provided about lighting shifts and baseline data augmentations.
title: title: A saliency-guided augmentation strategy for BC
paper_summary: paper_summary: The authors proposes to guide superimposition-based augmentation with model saliency with validation on simulation/real-world manipulation tasks
main_review: main_review: The idea of using saliency to guide superimposition-based image augmentation makes intuitive sense -- When introducing augmentation signals, the method aims to reduce the obscuring of the task-related information by avoiding excessive overlay in regions of interest. The saliency is extracted by the Fullgrad method. These augmentation methods are used to improve the performance/robustness of BC-based policy learning.

Experimental validation is conducted on several simulation-based robotic manipulation environments and a real-world robotic manipulation task. The proposed method generally shows better performance than an overlay baseline and another augmentation method -- SODA.

The proposed method is of interest to the community in achieving generally applicable manipulation policies in diverse visual environments. This reviewer believes the notion of task-related information could be quite extendable -- e.g. when having access to a very good segmentation model, the segmentation map could also be used for deciding where to do overlay or not. As the authors also noted, the saliency map is not always a good proxy for task relevance. 

Overall, the paper is well-written and shows convincing/interesting results. The discussions on limitations are appreciated.
questions: questions: I think the paper in its current form may have a limited impact due to the potential problems with saliency quality. It would be interesting to extend the scope of the paper to more general task-relevance notions which this reviewer belives would greatly improve the applicability/significance of the paper.

For the simulated environment task, would it be possible to extract the ground-truth task-relevant pixels? This would be an interesting baseline to consider and somewhat establish an upper bound of task-relevance-guided overlay. In practice, such maps could potentially be obtained through image segmentation models in certain domains.
limitations: limitations: 3
review_summary: review_summary: Recommending weak accept -- happy to raise my scores if there would be clarifications/experiments to the questions/comments above.
title: title: Application of well-studied saliency based augmentation for behavior cloning
paper_summary: paper_summary: The authors' primary assertion is that traditional augmentations like pixel shift are effective in boosting in-domain performance; however, these methods are not as effective for visual domain shifts (i.e. shifts where there are other objects/background is different). To improve performance for visual domain shifts, it is crucial to superimpose additional objects or images onto the training image. However, this approach can result in information loss for behavior cloning. Therefore, the authors propose a saliency-guided approach to augmenting existing data.
main_review: main_review: The bird's eye view of this paper is a strategy to apply overlay based augmentation techniques that do well in computer vision tasks to improve training performance for the task of behavior cloning. However, since you want to be careful and not augment parts of the image which are important, the main idea is to compute saliency maps over the image to identify the important regions, and then weigh the augmentation strategy based on this saliency map.

Overall, this paper is clear and well-explained. The authors apply a method similar to KeepAugment specifically for behavior cloning and show that it improves accuracy tremendously when there are visual domain shifts. The results and findings are quite interesting 

Strengths:
1. The technical approach is well-explained with good explanations on the motivations behind each experiment.
2. There aren’t many papers on such augmentation techniques for behavior cloning, which might make this informative to the community as a whole
3. They also demonstrate their results on hardware and show that there are significant improvements

Weaknesses:
1. The method used is very similar to KeepAugment, which is already a well-known method in computer vision
2. Further analysis/references need to be provided for the other claim of the paper that non-overlay techniques are not effective. Between overlay techniques, the claims made in this paper hold but it's unclear how this augmentation would perform relative to other techniques.
questions: questions: A few major points:
1. A discussion on fullgrad in the related work section would help the reader understand the motivation behind using this over other saliency methods.
2. As per my understanding, consecutive images within a trajectory could get very different augmentations, i.e. different backgrounds which is unlikely to happen in real-world scenarios. Wouldn’t this mean that data could be out of distribution for consecutive images making saliency maps for images later in the trajectory a little arbitrary, especially for BC-RNN and Diffusion Policy?
3. It would be interesting to see how the saliency maps look for time dependent networks, especially BC-RNN and Diffusion Policy in the main paper as well
4. Does it make sense to use this method in the full training cycle? The saliency map might not make sense in the very beginning - would this cause slower convergence?
5. The abstract mentions that pixel shift is not effective for visual domain shifts, but would like to see this backed up by citations or experiments. All the experiments compare only overlay based methods with each other

More minor changes/typos:
1. Line 211 - "Another unspecified method" is a typo
2. Line 255 - "Distil" -> "Distill"
3. Line 166 - "Thus" should be "this"
limitations: limitations: 3
review_summary: review_summary: The paper is very clear and easy to follow. The performance improvement is fairly significant and the experiments performed are thorough. Some additional analysis could be provided to improve the overall quality. In terms of originality though such methods have already been explored or published in the field of computer vision, but not in the context of behavior cloning. Hence my recommendation for a weak accept.