Stem-OB: Generalizable Visual Imitation Learning with Stem-Like Convergent Observation through Diffusion Inversion

## Abstract

Abstract Visual imitation learning methods demonstrate strong performance, yet they lack generalization when faced with visual input perturbations like variations in lighting and textures. This limitation hampers their practical application in real-world settings. To address this, we propose Stem-OB that leverages the inversion process of pretrained image diffusion models to suppress low-level visual differences while maintaining high-level scene structures. This image inversion process is akin to transforming the observation into a shared representation, from which other observations also stem. Stem-OB offers a simple yet effective plug-and-play solution that stands in contrast to data augmentation approaches. It demonstrates robustness to various unspecified appearance changes without the need for additional training. We provide theoretical insights and empirical results that validate the efficacy of our approach in simulated and real settings. Stem-OB shows an exceptionally significant improvement in real-world robotic tasks, where challenging light and appearance changes are present, with an average increase of 22.2% in success rates compared to the best baseline. See our website for more info.

## 1 Introduction

Visual Imitation Learning (IL), where an agent learns to mimic the behavior of the demonstrator by learning a direct mapping from visual observations to low-level actions, has gained popularity in recent real-world robot tasks (Chi et al., ; Zhao et al., ; Wang et al., ; Chi et al., ; Ze et al., ). Despite the versatility demonstrated by visual IL, learned policies are often brittle and fail to generalize to unseen environments, even minor perturbations such as altering lighting conditions or changing the texture of the object may lead to failure of the learned policy (Xie et al., ; Yuan et al., ).
The underlying reason is that the high-dimensional visual observation space is redundant with virtually infinite variations in appearance that are irrelevant to the task and hard to generalize.

As human beings, we can easily manipulate objects that have different appearances. For example, we can pick up a cup of coffee regardless of its color, texture, or the lighting condition of the room. This is partially because our visual system is capable of abstracting the high-level semantics of the scene, such as the silhouette of the object, the structure and arrangement of different objects, etc in a hierarchical manner (Hochstein & Ahissar, ), effectively merging scenes with perceptual differences to similar “meta” observations.

Augmentation techniques such as Spectrum Random Masking (SRM) (Huang et al., ) and Mixup (Zhang et al., ) remove details from observations to encourage the model to focus on structural features; however, they lack the ability to distinguish between low-level and high-level features. It is preferable if we can sweep the photometrical differences while maintaining the high-level structure for the scene. Achieving this requires a semantic understanding of the observations, and naively perturbing the data with Gaussian noise can lead to irreversible information loss.

Pretrained large image diffusion models, such as Stable Diffusion (Rombach et al., ; Esser et al., ), embed essential world knowledge for visual understanding. Apart from synthesizing new images from random noise, these models are capable to perform a reverse procedure called inversion (Song et al., ), which converts an image back to the space of random noises. A recent study (Yue et al., ) indicates that this inversion process selectively eliminates information from the image.
Rather than uniformly removing information from different semantic hierarchies, it will push those images with similar structures closer in the early stages of the inversion process. Inversion is like the reprogramming of a differentiated cell back to a stem cell, which bears the totipotency to differentiate into any cell type.
This characteristic aligns perfectly with our will of enhancing the robustness and generalizability of visual IL algorithms to visual variations.
To distill such property into a visual IL policy, we propose an imitation learning pipeline which applies diffusion inversion to the visual observations.
We name our method Stem-OB to highlight the similarity between the inversed observation and the stem cell in biology, as illustrated in Figure 1.

To be specific, our method is as simple as inverting the image for reasonable steps before sending them to the downstream visual IL algorithms, effectively serving as a preprocessing step that converges low-level appearance differences into similar high-level structures.
From this perspective, our approach fundamentally distinguishes from generative augmentation methods, which aim to enrich the training dataset with more unseen objects and appearances (Yu et al., ; Mandlekar et al., ). Moreover, Stem-OB is indifferent to many unspecified appearance changes, in contrast to augmentation-based methods that must concentrate on a few selected types of generalization, thereby introducing inevitable inductive biases.

We provide theoretical analysis and a user study to support our claim that Stem-OB can effectively merge scenes with perceptual differences to similar “stem observations”.
Empirical study demonstrates the effectiveness of our approach in a variety of simulated and real-world tasks and a range of different perturbations. Stem-OB proves to be particularly effective in real-world tasks where appearance and lighting changes hamper the other baselines, establishing an overall improvement in the success rate of 22.2%. What’s better, no inference time inversion is required for Stem-OB to take effect, making the deployment of our method virtually free of computational cost.

## 2 Preliminary

### 2.1 The Inversion of Diffusion Models

We begin by outlining the fundamentals of Diffusion Inversion. A diffusion model operates with two passes: a backward denoising pass, which generates an image from noise, and a forward pass, where noise is incrementally added to an image until it becomes pure Gaussian noise. This forward process is a Markov chain that starts with $\bm{x}_{0}$, and gradually adds noise to obtain latent variables $\bm{x}_{1},\bm{x}_{2},...,\bm{x}_{T}$. Each step in this process is a Gaussian transition following the common form

$$ $\bm{x}_{t}=\sqrt{{\alpha}_{t}}\bm{x}_{t-1}+\sigma_{t}\bm{{\epsilon}}_{t}\sim \mathcal{N}(\bm{x}_{t}|\sqrt{\alpha_{t}}\bm{x}_{t},\sigma_{t}^{2}\bm{I})$ (1) $$

where $\alpha_{t}\in(0,1)$ represents the scheduler parameter at each step $t$, while $\sigma_{t}$ characterizes the variance of the Gaussian noise $\bm{\epsilon}_{t}$ introduced at each step. In Denoising Diffusion Probabilistic Models (DDPM) (Ho et al., ), $\sigma_{t}=\sqrt{1-\alpha_{t}}$. Consequently, equation Eq. (1) can be reformulated as Eq. (2) by applying the cumulative product $\bar{\alpha}_{t}=\prod_{i=1}^{t}\alpha_{i}$

$$ $\bm{x}_{t}=\sqrt{\bar{\alpha}_{t}}\bm{x}_{0}+\sqrt{1-\bar{\alpha}_{t}}\bm{{ \epsilon}}_{t}\sim\mathcal{N}(\bm{x}_{t}|\sqrt{\bar{\alpha}_{t}}\bm{x}_{t-1},( 1-\bar{\alpha}_{t})\bm{I})$ (2) $$

Diffusion inversion is similar to the forward process in that they both maps an image to a noise, however, inversion tries to preserve the image’s information and obtain the specific noise that can reconstruct the image during a backward denoising process.

DDPM inversion. We follow the DDPM inversion proposed in Huberman-Spiegelglas et al. (),

$$ $\bm{x}_{t}=\sqrt{\bar{\alpha}_{t}}\bm{x}_{0}+\sqrt{1-\bar{\alpha}_{t}}\tilde{ \bm{\epsilon}}_{t}$ (3) $$

The DDPM inversion we consider here differs slightly from Eq. (2), as $\tilde{\bm{\epsilon}}_{t}\sim\mathcal{N}(\bm{0},\bm{I})$ are mutually independent distributions, in contrast to the highly correlated $\bm{\epsilon}_{t}$ in Eq. (2). As mentioned by Huberman-Spiegelglas et al. (), the independence of $\tilde{\bm{\epsilon}}_{t}$ results in a sequence of latent vectors where the structures of $\bm{x}_{0}$ are more strongly imprinted into the noise maps. An error reduction step is conducted in reverse order after the diffusion forward process to improve image reconstruction accuracy during the denoising process:

$$ $\bm{z}_{t}=\bm{x}_{t-1}-\hat{\mu}(\bm{x}_{t})/\sigma_{t},\quad\bm{x}_{t-1}= \hat{\mu}(\bm{x}_{t})+\sigma_{t}\bm{z}_{t}$ (4) $$

DDIM inversion. We follow the DDIM inversion proposed in Song et al. (), where at each forward diffusion step

$$ $\bm{x}_{t}=\sqrt{\frac{\bar{\alpha}_{t}}{\bar{\alpha}_{t-1}}}\bm{x}_{t-1}+\Big {(}\sqrt{\frac{1}{\bar{\alpha}_{t}}-1}-\sqrt{\frac{1}{\bar{\alpha}_{t-1}}-1} \Big{)}{\bm{\epsilon}}_{\bm{\theta}}(\bm{x}_{t-1},t,\mathcal{C})$ (5) $$

Note that the noise $\bm{\epsilon}_{\bm{\theta}}(\bm{x}_{t-1},t,\mathcal{C})$ is now generated by a network trained to predict the noise based on the previous vector $\bm{x}_{t-1}$ and the text embedding $\mathcal{C}$ which, in our case, is $\emptyset$.

### 2.2 Visual Imitation Learning

Visual Imitation Learning (VIL) is a branch of Imitation learning (IL) that focuses on learning action mappings from visual observations. Given a dataset of observation $\mathcal{O}$ and action $\mathcal{A}$ pairs, the goal of VIL is to learn a policy $\pi_{\theta}(\mathcal{A}|\mathcal{O})$ that maps observations to actions. The policy is typically parameterized by a neural network with parameters $\theta$, and is trained to minimize the negative log-likelihood of the actions taken by the expert.

## 3 Related Works

Figure: Figure 2: Overview of Stem-OB: (a). Stem-OB has been evaluated in both real-world and simulated environments. (b). The trained visual IL policies are directly applied to the original observation space $\scriptstyle\mathcal{O}$, demonstrating robustness to unseen environmental disturbances. (c). We train the visual IL policy $\bm{\pi}$ on the diffusion-inversed latent space $\scriptstyle\hat{\mathcal{O}}^{\hat{t}/T}$, where $\hat{t}$ denotes a specific inversion step out of a total of $T$. Each composite rectangle in the diffusion inversion process, made up of three smaller sections, represents the latent vector of an image, with the smaller section depict the finer attributes (gray). During the inversion process, finer attributes converge earlier than coarser ones.
Refer to caption: extracted/5982770/images/Pipeline/pipeline.png

### 3.1 Visual Imitation Learning and Generalization

Visual imitation learning typically follows two approaches: directly imitating expert policies, as in behavior cloning and DAgger (Ross et al., ), or inferring a reward function that aligns the agent’s behavior with expert demonstrations, like inverse reinforcement learning (Ng & Russell, ) and GAIL (Ho & Ermon, ). The former approach, favored in recent works due to its scalability and practicality in complex, real-world tasks, has led to several advancements. Notable methods include Diffusion Policy (Chi et al., ), which leverages a diffusion model (Ho et al., ) to maximize the likelihood of expert actions, and Action Chunk Transformer (Zhao et al., ) that uses a Transformer (Vaswani, ).

To enhance the generalization and robustness of visual imitation learning algorithms, various approaches have been explored (Cetin & Celiktutan, ). For instance, Li et al. () leverage trajectory and step level similarity through an estimated reward function, while Wan et al. () improve visual robustness by separating task-relevant and irrelevant dynamics models before applying the GAIL framework (Ho & Ermon, ). Zhang et al. () use mutual information constraints and adversarial imitation learning to create compact representations that generalize to unseen environments. However, these approaches are not directly applicable to methods like diffusion policy, which focus on imitation without reward functions or dynamics models.  Zheng et al. () propose to filter extraneous action subsequence, yet their focus is not on visual perturbations. Most relevant to our setting, several works in robust visual reinforcement learning have explored adding noise in image or frequency domain to improve generalizability (Huang et al., ; Lee & Hwang, ; Yuan et al., ), however, they lack the semantic understanding of the augmentation process.

### 3.2 Inversion of Diffusion Models and its Application

Diffusion model inversion aims to recover an initial noise distribution from a given image, enabling image reconstruction from the same noise via backward denoising. A common approach is DDIM inversion (Song et al., ), which estimates the previous noise using predictions from the current diffusion step, though the approximation will introduce cumulative errors. To address this issue, several methods employ learnable text embeddings (Mokady et al., ; Miyake et al., ), fixed-point iterations (Pan et al., ; Garibi et al., ), or gradient-based techniques (Samuel et al., ) to refine the result. Another approach, based on the stochastic DDPM scheduler (Ho et al., ), reconstructs noisy images at each diffusion step (Huberman-Spiegelglas et al., ; Brack et al., ). In contrast to the DDIM inversion methods, the noise $\tilde{\epsilon}_{t}$ of each step is statistically independent, making them ideal for our application since we can obtain the noisy image of a certain step without the need to recover other steps, greatly reducing the calculation cost.

Diffusion Inversion is the crucial part of diffusion-based image editing methods (Meng et al., ; Kawar et al., ), which typically involves first inverting the diffusion process to recover the noise latent, then denoise the latent with desired editing conditions. Recent works also explore to apply attention control over the denoising process to improve the fidelity of the edited image Hertz et al. (); Tumanyan et al. (), and have shown promising application in robot learning tasks Gao et al. (). Beyond that, inversion is also used in tasks like concept extraction Huang et al. () and personalization Gal et al. (). Most recently,  Wang & Chen () also propose the use of diffusion inversion to interpolate between image categories to improve classification performance.

## 4 Method

In this section, we introduce the intuition and implementation of our framework. We first propose the intuition of applying inversion on observations through theoretical analysis based on attribute loss, a diffusion-based measurement of image semantic similarity. Then, we conduct an illustrative experiment and a user study to validate our intuition. Finally, we explain how to practically implement Stem-OB and incorporate diffusion inversion into a visual imitation learning framework.

### 4.1 Intuition Deviation by Attribute loss

Intuitively, given a source image and its variation, distinguishing between them becomes increasingly difficult during the diffusion inversion process.
If there are two different variations, we want to show that as the inversion step increases, the pair with minor alterations will become indistinguishable sooner than the pair with larger and structural changes.
We borrow the definition of attribute loss from Yue et al. () to quantify the semantic overlapping of the two images at time step $t$ during a inversion process:

$$ $\displaystyle loss(\bm{x}_{0},\bm{y}_{0},t)=\frac{1}{2}\text{OVL}(q(\bm{x}_{t} |\bm{x}_{0}),q(\bm{y}_{t}|\bm{y_{0}}))$ (6) $$

where $\bm{x}_{0}$ and $\bm{y}_{0}$ are the latent variables of the two images, and OVL is the overlapping coefficient quantifying the overlapping area of probability density functions. For an inversion process where each step follows a Gaussian transition, it takes the form $\bm{x}_{t}=\sqrt{\bar{\alpha}_{t}}\bm{x}_{0}+\sigma\epsilon\sim\mathcal{N}(
\sqrt{\bar{\alpha}_{t}}\bm{x}_{0},\sigma^{2}\textbf{I})$. The OVL can be further calculated as the overlapping area of two Gaussian distributions, i.e.,

$$ $\displaystyle loss(\bm{x}_{0},\bm{y}_{0},t)=\frac{1}{2}\Big{[}1-\text{erf}( \frac{||\sqrt{\bar{\alpha}_{t}}(\bm{y}_{0}-\bm{x}_{0})||}{2\sqrt{2}\sigma}) \Big{]}$ (7) $$

where $erf$ is the error function, which is strictly increasing. Given a source image $\bm{x}_{0}$ and its variations $\hat{\bm{x}}_{0}$ and $\tilde{\bm{x}}_{0}$, with $\tilde{\bm{x}}_{0}$ undergoing a larger variation than $\hat{\bm{x}}_{0}$, the following conclusion can be easily observed under the same diffusion scheduling:

$$ $\displaystyle\tau(\bm{x_{0}},\hat{\bm{x}}_{0},\rho)$ $\displaystyle<\tau(\bm{x_{0}},\tilde{\bm{x}}_{0},\rho),\ s.t.||\hat{\bm{x}}_{0 }-\bm{x}_{0}||<||\tilde{\bm{x}}_{0}-\bm{x}_{0}||$ (8) $$

Here, $\tau(\bm{x}_{0},\bm{y}_{0},\rho)=\inf\{t>0\ |\ \text{loss}(\bm{x}_{0},\bm{y}_{
0},t)>\rho\}$ represents the earliest step where the loss between $\bm{x}_{0}$ and $\bm{y}_{0}$ exceeds the threshold $\rho$, and $||\cdot||$ measures the difference between an image and its variation. Eq. (8) provides a theoretical grounding for our intuition: images with fine-grained attribute changes tend to become indistinguishable sooner than those with coarse-grained modifications under identical diffusion schedules.

We can further derive the attribute loss for DDPM inversion

$$ $\displaystyle loss_{DDPM}(\bm{x}_{0},\bm{y}_{0},t)=\frac{1}{2}\Big{[}1-\text{ erf}(\frac{||\sqrt{\bar{\alpha}_{t}}(\bm{y}_{0}-\bm{x}_{0})||}{2\sqrt{2(1-\bar {\alpha}_{t})}})\Big{]}$ (9) $$

Additionally, we derive that the attribute loss for DDIM inversion exhibits a similar form under certain assumptions. The detailed derivation can be found in Appendix A.2.

$$ $loss_{DDIM}(\bm{x}_{0},\bm{y}_{0},t)=\frac{1}{2}\Bigg{[}1-\text{erf}\Big{(} \frac{||(\bm{y}_{0}-\bm{x}_{0})||}{2\sqrt{2{\sum_{i=1}^{t}\frac{1}{\bar{\alpha }_{i}}\Big{(}\sqrt{\frac{1}{\bar{\alpha}_{i}}-1}-\sqrt{\frac{1}{\bar{\alpha}_{ i-1}}-1}\Big{)}^{2}}}}\Big{)}\Bigg{]}$ (10) $$

Because $\bar{\alpha}_{t}\in(0,1)$ is strictly decreasing, the attribute loss tends to increase as the time step increases. Furthermore, as discussed in Yue et al. (), this attribute loss is equivalent to how likely the DM falsely reconstruct $\bm{x}_{t}$ sampled from $q(\bm{x}_{t}|\bm{x}_{0})$ closer to $\bm{y}_{0}$ instead of $\bm{x}_{0}$, and vise versa.

### 4.2 Illustrative Experiment

In Sec. 4.1, we made a key assumption that similar images are closer in latent space, leading to the conclusion that such images exhibit higher attribute loss at a given inversion step (Eq. (8)). While this claim is intuitive, we conducted an illustrative experiment to further support it. To validate this assumption, we used a set of images, denoted as $\mathcal{I}$, from the real-world task variations described in Sec. 5.1.1. Specifically, we selected several images of objects from 5 categories and calculated the pairwise distance between images on their latent representations, both for intra-category and cross-category image pairs. The computed image distances are presented in Tab. 1, where each entry represents the average similarity between two categories and the diagonal entries indicate intra-category similarity. As indicated in this table, intra-category images exhibit lower similarity compared to cross-category images, which justifies our claim.

Figure: Table 1: By-category image semantic similarity.
Refer to caption: x1.png

We then conducted a user study to validate that similar images exhibit higher attribute loss at a given inversion step. We recruited 51 participants and presented each with image pairs from $\mathcal{I}$ after specific steps of diffusion inversion. The inversion steps were systematically sampled from 15 to 45, out of a total 50 steps inversion and with intervals of 5. Each participant was asked to determine whether the image pairs depicted the same object, and we recorded the proportion of incorrect responses.

The experimental results in Fig. 3 show that at inversion step $t_{1}=20$, the incorrect response rate within the same category starts to increase. In contrast, the incorrect responses for objects from different categories only started to rise at the inversion step $t_{2}=35$. This indicates that inversion makes objects within the same category harder to distinguish earlier than affecting the distinction between objects from different categories. Therefore, these results support our claim that similar images exhibit higher attribute loss at a given inversion step.

### 4.3 Implementation of Stem-OB

Following the theoretical analysis and experimental validation, we propose a practical implementation of our framework. Our method applies diffusion inversion to converge different observations during training, and we find that the model gains robustness improvement in the original observation space during test time. The detailed implementation is described below.

Training.
The training process begins by applying diffusion inversion to each observation. We define the following partial diffusion inversion process for an observation $\bm{o}_{i}\in\mathcal{O}$:

$$ $\hat{\bm{o}}^{t/T}_{i}=\bm{f}(\bm{o}_{i},t,T)\vspace{-8pt}$ (11) $$

Here, $\hat{\bm{o}}^{t/T}_{i}\in\hat{\mathcal{O}}^{t/T}$ denotes the observation after $t$ out of $T$ steps of diffusion inversion from the original observation $\bm{o}_{i}\in\mathcal{O}$. The function $\bm{f}(\cdot)$ applies $t$ steps of diffusion inversion to $\bm{o}$ using any inversion methods. We select the DDPM inversion method for its efficiency and effectiveness in our experiments, and the selection of $t$ and total inversion step $T$ is discussed in Sec. 5.5.
The visual imitation learning algorithms are then trained on the inversion-altered space $\hat{\mathcal{O}}^{t/T}$.
Note that we empirically find the error reduction step of DDPM inversion in Eq. (4) is not significant for the performance, so we omit it and approximate the partial inversion process with regard to Eq. (3) only. In this way, we avoid the full inversion process or reverse-order corrections that involve extensive Diffusion Model inference. This approach significantly reduces preprocessing time with minimal performance impact, achieving an average time for preprocessing of 0.2s per image.

Testing.
Despite training is entirely conducted on the inversion-altered space, the imitation learning algorithm of our choice showcases surprising zero-shot adaptation ability to the original observation space. We adhere to the original observation space $\mathcal{O}$ during testing, which means essentially no changes are made to the downstream visual imitation learning algorithms. This approach demonstrates improved robustness to environmental variations without any inference-time overhead, and is suitable for any other test-time augmentation techniques.

## 5 Experiments

In this section, we present a comprehensive set of experiments using Diffusion Policy (DP) (Chi et al., ) as our visual imitation learning backbone. Since Stable Diffusion is pretrained on real-world images (Chi et al., ; Rombach et al., ), and a increasing interest in deploying visual imitation learning in real-world scenarios in the community (Paolo et al., ),
we focus primarily on evaluating Stem-OB on real-world tasks. We extend the testing to simulated environments as well for further benchmarking. To assess the robustness of our method against visual appearance variations, we design experiments featuring different object textures and lighting conditions. We compare Stem-OB against several baselines to validate its effectiveness.

Figure: Figure 4: Realworld Setup: (a) Overview of the whole setup. (b)(c) These tasks are performed by the robot in a real-world environment, from left to right: Cup2Plate, Turn on Faucet, Open Drawer, and Duck2Bowl. The figure showcases the initial and final states of the tasks.
Refer to caption: extracted/5982770/images/Realworld/real-world_setup_overview.png

### 5.1 Experiment Setup

#### 5.1.1 Real-World Tasks

We conduct real-world experiments using a Franka Emika Panda Arm, and two RealSense D435I cameras positioned at different angles for RGB inputs. The setup is shown in Fig. 4. Our experiments focus on four tasks, as described below:

Cup to Plate (C2P): The robot arm picks up a cup and places it on a plate, with variation introduced by changing tablecloth patterns.
Duck to Bowl (D2B): The robot grasps a toy duck and places it into a bowl, with variations introduced by altering the duck’s appearance.
Open Drawer (OD): The robot arm grabs a drawer handle and pulls it open, with variations introduced by modifying the drawer’s visual characteristics.
Turn on Faucet (ToF): The robot arm turns on a faucet, with variations introduced by altering the appearance of the faucet and bucket.

In addition to the visual variations mentioned above, all four tasks above involve changes in lighting conditions, i.e., cool vs. warm light. All the variations only happen during testing time, with the training set contain only a basic setting. The object locations in training set are randomly initialized within a specified area, and 100 demonstrations are collected per task. For testing, nine predefined target positions are used. Further details of the environmental variations can be found in Appendix C.1.

#### 5.1.2 Simulation Tasks

Our simulation experiments consider different tasks within two frameworks: a photorealistic simulation platform SAPIEN 3 (Xiang et al., ) and a less realistic framework MimicGen (Mandlekar et al., ).

SAPIEN 3 provides physical simulation for robots, rigid bodies, and articulated objects. It delivers high-fidelity visual and physical simulations that closely approximate real-world conditions. We leverage the ManiSkill 3 dataset (Gu et al., ; Tao et al., ), collected on SAPIEN 3, for benchmarking. Specifically, we select four tasks from ManiSkill for evaluation:

PushCube: The robot arm pushes a cube to a target location.
PegInsertionSide: The robot arm inserts a peg into a hole.
PickCube: The robot arm picks up a cube and places it on a target location.
StackCube: The robot arm stacks one cube over the other.

During testing, we vary the background and lighting conditions in all tasks to generate different visual appearances. 50 episodes are tested for each setting. Details of the environmental variations can be found in Appendix C.2.

Figure: (a) PushCube
Refer to caption: extracted/5982770/images/Sapien/PushCube.png

MimicGen is a system for generating large diverse datasets from a small number of human demonstrations.
We utilize the benchmark to evaluate the performance of our approach and conduct experiments on a variety of tasks: MugCleanup, Threading, ThreePieceAssembly and Coffee. MimicGen offers numerous task variations, each characterized by different initial distributions. We adopt the default initial distribution (D0) for both training data and test environments. For evaluation, we employ a single image as the input to the policy, using 500 samples out of a total of 1000 demos for training. We alter the table texture and object appearances to create different test environments. 300 episodes are tested for each setting of all the tasks. Detailed descriptions of the environmental variations can be found in Appendix C.3.

### 5.2 Baselines

We compare Stem-OB with several data augmentation methods aimed at improving the generalizability of visual imitation learning. The baselines include SRM (Huang et al., ), which modifies images by adding random noise in the frequency domain, Mix-Spectrum (Mix) (), which enhances the agent’s focus on semantic content by blending original images with randomly selected reference images from the same dataset in the frequency domain, and original images (Org) without any modification. Additionally, as highlighted in (Yuan et al., ), Random Overlay (RO) improves generalization in real-world experiments by blending original images with random real-world photos in the image domain. Therefore, we include an additional study using Random Overlay in our real-world experiments.

### 5.3 Experiment Results

#### 5.3.1 Real-World Experiments

In this section, we report the average success rate across all predefined test positions for each task. As shown in Tab. 2, Stem-OB consistently outperforms the baseline models in all settings, demonstrating superior generalization capability. Under the training conditions, all methods achieve relatively high success rates, with our method performing slightly better than the baselines. However, in generalization testing, baseline methods exhibit significant performance drop, while our approach maintains a high success rate, showcasing the superior adaptability of Stem-OB to complex and noisy real-world visual disturbances.

The experiment results demonstrate that previous visual augmentation methods, such as SRM and Mix-Spectrum, struggle to generalize in real-world scenarios, which could be attribute to the complexity of real-world environments. Real images contain more redundant information, complicating the frequency domain and potentially leading to the failure of these augmentation methods. The light disturbance introduced in our experiments is a typical example, where wide-range but low-intensity noise is added to the images. Our approach effectively handles real-world noise by extracting high-level structures from the appearances, resulting in better generalization. Even in challenging scenarios like D2B, where baseline methods mostly fail, our method maintains a high success rate.

**Table 2: Evaluation of real-world experiments. Train.: evaluations in the same settings as the training dataset. Gen.: evaluations under different visual perturbations for generalizability analysis. All: evaluations including both Train. and Gen. The tasks are C2P, D2B, OD, and ToF. We report the mean and standard deviation of the success rate (%) over 6 settings for each task, and the best results are highlighted in bold.**
| Task $\backslash$ | C2P | D2B | OD | ToF |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Algorithm | Train. | Gen. | All | Train. | Gen. | All | Train. | Gen. | All | Train. | Gen. | All |
| Stem-OB | 89.0 | 93.3$\pm$6.1 | 92.6$\pm$5.7 | 78.0 | 44.4$\pm$22.2 | 50.0$\pm$24.1 | 100.0 | 100.0$\pm$0 | 100.0$\pm$0 | 100.0 | 88.9$\pm$19.2 | 90.7$\pm$17.8 |
| Org | 89.0 | 77.8$\pm$26.1 | 79.6$\pm$23.7 | 56.0 | 13.3$\pm$14.4 | 20.4$\pm$21.6 | 89.0 | 13.3$\pm$18.3 | 25.9$\pm$34.9 | 100.0 | 22.2$\pm$19.2 | 35.2$\pm$36.1 |
| SRM | 56.0 | 73.3$\pm$9.9 | 70.4$\pm$11.5 | 44.0 | 8.9$\pm$9.3 | 14.8$\pm$16.7 | 89.0 | 15.6$\pm$21.7 | 27.8$\pm$35.7 | 89.0 | 31.1$\pm$30.8 | 40.7$\pm$36.3 |
| Mix | 89.0 | 55.6$\pm$13.6 | 61.1$\pm$18.3 | 44.0 | 22.2$\pm$17.6 | 25.9$\pm$18.1 | 33.0 | 35.6$\pm$5.0 | 35.2$\pm$4.5 | 100.0 | 68.9$\pm$30.8 | 74.1$\pm$30.4 |
| RO | 56.0 | 53.3$\pm$9.3 | 53.7$\pm$8.4 | 44.0 | 15.6$\pm$12.7 | 20.4 $\pm$16.4 | 89.0 | 88.9 $\pm$7.9 | 88.9 $\pm$7.0 | 78.0 | 82.2$\pm$20.2 | 81.5$\pm$18.1 |

Interestingly, DP without any image modification (Org) outperforms the baselines in some tasks, such as OD and ToF, suggesting that DP has inherent generalization capabilities. However, this generalizability is inconsistent across tasks. For instance, in C2P, we observe that object appearance (cup and plate) had little impact on DP’s performance, while the tablecloth pattern significantly affects it. In D2B, the duck’s appearance is critical, whereas the table cloth variation is more influential. In contrast, our method exhibits consistent generalization across diverse scenarios.

### 5.4 Simulation Experiments

**Table 3: Evaluations in simulated environments. We compare the performance of Stem-OB, Org, SRM, and Mix in simulated environments. The left side of the table shows the results on the SAPIEN 3 environment, while the right side is about the MimicGen environment. The results are reported as the mean and standard deviation of the success rate(%) over their own various test settings. The performance of Stem-OB is highlighted in orange for better visibility. The best performance of each task is highlighted in bold.**
| Task $\backslash$ | SAPIEN 3 | MimicGen |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Algorithm | PushCube | PegInsertionSide | PickCube | StackCube | MugCleanup | Threading | ThreePieceAssembly | Coffee |
| Stem-OB | 99.1$\pm$1.8 | 0.4$\pm$0.8 | 25.1$\pm$20.9 | 27.2$\pm$31.3 | 19.5$\pm$12.5 | 16.4$\pm$12.8 | 13.1$\pm$6.9 | 50.5$\pm$19.6 |
| Org | 61.8$\pm$33.1 | 0.0$\pm$0.0 | 8.6$\pm$18.2 | 4.3$\pm$19.2 | 16.4$\pm$12.0 | 18.1$\pm$12.8 | 14.8$\pm$8.4 | 43.3$\pm$17.3 |
| SRM | 62.5$\pm$30.5 | 0.0$\pm$0.0 | 8.4$\pm$16.1 | 0.7$\pm$3.4 | 15.4$\pm$0.08 | 25.8$\pm$11.7 | 13.6$\pm$7.6 | 39.4$\pm$12.0 |
| Mix | 97.2$\pm$3.35 | 0.6$\pm$0.1 | 12.9$\pm$16.3 | 8.2$\pm$18.0 | 22.3$\pm$11.8 | 18.9$\pm$8.5 | 14.2$\pm$6.6 | 41.8$\pm$9.9 |

Sapien 3 The evaluation results on Sapien 3 are presented on the left side of Tab. 3. The complexity of the tasks poses challenges for DP in generalizing across various conditions. However, our method achieves a higher success rate than the baseline methods on most of the tasks. In PushCube, both Mix-Spectrum and our method perform well, but our approach is more robust and reach nearly 100% success. These results demonstrate that in more photorealistic simulation environments, our method generalizes more effectively across diverse tasks and conditions.

MimicGen. The evaluation results on the MimicGen benchmark are presented on the right side of Tab. 3. At first glance, the results seem unanticipated, asresults seem unanticipated, as Stem-OB does not perform best in most settings. This can be attributed to the fact that is can be attributed to the fact that MimicGen evironments are less photorealistic, with nearly texturexture-free and -ow-resolutions, limiting DP from fully leveraging images. Consequently, diffusion inversion processing complicates the observations, limiting DP from fully leveraging the advantages of Stem-OB. It is important to emphasize that our method is primarily designed for real-world sccenarios, where diverse noise is inevitable, in contrast to the controlled controlled Gen environment.

### 5.5 Ablation

In this section, we test with several design choices of Stem-OB, providing a better understanding of their impact on the final performance. We consider the following choices: the number of inversion steps with fixed total steps, the number of total steps with the constant proportion of inversion steps to the total steps, and the selection of inversion methods (DDPM or DDIM inversion). The experiments are conducted on SAPIEN 3. We choose PushCube, PickCube and StackCube, since they have a more significant performance variance in the main experiments.

**Table 4: Ablation study on diffusion steps. On the left side of the table, we increase the inversion step with fixed total number of steps, to intensify the effect of the diffusion process. Additionally, we compare the performance of the model with fixed ratio of steps, where the intensity of inversion is approximately the same. The results are reported as mean and standard deviation over 21 kinds of settings. The best performance is highlighted in orange and the second best is highlighted in pink.**
| Task $\backslash$ | Fixed Total Number of Steps (50) | Fixed Ratio of Steps (30%) |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Settings | 0/50 | 5/50 | 10/50 | 15/50 | 20/50 | 25/50 | 9/30 | 15/50 | 30/100 |
| PushCube | 61.8$\pm$33.1 | 97.1$\pm$4.0 | 91.1$\pm$18.9 | 99.1$\pm$1.8 | 98.5$\pm$3.2 | 96.3$\pm$3.8 | 98.1$\pm$3.8 | 99.1$\pm$1.8 | 98.3$\pm$2.9 |
| PickCube | 8.6$\pm$18.2 | 12.0$\pm$20.0 | 28.0$\pm$26.3 | 25.1$\pm$20.9 | 12.7$\pm$14.0 | 8.29$\pm$6.2 | 23.7$\pm$24.4 | 25.1$\pm$20.9 | 25.6$\pm$22.3 |
| StackCube | 4.3$\pm$19.2 | 20.7$\pm$29.6 | 27.2$\pm$31.3 | 19.6$\pm$24.9 | 11.6$\pm$12.8 | 2.0$\pm$3.6 | 19.5$\pm$20.5 | 19.6$\pm$24.9 | 20.2$\pm$27.0 |

Number of Inversion Steps.
We compare the performance of Stem-OB with varying numbers of inversion steps, fixing the total steps to 50 and adjusting the inversion steps from 5 to 25 in increments of 5. The results, shown in Tab. 4, indicate that performance generally increases up to 15 inversion steps before declining. This can be attributed to insufficient removal of low-level appearance features with too few inversion steps, while excessive steps eliminate high-level structural information, hindering task performance. Optimal performance is observed around 10 to 15 inversion steps, depending on the complexity of the tasks.

Number of Total Steps.
With the proportion of inversion steps to total steps fixed, we varied the total number of steps to 30, 50, and 100. The results, shown on the right side of Tab. 4, indicate that performance remains consistent regardless of the total number of steps. This suggests that the proportion of inversion steps is more critical to performance than the total number of steps.

**Table 5: Ablation study on different inversion methods. We compare the performance of DDPM and DDIM on the tasks of PushCube and PickCube. The results are reported as the success rate (%) of the task. The best performance is highlighted in bold.**
| Task | DDPM | DDIM |
| --- | --- | --- |
| PushCube | 99.1$\pm$1.8 | 81.3$\pm$15.7 |
| PickCube | 25.1$\pm$20.9 | 5.4$\pm$12.0 |

Diffusion Inversion Methods.
We compare the performance of two inversion methods: DDPM inversion and DDIM inversion, with results presented in Tab. 5. For DDIM inversion, we use 5/50 inversion steps instead of 15/50 due to differences in noise scheduling compared to DDPM inversion. The results show that DDPM inversion outperforms DDIM inversion on the benchmark datasets, supporting our choice of DDPM inversion as the primary diffusion inversion method.

## 6 Conclusion

In this work, we propose Stem-OB, a straightforward preprocessing method for visual IL. By inverting the observation in the diffusion latent space for certain steps, we effectively converge different observation variations to the node it stems from, Making it invariant to unseen low-level appearance changes in the observation space.
Though we only test our method on the diffusion policy method, our method is general and compatible with any visual IL baselines in theory, and the plug-and-play nature of our method makes it easy to integrate. We plan to test our method with other visual IL baselines in simulation and real tasks in the future.

Reproducibility: The main algorithm of our method is simple as applying the open-sourced DDPM inversion method on the dataset before training. We’ve provided the code for our method in the supplementary material.

## Appendix A Appendix

### A.1 Derivation of DDPM inversion attribute loss

The derivation of DDPM inversion is straightforward by simply letting $\sigma=\sqrt{1-\bar{\alpha}_{t}}$ in Eq. (7).

### A.2 Derivation of DDIM inversion attribute loss

We first rewrite Eq. (5) as a linear combination of $\bm{x}_{0}$ and a noise variable. To do so, we expand the recursive equation in Eq. (5). Note that we assume that ${\bm{\epsilon}}_{\bm{\theta}}(\bm{x}_{t-1},t,\mathcal{C})$ is a Gaussian distribution with mean $\bm{\mu_{t}}$ and use $\bm{\epsilon}_{t}$ for simplicity in the following derivation.

$$ $\displaystyle\bm{x}_{t}$ $\displaystyle=\sqrt{\frac{\bar{\alpha}_{t}}{\bar{\alpha}_{t-1}}}\bm{x}_{t-1}+ \left(\sqrt{\frac{1}{\bar{\alpha}_{t}}-1}-\sqrt{\frac{1}{\bar{\alpha}_{t-1}}-1 }\right)\bm{{\epsilon}}_{t}$ (12) $\displaystyle=\sqrt{\frac{\bar{\alpha}_{t}}{\bar{\alpha}_{t-1}}}\left(\sqrt{ \frac{\bar{\alpha}_{t-1}}{\bar{\alpha}_{t-2}}}\bm{x}_{t-2}+\left(\sqrt{\frac{1 }{\bar{\alpha}_{t-1}}-1}-\sqrt{\frac{1}{\bar{\alpha}_{t-2}}-1}\right)\bm{ \epsilon}_{t-1}\right)+\left(\sqrt{\frac{1}{\bar{\alpha}_{t}}-1}-\sqrt{\frac{1 }{\bar{\alpha}_{t-1}}-1}\right)\bm{{\epsilon}}_{t}$ $\displaystyle=\sqrt{\frac{\bar{\alpha}_{t}}{\bar{\alpha}_{t-2}}}\left(\sqrt{ \frac{\bar{\alpha}_{t-2}}{\bar{\alpha}_{t-3}}}\bm{x}_{t-3}+\left(\sqrt{\frac{1 }{\bar{\alpha}_{t-2}}-1}-\sqrt{\frac{1}{\bar{\alpha}_{t-3}}-1}\right)\bm{ \epsilon}_{t-2}\right)$ $\displaystyle\qquad\qquad\qquad\qquad\qquad+\sqrt{\frac{\bar{\alpha}_{t}}{\bar {\alpha}_{t-1}}}\left(\sqrt{\frac{1}{\bar{\alpha}_{t-1}}-1}-\sqrt{\frac{1}{ \bar{\alpha}_{t-2}}-1}\right)\bm{\epsilon}_{t-1}+\left(\sqrt{\frac{1}{\bar{ \alpha}_{t}}-1}-\sqrt{\frac{1}{\bar{\alpha}_{t-1}}-1}\right)\bm{\epsilon}_{t}$ $\displaystyle...$ $\displaystyle=\sqrt{\bar{\alpha}_{t}}\bm{x}_{0}+\sum_{i=1}^{t}\sqrt{\frac{\bar {\alpha}_{t}}{\bar{\alpha}_{i}}}\left(\sqrt{\frac{1}{\bar{\alpha}_{i}}-1}- \sqrt{\frac{1}{\bar{\alpha}_{i-1}}-1}\right)\bm{\epsilon}_{i}$ $\displaystyle=\sqrt{\bar{\alpha}_{t}}\bm{x}_{0}+\sum_{i=1}^{t}\sqrt{\frac{\bar {\alpha}_{t}}{\bar{\alpha}_{i}}}\left(\sqrt{\frac{1}{\bar{\alpha}_{i}}-1}- \sqrt{\frac{1}{\bar{\alpha}_{i-1}}-1}\right)\bm{\mu_{t}}+\sqrt{\sum_{i=1}^{t} \frac{\bar{\alpha}_{t}}{\bar{\alpha}_{i}}\left(\sqrt{\frac{1}{\bar{\alpha}_{i} }-1}-\sqrt{\frac{1}{\bar{\alpha}_{i-1}}-1}\right)^{2}}\bm{\epsilon}$ $$

where the last equality holds considering the property of sum of gaussian distributions, i.e., $\mathcal{N}(\bm{\mu}_{1},\sigma_{1}^{2}\bm{I})+\mathcal{N}(\bm{\mu}_{2},\sigma
_{2}^{2}\bm{I})\sim\mathcal{N}(\bm{\mu}_{1}+\bm{\mu}_{2},(\sigma_{1}^{2}+
\sigma_{2}^{2})\bm{I})$. And $\sum_{i=1}^{t}\sqrt{\frac{\bar{\alpha}_{t}}{\bar{\alpha}_{i}}}\left(\sqrt{
\frac{1}{\bar{\alpha}_{i}}-1}-\sqrt{\frac{1}{\bar{\alpha}_{i-1}}-1}\right)\bm{
\mu_{t}}$ represents the mean shift resulting from the mean bias of model ${\bm{\epsilon}}_{\bm{\theta}}$. For two similar images, the biases are approximately equal and thus cancel out when substituted into Eq. (7). Consequently, we can approximately derive the attribute loss for DDIM inversion.

$$ $\displaystyle\quad loss_{DDIM}(\bm{x}_{0},\bm{y}_{0},t)$ $\displaystyle\approx\frac{1}{2}\Big{[}1-\text{erf}\Big{(}\frac{||\sqrt{\bar{ \alpha}_{t}}(\bm{y}_{0}-\bm{x}_{0})||}{2\sqrt{2{\sum_{i=1}^{t}\frac{\bar{ \alpha}_{t}}{\bar{\alpha}_{i}}\left(\sqrt{\frac{1}{\bar{\alpha}_{i}}-1}-\sqrt{ \frac{1}{\bar{\alpha}_{i-1}}-1}\right)^{2}}}}\Big{)}\Big{]}$ (13) $\displaystyle=\frac{1}{2}\Big{[}1-\text{erf}\Big{(}\frac{||(\bm{y}_{0}-\bm{x}_ {0})||}{2\sqrt{2{\sum_{i=1}^{t}\frac{1}{\bar{\alpha}_{i}}\left(\sqrt{\frac{1}{ \bar{\alpha}_{i}}-1}-\sqrt{\frac{1}{\bar{\alpha}_{i-1}}-1}\right)^{2}}}}\Big{) }\Big{]}$ $$

## Appendix B Training Details

Table: Table 6: Hyperparameters used for Diffusion Policy. We use the same hyperparameters of diffusion policy for all the experiments.

**Table 7: Tasks and inversion steps used for training the Diffusion Policy. We all use DDPM as the inversion method.**
| Task | Step | Task | Step | Task | Step |
| --- | --- | --- | --- | --- | --- |
| Cup2Plate | 15/50 | PushCube | 15/50 | MugCleanup | 10/50 |
| Duck2Bowl | 15/50 | PegInsertionSide | 15/50 | Threading | 5/50 |
| OpeningDrawer | 15/50 | PickCube | 15/50 | ThreePieceAssembly | 5/50 |
| Turn on Faucet | 15/50 | StackCube | 10/50 | Coffee | 5/50 |

The hyperparameters for Diffusion Policy across all experiments are listed in Tab. 6. We use DDPM inversion in all the basic experiments, with the specific inversion steps for each experiment detailed in Tab. 7.

## Appendix C Experiment Details

### C.1 Realworld Details

Figure: Figure 6: Realworld Camera Views. Each task is represented by two rows, with each representing a different camera view angle. For each task, we perform experiments on three instances with different object appearances, and two light conditions, as displayed along the columns. The first column represents the training setup.
Refer to caption: extracted/5982770/images/Realworld/cam_view.png

Fig. 6 shows the camera views of the real-world environments. Each task is represented by two rows, corresponding to different camera angles. The columns represent task settings, with the first being the default. As lighting conditions vary across tasks, images are grouped in pairs: the odd columns are under warm light and the even columns are under cold light. The low-level object appearance changes along each row.

### C.2 Sapien environments settings

Figure: Figure 7: Sapien Testing Settings. Each task consists of 21 distinct settings, arranged in two rows. The images are grouped in threes, with each group sharing the same tablecloth texture but differing in lighting conditions: white, yellow, and red light. A total of seven unique tablecloth textures are used. The training environment is the first setting in the first row.
Refer to caption: extracted/5982770/images/Sapien/sapien_settings.png

Figure: Figure 8: MimicGen Testing Settings. Each row represents a specific task, with eight different table textures across the columns. The training environment is the first column. Additionally, MugCleanup, Threading, and ThreePieceAssembly employ alternate object textures in the last column.
Refer to caption: extracted/5982770/images/Mimicgen/mimicgen_settings.png

Each task includes 21 distinct testing environments, as shown in Fig. 7.
We alternate between seven unique tablecloth textures and three lighting conditions: white, yellow, and red light.

### C.3 MimicGen environments settings

Figure: (a) MugCleanup
Refer to caption: extracted/5982770/images/Mimicgen/mug_cleanup_d0_.png

Fig. 9 demonstrates the four MimicGen environments used in the experiments, and each row in Fig. 8 represents a task. In MugCleanup, the robot arm opens a drawer, retrieves a mug, and places it back. Threading involves precise insertion of a stick into a hole. ThreePieceAssembly requires collecting and assembling three components in a specific order. In Coffee, the robot opens the lid of the coffee machine and places a cup inside. Eight different table textures are used across all tasks (first eight columns). Additionally, MugCleanup, Threading, and ThreePieceAssembly employ alternating object texture, as indicated in the last column, creating distinct testing environments.

### C.4 Details of experiment results

Figure: Figure 10: Realworld Success Rate. This figure presents the success rates for real-world tasks. The groups are labeled as W1, C1 through W4, C4. W1 to W4 are conducted under the same warm lighting condition kz: as the training set with different object appearances. C1 to C4 kz: C3 are performed under identical cold lighting conditions, with varying object appearances. The order of the groups is consistent to that shown in Fig. 6.
Refer to caption: extracted/5982770/images/SuccessRate/realworld.png

Figure: Figure 11: Sapien Success Rate. This figure presents the success rates for tasks of the SAPIEN 3 benchmark. The groups are labeled as T1 to T7 and L1 to L3. The settings are varied across 7 different tablecloth textures (T1 to T7) and 3 distinct lighting conditions (L1 to L3). For T1 to T7, the mean and standard deviation of the success rates are calculated over different lighting conditions with 3 lighting conditions combined for one texture. Conversely, for L1 to L3, the mean and standard deviation are computed over various tablecloth textures under a fixed lighting condition, with 7 textures evaluated for each light condition. The order of tablecloth textures and lighting conditions aligns with that presented in Fig. 7.
Refer to caption: extracted/5982770/images/SuccessRate/sapien.png

Figure: Figure 12: MimicGen Success Rate. This figure presents the success rates of Stem-OB and baselines across four distinct tasks. The mean and standard deviation are computed over 300 episodes. Each group of bars corresponds to one experimental setting. The bar order is consistent with the arrangement shown in Fig. 8.
Refer to caption: extracted/5982770/images/SuccessRate/mimicgen.png

Fig. 10, Fig. 11, and Fig. 12 illustrate the success rates of Stem-OB and baseline methods in real-world, SAPIEN 3, and MimicGen environments, respectively. These three figures display the performance of Stem-OB under each setting with more details.

### C.5 Details of Illustrative Experiments

Figure: Figure 13: Illustrative Experiment Objects.
Refer to caption: extracted/5982770/images/Illustrative/user_object.png

Image Distance Calculation: We use the 2-norm to compute the image distance $D(\bm{x},\bm{y})$ in the latent space, where a smaller distance indicates greater similarity between the two images.

Intra-Category Distance Calculation: the intra-category distance is calculated as the mean of the pairwise distances between images within the same category $\mathcal{I}$

$$ $\displaystyle D_{intra}=\frac{2}{N(N-1)}\sum_{i=1}^{N}\sum_{j=i+1}^{N}D(\bm{x} _{i},\bm{x}_{j})$ (14) $$

where $N$ is the number of images within $\mathcal{I}$ and $\bm{x}_{i}$ is the $i$-th image.

Cross-Category Distance Calculation: the cross-category distance is calculated as the mean of the pairwise distances between images from two different categories $\mathcal{I}_{1}$ and $\mathcal{I}_{2}$

$$ $\displaystyle D_{intra}=\frac{1}{MN}\sum_{i=1}^{M}\sum_{j=1}^{N}D(\bm{x}_{i}, \bm{y}_{j})$ (15) $$

where $M$ and $N$ are the size of $\mathcal{I}_{0}$ and $\mathcal{I}_{1}$, respectively, $\bm{x}_{i}\in\mathcal{I}$ and $\bm{y}_{i}\in\mathcal{I}_{2}$.

### C.6 Details of User Study

Figure: Figure 14: Example of User Study Questions. The upper two images represent one example from the first section, and the lower two images represent one example from the second section. Each pair is generated through a randomly selected inversion step. To prevent users from making judgments based on image size or orientation, these images have been randomly rotated, cropped, and zoomed.
Refer to caption: extracted/5982770/images/Illustrative/user_example.png

We conducted a user study to evaluate the quality of the inversion results through a questionnaire-based approach. In the first section, consisting of 42 questions, participants compare two images per question and are asked to determine whether the images depict the same object, different objects of the same category, or entirely different objects. Participants are informed that the images have been randomly rotated, cropped, and zoomed, so image size or orientation should not influence their judgment of object similarity. All images are snapshots of items like bowls and faucets that are used in the real world experiment, with each pair inverted by a randomly selected inversion step.

The second section, consisting of 56 questions, presented images from real-world tasks, as seen in Fig. 6. In this section, participants evaluate whether the images represent the same task, different scenes from the same task (with variations such as lighting or object appearance), or entirely different tasks, where the target of the task changed (for example, a cup versus a drawer). Participants are informed that scene variations could include changes in environmental conditions, such as lighting, or the appearance of objects, like a change in a tablecloth or the texture of a drawer, but the underlying task or function remains the same. To avoid bias from image size or angle, we also randomly cropped, rotated, and zoomed the images. The example of images from the two sections is shown in Fig. 14. A total of 51 valid questionnaires were collected, with the results presented in Fig. 3.

## Appendix D Visualization

### D.1 Inversion Visualization

Figure: Figure 15: Realworld Inversion Results. Each task corresponds to two rows, with each representing a distinct camera view. The first column of each row is the original image, while the other columns are the DDPM inversion results. Starting from the second column, the diffusion steps are increased from 5 to 50, with the incremental step being 5.
Refer to caption: extracted/5982770/images/Realworld/visual_real.png

Figure: Figure 16: Simulation Inversion Results. The first four rows correspond to the tasks in SAPIEN 3, while the last four rows correspond to the tasks in MimicGen. Each row represents a task, with the first column displaying the original image and the subsequent columns illustrating the DDPM inversion results. Starting from the second column, the diffusion steps are increased from 5 to 50, with the incremental step being 5.
Refer to caption: extracted/5982770/images/Sapien/visual_simulation.png

Fig. 15 and Fig. 16 display the inversion results for real-world and simulation environments, respectively. In the real-world environments, each task spans two rows, corresponding to the same camera view as in Fig. 6. In the simulation environments, each row represents a task, with the first four rows showing SAPIEN 3 environments and the last four showing MimicGen environments. The first column in each row is the original image, while subsequent columns present the inverted results. From the second column onward, the inversion results are generated by DDPM inversion, with the 50 total inversion steps and 5 incremental steps from 5 to 50.

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 2
questions: Please refer to Weakness.
soundness: 3
strengths: * This paper focuses on enhancing the robustness of visual imitation learning, addressing a practical and impactful topic. The approach holds significant potential for advancing research in the field of robotics.

* The idea of using diffusion inversion to remove low-level visual variations while preserving high-level scene structures is both novel and intriguing.

* The evaluations are thorough, with experiments conducted in both simulated environments and on real-world robots. The real-world experiments highlight the method's strong generalization capabilities.
summary: This paper introduces Stem-OB, a method that enhances visual imitation learning by using image inversion from pretrained diffusion models to reduce low-level visual variations while preserving high-level scene structure. Stem-OB creates a shared representation that is robust to various appearance changes without additional training. Empirical results on several benchmarks demonstrate the effectiveness in challenging environments with lighting and appearance changes.
weaknesses: * I found the structure of this paper somewhat difficult to follow, as certain sections lacked clarity. For instance, in the preliminaries, the definition of diffusion inversion seemed to be a combination of both forward and backward diffusion. But in Figure 2, it seems that proposed method uses the noised observations as the policy input. If the proposed method is trained on the inversion-altered space, why using the noised version of the observation as input can improve the performance? How can the authors guarantee that applying forward diffusion to the image improves generalization?

* There seems to be a trade-off when choosing the inversion step, which can not be either too large or too small. Is there any explanation about this phenomenon? 

* The theoretical analysis section was also challenging to understand. The authors discuss a loss between two latent variables,  $x_0$ and $y_0$. What  is the intuition of calculating the loss of two different images? From my understanding, the attribute loss here should refer to the inversed data $\hat{x}_0$ and its original version $x_0$. Could the authors also clarify the statement, “images with fine-grained attribute changes tend to become indistinguishable sooner than those with coarse-grained modifications under identical diffusion schedules”? 

* Section 4.1 reminded me of another study [1], which employs diffusion models to purify noise within noisy demonstrations while preserving the optimal structure. Are there any conceptual similarities between that approach and the proposed method?

* Typos: Line 242: $er$f should be $erf$.

[1] Imitation Learning from Purified Demonstrations, ICML 2024.

---

confidence: 3
contribution: 4
initial_rating: 8
presentation: 3
questions: The most significant questions are already listed under weaknesses. In addition, I would be interested to get a clarification on the following points:

- line 272: How many images?
- line 428: Can the authors explain how they get to this conclusion? The training setting performance of RO in particular looks to me like it was not trained correctly.
- line 430: Given the result in D2B, I don't think the conclusion can be that there is always a high success rate?
soundness: 3
strengths: - The idea is, as the authors write, 'simple yet effective' and the authors manage to neatly package a quite theoretical idea and a very practically result measured in real-world robotic experiments all in 1 paper
- While I lack the background to assess the theoretical grounding, the idea itself is well explained
- The authors combine experiments on simulated datasets with actual real-world tests. Real-world tests are incredibly important for visual methods, yet in case of robotic applications very hard to make reproducible. The combination of both is a commendable experimental design. [**update after discussion phase** the extend and rigour of experiments has even further improved during the discussion phase]
- Similarly, the authors conduct real-world user studies to investigate their hypothesis that diffusion inversion progresses along semantic hierarchies.
summary: This paper proposes to use diffusion inversion as input preprocessing for visual imitation learning methods in order to improve their generalization to visual changes between demonstration and test setup. The proposed approach is evaluated in real-world robotic experiments as well as on 2 simulated datasets.
weaknesses: - [**update after discussion: this point was fixed**] I find the derivation in Section 4.1 very hard to follow. Only the first sentence of Section 4.2 makes it clear that the whole derivation is based on the assumption that semantically similar images are closer in latent space. That is a HUGE assumption and the following experiments on intra- and inter-class confusion do not show this over multiple levels of semantic hierarchies, but only for the semantic leafs. The introduction (lines 60-65) however argues that much more than leaf classes, abstract semantic hierarchies are important to generalize over perceptual differences.
In the empirical study on diffusion inversion, one might argue that out of the 5 empirical examples, "bowl" and "cup" are most similar. However, Table 1 shows that the latent representation of 'cup' samples is actually on average closer to any of {'drawer', 'duck', 'faucet'} than to 'bowl'. To me this makes the assumption 'semantically similar images are closer in latent space' quite unbelievable. For sure, this assumption and therefore the formulation of semantic overlap as overlap of latent gaussians is anything but 'intuitive' (line 270) in presence of this data.
- [**update after discussion: this point was fixed**] In relation to the above, I miss a clear definition of what kind of variation should be compensated through diffusion inversion. The abstract and introduction repeatedly claims that diffusion inversion can extract the 'high-level structure' of the scene, suggesting generalization over different object instances, shapes, appearances, relative placements, and robot arms. Section 4.1 considers 'variation', 'fine-grained attribute change', 'coarse-grained modifications', and 'semantic overlap' without defining any of these. The investigation in Section 4.2 is focused on variations within a semantic object class, i.e. a demonstration with 1 cup should be repeated with a different cup. The experiments then consider lighting change and a limited set of object appearance change of sometimes multiple object instances, while locations are fully fixed. The problem is that all of this currently does not fully fit together and it would be good if the authors can define more accurately what kind of variations they expect diffusion inversion to abstract / generalize over, and then design experiments accordingly to show improvement with exactly these variations.
- There are a couple of odd aspects about the simulation experiments that raise questions about the soundness of the results:
  - For the benchmarks from ManiSkill and MimicGen, why were not all tasks evaluated? For ManiSkill, plug-charger seems to be specifically excluded and for MimicGen 4 out of 12 tasks were picked without any explanation.  
[ **update after discussion: now there are 8 out of 12 tasks from MimicGen. It is still a subset, but this indeed makes the results more thorough**]
  - While I did not quickly find comparable numbers for ManiSkill, the MimicGen paper reports much higher success rates for the investigated tasks. E.g. for Threading, MimicGen reports around 19% sucess from just 10 videos and 98% success from 1000 videos. For the 500 videos used in the experiments here, the success is below 19% for 3 out of 4 variants, including the proposed method. Why are the achieved success rates so low? And can the proposed method actually improve anything in a more state-of-the-art setting? [**update after discussion: This concern partially remains. While the authors show that their success rates for 500 trials are mostly between the baseline MimicGen results of 10 and 1000 trials, there is no case where the authors can demonstrate to improve over these prior results. If one expects the relation between number of trials and success rate to follow a saturation trend rather than a linear trend, their results are still a bit below those of prior works.**]
  - Why is the RO baseline excluded from the simulation experiments? [**update from discussion: this was resolved**]
- In all experiments (real and simulated), most of the differences between methods are within the standard deviation, so it is very hard to say if any conclusion can be drawn. Why is the standard deviation so high? [**update from discussion: this has been thoroughly addressed by the authors. The updated results show, similarly to the question about success rates above, that Stem-OB does not push the state-of-the-art further than where it is. But it does more importantly also not appear to be much worse, while the method in some sense is simpler.**]

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 4
questions: The current method conducts experiments solely on Stable Diffusion. It would be valuable to understand if the conclusions drawn from Stem-OB are applicable to other generative models, such as Flux or SD 3 (which uses flow instead of diffusion). Demonstrating that Stem-OB generalizes across different models would strengthen the claim of robustness and broaden the impact of this approach beyond a single model framework.
soundness: 3
strengths: 1. The paper is well-written, with clear motivation and solid experimental validation, including both real-world and simulated experiments.

2. The central idea resonates well: robotic observations often include excessive low-level details, while effective scene understanding requires capturing high-level structural information. This paper draws on an intriguing insight from Yue et al. (2024): "Rather than uniformly removing information from different semantic hierarchies, the process brings structurally similar images closer in the early stages of inversion." Building on this, Stem-OB leverages this observation to project data into a space that prioritizes inherent structural features.
summary: This paper proposes Stem-OB, a method that uses diffusion inversion to transform observations into a shared, robust representation. This approach enhances robustness to various appearance changes, such as shifts in visual patterns and lighting conditions. During inference, Stem-OB approximates the partial inversion process to maintain efficient inference speed.
weaknesses: Stem-OB enhances the generalization capabilities of robotic systems by using diffusion inversion to map observations into a shared representation. However, another promising line of research—self-supervised representation learning—also aims to unify raw observations into a common representation space, reducing low-level visual discrepancies. I suggest that the authors consider benchmarking Stem-OB against these approaches, particularly methods that leverage self-supervised learning (SSL) for robotic representations, to offer a more comprehensive comparison.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: See above. I currently give a weak accept to the paper but am inclined to vote for acceptance should my points above be taken into consideration during the rebuttal.
soundness: 3
strengths: * The paper relies on a central observation from a previous paper (Yue et al. 2024), which found that during the inversion process of a diffusion model, fine-grained (or high-frequency) details of an image are destroyed first before the low-level semantic concepts. This paper builds on this basic concept and proposes a new preprocessing step that uses this theoretical property of diffusion models as a preprocessing step, which, in the process, improves the generalization of the IL algorithm. 
* The technical principle seems well established at this point, but this paper takes these theoretical findings and applies them to a new problem. While the whole approach is very simple from a technical standpoint, the paper re-introduces the necessary background in section 2 before developing the reader's intuition as to why this preprocessing might work in section 4.2 through theoretical explanation and small experiments. I believe that this paper warrants publication because it shows that diffusion model inversion can be useful for additional tasks. 
* The presentation of the paper is clear and easy to follow overall. 
* The results nicely highlight the drawbacks of previous methods and that the proposed method offers much better generalization capabilities. 
* I appreciated the detailed appendix, with theoretical derivations, additional details for reproducibility, and additional experiments.
summary: Visual imitation learning is a method for robots to learn tasks through visual human demonstrations. While multiple methods exist for preprocessing visual data that increase the generalizability of VIL, these methods are only partially efficient. To address the generalization issue, this paper proposes to use the inversion process of a pretrained diffusion model as a preprocessing step to increase the generalizability of visual imitation learning methods. The proposed method relies on the intuitive insight that images with only small perturbations between them are closer in latent space (and therefore become indistinguishable under diffusion faster), than data that exhibits large differences. The method is tested on real and synthetic data and shows that the proposed preprocessing significantly increases the models generalization capabilities.
weaknesses: I believe that the paper's technical novelty is small, as the main concept has already been introduced in other papers. That being said, I also believe that the paper is worth publishing, as it adds to the growing body of literature showing the usefulness of diffusion model inversion for various tasks, and the empirical validation is well executed.

I have additional suggestions to improve the paper's readability and make it easier for the reader to understand.
1. Section 2.2 can be cut (title), and the paragraph can be integrated into Section 3.1. 
2. As a reader, I would really like a short problem formulation that properly introduces the problem the paper addresses. This should properly define the inputs and outputs. I would also suggest rearranging the sections/subsections of the paper to something like this:
* Introduction
* Related Work
* Problem Definition
* Preliminaries
* Method
This would greatly improve the flow of the paper. 
3. While the paper is generally easy to follow, some sentences are should be simplified or run through a writing program. I have added some examples below:
* Line 85: "To be specific, our method is as simple as inverting the image for reasonable steps before" What is reasonable? Maybe rephrase?
* Line 226: "Intuitively, given a source image and its variation, distinguishing between them becomes increasingly difficult" This sentence does not make a lot of sense (although I can guess what is meant).
There are many more of these convoluted sentences. Cleaning up the writing a bit would go a long way to improve the paper.

There

Meta-review: This paper proposes Stem-OB, which suppresses low-level visual differences while maintaining high-level scene structures. The idea is to apply a diffusion inversion process to transform images into a shared image space without additional training. The approach is demonstrated with robust visual imitation learning. Given the compelling results and detailed description of the proposed approach, AC confirms that this paper can shed light on the possibility of the practical use of diffusion priors for visual imitation learning. However, due to image degradation, AC also sees that the fine details of the original image can be lost. The paper shows mainly the picking tasks for the large and few objects in the scene whose main structure is not affected much after the inversion. AC recommends extending the proposed idea for more complex manipulation cases that exhibit tiny and thin structures as future work.

Decision: Accept (Spotlight)