Blindfolded Experts Generalize Better:  Insights from Robotic Manipulation and Videogames
Authors: Ev Zisselman
Mirco Mutti
Shelly Francis-Meretzki
Elisei Shafer
Aviv Tamar
Venue: NeurIPS 2025 poster ()
Keywords: Imitation Learning
Robotics
Sequential Decision-making
Generalization
Video Games
Primary area: reinforcement_learning
TL;DR: We propose to clone the behavior of ``blindfolded'' experts that are compelled to employ non-trivial exploration to solve the task, which leads to better generalization.

Abstract: Behavioral cloning is a simple yet effective technique for learning sequential decision-making from demonstrations. Recently, it has gained prominence as the core of foundation models for the physical world, where achieving generalization requires countless demonstrations of a multitude of tasks. Typically, a human expert with full information on the task demonstrates a (nearly) optimal behavior. In this paper, we propose to hide some of the task's information from the demonstrator. This ``blindfolded'' expert is compelled to employ non-trivial *exploration* to solve the task. We show that cloning the blindfolded expert generalizes better to unseen tasks than its fully-informed counterpart. We conduct experiments of real-world robot peg insertion tasks with (limited) human demonstrations, alongside videogames from the Procgen benchmark. Additionally, we support our findings with theoretical analysis, which confirms that the generalization error scales with $\sqrt{I/m}$, where $I$ measures the amount of task information available to the demonstrator, and $m$ is the number of demonstrated tasks. Both theory and practice indicate that cloning blindfolded experts generalizes better with fewer demonstrated tasks. Project page with videos and code: 
[https://sites.google.com/view/blindfoldedexperts/home](https://sites.google.com/view/blindfoldedexperts/home)

Blindfolded Experts Generalize Better: Insights from Robotic Manipulation and Videogames
Zisselman, Mirco Mutti, Shelly Francis-Meretzki, Elisei Shafer, Aviv Tamar Technion Israel Institute of Technology
Abstract
Behavioral cloning is a simple yet effective technique for learning sequential decision-making from demonstrations. Recently, it has gained prominence as the core of foundation models for the physical world, where achieving gener- alization requires countless demonstrations of multitude of tasks. Typically, a human expert with full information on the task demonstrates a (nearly) opti- mal behavior. In this paper, we propose hide some of the task's information from the demonstrator. This "blindfolded" expert is compelled to employ non- trivial exploration to solve the task. We show that cloning the blindfolded expert generalizes better to unseen tasks than its fully-informed counterpart. We con- duct experiments of real-world robot peg insertion tasks with (limited) human demonstrations, alongside videogames from the Procgen benchmark. Addition- ally, we support our findings with thcoretical analysis, which confirms that the generalization error scales with /I/m,, where measures the amount of task information available to the demonstrator, and m is the number of demonstrated tasks. Both theory and practice indicate that cloning blindfolded experts gener- alizes better with fewer demonstrated tasks. Project page with videos and code: httpv//ewwwwnnoooedeeeeddeeeeeee
https://sitee: htw.ssitsssgggllleee..
Introduction
Behavioral cloning (BC) is simple yet effective method for training policies in sequential decision- making problems [32 6. In BC, an expert demonstrates how to perform a task, and the sequence of observation-action data is input to supervised learning algorithm for training policy.
key question in BC is generalization--hvv many demonstrations are required to train an effective policy. For single tasks, a well-investigated challenge is compounding errors-small mistakes in the trained policy may lead to visit states that the expert did not visit, further increasing the prediction errors [43 However, recent results show that using appropriate neural-networ architectures. BC can learn to solve complex tasks even with a modest number of demonstrations (6] [59), and these results are reinforced by recent theory [13]. For multiple tasks (or significant variations of single task), on the other hand, BC still requires abundant data, and recent methods for mitigating data requirements include augmentations 26). simulation [52], and fine tuning foundation models trained on large scale In this work, we hence focus on generalization to task variations.
demonstration data
49 21
While various works study how to improve generalization via the BC algorithm (43, the policy representation (6), and the data diversity [23], one aspect that remains unexplored is the experts themselves. Many tasks can be solved in various ways-can some behaviors generalize better than others? Recently, in the context of zero-shot reinforcement learning, Zisselman al. [61] showed that certain exploratory behaviors generalize better than goal-oriented, wwward-maximizinn behavior. Intuitively. since exploratory behavior less gral-oriented, it is less dependent on any particular task
"Conresonnencce E-mail: ww_zisscampus..o wechnion..en...e. ac
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Masked Observations (Xmaah))
Original Observations (Korigtnal)
+ET
Blindfolded Expert (BF-Experr)
Expert
1tp--BC
Trajectories otayy-ota
MBc
Trajectories
BC
Mar Sxpert
BC
Trxpert
ptab.oct
Tm 23.4
+s Xeriginal
+
a gF Expert
Expert
Xorigtnal
Figure I: Illustration the learning process. Note that the mask on observations only applies to the blindfolded expert, while the observations in the logged trajectories are unmasked in both cases.
instance and therefore, more likely to generalize to novel tasks. In this work we ask-can a similar principle be useful also for imitation learning?
Our main idea (depicted in Figure D is that by introducing a blindfold an information bottleneck on the expert's ''s observation that makes it harder to identify the particular task, to express a more exploratory and less task-dependent behavior, which we conjecture will generalize better. Importantly,
induce the expert
method does not change the observations used for training the policy but only the expert's behavior, and therefore compliant with any BC algorithm, and complementary to the methods for improving generalization mentioned above. Since the expert's s exploratory behavior is typically history dependent, our method makes use policy architectures that can process a sequence of observations, such recurrent neural networks or transformers 7 53
Theoretically, we prove an upper bound on the generalization that scales with Egen + /I ///m. where measures the amount of task information available the demonstrator, m is the number of demonstrated tasks, and Egen is a cost associated with the expert not taking optimal actions. For domains where the exploratory behavior can still solve the task, Egen is zero, and thus by lowering using a "blindfold" we reduce the generalization eтror without paying any price. To our knowledge, this result the first of its kind in relating non-trivial properties of the expert's behavior multi-task generalization of the resulting BC policy.
Empirically, we demonstrate standard benchmark for task generalization, and on real-robot peg insertion task, based on the FMB challenge (25), where the shapes the peg and the hole define the task. For the Procgen games, our blindfold hides the observation and reveals only the agent's immediate surroundings to the expert. For the peg insertion domain, we let the expert teleoperate the robot by observing images from robot-mounted cameras, and mask out the hole shape from the image. In both domains, we find that the blindfold induces more exploratory behavior from the expert, which turn yields significantly better generalization to different tasks.
approach on simulated games from the Procgen suite g
Our results pave the way to new and principled approach for collecting demonstrations, both for specific problems, and also for more general foundation-mont scale endeavors.
Problem formulation
Throughout the paper, we will focus on multi-task imitation learning setup in which we aim to clone an expert's behavior from demonstrations of (small) selection of tasks with the goal to generalize the behavior to (many) unseen tasks. As we shall see both theoretically and empirically, the generalization not only affected by the number of available demonstrations, but also by the information available to the expert when performing demonstrations, First, let us introduce the setting formally.

---

Setting, We consider a set of tasks θ ;= [0 and a task distribution Po E A(O), where A(S) denotes the probability simplex over set S. Each task A E θ is defined through a Markov Decision Process (MDP 33 Ma= == (X, A. ro, H), where X, A. H respectively denote the observation space, the action space, and the horizon of an episode, which we assume common across all the tasks in θ Instead, each task may have their own transition model pe X x A - ( and reward function rg X' x A + [0, 1]. A history-basee randomized policy is a sequence of functions π = (Th : Th + ((A)) and T = following process. An initial observation is sampled x0 ~ po(.). Then, for every step h 2 0, an action is sampled from the policy an ~ =(>"), the reward n = rg (a", .a") is collected, and the MDP emits the next observation r^+11 ~ (o(x", a²). The process goes on until the step H is reached The Reinforcement Learning (RL (47) objective for an MDP Ma is the cumulative sum of rewards
I h=0 where Th is the set of ht-teps trajectories ~^ = (x",a", r....... x²) -f-I - Th A policy π on the MDP Ma induces a distribution 16 over trajectories with the
(TT) Epg = H-1 r*), where the sequence (r" .o......'11) is taken expectation over trajectories 17.. An optimal policy for Ma is denoted as x² arg max J (T). For some R N, we assume J(x²) S R, where typically R = 1 when rewards are sparse, as large as H when rewards are dense.
Behavioral cloning, In the setting described above, we assume to have access to a dataset of expert demonstrations F {0,- Po: (Tr,... Tin)) ~ ~P ym where Tij (= oiy ro .....) . a H -tstss trajectory sampled independently from a policy я E in the MDP 0,. Thus, the total number of trajectories is E mmn and the total number of transitions is m.H. With the available data, we aim to clone the expert's behavior x a problem that is known as behavioral cloning [T.. The idea is to train a policy क mimic the expert's 's policy x by minimizing supervised learning loss on the demonstrations. While several choice of loss functions could be made [55), here we opt for the negative log likelihood as in [13]. The behavioral cloning problem is then
a r
H
1
ã E arg min [(T)
log
((a (T
=en
1=
K=U
where 11 is a policy space of our choice and rij is h-steps chunk of the trajectories Ta in the dataset of demonstrations E, J is the action taken at step h in Tij. While a sufficiently expressive policy space 1I may allow for a cloned policy के that closely approximates the expert on the training data C(+) we typically aim for policy f that can mimic the expert's 's behavior on unseen data as well. Differently from the common setting [4 42 56 38 36 generalization across unseen observations in X, but also across unseen tasks in O. Before proceeding with the study of generalization in the next section, we introduce additional notation for later use.
37 13, here we are not only concerned with
Additional notation. In our behavioral cloning problem [I. single data point is given by the triplet (0., = ab which we intend as realizations from the random variables (T, X, A) distributed
as T ~ Po and AX ~ P7" respectively. We will turn to one or the other notation when convenient. For random variable A taking values 01, ...... with probabilities p(a1) ). p(az)..... we denote its entropy (A) = Σ, p(a,) log p(a,). For two random variables A, B, we denote their mutual information T JABB (A) (A) (BB) H (B] A), where H (AA () is the conditional entropy. Finally, we will use the symbol S to hide constant and lower order terms from inequalities.
3 Generalization analysis
In the previous section, we detailed how an expert's policy can be "cloned" from data by solving the optimization problem ( Obviously, fully cloning the expert's behavior is a far fetched objective when limited demonstrations are available: When training data spans only small portion of the observation space X' and the set of tasks θ how can we extract information on what would the expert do in unseen observations and tasks? Nonetheless, we aim for our cloned policy के transfer least part of the expert's behavior beyond the demonstrated observations and tasks. In this section, we
Note that this does not hinder generality, as we can always take X = Usger when observation spaces vary across tasks (ditto for the action space) and H maxage Hs when the episode horizons vary. *Oftentimes, the episode horizon is an upper bound to the episode length, while secondary termination conditions may end the episode early, as it will be the case in our experimental setting. For the ease of presentation, we ignore early termination in our setup and consider episodes of length H.
3

---

provide formal study of the generalization guarantees of the cloned policy म, showing an original dependence with the information available to the expert when collecting demonstrations. To specify what do we mean by "information" in this setting, let us consider the figure below.
observation
X
expert's representation
A
action
task
T
lz information bottleneck
The latter is a graphical illustration of the expert's 's behavior At each step, the expert takes as input an observation (X) which depends on the task they are going to demonstrate (I"). processing them into an internal representation Z. The demonstrated action (A) then conditioned on Z. Note that Z recurrent and retains any information that is relevant select A, including the task information that may be available in the observation X. The mutual information Iz,r measures the task-related information that goes into Z and, consequently, how much the strategy select A relies it.
f
We typically expect an expert with full information on the task-large / 7 to demonstrate an optimal policy specific to the task, without any exploratory actions. Instead, whenever there is an information bottleneck between the task information and the expert-small 7 12,T-we we expect them to take exploratory actions to first "understand" the task in order to solve it. We still call them experts as we assume their behavior to be optimal with the given information, a concept that has been formalized with Bayes-optimal policies [T6.. We conjecture that the latter behavior may generalize better to new tasks, as the process of understanding the task is more general than just solving it. In the following, provide formal result based this conjecture, in which analyze the generalization gap of the cloned policy as a function of the information I2,r available to the demonstrator.
we are interested in deriving upper bound on the performance gap between an optimal policy for each task (T*) and the cloned policy (T). Since we are considering multi-task setting, we average the gap across the task distribution Po.
Analogously to previous works [e.g., 41 13,
E -
J
E
E
IE
TwP
h=
Before stating the result, we introduce few technical assumptions. First, we define the generulization error of policy
Egen (T) B # A)]
E
T~Po XA~
for single point indicator loss. We make the following assumptions on the expert's policy, Assumption 1.
The expert's 's policy π is deterministic.
Assumption 2. The generalization error of the expert's 's policy is given by Eyen (
Note that we allow the expert's policy depend on the history, for which assuming determinism reasonable even when information bottleneck is applied to the expert. Whereas it is standard in the literature to assume the expert is optimal, i.c., Eyen (T = 0, in the presence of an information bottleneck we do not take for granted that the expert optimal in all the tasks. Nonetheless, if the expert's 's behavior is Bayes-optimal, non-trivial worst-case bounds on Enen( hold [5], for which the generalization error only scales with log( (H) under
assumptions. Moreover, in settings where the reward is sparse denoting task success, i..., R = 1, the optimal policy, albeit inefficient in the number of steps.
policy may still have
Egen (I ) 0 w.r.t.
Then, regarding the behavioral cloning problem [ we make a pair of assumptions as follows. Assumption 3. The expert's policy is realizable in the policy space II, i.e., x E II. Assumption 4. We have access to an optimization oracle that solves problem I with bounded error
m 12 H-1
Eopt
mnH ΣΣΣ
((((7)) # ot))
K=
4 "Note that this is not related to the architecture the cloned policy, which will be discussed later on. Note that the optimal policy x² depends the task I whereas ã is single policy cloned from data.

---

In principle, one can fulfill Asm. 3 by cloning the demonstrations into a rich enough policy space II. However, more expressive policy space may lead harder optimization problem, especially when the policies are represented through large neural networks, for which ① is non-convex. We now have all the ingredients state our main result.
Theorem 3.1. For a confidence ó E (0, 1), it holds with probability at least 1 - 26
E (J(m))- - J(F) & RH (x) Eopt. (7) + 11,2 AA (og(1A1/6) log( [1 mm/6)) T~Po
All of the derivations and the hidden constants can be found in Appendix Instead, here we unpack the bound and discuss the meaning of each term. First, the RH factor accounts for the cost of a "mistake" of the cloned policy, i.c., choosing an action different from π²_ The terms Egen (T²) and Eope(Tt) depends on the quality of the expert's policy and the solver for (I). hence they cannot be reduced with additional data. The last term, scaling with the number of trajectories in each demonstrated task n -1 comes from a typical behavioral cloning analysis of generalization within the training task [e.g..13] The more demonstrations we have from a task, the better we can clone the expert's policy in that task. The third term, scaling with the number of demonstrated tasks m=!, controls the generalization across tasks and comes from the analysis of generalization induced by an information bottleneck (20), which is expressed by IZ,T. The most important finding of our result lies in this term: To improve generalization of the cloned policy, we can either increase the number of tasks m or apply an information bottleneck- a "blindfold" to the demonstrator to reduce I2:r without paying any meaningful price, as is typically Egen (T²) = 0 for tasks with sparse rewards and Egen (T X log(H) for dense rewards, while other terms remain the same.
In the next sections, we provide an extensive empirical evaluation showing that this result is far from being a theoretical fluke, but translates to practical scenarios as well.
Overcoming assumptions. The result presented in this section holds for deterministic expert's policies (Asm. []. finite action space (due to |A| dependency), and finite policy class (due to [II| dependency). Deterministic expert's 's policy and finite policy class can be easily overcome by extending the in-task generalization result stochastic policies and infinite policy classes, done in [3]. Instead, the dependency on |AA| comes from reducing the cloning problem to classification, in order to invoke information bottleneck generalization results [20]. In principle, extending the analysis to
continuous action requires an analogous generalization bound for the regression problem (29] and to circumvent established negative results for this setting [46].
4 Experiments
In this section, we report an experimental campaign to validate the results of previous sections and to demonstrate the importance of the expert's behavior for multi-task BC generalization. To this end, we train two policies, with the same BC algorithm, on human demonstrations collected by either a traditional expert or a blindfolded exper We refer to the resulting policies BC and BBC BF respectively. We compare them in the success rate achieved on both the demonstrated tasks and unseen test tasks. We repeat the experiment twice, first in simulation on the Procgen maze and heist (Section 1. then on a real robot peg insertion task (Section 4.2). For both domains, we describe how the information bottleneck for the blindfolded expert obtained in practice.
4.1 Procgen maze and heist
Proegen is popular benchmark for measuring sample efficiency and generalization 35 34
61]. It consists of 16 different procedurally generated environments in which new levels randomly generated for every episode, forcing agents to handle changing layouts, colors, and textures. Here we focus the maze and heist games in the "casy" setting, in which each level is a 2D make-like layout that the agent navigates. These tasks are the most challenging tasks for generalization in the Procgen suite, and until the results of [61] have seen only minor improvements over random walk. We test how training on a set of demonstrated maze and heist tasks generalizes to unseen tasks for the different experts.
OOr human demonstration dataset for procgen maze and heist is available on the project website.
5

---

Maze
Heist
Peg Insertion
T
Full Obs
Masked Obs
Full Obs
Masked Obs
Full Obs
Masked Obs
Demonstration of actual experts' observations. For the Procgen maze (left-pair), Proegen heist (anddle-pair), and robotic peg insertion (right-pair) We show the full observation of the Expert and the masked observation of the Blindfolded-Expp Expeett.
Figure 2:
Maze Train
Heist Train
Maze Test
Heist Test
Train Score
Score
Test Score
Train
Ita 8C
Epochs
Epochs
Epochs
Epochs
Figure 3: Game score as a function of training epochs in the Procgen maze and heist. Left-pair: performance on 200 training levels. Right-pait pair: performance on unseen test levels. The mean and standard deviation are computed over 10 seeds.
Maze. In Procgen maze, the agent (mouse) must find and reach single goal (piece of cheese) in 2D maze. Procedural level generation produces variation in maze layout (sprawling corridors), goal positions, and the color and texture of navigable spaces.
Heist. The Procgen heist is based on maze-like level, in which the agent has to additionally solve sequence of sub-goals in order solve the task (reaching the gem). At each level, the goal (gem) is hidden behind a series of color-coded locks, accessible through keys of corresponding colors, that are scattered throughout the level (see Figure 2 This task requires multi-stage planning in the form of lock-war waykointssssss and to overcome variations in sub-task ordering, making the task more complex and challenging than regular maze.
Setup. For training, we consider 200 procedurally generated levels. There are discrete actions, move up, down, left, and right in the maze game, and additional 4 actions in heist: left-down, left-up, right-down, and right-up. The observations are 64 x 64 RGB images of the current state as top-down view of the maze.
Data collection. We collect a total of 4K demonstrated trajectories from 200 different training levels (20 trajectories per level). The environment allows for at most 500 input steps for maze and 1000 for heist, and we only retain trajectories of successful expert demonstrations. The standard experts are humans who see the top-view of the entire game, and therefore are likely to follow the shortest path the goal. The blindfolded experts are humans playing observations of its layout, such that only the immediate proximity of the agent is visible and the rest of the level is concealed (see Figure 2 middle and left). As a result, human experts cannot directly plan path the goal location, and exploration of the level is needed. Note that while the observations are masked to the expert, the stored data contains the original (unmasked) observations for training the cloning algorithm. The trajectories are tuples of observation, action, reward, and done flag (Ot, at, Ft. done).
game with occluded
Policy architecture and training. Training is conducted from scratch on the demonstrated trajec- tories by minimizing the negative log likelihood We use architecture from [27 for both the Expert (C) and BF-Expert (TBF-BC)~ nc) ResNet 17 to encode the observations, which are then processed by two fully-connected layers. capture the exploratory behavior, we add single GRU
7 before the Softmax policy layer (further details are in Appendix C).

---

Table 1: Average number of steps + std in the trajectories demonstrated by standard Experts and BF-Expert The latter takes more steps due to the information bottleneck, which forces exploration.
Mode
Maze
Experts
61.2 +
69.4 + 3.3
64.8 2.1
96.6 + 4.9
68.8 + 2.2
28.6 + 1.2
BF-Experts
74.3 + 5.9
165.6 + 42.9
163.6 + 43.6
148.6 + 33.1
194.4 55.0
51.6 + 1.8
Train Score
Test Score
Tac
Mac ext
faF
Epochs
Epochs
Figure 4: Procgen maze score as function of training epochs. Left: performance on 100 training levels. Right: performance on unseen test levels. Policies тBC and TBF-BC are trained on 2K trajectories, the former clearly overfits. We also train BBC-ent on 4.6K non-blindfolded experts' trajectories, matching the total number of steps of TBF-BC- The resulting BBC overfits despite access to twice the amount of data. The mean and standard deviation are computed over 10 seeds.
Results. In Figure we compare the game score over the training epochs achieved by policy cloning of the standard Expert (TTCC) and the BF-Expert (# As we can see from Figure 3 (left-aiir) the performance of the two policies is similar on the training levels, approaching the maximum task score. However, as evident from Figure (right-pair) while the TBC- BF gracefully improves the test score, the BBC policy overfits to the training-set, and its test score slightly degrades with further training. This a testament to the inherent generalization capabilities granted by the BF-Expert w.r.t. standard Expert.
Number of steps. Table [ (left-most most column) details the average number of steps taken by the experts (Experts) and the blindfolded experts (BF-Experts) across 100 demonstrated levels, accruing 2K maze trajectories. Trajectories taken by the Experts are shorter on average than the BF-Experts, supporting the assumption that the BF-Experts exhibit a more exploratory behavior, whereas the Experts take the shortest path to the goal.
To further highlight the contribution of exploration toward generalization and to show that it isn't t merely the result of additional training steps, we train a policy BCC-en on double the amount of non-blindfolded experts' trajectories- match the total number of steps produced by the blindfolded experts. Figure 4 shows that even with an equal number of total steps from both experts. TBC- ext overfits. Further details on the number of trajectories and steps are provided in Appendix
4.2 Robotic peg insertion
Peg insertion is a standard problem in robotic manipulation. Here, we consider the insertion task in the Functional Manipulation Benchmark [FMB, 25 which focuses on inserting variously shaped
pegs into tightly matching holes. Different from 25 however, we investigate generalization: How training on a fixed set of shapes generalizes to inserting previously unseen shapes. We simplify less relevant technical aspects of the benchmark by fixing the peg to the robot gripper, and 3D-printing individual holes, so that discerning the target hole becomes trivial (see Figure 5 In addition, we added several new shapes to the benchmark to increase its variations.
Setup. The task comprises ten pegs of various shapes with corresponding slots. In our setting, the robot initiates with the peg already in its grip and needs to insert it into single-slotted board anchored the surface. The robot setup is shown in Figure 5 full, alongside few examples of peg shapes. We use a Franka Emika Panda robot arm and teleoperate the robot using SpaceMouse. For operating the robot and the Spuse we use the SERL open-sourced package [24) with the same settings as the authors. The actions are input as SpaceMouse commands: 6-DoF end-effector move and twist (location and Fuler angles) at 10Hz, tracked by a low-level impedance controller running at IKHz. The observations obtained as RGB-only images from two Intel RealSense
7

---

Franka Emika
Figure 5: The robotic arm configured for peg insertion (left). Close-up view of various peg shapes and their boards (right).
D405 cameras, mounted on the robot end-effector, which simultancously capture images To avoid background distractions in image observations, we place each shaped board inside a black bin. In addition to the 6-DoF pose of the robot end-effector, the framework also includes the force, torque, and velocity information provided by the Franka Panda robot.
Data collection. We split the ten shapes into training and test shapes. When collecting demonstra- tions, only the training shapes used, while the test shapes serve as withheld subset for evaluation. We collect 400 trajectories from human demonstrations of each training shape, first with a standard Experts and then with BF-Experts. Each trajectory begins random initial pose. When the full information from the wrist cameras is provided to the human experts, they easily succeed in inserting all training shapes. The blindfolded experts are human experts exposed to redacted observations from the wrist cameras, which occlude the articulation of the slot as they attempt to insert the peg (see Figure 2) We use SegmentAnything2 (SAM2) (39] modified to segment live video stream and prompt it to mask out the shape of the target hole. In addition to the masked-out images from the wrist cameras, the initial robot pose varies with each insertion attempt, thus preventing the expert from memorizing or inferring the articulation of the target hole. Note, however, that the recorded trajectories collected by both the Expert and BX-Expert contain the full unmasked observation. The distinction is then the behavior of the two experts, with blindfolded experts taking exploratory actions to cope with masked-out images an attempt complete the task. The trajectories are collected on the training shapes only. Table T details the training shapes and the average number of steps taken by the standard Expert and Ex-Ext until successful peg insertion. Clearly, the trajectories demonstrated by the BF-Expert are longer, indicating more exploratory behavior, whereby the expert must rely on masked-out images until resolving the correct articulation for inserting the peg. In the next sections, show that the resulting exploratory behavior is useful for generalization.
Policy architecture and training. Training was conducted for i E 32, 5} peg shapes, with the remaining shapes serving as withheld test set. We use the same network architecture for cloning both the expert BB and the blindfolded expert BB--BC- Specifically, we use weight-shared frozen ResNet- 10 encoder [17] pretrained on the ImageNet dataset [00 for encoding the incoming images from both wrist cameras. The resulting embeddings are concatenated with the MLP-embedding the proprioceptive information before entering single GRU [7] that outputs a Gaussian policy. The use of memory-based architecture crucial to fully capture the non-Markovian exploratory behavior of the blindfolded expert (28). For more detailed specifications regarding our experimental setup and hyperparameters, please refer to Appendix B
Results. Figure 6 shows the success rate for & = 2, training shapes (results for l = 3, 4 are in Appendix B The success rate is the average of 24 insertion attempts per peg shape by the robotic arm. The results demonstrate that cloning the BF-Expert with #BF- BC achieves better generalization compared with the standard Expert, cloned with TBC, across all peg shapes and over all test subsets. Importantly, the advantage of the proposed blindfolding approach is more significant when fewer
Following the conclusions of FMB 25
omit the depth data, they showed it has marginal benefit.

---

2 Training Shapes
Trsining Shapes
Train
-last
Rac
Per-ac
Train
Test
Success rate
4 Success
Figure 6: Success rate of robotic peg insertion for 10 peg shapes (horizontal axis). We train on a subset of shapes & {2, 5). with the remaining shapes withheld as test-set. Both for & = 2 (left) and 5 (right), cloning blindfolded experts generalizes better than cloning standard experts.
shapes are used train the model. i..., when larger portion of shapes are withheld during training. Interestingly, the figure shows that even for shapes encountered in training, the TBF-BC policy generalizes better than cloning the standard expert тBC- This result of the limited ability of expert demonstrations to account for all real-world variations, e.g.. lighting and control-loop errors. In comparison, the TBF-BC is more robust to these kinds of errors, since it imitates blindfolded experts who cannot rely on visual cues, and must compensate in order to solve the task. Additionally, is worth mentioning that certain shapes are more challenging than others. observe that shapes with greater radial symmetry and convexity (such as the circle, triangle, hexagon) are simpler to learn than shapes with non rodial symmetry (such as the ellipse, rectangle) or non-convex shapes (such as the out-out-rectangee
they require more specific orientation for insertion.
The sensitivity to the choice of specific blindfold. To demonstrate the generality of our approach, we conduct an experiment of robotic peg insertion, where we apply random noise to the experts' observations instead of masking the shape socket. We collected total of 1600 expert trajectories with random noise
L'O0, P] added to all pixel values, from varying maximum noise level PE [0, 100, 170, 200] and 2 training shapes (square and star). We then train network on each noise level P (i.e., 400 trajectories per level). Table details the Success Rate (SR) [%] and State Entropy (SE) of the demonstrated trajectories, alongside the networks' performance on train (square and star) and test shapes (plus and ellipse). The table shows several interesting results:
1. Introducing more noise increases the State Entropy (SE), but
noise corruption (P = 200) impairs the expert, lowering the Success Rate (SR) of the demonstrations, and degrading the robot's performance.
2. Adding the "right" amount of noise (P 100) helps generalize unseen shapes. Although less effective than masking, it still significantly outperforms the traditional approach, i.c., without any blindfolding (0 noise).
3. This experiment demonstrates that a general form of blindfold (non-task specific) may still be superior to the conventional imitation learning approach.
Table 2: Success Rate (SR) [%] and State Entropy (SE) for the various blindfold types: range of noise levels or SAM2 mask. The column "Demos" details the SR and SE experts' demonstration subject to the blindfold. The left-most columns are their corresponding evaluation performance [%] Train (square, star) and Test (plus, ellipse).
Demos
Train
Test
Max-Noise Level
SR
SE
square
star
plus
ellipse
100%
3.17
71%
67%
20%
17%
100
96%
3.26
100%
88%
96%
79%
170
95%
3.39
92%
71%
58%
62%
200
58%
3.46
46%
38%
21%
54%
SAM2 Mask
100%
3.52
96%
96%
100%
96%

---

5
Related works
Our work closely relates to imitation learning, information bottleneck, and robotic manipulation.
Imitation learning theory. The imitation learning literature counts a plethora of contributions, for
which we refer to recent surveys 19 60). Here we are concerned with the generalization of behavioral
cloning - dominant imitation learning technique. Previous works [41 42] [56] 38 336 (3 50 [3
have studied the theoretical limits of behavior cloning and settled the generalization gap on the
training task as J( ( X R (og( Markovian policy in a single MDP. Other works have considered more general settings, including cloning history-based policies 4 and imitation learning under partial observability 48]. However, the generalization of behavior cloning in meta learning setup, which is popular in empirical works [1] [2], is understudied. The only generalization analysis that strikes close to this setting is, to the best of our knowledge, the one in [40]. Differently from ours, they assume online access to the set of tasks to further fine-ture tin-tune the cloned policy and they do not study how the information available to the demonstrator affects generalization, which is our main theoretical contribution.
[3]. These results are mostly limited cloning a
Imitation learning in robotics. Imitation learning has been fundamental to various robotic domains, including autonomous driving [3). locomotion [1, flight M. and manipulation [12]. A recent survey on manipulation, our case of interest here, is provided in (2]. Several works showed that imitation is useful for learning complex visuo-motor policies for robotic manipulation, where key ideas include predicting a sequence of future actions, and using diffusion generative model to learn distribution over the action sequence G 55 [4]. However, it is known that imitation learning requires a large number of demonstrations in order to generalize to variations in the task. Previously studied mitigations include 3-dimensional priors in the representation (58), automatic
data augmentation [26 15 57], interactive data collection [88,, and using simulations [52] Another
approach is leveraging large-scale data, either by collecting diverse task variations [23] or by fine
tuning robotics foundations models 30 49) 21. Differently from the approaches above, we postulate that the way demonstrator performs a task affects generalization, showing that by blindfolding the demonstrator we obtain an exploratory behavior, which induces better generalization when cloned. The generalization of exploratory behavior has been demonstrated in zero-shot reinforcement learning [61]. In comparison, we apply this idea to imitation learning, which requires a different approach, and also develop a theoretical explanation for the improved generalization. The idea that exploration at test time helps generalization has also been explored in the sim-to-real context in (54)
Information bottleneck. When learning X Y relationship between random variables, the information bottleneck (51) prescribes to "squeeze" X into a representation that only retains informa- tion to predict Y. This principle is believed to be factor beyond the generalization capabilities of deep learning [4 and formal generalization bounds through the information bottleneck have been derived [45] 29 20 In imitation learning, the information bottleneck has been used to analyze gener- alization in 3 However, they consider generalization on the training task only and the information bottleneck is applied to the representation of the cloned policy. Our work advocates for applying an information bottleneck to the demonstrator to improve generalization of the cloned policy.
6 Conclusion
We showed that cloning the behavior of blindfolded experts leads to better generalization to unseen tasks. We supported this with theoretical analysis and conducted empirical tests that, for the first time, explored the concept of blindfolding experts in the context of a real-world robotic task, as well as maze videogame. We observed that in both peg insertion and maze-solving tasks, blindfolding the experts encouraged them to enact more exploratory behavior, cloned to produce policies that better generalize. Importantly, our approach achieves better generalization while accommodating any imitation learning algorithm.
Finally, we point out a limitation of the proposed approach: each domain may require a different kind of blindfold (e.g., concealing the field of view in the maze videogame, or masking out the shape of the hole in the peg insertion task). Too little obstruction does not elicit exploration, while redacting too aggressively would impede any informative exploratory behavior (may resort to near random walk). An interesting question is how to find the optimal balance, which we reserve for future research.
10

---

Acknowledgments
This research was Funded by the European Union (ERC, Bayes-RI, 101041250). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union the European Research Council Executive Ageney (ERCEA). Neither the European Union nor the granting authority can be held responsible for them.
References
[1] Pieter Abbeel, Adam Coates, and Andrew Y Ng. Autonomous helicopter aerobatics through apprenticeship learning. The International Journal of Robotics Research, 2010.
[2] Shan An, Ziyu Meng, Chao Tang, Yuning Zhou, Tengyu Liu, Fangqiang Ding, Shufang Zhang, Yao Mu, Ran Song, Wei Zhang, et al. Dexterous manipulation through imitation learning: A survey, arXiv preprint arXiv:2504.03515, 2025.
[3] Shuanghai Bai, Wanqi Zhou, Pengxiang Ding, Wei Zhao, Donglin Wang, and Badong Chen. Rethinking latent representations in behavior cloning: An information bottleneck approach for robot manipulation. arXiv preprint arXiv:2502.0285.. 2025.
[4] Adam Block, Daniel Pfrommer, and Max Simchowitz. On the imitation of non-man non-markovian demonstrations: From low-level stability to high-level planning. In ICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems, 2023.
[5] Xiaoyu Chen, Jiachen Hu, Chi Jin, Lihong Li, and Liwei Wang. Understanding domain randomization for sim-to-real transfer. In International Conference on Learning Representations, 2022.
[6] Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, page 027836492411273668, 2023.
[7] Kyunghyun Cho, Bart Van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv: 1259, 2014.
[8] Karl Cobbe, Chris Hesse, Jacob Hilton, and John Schulman. Leveraging procedural generation to benchmark reinforcement learning. In International conference on machine learning, pages 2048-2056. PMLR, 2020.
[9] Karl W Cobbe, Jacob Hilton, Oleg Klimov, and John Schulman. Phasic policy gradient. In International Conference Machine Learning, pages 2020-2027. PMLR, 2021.
[10] Ji Deng, Wei Dong. Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255. Iece, 2009.
[11] Yan Duan, Marcin Andrychowicz, Bradly Stadie, Jonathan Ho, Jonas Schneider, llya Sutskever, Picter Abbeel, and Wojciech Zaremba. One-shot imitation learning. Advances in neural information processing systems, 2017.
[12] Chelsea Finn, Tianhe Yu, Tianhao Zhang, Pieter Abbeel, and Sergey Levine. One-shot visual imitation learning via meta-learning. In Conference on robot learning, pages 357-368. PMLR, 2017.
[13] Dylan J Foster, Adam Block, and Dipendra Misra. Is behavior cloning all you need? under- standing horizon imitation learning. Advances in Neural Information Processing Systems, 2024.
[14] Zipeng Fu, Tony Zhao, and Chelsea Finn. Mobile aloha: Learning bimanual mobile manipu- lation with low-cost whole-body teleoperation. arXiv preprint 470.022401.02117, 2024.
11

---

[15] Caelan Garrett, Ajay Mandlekar, Bowen Wen, and Dieter Fox. Skillmimicgen: Auto- mated demonstration generation for efficient skill learning and deployment. arXiv preprint arXiv:2410. 18907, 2024.
[16] Mohammad Ghavamzadeh, Shie Mannor, Joelle Pineau, Aviv Tamar, al. Bayesian reinforce- ment learning: A survey. Foundations and Trendy@ in Machine Learning, 8(5-6):3594483, 2015.
[17] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pultern recognition, pages 770-778, 2016.
[18] Ryan Hoque, Ajay Mandlekar, Caelan Garrett, Ken Goldberg, and Dieter Fox. Intervengen: Interventional data generation for robust and data-efficient robot imitation learning. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2840-2846. IEEE, 2024.
[19] Ahmed Hussein, Mohamed Medhat Gaber, Eyad Elyan, and Chrisina Jayne. Imitation learning: A survey of learning methods. ACM Computing Surveys, 50(2):1-35, 2017.
[20] Kenji Kawaguchi, Zhun Deng, Xu Ji, and Jiaoyang Huang. How does information bottleneck help deep learning? In International Conference on Machine Learning, 2023.
[21] Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2446.009246, 2024.
[22] Robert Kirk, Amy Zhang, Edward Grefenstette, and Tim Rocktäschel. A survey of generalisation in deep reinforcement learning. arXiv preprint arXiv:2111.0979) 1:16, 2021.
[23] Fanqi Lin, Yingdong Hu, Pingyue Sheng, Chuan Wen, Jiacheng You, and Yang Gao. Data scaling laws in imitation learning for robotic manipulation. arXiv preprint 4rX.v:2410.18647, 2024.
[24] Jianlan Luo. Zheyuan Hu, Charles Xu, You Liang Tan, Jacob Berg, Archit Sharma, Stefan Schaal, Chelsea Finn, Abhishek Gupta, and Sergey Levine. Serl: A software suite for sample- efficient robotic reinforcement learning. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 16961-16969. IEEE, 2024.
[25] Jianlan Luo, Charles Xu, Fangchen Liu, Liam Tan, Zipeng Lin, Jeffrey Wu, Pieter Abbeel, and Sergey Levine. FMB: a functional manipulation benchmark for generalizable robotic learning. The International Journal of Robotics Research, page 027836492412760177 2023.
[26] Ajay Mandlekar, Soroush Nasiriany, Bowen Wen, Iretiayo Akinola, Yashraj Narang. Linxi Fan, Yuke Zhu, and Dieter Fox. Mimicgen: A data generation system for scalable robot learning using human demonstrations. arXiv preprint urXiv:23/0./75000 2023.
[27] Ishita Mediratta, Qingfei You, Minqi Jiang, and Roberta Raileanu. The generalization gap in offline reinforcement learning. arXiv preprint arXiv:23/2.05742, 2023.
[28] Mirco Mutti, Riccardo De Santi, and Marcello Restelli. The importance of non morkiaaaoyyy in maximum state entropy exploration. In International Conference on Machine Learning, pages 16223-16239. PMLR, 2022.
[29] Vudtiwat Ngampruetikorn and David J Schwab. Information bottleneck theory of high- dimensional regression: relevancy, efficiency and optimality. Advances in Neural Information Processing Systems, 2022.
[30] Abby O'Neill, Abdul Rehman, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, et al. Open x- embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 6802-6903. IEEE, 2024.
12

---

[31] Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel Van de Panne. Deepmimic: Example- guided deep reinforcement learning of physics- based character skills. ACM Transactions On Graphics (TOG), 37(4))1-14, 2018.
[32] Dean A Pomerleau. Alvinn: An autonomous land vehicle in a neural network. Advances in neural information processing systems, 1, 1988.
[33] Martin L. Puterman. Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons, 2014.
[34] Roberta Raileanu and Rob Fergus. Decoupling value and policy for generalization in reinforce- ment learning. In International Conference on Machine Learning, pages 8787-8798. PMLR, 2021.
[35] Roberta Raileanu, Maxwell Goldstein, Denis Yarats, Ilya Kostrikov, and Rob Fergus. Automatic data augmentation for generalization reinforcement learning. Advances in Neural Information Processing Systems, 34:5402-5415, 2021.
[36] Nived Rajaraman, Yanjun Han, Lin Yang, Jingbo Liu, Jiantao Jiao, and Kannan Ramchandran. On the value of interaction and function approximation imitation learning. Advances in Neural Information Processing Systems, 2021.
[37] Nived Rajaraman, Yanjun Han, Lin F Yang, Kannan Ramchandran, and Jiantao Jiao. Provably breaking the quadratic error compounding barrier in imitation learning, optimally. arXiv preprint SEii::2002.12448, 2021.
[38] Nived Rajaraman, Lin Yang, Jiantao Jiao, and Kannan Ramchandran. Toward the fundamental limits of imitation learning. Advances Neural Information Processing Systems, 2020.
[39] Nikhila Ravi, Valentin Gabeur, Yuan- Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything images and videos. arXiv preprint arXiv:2408.000 00008000144 14, 2024.
[40] Allen Ren, Sushant Veer, and Anirudha Majumdar. Generalization guarantees for imitation learning. In Conference Robot Learning, 2021.
[41] Stéphane Ross and Drew Bagnell. Efficient reductions for imitation learning. In International Conference on Artificial Intelligence and Statistics, 2010.
[42] Stephane Ross and J Andrew Bagnell. Reinforcement and imitation learning via interactive no-regret learning. arXiv preprint arXiv: 59799, 2014.
[43] Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth interna- tional conference on artificial intelligence and slatistics, pages 627-635. JMLR Workshop and Conference Proceedings, 2011.
[44] Ohad Shamir, Sivan Sabato, and Naftali Tishby. Learning and generalization with the informa- tion bottleneck. Theoretical Computer Science, 41 411029-30)226662211,22 2010.
[45] Ravid Shwartz-Ziv, Amichai Painsky, and Naftali Tishby. Representation compression and generalization in deep neural networks. OpenReview, 2018
[46] Max Simchowitz, Daniel Pfrommer, and Ali Jadbabaie. The pitfalls of imitation learning when actions are continuous. arXiv preprint trXiiv2503.09722 2025.
[47] Richard S Sutton, Andrew G Barto, et al. Reinforcement learning: An introduction. MIT press Cambridge, 1998.
[48] Gokul Swamy, Sanjiban Choudhury, Bagnell, and Steven Z. Wu. Sequence model imitation learning with unobserved contexts. Advances in Neural Information Processing Systems, 2022.
[49] Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy, arXiv preprint 2405 2024.
13

---

[50] Daniil Tiapkin, Denis Belomestny, Daniele Calandriello, Eric Moulines, Alexey Naumov, Pierre Perrault, Michal Valko, and Pierre Menard. aamoonstratio-regglaaizzz rl. In International Conference on Learning Representations, 2024.
[51] Naftali Tishby, Fernando C Pereira, and William Bialek. The information bottleneck method. arXiv preprint physics/00040577 1999.
[52] Marcel Torne, Anthony Simeonov, Zechu Li, April Chan, Tao Chen, Abhishek Gupta, and Pulkit Agrawal. Reconciling reality through simulation: A real-to-sim-tto-ral approach for robust manipulation. arXiv preprint arXiv: urXiv:2403.03949 2024.
[53] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
[54) Andrew Wagenmaker, Kevin Huang. Liyiming Ke, Kevin Jamieson, and Abhishek Gupta. Overcoming the sim-to-real gap: Leveraging simulation to learn to explore for real-world Advances in Neural Information Processing Systems, 37:78715-78765. 2024.
[55] Qi Wang, Yue Ma, Kun Zhao, and Yingjie Tian. A comprehensive survey of loss functions in machine learning. Annals of Data Science, 9(2):187-21,, 2022.
[56] Tian Xu, Ziniu Li, and Yang Yu. Error bounds of imitating policies and environments. Advances in Neural Information Processing Systems, 2020.
[57] Zhengrong Xue, Shuying Deng, Zhenyang Chen, Yixuan Wang, Zhecheng Yuan, and Huazhe Xu. Demogen: Synthetic demonstration generation for data-efficient visuomotor policy learning. arXiv preprint 169322 2025.
[58] Yanjie Ze, Gu Zhang, Kangning Zhang, Chenyuan Hu, Muhan Wang, and Huazhe Xu. 3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations. arXiv preprint arXiv:2403.03994, 2024.
[59] Tony Z Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware. In Robots:Science and Systems (RSS), 2023.
[60] Boyuan Zheng, Sunny Verma, Jianlong Zhou, Ivor W Tsang, and Fang Chen. Imitation learning: Progress, taxonomies and challenges. IEEE Transactions on Neural Networks and Learning Systems, 35(5):6322-6337, 2022.
[61] Ev Zisselman, Itai Lavie, Daniel Soudry, and Aviv Tamar. Explore generalize zero-shot Advances in Neural Information Processing Systems, 2023.
14

---

A Proofs
Here provide the derivations for the generalization bound in Section 3 non-trivial combination of previous results in provable imitation learning. mostly from [13. and the generalization guarantees of information bottleneck [20]. These results are reported in Lemma A.3
The proof requires
A.4 Before going through the proofs, we state the main theorem again for convenience.
Theorem A.1 (Theorem 3.1 For confidence 5 E (0, 1), it holds with probability least 28
E TwPb
(J J(T)
Ir:2 AA log(1Al/6)
8 8log mm
S RH
Egen(TT)) + Eope ( CCEE
V
where
R is an upper bound to the cumulative reward of any policy in any task 0 E θ (Section
Egen(x' (Fe is the gemeralization error of the expert's 's policy (Asm. 2
Eapt(it (T) is bound to the optimization error of solving ① (Asm.
. CCFF, 7) is a constant that depends on the training data F, the expert's policy B the cloned policy n. and other absolute constants as detailed Lemma A.4
Proof.
We derive the result as follows
E [J(TT) - J(+)
T-Po
< RH E T~Po
E T2
(TT
+ {()))()))
(4)
RHE gen (T)
(5)
RH Exen(=
E T~Po XAMPp
E
[( # A)]
(6)
1r. |A| log((AA/8)
8 ll0g(1)mm/0))
S RH (T) Eopt (
m71
where and 5 straightforward from the definitions of the performance ((r) and the general-
ization error Egen(T) (see Section and (3) respectively), (6) follows from Assumption 2 and holds with probability least 26 through Lemma A.2
We provide below the lemmas we need to prove the result above. Lemma A.2. For confidence å (0, 1), it holds with probability least 1 - 25
IT
8 (l0g(()(mm/)) Eopt(A)
E E (1 T-Po XANBEE
A) V
m
where IT. is the mutual information between the task T and the internal representation of the demonstrator Z.
Proof. We derive the result as follows
E TsPo
E XA
[1( + A)]
<
E
E
1 ( # A)
Σ 1(7(7))
a + Eapt (7)
(8)
TrPa x B
mnH
i=1 j=1 h=0
m
logg [l + Eopt (7)
<
E
1
# A)
1 A)]
T~Pa XA=PE"
m
XA~PE
n
(9)
15

---

(T:z)|/|log((||/// 8 log( [I mm/6) + Eopt(T)
(10)
n
where ( is trivial consequence of Assumption on the optimization error for solving I holds with probability at least 1 through Lemma A.3 and union bound on the m training tasks, and (TO) holds with probability - 26 from LemmalA. by omitting constant and lower order terms and applying a union bound.
Lemma A.3 (Sample complexity of behavioral cloning [3].
For a confidence 8 (0, 1), an MDP 9, and a deterministic expert's 's policy π E holds with probability at least -
8 8log( (0)))
E XA~28
[F((X) + A)) S
71
Proof. This result can be obtained through combination of results in [13. First, for policy î obtained by minimizing the negative log likelihood of the data, as in (T. from Proposition 2.1 [13 have with probability least 5 that
Dd (P P ) (
where (P,
is the squared Hellinger distance between the probability measures P and Q Then, through Lemma F.3 [13 we have
(VdP v [
E XA-PE
+ A)] =
which concludes the proof.
Lemma A.4 (Information bottleneck generalization gap (20)). For dataset E = (0; ~ Polm of m tasks and a single-point convex loss ((*(X), A), let us define the generalization gap across the prior Po
T(E)
E
E
(X((X,, A)]
E
[((()),
T~Po XA
m
XA-PE
For a confidence E (0, ((E) is upper bounded with probubility at least 1 by
BV IT,2AA log 2 + log 2 + H( A) log(21A1/8)
[(T) 27| log(llog2211) (2g(2
79(#)
m
3/4
1/2
where an, 3, constant values, (()) max4e[m) B PAA ((()(X, A)] is the maximum
training loss, g( = supx (X)(), A) is the maximum generalization loss, and H (Z(T, A) is the entropy of the demonstrator internal representation given TAA
Proof. This result is based on Theorem in 200 in which the notation adapted to our setting of
interest. All of the derivations can be found in 20 A coarser version of the bound is given
T(F) s (8 (1 H( + +((+(()))))))))))))) ( 2 (9(3))
IT:2IA AA] log(2 (|/8))
m
where the first factor can be incorporated into a constant B 7). We note that the term (Z T,, A) assumption in our setting. Further, the maximum training eгтоr f(ā) is close to zero and upper bounded by the optimization error Eopt(f) The value g upper bounded by for the indicator ITIZIA- With these considerations,
whenever the demonstrator internal representation deterministic, which is fair
loss E(# ((+(X), A) = (x) * A). Finally, we note that Ir.2 by omitting all of the constants, we have
Tri
T(E) 2
as it is reported elsewhere in the paper.
16

---

B Peg insertion extended results
This section describes detail the setting, hyperparameters, and constants used for the peg insertion task, as well as provides extended evaluation results.
B.1 Experimental setup
Figure 7 shows close-up view of all peg shapes (10 shapes in total) and their corresponding boards, where the training shapes are in the top row and the test shapes are in the bottom row. Each peg insertion attempt starts from 10cm above the hole axis) and a random reset position within 0.5cm box in the XY plane, centered above the hole position. In this experiment, initial rotations about the and Y axes are fixed (0 degrees), while the angle starts from a random rotation ranging from -60 to 60 degrees. Two Realsense cameras are mounted the robot's ' wrist. For the blindfolded expert experiment, we mask out the hole hide its orientation from the experts, such that they cannot infer the orientation of the peg and must explore the domain in order to insert the peg.
Figure 7: Close-up view of all peg insertion tasks. Top row: Training pegs.
Bottom row: Test pegs.
B.2 Hyperparameters and constants
architecture is used for learning both mh and RFF-BC BC- Specifically, we use NesNet-10 10 [17 encoder pretrained on the ImageNet dataset [10], and a GRU of 1024, which we found to produce the best performance for both policies TBC and BBF-BC independently. Throughout our experiments, we train our networks using the Adam optimizer. The hyperparameters of the networks are the learning rate, learning rate decay, and the batch size. To ensure that the best performance of each approach is achieved, we perform a separate hyperparameter search for each policy, trained on each subset of shapes. The best hyperparameters are listed in Table 3] The network outputs a 6-dimensional vector for the mean and a 6-dimensional vector for the diagonal covariance matrix of Gaussian policy (for 6-DoF action space). We train our networks using the log-likelihood loss. In all our evaluations, the action is chosen as the maximum likelihood of the distribution.
As described in section 4.2 the
Table 3: List of hyperparameters used in the peg insertion experiment. The learning rate (Ir) schedule indicates the iteration number for multiplying the Ir by 0.5.
Hyperparameter batch size
BB
BBF BC
1024
1024
hidden size
1024
1024
initial Ir Ir schedule
0.0003
0.0003
Ir x0.5 at [10, 100, 150, 200 K
Ir x0.5 at (50, 100, 150, 200 K
B.3 Measuring exploratory behavior
We compute two additional measures for the exploratory behavior of the different experts: the map coverage score (Table 4 and the entropy of state visitation (Table
Map coverage score is the ratio C Nu/Niotal given by the number of visited states N, divided by all accessible states Nrotll, averaged over all episodes. For the peg insertion experiment, we consider
17

---

3 Training Shapes
Trsining Shapes
Train
Rec
Ren-SC
Train
Test
Success rate
Success
Figure 8: Success rate of robotic peg insertion for 10 peg shapes (horizontal axis). We train on subset of shapes & f3, 4). with the remaining shapes withheld as test-set. Both for & = 3 (left) and (right), cloning blindfolded experts generalizes better than cloning standard experts.
the rotation of the robotic arm around the Z-axis the crucial component of the state space for obtaining the correct articulation for insertion. We compute the ratio of the rotation performed (in radians) divided by 2n in each trajectory, averaged over all trajectories.
The entropy of state visitation is defined by H E. p(s) log p(s). We calculate p(s) using histogram (with 20 bins) of rotation angles along the trajectory, averaged over all trajectories.
The results confirm that blindfolded experts explore larger portion of the state space compensate for the redacted information in the observations.
Table 4: Map coverage score of the trajectories demonstrated by fully-informed experts (Experts) and blindfolded experts (BF-Experts).
Mode
hexagon
star
square
plus
triangle
Average
Experts
0.078
0.113
0.150
0.153
0.191
0.137
BF-Expsrts
0.114
0.259
0.267
0.270
0.327
0.247
Table Entropy of the state visitation the trajectories demonstrated by fully-informed experts (Experts) and blindfolded experts (BF-Experts).
Mode hexagon
star
square
plus
triangle
Average
Experts
2.876
3.121
3.266
3.228
3,418
3.182
BF-Expsrts
3.100
3,416
3.546
3.577
3.631
3.454
B.4 Results for different combinations of training shapes
Figure shows the success rate for i training shapes (and the rest serve as set, of total of 10 peg shapes). The results on the varying amounts of training shapes, further support that cloning blindfolded experts generalizes better than the standard BC approach.
C Procgen maze and heist extended results
This section describes in detail the hyperparameters and constants used for the Procgen maze task.
C.1 Hyperparameters and constants
For fair comparison, we conduct separate hyperparameter search for both TBC and TBF-BC- We perform hyperparameter search for the batch size b {128, 256, 512, 1024), for the learning rate (1e-3, le-4, le-5, -3 -4 5e 55) and for hidden size E [128, 256, 512, 1024). We also evaluate the performance with and without using learning decay schedule. Our networks are trained using the Adam optimizer. The best hyperparameters are chosen based on the lowest training loss and the highest training success rate. We evaluate performance
average of 10 different random
18

---

Procgen Maze Loss Results
Procgen Heist Loss Results
1.5
Score
Score
1.0
0.5
Loss-Itac
Loss-flac Loss-rto- dc
Loss-rler 8C
50
100
150
100
150
Epochs
Epochs
Figure 9: Loss as a function of training epochs for the Procgen maze (left) and heist (right). Mean
(color line) and std (shaded region) computed across 10 seeds.
training seeds. Figure that in most cases, the best hyperparameters for Nn and TBF-BC BC turned out be fairly similar.
D and Table show the loss function and the chosen hyperparameters. Note
Table 6: List of hyperparameters used in the Procgen Maze experiment.
Hyperparameter
TBC
TBF- BC
batch size
256
256
hidden size
1024
1024
learning rate
10-5
10
learning rate schedule?
No
No
Table 7: List of hyperparameters used in the Procgen Heist experiment.
Hyperparameter
TBC
TBF BC
batch size
128
128
hidden size
512
1024
learning rate
10-b
10
learning rate schedule?
No
No
C.2 Number of steps number of trajectories
As described Table the blindfolded expert takes more steps on average to complete each trajectory. When comparing the different approaches, match the number of trajectories, which leads to greater total number environment steps for the blindfolded expert. Table showw total number of steps available for training the different BC policies, alongside their performance. We also compare our results to a standard BC approach with twice the number trajectories (from the same 100 seeds) to match the number of environment steps produced by the blindfolded expert. In addition, compare our results to the results reported by [27], who train a BC policy on the Procgen maze with dataset of M environment steps taken from trained PPO expert, 200 training seeds
We can see in Table that TBF-BC achieves better performance than all other contending policies. When compared to the results reported in [27], we can see that our results are better despite signifi- cantly less training data (an order of magnitude fewer trajectories) and half the number of training seeds.
For LM expert dataset, report the results from 2 who evaluated over seeds.
19

---

Table 8: Performance comparison on the Procgen maze experiment. Our results are reported at epoch 40 (early stopping) when training performance plateaus. The mean and std are computed over 10 seeds. Top performer in bold.
Parameter
IM Expert Dataset in [27]
BC
TBC-ezt
TBF-BC BC
of trajectories
15385
2000
4608
2000
# of total env steps
1000000
57166
115290
103238
of seeds
200
100
100
100
Test performance
4.46 0.16
3.35 1:55:00 0.29
3.65 + 0.3
6.37 0.47
D Data collection
Peg insertion. For both BC and BBF-BC- we use 400 trajectories for each of the training shapes. We, the authors, collected the data by operating the robot manually using Spacemouse control. Recall that the blindfolded expert observes a masked-out view of the board (through the robot wrist cameras) such that the orientation of the peg is not directly visible and must be inferred through exploration. However, recorded observations in favor of cloning the blindfolded policy π TBF- unmasked, i.e., only the human expert is blindfolded. In addition, rescale the images from 480 x 480 to 128 x 128 for both BBC and #BF-BE BC to facilitate computations.
Procgen maze and heist.
To train our experts (BC) and blindfolded experts (BF-BC) policies, we collected 4000 human demonstrations on 200 levels. We conducted crowd-sourced data collection for the maze and heist videogames by recruiting 20 volunteers who played the games. Each expert played 200 levels of maze and all 200 levels of heist twice, once with the mask and once without the mask. Their game trajectories were recorded to serve toward the imitation leaatiion---aninn of the experts' policy ивс and blindfolded experts' policy TBF-BC. The participants moved the mouse using the keyboard's arrow keys and relied on the Procgen "interactive" GUI for maze and heist observations in full resolution (512 512). For the blindfolded experts' data, their observations agent's 's immediate surroundings (a diameter of of the width for maze and for heist) with the rest of the observation masked out. Note that the state observations that provided to the cloning networks are lower resolution of 64 x 64 of the unmasked observations.
modified reveal only the
All participants were compensated with vouchers for their efforts. The experiment's GUI environment, alongside the training code and the recorded data, are available at: https://gg http//ggtttu.....e/ee
thub. www./oooeeeeeeeeeeccecceo bindfff///idieeedeeeeeeie inddffoon olded- oddee-epertttttt
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract claims that the paper shows that cloning blindfolded experts leads to better generalization. We support this claim both theoretically (see Theorem 3.1 in Section 3 and empirically (see Section 4.1 and Section 4.2 where we provide the Proegen and peg insertion experiments advertised in the abstract).
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: While the paper does include a "Limitations" section, limitations are discussed in the paper. On the theoretical side, discussion on the limitations given by the considered assumptions, and how they may be overcome, is provided at the end of Section Bl For the general approach, a limitation on the design of the information bottleneck for the blindfolded expert reported in the Conclusion section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "I "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
21

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: The paper has thcoretical section (Section 3) with main theoretical result (Theorem B.T). The assumptions to derive the result are explicitly stated in the same section,
above the theorem (Asm. 1234. The proof of the theorem, together with the necessary lemmas, is provided in the Appendix A
Guidelines:
The answer NA means that the paper does not include theoretical results. All theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in the appendix or supplemental material. Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes].
Justification: The experimental setup, network architecture and hyperparameters, data collection and training schemes are all detailed in Sections B for full reproducibility. In addition, the abstract provides a link to the project home page that contains the full source code and documentation.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided instructions for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
22

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes].
The abstract section provides a link the project home page that will contain the full source code, environment setup and documentation, tutorial, components CAD and network weights (complete soon).
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//iips.c/ppil wttpcccppppppppeeeeeeeeeeeeeeeee wwweeee/giisss//eee httpa////ggieee/edeeuubbbssssss sonoooooic for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification:
The experiments, their setup, parameters and process are detailed at length in the Experiments section (4). Also, the abstract provides link to the project's page with all the necessary code, documentation, components CAD and tutorials.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes].
23

---

Justification:
The paper contains error bars in table and figure in the form of mean and standard deviation and the method for their computation is detailed the corresponding captions.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes].
Justification: For both experiments, the compute-resources are detailed in sections B and
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: http://eeuiip.p.eeeee wttpppp...pppuuiee cttppuulicettiicccuiiileellaeeea
Answer: [Yes]
Justification:
The paper fully respect the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
24

---

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
The paper can categorized as foundational research. At the current stage of development, it is not tied to any product that may be deployed in the real world and have an impact on society.
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not pose such risks such as potential for dual-use or risk to society.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes].
25

---

Justification: All relevant papers and works mentioned are properly cited. and supportive code implementations are recognized and accredited (Proegen SERL 25
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. www.coo///aaaaa com/aataeet has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes].
Justification; Our dataset of human expert trajectories for the Procgen maze and peg- veg-insertion tasks will be made available online on the project page https://sites google e. com/
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes].
Justification: In section D we detail how data collection was conducted, including the instructions for the participants and the compensation provided.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
26

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification: There potential risks incurred by study participants, and we verified our study conforms to the guidelines in our institution.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification:
This paper does not involve LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non non-standard components.
Please refer to our LLM policy /tap:::///eurraii cc. cc//oonferences//225/11 for what should or should not be described.
27


Reviews:
[{"review_id": "cK2dL4NbCb", "reviewer": "Reviewer_bJoN", "summary": "In this paper, the authors propose a method called blindfolded expert, in which experts are intentionally provided with only partial information. As a result, the expert cannot fully grasp the given task, prompting more exploratory behavior. The central hypothesis is that this exploratory behavior enables better generalization to previously unseen tasks.\nBuilding on this hypothesis, the authors conduct a theoretical analysis and derive a relationship showing that the generalization error depends on the amount of task-specific information available to the expert. The empirical results show that this hypothesis through experiments on a real-world robotic peg insertion task and a video game environment (Procgen Maze).", "questions": "The strength of the information bottleneck (i.e., the degree of blindfolding) must be carefully tuned to encourage exploration while still allowing task success. This appears to require domain-specific tuning. Do the authors have any principled method or insight for finding this optimal trade-off?", "limitations": "The theoretical bound is derived under a classification setting and may not hold in continuous action spaces. Also, strong bottleneck could degrade expert performance. (the authors do acknowledge both of these limitations in the manuscript).", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n- Behavior cloning is an active area of research in robotics and decision making. While much work has focused on scalability with respect to the number of tasks or demonstrations, this paper presents a novel direction by investigating how performance scales with the mutual information between the expert’s internal representation and the task.\n\n- In the case of the blindfolded expert, the trajectories tend to have a higher average number of steps. The authors acknowledge that when comparing with vanilla BC, the total number of transitions may differ even with the same number of episodes. To address this, they provide an additional baseline, BC-ext, which corrects for this discrepancy.\n\n- The paper includes both real-world experiments and theoretical analysis.\n\nWeaknesses\n\n- The experimental environments are relatively simple. The impact of the proposed blindfolded expert approach would be significantly greater if it could be shown to work on more complex manipulation tasks, where imitation learning is more commonly applied.\n\n- It would be beneficial to provide more qualitative results beyond just the number of steps. For instance, does the blindfolded expert actually explore a meaningfully larger portion of the state space?\n\n- For the theoretical analysis, the authors reformulate the problem as a classification task. As a result, it is unclear how well the derived bound extends to continuous action spaces, which are typical in robotic manipulation.\n\n- While blindfolding promotes generalization by encouraging exploratory behavior, it can also result in sub-optimal expert demonstrations. This increases the expert’s sub-optimality term in the generalization bound, which could potentially weaken the bound overall.", "was_revised": true, "final_justification": "The authors provides quantitative results regarding the exploration of the blindfolded expert. I believe this paper is solid and novel overall. Also, I encourage the authors to include a more explicit discussion of the limitations, especially the restriction to discrete action spaces in theoretical analysis. Also, it would be interesting how much this technique gives additional gain in different levels of difficulty.", "rebuttal": "We thank the reviewer for the thoughtful comments and for recognizing the key strengths of our work in its novelty, theoretical analysis, and experiments. Below we provide detailed answers to the questions and comments.\n\n**\"Experimental environments are relatively simple\".**\n\nOur work tackles generalization across tasks, and therefore, *we focus on domains in which generalization is the main challenge*, rather than other factors that may confound the analysis of generalization (such as control or the need to plan for long horizons). In our experiments, most of the complexity stems from the limited variability of training tasks, which produce a cloned policy that memorizes the data. The resulting model lacks the ability to generalize to unseen test tasks - which we show can be resolved with the introduction of blindfolds (Figures 3 and 5).\n\nNevertheless, note that the peg insertion task is not as easy as it seems: It is a contact-rich manipulation task -- not trivial, and much harder than common pick-and-place manipulation tasks.\n\nRegarding the maze experiment, **we provide an additional experiment of Procgen Heist game**, in which the levels are more challenging compared to a standard maze. At each level, the agent needs to collect a gem hidden behind a network of color-coded locks accessible through keys (of corresponding colors) that are scattered throughout the level (See [1] for more details on the environment setting). This requires multi-stage planning in the form of lock-waypoints and overcoming variation in sub-tasks (key color) ordering. We apply the blindfold similarly to the Procgen maze and follow the same experimental setting as in the paper. In Table 1 below, we compare the results of the agent trained with the fully informed expert's data ($\\pi_{BC}$) to the results of the agent trained on the blindfolded experts' data ($\\pi_{BF-BC}$). While the training performance is above 90% for both agents, $\\pi_{BF-BC}$ significantly outperforms $\\pi_{BC}$ on unseen test levels (43% vs. 7%), which indicates better generalization performance (results obtained over $4K$ trajectories from $200$ different seeds).\n\n*Table 1: Performance comparison on the Procgen Heist experiment. The mean and std are computed over 10 seeds. Top performer in bold.*\n| Method | Train | Test |\n|--------|-------|------|\n| $\\pi_{BC}$ | 9.41 ± 0.20 | 0.69 ± 0.44 |\n| $\\pi_{BF-BC}$ | **9.91 ± 0.04** | **4.27 ± 1.01** |\n\n**\"It would be beneficial to provide more qualitative results\".**\n\nThanks for the valuable suggestion. **We computed two additional metrics**: Map coverage score (Table 2) and the entropy of state visitation (Table 3).\n\n*Map coverage score* is the ratio $C=N_v/N_{total}$ given by the number of visited states $N_v$ divided by all accessible states $N_{total}$, averaged over all episodes. For the peg insertion experiment, we consider the rotation of the robotic arm around the Z-axis as the crucial component of the state space for obtaining the correct articulation for insertion. We compute the ratio of the rotation performed (in radians) divided by $2\\pi$ in each trajectory, averaged over all trajectories.\n\n*Entropy of the state visitation* is defined by $H= -\\sum_s p(s)\\log p(s)$. We calculate $p(s)$ using a histogram (with 20 bins) of rotation angles along the trajectory, averaged over all trajectories.\n\nThe results confirm that blindfolded experts explore a larger portion of the state space to compensate for the redacted information in the observations.\n\n*Table 2: Map coverage score of the trajectories demonstrated by fully-informed experts (expert) and blindfolded experts (BF-expert).*\n\n| Mode | hexagon | star | square | plus | triangle | Average |\n|------|---------|------|--------|------|----------|---------|\n| expert | 0.078 | 0.113 | 0.150 | 0.153 | 0.191 | 0.137 |\n| BF-expert | **0.114** | **0.259** | **0.267** | **0.270** | **0.327** | **0.247** |\n\n*Table 3: Entropy of the state visitation of the trajectories demonstrated by fully-informed experts (expert) and blindfolded experts (BF-expert).*\n\n| Mode | hexagon | star | square | plus | triangle | Average |\n|------|---------|------|--------|------|----------|---------|\n| Expert | 2.876 | 3.121 | 3.266 | 3.228 | 3.418 | 3.182 |\n| BF-Expert | **3.100** | **3.416** | **3.546** | **3.577** | **3.631** | **3.454** |\n\n**Extension of the theory to continuous action space.**\n\nRobotic manipulation tasks typically allow for modeling with discrete action spaces due to the inherent smoothness of the physical world. Notably, even advanced state-of-the-art models (e.g., OpenVLA [2]) are trained by mapping continuous actions into discrete tokens. This makes our theory applicable to the experimental setting in the paper and many other settings. We do, however, see the value of the reviewer's suggestion to extend the analysis to a continuous action space. That said, the theory of imitation learning with continuous actions is known to be highly non-trivial, and hardness-analysis has been derived recently [3]. We leave settling this open question to future work.\n\n**Q1. Optimal trade-off between generalization and sub-optimal behavior. Principled method or insight for finding this optimal trade-off?**\n\nAny BC experiment requires some engineering (of the data collection setup, the human demonstrator instruction, etc.), and we believe blindfolds will become another factor in this list.\n\nWe can provide some insights on how we designed the blindfold in practice. Take the peg insertion case, in which the blindfold is a mask on the image observation. We tweaked the mask to hide most information that still allows the human to solve the task. At that point, removing even more information would not have produced meaningful demonstrations.\n\nFurther studies on principled methods for the automatic design of the blindfold are a good direction for future work.\n\n\n**References**\n\n[1] Cobbe, Karl, et al. \"Leveraging procedural generation to benchmark reinforcement learning.\" International conference on machine learning. PMLR, 2020.\n\n[2] Moo Jin Kim et al. OpenVLA: An Open-Source Vision-Language-Action Model. 2024\n\n[3] Thomas T. Zhang et al. Imitation Learning in Continuous Action Spaces: Mitigating Compounding Error without Interaction. 2025"}, {"review_id": "5ACVdmms5Y", "reviewer": "Reviewer_LUkx", "summary": "This paper studies the effect of information bottlenecks on the expert’s observation in behavior cloning (BC), with a conjecture that such bottlenecks yield better generalization of BC policies across multiple tasks. The paper provides a theoretical result proving that an upper-bound on the generalization error scales with the sum of two terms: the cost of suboptimal experts and the square-root of mutual-information between the task information and the expert’s internal representation of the task. This theory suggests that lower mutual information for the expert leads to more generalizable multi-task BC policies. The paper empirically verifies this theory in a simulated Procgen domain and a real-world robotic manipulation domain.", "questions": "In the proof of Theorem 3.1, the authors note that equation (6) follows from Assumption 2. Some elaboration would be appreciated since the transition from (5) to (6) was not trivial to me.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n* Multi-task generalization of behavior cloning is an important open research question. The paper poses a very interesting conjecture that information bottlenecks for the expert demonstrator will improve the generalizability of BC agents. This conjecture is convincingly supported by both theoretical and empirical results.\n\n* Overall, the experimental setup is well-designed. The ablation of $\\pi_{\\text{BC-ext}}$ is a nice addition to the performance comparison, since it effectively eliminates the possibility that the blind-folded policy generalized simply due to increased amount of data.\n\n* Despite the inherent complexity of the theoretical development, the paper is well-written and is relatively straightforward to follow.\n\n**Weaknesses:**\n* The main theoretical results (Theorem 3.1) essentially illustrate a trade-off between the degree of suboptimality of the expert and the amount of mutual information. As the paper admits in the Conclusion section, finding the best trade-off between the two conflicting terms can be task-specific and challenging to achieve in practice.\n\n* Related to the point above, it is difficult to quantify the mutual information $I_{Z,T}$ since we don't know the expert's internal representation $Z$. Therefore, specific means to reduce this mutual information have to resort to some heuristic choices (such as masking parts of observations, as done in this paper), rather than some principled guidelines to design effective interventions based on quantified mutual information.\n\n* Although the empirical results are generally convincing, the variation of tasks in the empirical study is somewhat small. Specifically, each maze layout and background is considered a distinct task in the Procgen domain, and each peg shapes in the manipulation domain. In future work, it would be interesting to see more diversity across different axes of generalization, including spatial, object, and visual variations, as well as task-specification diversities for language-conditioned policies. There are some simulation environments specifically designed to test multi-task policies in the robotics domain (such as the Meta-World [1] and LIBERO [2] benchmarks, to name a few).\n\n[1] YU, Tianhe, et al. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In: Conference on robot learning. PMLR, 2020. p. 1094-1100.\n\n[2] LIU, Bo, et al. Libero: Benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems, 2023, 36: 44776-44791.", "was_revised": true, "final_justification": "The rebuttals by authors have addressed most of my major concerns and questions that I raised in my original review. While the authors admit how to reduce task information in language-conditioned settings is still an open problem, such an acknowledgement of limitation should be rewarded rather than punished since it improves clarity of the paper. The new Procgen Heist experiment demonstrates strong generalizability of blindfolded experts. Thus, I have raised the clarity and the significance scores. I believe that the paper is solid.", "rebuttal": "We appreciate the reviewer's constructive comments. We discuss the raised points below.\n\n**It is difficult to quantify the mutual information since we don't know the expert's internal representation Z.**\n\nThis is a very good point. We can still lower bound the information between *observations* and *tasks* using a variational approach -- a classifier that predicts task identity from the observations [1]. We believe that this bound will be lower when we add the mask, and may be used in the future to devise better masks. Of course, whatever information the human gleans from the observation is not as easy to quantify.\n\n**Task diversities and more variation of tasks.**\n\nAs we discuss in the paper, not all task variability can be improved by our method. Following Theorem 3.1, we need the blindfold to reduce task information, but still allow the human to solve the task. In robotics, a simple \"rule of thumb\" for such tasks is -- can you solve it in the dark? Many important tasks fall under this criterion, including grasping, assembly, and disassembly. The tasks in Meta-World and Libero do not -- the observation contains the task description itself -- what to do and where to move, and they are impossible to solve without a very detailed observation. How to reduce task information in such domains is still an open problem.\n\nIn the meantime, **we provide an additional experiment on Procgen Heist game** to have more variation of tasks in the paper. In Heist, the agent needs to collect a gem hidden behind a network of color-coded locks accessible through keys (of corresponding colors) that are scattered throughout the level (See [2] for more details on the environment setting). This requires multi-stage planning in the form of lock-waypoints and to overcome variations in sub-task ordering, making it significantly more complex than the regular maze. For training the BC policies, we collected human demonstrations as in the Maze experiment, applying a similar blindfold to the blindfolded experts. In Table 1 below, we compare the results of the agent trained with the fully informed expert's data ($\\pi_{BC}$) to the results of the agent trained on the blindfolded experts' data ($\\pi_{BF-BC}$). While the training performance is above 90% for both agents, $\\pi_{BF-BC}$ significantly outperforms $\\pi_{BC}$ on unseen test levels (43% vs. 7%), which indicates better generalization performance (results obtained over $4K$ trajectories from $200$ different seeds).\n\n*Table 1: Performance comparison on the Procgen Heist experiment. The mean and std are computed over 10 seeds. Top performer in bold.*\n| Method | Train | Test |\n|--------|-------|------|\n| $\\pi_{BC}$ | 9.41 ± 0.20 | 0.69 ± 0.44 |\n| $\\pi_{BF-BC}$ | **9.91 ± 0.04** | **4.27 ± 1.01** |\n\n\n**Q1. Elaboration on the transition from (5) to (6) in Theorem 3.1.**\n\nThe derivation in Eq. 6 is \n$$\\mathcal{E}\\_{gen} (\\hat{\\pi}) \\leq \\mathcal{E}\\_{gen} (\\pi^E)+ \\mathbb{E}\\_{T \\sim P\\_0} \\mathbb{E}\\_{XA \\sim \\mathbb{P}^{ \\pi^E}\\_{T}} [1 (\\hat\\pi (X) \\neq A)].$$\nThis is obtained by noting that the suboptimality of the cloned policy $\\hat \\pi$ cannot be larger than the suboptimality of the expert's policy $\\pi^E$ plus a term that accounts for every time $\\hat\\pi$ is different from $\\pi^E$. Then, Assumption 2 allows us to write the suboptimality of the expert's policy as $\\mathcal{E}_{gen} (\\pi^E)$.\n\n**References**\n\n[1] Chen et al., Infogan: Interpretable representation learning by information maximizing generative adversarial nets. NeurIPS 2016.\n\n[2] Cobbe, Karl, et al. \"Leveraging procedural generation to benchmark reinforcement learning.\" International conference on machine learning. PMLR, 2020."}, {"review_id": "vaexDBUFvN", "reviewer": "Reviewer_29eA", "summary": "This paper presents a simple yet effective paradigm for behavioral cloning (BC). During demonstration data collection, the approach restricts the information available to human experts (observation masking), forcing them to adopt more exploratory behaviors to complete tasks. Crucially, while the expert operates with limited information, the recorded observations remain complete and unmasked. When these demonstrations are used for behavioral cloning, the learned policy inherits the expert's exploratory behavior. This data collection methodology significantly improves the generalization performance of the resulting policies.", "questions": "- How might you automatically determine the optimal \"blindfolding\" strategy for a new domain without domain-specific engineering?\n- How does this approach compare to simply collecting more diverse demonstrations from standard experts? Is there a point where additional task diversity would make the blindfolding unnecessary?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- Simple but effective way for improving BC: The idea of manipulating the expert's information access rather than modifying the learning algorithm is effective and orthogonal to existing approaches for improving BC.\n- Solid theoretical analysis: The paper provides formal analysis showing why blindfolding experts leads to better generalization, connecting information theory to multi-task generalization.\n- Good results: The experiments on both simulated and real-world tasks show substantial improvements in generalization performance, especially when trained on fewer task variations.\n\nWeaknesses:\n- Task-specific blindfolding: As acknowledged by the authors, each task requires a custom blindfolding strategy, and finding the right balance of information restriction is non-trivial.\n- Limited task diversity: While the experiments cover both virtual and physical domains, they're limited to navigation and insertion tasks. It's unclear how the approach would generalize to more complex or diverse task families.\n- Increased demonstration effort: Blindfolded experts take longer to complete tasks (as shown in Table 1), which means more time is required for data collection, potentially offsetting some of the data efficiency gains.", "was_revised": true, "final_justification": "The authors' rebuttal has addressed several of my concerns; however, I will maintain my original score due to the limitations regarding task-specific blindfolding.", "rebuttal": "We thank the reviewer for the insightful comments and for considering our approach to be effective and well-founded. \nWe are reporting below detailed replies to the questions and concerns. \n\n**Task diversity.**\n\nWe include additional results on Procgen Heist to increase the task diversity in the paper. In Heist, the agent needs to collect a gem hidden behind a network of color-coded locks accessible through keys (of corresponding colors) that are scattered throughout the level (See [1] for more details on the environment setting). This task requires multi-stage planning in the form of lock-waypoints and to overcome variations in sub-task ordering, making it significantly more complex than the regular maze. For training the BC policies, we collected human demonstrations as in the Maze experiment, applying a similar blindfold to the blindfolded experts. In Table 1 below, we compare the results of the agent trained with the fully informed expert's data ($\\pi_{BC}$) to the results of the agent trained on the blindfolded experts' data ($\\pi_{BF-BC}$). While the training performance is above 90% for both agents, $\\pi_{BF-BC}$ significantly outperforms $\\pi_{BC}$ on unseen test levels (43% vs. 7%), which indicates better generalization performance (results obtained over $4K$ trajectories from $200$ different seeds).\n\n*Table 1: Performance comparison on the Procgen Heist experiment. The mean and std are computed over 10 seeds. Top performer in bold.*\n| Method | Train | Test |\n|--------|-------|------|\n| $\\pi_{BC}$ | 9.41 ± 0.20 | 0.69 ± 0.44 |\n| $\\pi_{BF-BC}$ | **9.91 ± 0.04** | **4.27 ± 1.01** |\n\n**How the approach generalizes to more complex or diverse task families.**\n\nFor robotic applications, a simple \"rule of thumb\" we found is -- can you solve it in the dark? Many important tasks fall under this criterion, including grasping, assembly, and disassembly. This is not exhaustive, and additional blindfold techniques may be found in future studies to extend the applicability of the idea.\n\nOn this line, we mention an interesting (yet speculative) application to LLM reasoning. When answering a complex question (e.g., \"what is the tallest tower in Europe\"), we must balance between recalling from memory, and performing some reasoning computation (\"what are the countries in Europe, what is the tallest tower in each European country, etc.\"). For supervised fine tuning, if we manage to induce the humans to provide more reasoning traces, they may lead to better generalization. We may \"blindfold\" the humans by asking them not to rely on memory, for instance, but more on tool use and reasoning.\n\n**Increased demonstration effort offsetting data efficiency gains**\n\nThis is a valid point, for which we dedicated an ablation study in the manuscript (Figure 3). Since the blindfolded experts' trajectories are longer, we collected more trajectories with fully-informed experts ($\\pi_{BC-ext}$) to match the total number of samples of the blindfolded expert. The results are mostly the same: This means that the blindfolded approach generalizes better **with the same number of samples** and therefore the same time for collecting the data.\n\n**Q1. How might you automatically determine the optimal \"blindfolding\" strategy for a new domain without domain-specific engineering?**\n\nThis is an interesting direction that would require substantial additional study, which we leave as future work.\n\nWe can provide some insights on how we designed the blindfold in practice. Take the peg insertion case, in which the blindfold is a mask on the image observation. We tweaked the mask to hide most information that still allows the human to solve the task. At that point, removing even more information would not have produced meaningful demonstrations. This heuristic can be replicated by practitioners in other domains.\n\nThe latter requires some engineering, but note that any BC experiment requires engineering anyway (of the data collection setup, the human demonstrator instruction, etc.), and we believe blindfolds will become another factor in this list.\n\n**Q2. How does this approach compare to simply collecting more diverse demonstrations from standard experts? Is there a point where additional task diversity would make the blindfolding unnecessary?**\n\nIf we understand the reviewer's intention correctly, \"more diverse demonstrations\" refers to \"demonstrations of more tasks\". The intuition is correct: **When demonstrating more and more tasks, the gap between blindfolded and standard experts shrinks**. This is supported both by the theory and the experiments.\n\nThe theory shows that the generalization error scales with $O (\\sqrt{I_{Z;T} / m})$, where $m$ is the number of demonstrated tasks. When $m$ is large enough, the effect of a smaller $I_{Z;T}$ is reduced.\n\nThe experiment on peg insertion shows that the generalization error of the policy cloned from regular demonstrations shrinks slightly when increasing the training tasks from 2 (Figure 5 left) to 5 (Figure 5 right). The gap with the blindfolded strategy remains significant, but we expect it to reduce with even more demonstrated tasks.\n\n**References**\n\n[1] Cobbe, Karl, et al. \"Leveraging procedural generation to benchmark reinforcement learning.\" International conference on machine learning. PMLR, 2020."}, {"review_id": "0goEsLWODV", "reviewer": "Reviewer_qfwW", "summary": "This paper introduces a novel and insightful perspective on imitation learning: the quality and nature of expert demonstrations are as critical for generalization as their quantity. The authors propose the concept of a \"blindfolded\" expert—an expert who is intentionally deprived of some task-specific information. This forces the expert to adopt a more general, exploratory behavior to solve the task. The core contribution is showing, both theoretically and empirically, that cloning this exploratory behavior leads to policies that generalize significantly better to unseen tasks. This central idea is supported by a formal generalization bound, which links the error to the mutual information between the expert's strategy and the task identity, and is validated with compelling experiments in a Procgen maze environment and on a real-world robotic peg insertion task.", "questions": "- Could the authors comment on the sensitivity of the method to the degree of \"blindfolding\"? The experiments show a case that works well, but what happens at the extremes? For instance, if the blindfold is so severe that the expert's success rate drops significantly, how does this affect the performance of the cloned policy? This speaks to the practical challenge of designing a good blindfold.\n- The choice of \"blindfold\" seems domain-specific (hiding the global maze vs. masking the hole shape). How sensitive were the results to the specific implementation of the mask? For the peg insertion task, for example, what would happen if instead of masking the hole, you added significant noise to the robot's proprioceptive state, forcing the expert to rely more on visual and force feedback to re-localize the peg? This could help understand if the benefit comes from any information bottleneck, or specifically from withholding task-identity information.", "limitations": "The experiments are conducted on tasks that, while challenging, have a relatively monolithic goal (find the cheese, insert the peg). In more complex, sequential tasks (e.g., \"pick up the cup, take it to the sink, turn on the faucet\"), it's unclear how to apply a meaningful blindfold. Masking the final goal might make the task entirely unsolvable, while masking an intermediate step might not induce the right kind of hierarchical exploration. The proposed blindfolding strategy may not scale intuitively to tasks with long-horizon, compositional logic.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths: \n+ The paper provides a refreshing and valuable lens on data collection for imitation learning. The community has largely focused on scaling data quantity, using data augmentation, or improving the learning algorithm itself. This work compellingly argues that we must also consider the behavioral patterns within the demonstrations. The concept of \"blindfolding\" to induce a more generalizable, exploratory strategy is both intuitive and powerful. This has immediate practical implications for robotics and other fields where collecting vast amounts of \"perfect\" demonstrations is prohibitively expensive.\n\nWeaknesses:\n- A primary limitation lies in the abstract nature of the expert's internal representation, Z, and the associated mutual information term, I(Z;T). This term is central to the theoretical argument, but it is not practically measurable. The authors make a reasonable assumption that masking the expert's input observation will reduce I(Z;T), but this is an indirect manipulation. The paper would benefit from a discussion on this gap, acknowledging that Z is an unobservable theoretical construct and the link to input masking is an assumption, albeit a well-motivated one.\n- Furthermore, the paper does not offer a principle for designing the optimal \"blindfold.\" There is a clear trade-off: too little masking may not induce exploration, while too much masking may render the task unsolvable for the expert, leading to poor or failed demonstrations. The success of the method hinges on finding a sweet spot, and the paper currently lacks guidance on how a practitioner might find this balance for a new task.", "was_revised": true, "final_justification": "This is an excellent paper that shed new light on the relationship between the demonstration data and the resulting imitation learning policy, which should be of great interest to the robot learning community.", "rebuttal": "We thank the reviewer for the insightful comments. We are pleased that our work is found to be \"intuitive and powerful\" and recognized for its \"immediate practical implications for robotics and other fields\". In the following, we provide detailed replies to the questions and concerns. \n\n**How to quantify the mutual information I(Z;T)?**\n\nThis is a very good point. We can still lower bound the information between *observations* and *tasks* using a variational approach -- a classifier that predicts task identity from the observations [1]. We believe that this bound will be lower when we add the mask, and may be used in the future to devise better masks. Of course, whatever information the human gleans from the observation is not as easy to quantify.\n\n**How to design the blindfold?**\n\nThe study of principled methods for automatic design of the blindfold is an interesting direction for future work. \n\nHere we can provide some insights on how we designed the blindfold in practice. Take the peg insertion case, in which the blindfold is a mask on the image observation. We tweaked the mask to hide most information that still allows the human to solve the task. At that point, removing even more information would not have produced meaningful demonstrations.\nThis heuristic can be replicated by practitioners in other domains.\n\n**Question 1.**\n\nWe expect the sensitivity to the degree of blindfolding to follow the theoretical result in Theorem 3.1: More blindfolding leads to more generalization. However, if the blindfold is so severe that the expert's success rate drops, we do not expect the cloned policy to work well. Recall that we are doing behavioral cloning (and not offline RL), so the performance of the policy is limited by the performance of the expert.\n\n**Question 2.**\n\nWe tested a single implementation of the mask, so we do not have a sensitivity analysis.\nThe idea of adding noise is interesting for this matter: We can add experiments with different noise added to the observations (proprioceptive noise is less relevant as the demonstrator can see the robot in the image, but we can add image noise instead of masking the hole). \n\n**Address limitation: Beyond monolithic goals.**\n\nWe acknowledge that future studies are required to determine the full potential of blindfolding. That being said, our method is not a silver bullet for all generalization problems.\n\nNevertheless, we provide an additional experiment with some glimpses of the challenges the reviewer is suggesting.\nIn Procgen Heist, the agent has to solve a sequence of sub-goals in order to solve the task. At each level, the agent needs to collect a gem hidden behind a network of color-coded locks accessible through keys (of corresponding colors) that are scattered throughout the level (See [1] for more details on the environment setting). This task requires multi-stage planning in the form of lock-waypoints and to overcome variations in sub-task ordering, making the task more complex and challenging than the regular maze. For training the BC policies, we collected human demonstrations as in the Maze experiment, applying a similar blindfold to the blindfolded experts. In Table 1 below, we compare the results of the agent trained with the fully informed expert's data ($\\pi_{BC}$) to the results of the agent trained on the blindfolded experts' data ($\\pi_{BF-BC}$). While the training performance is above 90% for both agents, $\\pi_{BF-BC}$ significantly outperforms $\\pi_{BC}$ on unseen test levels (43% vs. 7%), which indicates better generalization performance (results obtained over $4K$ trajectories from $200$ different seeds).\n\n*Table 1: Performance comparison on the Procgen Heist experiment. The mean and std are computed over 10 seeds. Top performer in bold.*\n| Method | Train | Test |\n|--------|-------|------|\n| $\\pi_{BC}$ | 9.41 ± 0.20 | 0.69 ± 0.44 |\n| $\\pi_{BF-BC}$ | **9.91 ± 0.04** | **4.27 ± 1.01** |\n\n**References**\n\n[1] Cobbe, Karl, et al. \"Leveraging procedural generation to benchmark reinforcement learning.\" International conference on machine learning. PMLR, 2020."}]

Author rebuttal:

Meta-review: (a) This paper proposes a data collection paradigm for behavioral cloning by intentionally blindfolding human experts by withholding some task specific information during demonstration. This encourages exploratory behavior to infer the task, leading to improved generalization to unseen tasks. The authors provide a generalization bound whose dominant terms increase with the amount of task info available to the expert and decreases with # of demonstrated tasks. A peg insertion task is used for testing, which is a hard continuous control problem, as well as a procgen game environment. 

(b) The idea of blindfolding human experts to encourage exploration is a smart idea and could be generally valuable and interesting with lots of follow up studies. There is a bound to show that generalization error grows with task‑information available to the expert and shrinks as the number of distinct training tasks increases. Map coverage and state‑visitation entropy show blindfolded experts explore a larger fraction of state space, aligning with the paper’s hypothesis. The rebuttal and discussion addressed technical questions and added new experiments.

(c) Despite the added Heist results, the task diversity remains limited (peg insertion & two Procgen families). More axes of generalization (object/spatial/visual, language‑conditioned tasks, compositional objectives) would strengthen the case. The main bound is developed under a finite‑action formulation and the extension to continuous control is discussed qualitatively  but not formalized. Perhaps the biggest limitation is that there are no principled, task‑agnostic procedure for selecting the right bottleneck strength. The approach currently requires domain knowledge and tuning. But it is not fair to expect this to be resolved - the space of tasks is extremely large in the real physical world.

(d) Three reviewers (qfwW, bJoN, LUkx) ultimately argue for accept, with upgrades after rebuttal. One reviewer (29eA) remains borderline accept due to task‑specificity of blindfolding. 

(e) Authors added Heist and argued why the chosen domains isolate generalization rather than long‑horizon control. This materially improved the paper - residual concern about breadth remains, but no longer blocks acceptance (raised by bJoN, LUkx, 29eA). Authors added map coverage and state‑entropy; this directly addressed the request and supports the mechanism (bJoN). Authors acknowledged limitations and positioned applicability via discretization practices in modern VLA models.
Decision: Accept (poster)