AED: Adaptable Error Detection for Few-shot Imitation Policy

Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-02 12:42:58

Abstract: We introduce a new task called Adaptable Error Detection (AED), which aims to identify behavior errors in few-shot imitation (FSI) policies based on visual observations in novel environments. The potential to cause serious damage to surrounding areas limits the application of FSI policies in real-world scenarios. Thus, a robust system is necessary to notify operators when FSI policies are inconsistent with the intent of demonstrations. This task introduces three challenges: (1) detecting behavior errors in novel environments, (2) identifying behavior errors that occur without revealing notable changes, and (3) lacking complete temporal information of the rollout due to the necessity of online detection. However, the existing benchmarks cannot support the development of AED because their tasks do not present all these challenges. To this end, we develop a cross-domain AED benchmark, consisting of 322 base and 153 novel environments. Additionally, we propose Pattern Observer (PrObe) to address these challenges. PrObe is equipped with a powerful pattern extractor and guided by novel learning objectives to parse discernible patterns in the policy feature representations of normal or error states. Through our comprehensive evaluation, PrObe demonstrates superior capability to detect errors arising from a wide range of FSI policies, consistently surpassing strong baselines. Moreover, we conduct detailed ablations and a pilot study on error correction to validate the effectiveness of the proposed architecture design and the practicality of the AED task, respectively. The AED project page can be found at https://aed-neurips.github.io/.

Main paper: 
 introduction: 
 Few-shot imitation (FSI) is a critical and practical framework for future human-robot collaboration. Specifically, a robot solves a set of everyday missions through learning from a few owner's demonstrations. By constructing a demonstration-conditioned (DC) policy, existing mainstream FSI methods (Duan et al., 2017; James et al., 2018; Bonardi et al., 2020; Dasari & Gupta, 2020; Dance et al., 2021; Yeh et al., 2022; Watahiki & Tsuruoka, 2022) have made a significant breakthrough in novel environments. A major barrier that still limits their ability to infiltrate our everyday lives is to detect errors in novel (unseen) environments. Unlike other few-shot visual perception tasks, policies in FSI that fail to perform tasks in real scenes will cause severe impairments of surrounding objects and humans, resulting in rarely conducting real-world experiments (or only simple tasks).Therefore, we propose a challenging and crucial problem: adaptable error detection (AED), which aims to monitor FSI policies and report their behavior errors. In this work, behavior errors are the states inconsistent with the demonstrations, and the policy must be timely terminated when they occur. We emphasize that addressing AED is as critical as enhancing FSI policies' performance.As illustrated in Figure 1 , our AED problem introduces three novel challenges not present in a related problem, video few-shot anomaly detection (vFSAD). Specifically, (1) AED monitors the policy's behavior in novel environments whose normal states are not seen during training. Although vFSAD also works in an unseen scenario, the anomalies they monitor are common objects. (2) In AED, no impulse signals (i.e., notable changes) appear when behavior errors occur. Either (2-a) minor visual difference or (2-b) task misunderstanding is hard to recognize through adjacent rollout frames. In contrast, the anomaly in vFSAD raises a notable change in the perception field (e.g., a cyclist suddenly appearing on the sidewalk (Mahadevan et al., 2010) ). (3) AED requires online detection to terminate the policy timely, lacking the global temporal information of the rollout. However, in vFSAD (Sultani et al., 2018; Feng et al., 2021) , the whole video is accessible so that its statistical information can be leveraged, which is straightforward for detecting anomalies.Unfortunately, previous methods cannot address all three challenges posed by our AED problem. First, one-class classification (OCC) methods, such as one-class SVMs (Zhou & Paffenroth, 2017) or autoencoders (Ruff et al., 2018; Park et al., 2018; Chen et al., 2020; Wong et al., 2021) , struggle due to their inability to handle unseen environments. Clearly, they are trained with normal-only samples and detect relying on a substantial error when encountering anomalies (errors), i.e., outof-distribution (OOD) samples. However, normal states in novel environments containing unseen backgrounds and objects are already OOD to these methods, leading to poor performance. Second, multiple instance learning (Maron & Lozano-Pérez, 1997) or patch-based methods (Sultani et al., 2018; Roth et al., 2022) , good at discovering tiny defeats, may alleviate the (2-a) case in the second challenge. Even so, the minor visual difference and target misunderstandings cannot be recognized in novel environments if no consistent comparison between the current rollout and demonstrations is made, even at the instance/patch level. At last, due to lacking the crucial component (global temporal information from a completed trajectory), existing vFSAD methods (Sultani et al., 2018; Feng et al., 2021) are unsuitable in AED, where only partial temporal information exists.As analyzed above, conventional methods run into insurmountable obstacles. To address AED, we discover the potential to leverage policy understanding, i.e., how policy extracts feature representations in different situations (normal and error). In addition to obtaining the information from visual inputs, learning in policy features provides additional knowledge related to FSI tasks. We hypothesize that the feature representations (embeddings) from trained policies have implicit discernible patterns within the frames of normal and error states. Hence, the Pattern Observer (PrObe) is then proposed. Precisely, we collect and augment successful and failed rollouts in the base (training) environments. Then, the trained policy computes feature embeddings of rollout frames. In order to mitigate the impact of shifting into novel environments (first challenge), PrObe uses an extractor to parse patterns in policy features to obtain additional task-related knowledge. Next, PrObe computes the pattern flow containing pattern changes over time, efficiently leveraging existing temporal information (third challenge). Finally, PrObe conducts the consistent comparison (second challenge) by a fusion of pattern flow and transformed task-embeddings and distinguishes when the feature embedding is extracted, i.e., the frame that policy encountered in successful or failed rollouts. With these designs, PrObe can effectively address the challenging AED problem.We designed seven challenging multi-stage FSI tasks (329 base and 158 novel environments in total) to validate PrObe's effectiveness. From the results, PrObe achieves the best performance and outperforms strong baselines with a significant gap of up to 40%. Meanwhile, it is the most robust method for different FSI policies. Also, we provide detailed ablations and the timing accuracy result to support our claims. Finally, a pilot study on addressing error correction after AED methods detect an error is conducted to demonstrate the practical scenario of the proposed problem.Contributions (1) We formulate the adaptable error detection (AED) problem, accelerating the development of human-robot collaboration.(2) We propose PrObe, addressing AED by learning the pattern flow in the policy's feature representations.(3) We develop challenging FSI tasks to evaluate PrObe's effectiveness. It outperforms various baselines and is robust for distinct policies. Our work is expected to become an important cornerstone for FSI to conduct future real-world experiments. 
few-shot imitation (fsi): 
 Policy With the recent progress in meta-learning (Finn et al., 2017a) , the community explores the paradigm for learning policies from limited demonstrations during inference (Finn et al., 2017b; Yu et al., 2018; 2019a) . Notably, these works either assume that the agent and expert have the same configuration or explicitly learn a motion mapping between them (Yu et al., 2018) . Conversely, DC methods (Dance et al., 2021; Yeh et al., 2022; Watahiki & Tsuruoka, 2022) develop a policy that behaves conditioned both on the current state and demonstrations. Furthermore, they implicitly learn the mapping between agents and experts, making fine-tuning optional. Thereby, effectively extracting knowledge from demonstrations becomes the most critical matter. In most FSI works, no environment interactions are allowed before inference. Hence, policies are usually trained by behavior cloning (BC) objectives, i.e., learning the likelihood of expert actions by giving expert observations. Recently, DCRL (Dance et al., 2021) trains a model using reinforcement learning (RL) objects and performs FSI tasks without interacting with novel environments.Evaluation tasks Since humans can perform complex long-horizon tasks after watching a few demonstrations, FSI studies continuously pursue solving long-horizon tasks to verify if machines can achieve the same level. A set of research applies policy on a robot arm to perform daily life tasks in the real world (Yu et al., 2018) or simulation (Duan et al., 2017) . The task is usually multistage and composed of primitives/skills/stages (Yeh et al., 2022; Hakhamaneshi et al., 2022) , such as a typical pick-and-place task (Yu et al., 2019a) or multiple boxes stacking (Duan et al., 2017) . Besides, MoMaRT (Wong et al., 2021) tackles a challenging mobile robot task in a kitchen scene.Our work formulates the AED problem to monitor the FSI policies' behaviors and proposes PrObe to address AED, which is valuable to extensive FSI research. Besides, we also built seven challenging FSI tasks containing attributes like realistic simulation, task distractors, and various robot behaviors.Problem setting FSAD is an innovative research topic. Most existing studies deal with FSAD at the image level (Roth et al., 2022; Sheynin et al., 2021; Pang et al., 2021; Huang et al., 2022a; Wang et al., 2022) and only a few tackle video anomalies (Lu et al., 2020; Qiang et al., 2021; Huang et al., 2022b) . Moreover, problem settings in the literature are diverse. Some works presume only normal data are given during training (Sheynin et al., 2021; Wang et al., 2022; Lu et al., 2020; Qiang et al., 2021) , while others train models with normal and a few anomaly data and include unknown anomaly classes during performance evaluation (Pang et al., 2021; Huang et al., 2022b) .Method summary Studies that only use normal training samples usually develop a reconstructionbased model with auxiliary objectives, such as meta-learning (Lu et al., 2020) , optical flow (Qiang et al., 2021) , and adversarial training (Sheynin et al., 2021; Qiang et al., 2021) . Besides, patch-based methods (Roth et al., 2022; Pang et al., 2021) reveal the performance superiority on main image FSAD benchmark (Bergmann et al., 2019) since the anomaly are tiny defeats. Regarding video anomaly detection, existing works access a complete video to compute the temporal information for determining if it contains anomalies. In addition, vFSAD benchmarks (Mahadevan et al., 2010; Lu et al., 2013) provide frame-level labels to evaluate the accuracy of locating anomalies in videos.In our work, we formulate the AED problem. As stated earlier, addressing AED is more challenging than vFSAD since it brings two more challenges, including no impulse signals and online detection. Moreover, previous error detection methods are infeasible for the proposed AED problem. 
preliminaries: 
 Few-shot imitation (FSI) FSI is a framework worthy of attention, accelerating the development of human-robot collaboration, such as robots completing pre-tasks alone or interacting with humans for highly complex tasks. Following Yeh et al. (2022) , a FSI task is associated with a set of base environments E b and novel environments E n . In each novel environment e n ∈ E n , a few demonstrations D n are given. The objective is to seek a policy π that achieves the best performance (e.g., success rate) in novel environments leveraging few demonstrations. Notably, the task in base and novel environments are semantically similar, but their backgrounds and interactive objects are disjoint. Besides, observations and demonstrations are RGB-D images that only provide partial information in recent studies (Dasari & Gupta, 2020; Yeh et al., 2022) .Additionally, there are several challenges when addressing FSI tasks in practice, including (1) the task is a multi-stage task, (2) demonstrations are length-variant and misaligned, and (3) the expert and agent have a distinct appearance or configuration, as mentioned in Yeh et al. (2022) . Developing a policy to solve the FSI problem and simultaneously tackle these challenges is crucial.Demonstration-conditioned (DC) policy As previously stated, the expert and agent usually have different appearances or configurations in practice. The DC policy π(a | s, D) learns implicit mapping using current states s and demonstrations D to compute agent actions a. Next, we present the unified architecture of DC policies and how they produce the action. When the observations and demonstrations are RGB-D images that only provide partial information, we assume that current history h (observations from the beginning to the latest) and demonstrations are adopted as inputs.A DC policy comprises a feature encoder, a task-embedding network, and an actor. After receiving the rollout history h, the feature encoder extracts the history features f h . Meanwhile, the feature encoder also extracts the demonstration features. Then, the task-embedding network computes the task-embedding f ζ to retrieve task guidance. Notably, the lengths of agent rollout and demonstrations can vary. The task-embedding network is expected to handle length-variant sequences by padding frames to a prefixed length or applying attention mechanisms. Afterward, the actor predicts the action for the latest observation, conditioned on the history features f h and task-embedding f ζ .Besides, an optional inverse dynamics module predict the action between consecutive observations to improve the policy's understanding of how actions affect environmental changes. At last, the predicted actions are optimized by the negative log-likelihood or regression objectives (MSE). 
aed problem: 
 We formulate AED to monitor FSI policies and report their behavior errors, i.e., states in policy rollouts inconsistent with demonstrations. The inevitable challenges faced in addressing AED is previously presented. Next, we formally state the problem.Problem statement Let c denote the category of agent's behavior modes, where c ∈ C, C = {normal, error}. When the agent (executing trained π θ ) performs FSI task in a novel environment e n , an adaptable error detector ϕ can access the agent's rollout history h and few expert demonstrations D n . Then, ϕ predicts the categorical probability ŷ of the behavior mode for the latest state in h byEQUATIONwhere enc denotes the feature encoder, and it may be enc ϕ (ϕ contains its own encoder) or enc π θ (based on policy's encoder). Next, let y represent the ground truth probability, we evaluate ϕ via the expectation of detection accuracy over agent rollouts X n in all novel environments E n :EQUATIONwhere A(•, •) is the accuracy function that returns 1 if ŷ is consistent with y and 0 otherwise. However, frame-level labels are lacking in novel environments since we only know if the rollout is successful or failed at the end. Hence, we present the details of A(•, •) in the Appendix's Section E.1.Protocol To train the adaptable error detector ϕ, we assume it can access the resources in base environments E b just like FSI policies, i.e., successful agent rollouts X b succ and a few expert demonstrations D b , which are collected by executing the agent's and expert's optimal policies π *1 , respectively. Besides, we collect failed agent rollouts X b f ail by intervening in the agent's π * at critical timing (e.g., the timing to grasp objects) so that we have precise frame-level error labels.  
pattern observer (probe): 
 Overview Our motivation is to detect behavior errors in the space of policy features to acquire additional task-related knowledge. Thus, we propose Pattern Observer (PrObe), an adaptable error detector, to discover the unexpressed patterns in the policy features extracted from frames of successful or failed states. Even if the visual inputs vary during inference, PrObe leverages the pattern flow and a following consistent comparison to alleviate the challenges in Figure 1 . As a result, it can address the crucial AED problem effectively. Furthermore, a rollout augmentation is designed to increase the collected rollout diversity and enhance method robustness, as we now present in detail.To obtain balanced (successful and failed) data and precise frame-level labels, we collect X b succ and X b f ail using the agent's optimal policy π * with intervention rather than the trained policy π θ . However, even if π θ is trained on the rollouts collected by the agent executing π * , the trained π θ will still deviate from agent's π * due to the limited rollout diversity, i.e., a behavior bias exists between the two policies, which should be reduced. Otherwise, an adaptable error detector trained on raw rollouts (X b succ and X b f ail ) will yield a high false-positive during AED inference. Accordingly, we augment agent rollouts so as to increase rollout diversity and dilute the behavior bias between trained π θ and agent's π * . Specifically, we iterate each frame (and its label) from sampled agent rollouts and randomly apply the following operations: keep, drop, swap, and copy, with probability 0.3, 0.3, 0.2, and 0.2, respectively. As a result, we inject distinct behaviors into collected rollouts, such as speeding up (interval frames dropped), slowing down (repetitive frames), and not smooth movements (swapped frames). We carefully verify its benefits in Section 6.As depicted in Figure 3 , PrObe comprises three major components: a pattern extractor, a pattern flow generator, and an embedding fusion. Since PrObe detects errors through policy features, the history features f h and task-embeddings f ζ from the trained policy π θ are passed into PrObe. At first, the pattern extractor aims to gather discriminative information from each embedding of history features f h (the features of a rollout sequence). In more detail, f h is transformed into unit embeddings divided by its L2 norm, which mitigates biases caused by changes in visual inputs.Then, a learnable gate composed of a linear layer with a sigmoid function determines the importance of each embedding cell. A Hadamard product between the sign of unit embeddings and the importance scores, followed by an instance normalization (IN), is applied to obtain the pattern features f p for each timestep. PrObe then feeds f p into a flow generator (a LSTM) to generate the pattern flow. Intuitively, the changes in the pattern flow of successful and failed rollouts will differ.On the other hand, the task-embeddings f ζ is transformed by an IN layer, a linear layer (FC), and a tanh function. This step tries to map the task-embeddings into a space similar to pattern flow. Then, a fusion process concatenates the pattern flow and transformed task-embeddings to compute the error predictions ŷ. We expect this fusion process to be able to compare the consistency between agent rollout and tasks and use this as a basis for whether behavior errors occur.Objectives PrObe is optimized by one supervised and two unsupervised objectives. First, since accurate frame-level error labels y are available during training, a binary cross-entropy (BCE) objective L cls is used to optimize the error prediction ŷ. Second, L pat is a L1 loss applied to the pattern features f p . The L pat objective encourages the pattern extractor to generate sparse pattern features, which are easier to observe the changes. In addition, a contrastive objective L tem is added to logit embeddings (the embedding before the final outputs) to reinforce the difference between the normal and error states in failed rollouts X b f ail , i.e., we sample anchors from X b f ail , and the positive and negative samples will be logits of the same or opposed states from the same rollout.However, the logits of adjacent frames will contain similar signals since they contain temporal information from the pattern flow, even if behavioral errors have occurred (cf. Appendix's Figure 7 ). Blindly forcing the logits of normal and error states to be far away from each other may mislead the model and have a negative impact. Hence, L tem is a novel temporal-aware triplet loss (Schroff et al., 2015) . The temporal-aware margin m t in L tem will be enlarged or reduced considering the temporal distance of anchor, positive and negative samples. The L tem objective is calculated by:EQUATIONwherem t = m * (1.0 -α * (d ap -d an )), m is the margin in the original triplet loss, and d ap , d an are the clipped temporal distances between the anchor and positive sample and the anchor and negative sample, respectively. And N is the number of sampled pairs. Ultimately, the total loss L total combines L pat , L tem , L cls objectives weighted by λ pat , λ tem , and λ cls . 
experiments: 
 We design the experiments to verify the following questions: (1) Is detecting behavior errors from policy features beneficial to addressing the novel AED problem? (2) Can the proposed PrObe precisely recognize the behavior errors at the accurate timing? (3) Do the embeddings learned by PrObe have discernable patterns? (4) How practical is our proposed AED research?Evaluation tasks Existing robotic manipulation benchmarks (Yu et al., 2019b; James et al., 2020; Fu et al., 2020; Mandlekar et al., 2021) do not suit our needs since we formulate the novel AED problem. We design seven challenging multi-stage FSI tasks (six indoor and one factory scene) that contain the challenges introduced in Section 3. Detailed task descriptions, schematic, the number of environments, and distractors are shown in the Appendix's Section D. We build the tasks on Coppeliasim software (Robotics) and use the Pyrep (James et al., 2019) toolkit as a coding interface.To verify if the AED methods can handle various policies' behaviors, we select three representative DC policies to perform FSI tasks, which are implemented following the descriptions in Section 3 and use the same feature extractor and actor architecture. The main difference between them is how they extract task-embeddings from expert demonstrations. NaiveDC2 (James et al., 2018) concatenates the first and last frames of demonstrations and averages their embeddings as task-embeddings; DCT3 (Dance et al., 2021) performs cross-demonstration attention and fuses them at the time dimension; SCAN (Yeh et al., 2022) computes the tasks-embeddings using stageconscious attention to locate the critical frames in demonstrations.Error detection baselines We compare our PrObe with four error detection methods that possess different characteristics. Unlike PrObe, all methods are policy-independent (i.e., containing their own encoders) and use the same encoder architecture. We briefly introduce them: (1) SVDDED, an one-class SVM modified from Ruff et al. (2018) , is a single-frame DC method trained with only successful rollouts. (2) TaskEmbED, a single-frame DC method trained with successful and failed rollouts, uses the NaiveDC (James et al., 2018) policy's strategy to extract task-embeddings when predicting errors. (3) LSTMED is a LSTM model (Hochreiter & Schmidhuber, 1997 ) predicts errors without demonstration information. (4) DCTED leverages the DCT (Dance et al., 2021) policy's strategy to predict errors. The detailed version can be found in the Appendix's Section C.The area under the receiver operating characteristic (AUROC) and the area under the precision-recall curve (AUPRC), two conventional threshold-independent metrics in error (or anomaly) detection literature, are reported. We linearly select 5000 thresholds spaced from 0 to 1 (or outputs of SVDDED) to compute the two scores. The accuracy function (presented in the Appendix's Section E.1) determines the error prediction results (i.e., TP, TN, FP, FN) at the sequencelevel so that we further conduct and illustrate the timing accuracy experiment in Figure 5 .Overall performance Our main experiment aims to answer the first question, i.e., is detecting the policy's behavior errors from its feature embedding beneficial? We let NaiveDC, DCT, and SCAN policies perform in dozens of novel environments for each task and record the results (rollout observations and the success or failure of the rollout). Policies behave differently, making it difficult for AED methods to effectively detect behavioral errors in all policies. We then let AED methods infer the error predictions and report the two metrics. It is worth noting that in this experiment, AUPRC has more reference value. Because the results of policy rollout are often biased (e.g., many successes or failures), AUPRC is less affected by the imbalance of labeling. Figure 4 : AED methods' performance on our seven challenging FSI tasks. The value under the policy indicates the policy's success rate in that task. The ranges of the two metrics [↑] are both 0 to 1. Our PrObe gets the best 13 AUROC and 15 AUPRC scores (out of 21 cases), demonstrating its superiority and robustness. We refer to the Appendix's Table 7 for detailed values.From Figure 4 , PrObe effectively reports the inconsistency between rollout frames and demonstrations (behavior errors) in novel environments, even some errors not included in the training data.Our PrObe obtained the best 13 AUROC and 15 AUPRC results out of 21 cases, which reveals its superiority among AED methods and robustness for different policies. Besides, we investigate the cases where PrObe got suboptimal performance and found that they can be grouped into two types:(1) Errors all occur at the same timing (e.g., at the end of the rollout) and can be identified without reference to the demonstrations (e.g., DCT policy in Organize Table task ).(2) Errors are not obvious compared to the demonstrations, and there is still a chance to complete the task, such as the drawer being closed to a small gap (SCAN and DCT policies in Close Drawer task). In these cases, the pattern flow will not change enough to be recognized, resulting in suboptimal results for PrObe.Ablations We conduct and summarize several ablations on the proposed components in the Appendix. First, Figure 11 indicates that the rollout augmentation (RA) strategy can increase the rollout diversity and benefits AED methods with temporal information. Second, Figure 12 proves that PrObe's design can effectively improve performance. Finally, Figure 13 shows PrObe's performance stability by executing multiple experiment runs on a subset of tasks and computing the performance variance. For an exhaustive version, please refer to the corresponding paragraphs in the Appendix.Timing accuracy Our main experiment examines whether the AED methods can accurately report behavior errors when receiving the complete policy rollouts. To validate if they raise the error at the right timing, we manually annotate a subset of policy rollouts (SCAN policy in Pick & Place task) and visualize the raw outputs of all AED methods, where SVDDED outputs the embedding distance between observation and task-embedding while others compute the probabilities, as shown in Figure 5 . In the successful rollout case, our PrObe and LSTMED stably recognize the states are normal, while other methods misidentify and increase the probability that inputs may be the error states. In the failed rollout case, PrObe detects the error after it has happened for a while (< 5 frames) and significantly increases the probability. We attribute this short delay to the fact that the pattern flow takes a short period to cause enough change. Even so, PrObe is the one that detects the error at the closest timing; other methods raise the probability too early or too late. We emphasize that this situation is ubiquitous and is not a deliberate choice.To analyze if the learned embeddings possess discernable patterns, we present the t-SNE transform (van der Maaten & Hinton, 2008) of three AED methods' embeddings in Figure 6 . The same 20 rollouts from the annotated dataset above are extracted by the three methods to obtain the embeddings. Obviously, the embeddings learned by TaskEmbED and LSTMED are scattered and have no explainable structure. In contrast, our PrObe's embeddings have task progress and homogeneous state clustering characteristics, i.e., the states with similar properties (e.g., the beginnings of rollouts, the same type of failures) are closer. This experiment supports the hypothesis that our PrObe can learn the implicit patterns from the policy's feature embeddings. 
conclusion: 
 In this work, we point out the importance of monitoring policy behavior errors to accelerate the development of FSI research. To this end, we formulate the novel adaptable error detection (AED) problem, whose three challenges make previous error detection methods infeasible. To address AED, we propose the novel Pattern Observer (PrObe) by detecting errors in the space of policy features. With the extracted discernable patterns and additional task-related knowledge, PrObe effectively alleviates AED's challenges. It achieves the best performance, which can be verified by our main experiment and detailed ablations. We also demonstrate PrObe's superiority using the timing accuracy experiment and the learned embedding visualization. Finally, we provide the pilot study on error correction in the Appendix to confirm the practicality of our AED problem. Our work is an essential cornerstone in developing future FSI research to conduct complex real-world experiments. 
 
 Appendix: This appendix presents more details of our work, following a similar flow to the main paper. First, Section A introduces the few-shot imitation (FSI) policies. Then, additional descriptions of PrObe's are provided in Section B. Besides, we introduce the implemented baselines of adaptable error detection in Section C. Next, in Section D, we exhaustively describe the designed multi-stage FSI tasks, including task configurations, design motivations, and possible behavior errors. In addition, more experimental results are shown in Section E to support our claims. At last, we responsibly present the limitations of this work in Section F.We follow the main paper's Section 3 to implement three state-of-the-art demonstration-conditioned (DC) policies, including NaiveDC (James et al., 2018) , DCT (Dance et al., 2021) , and SCAN (Yeh et al., 2022) , and monitor their behaviors.Architecture settings In our work, we employ the same visual head and actor architecture for all policies to conduct a fair comparison. Table 1 and Table 2 present the settings of general components (visual head and actor) and respective task-embedding network settings for better reproducibility.In the visual head, we added fully-connected (FC) and dropout layer after a single resnet18 to avoid overfitting the data from base environments. Furthermore, the input dimension of the actor module is dependent on the summation of the output dimensions from the visual head and the task-embedding network. For the DCT and SCAN policies, the task-embedding dimension matches the visual head's output. However, the NaiveDC policy has a task-embedding dimension that is twice the size.In terms of the task-embedding networks, NaiveDC concatenates the first and last frames of each demonstration and computes the average as the task-embedding, without involving any attention mechanism. DCT, on the other hand, incorporates cross-demo attention to fuse demonstration features and then applies rollout-demo attention to compute the task-embedding. SCAN uses an attention mechanism to attend to each demonstration from the rollout, filtering out uninformative frames, and fuses them to obtain the final task-embedding. Besdies, We did not employ the bi-directional LSTM, which differs from the original setting in Yeh et al. (2022) . Training details To optimize the policies, we utilize a RMSProp optimizer (Hinton) with a learning rate of 1e-4 and a weight regularizer with a weight of 1e-2. Each training epoch involves iterating through all the base environments. Within each iteration, we sample five demonstrations and ten rollouts from the sampled base environment. The total number of epochs varies depending on the specific tasks and is specified in In this work, we propose the Pattern Observer (PrObe) to address the novel AED problem by detecting behavior errors from the policy's feature representations, which offers two advantages: (1) PrObe has a better understanding of the policy because the trained policy remains the same during both AED training and testing. This consistency enhances its effectiveness in identifying errors.(2) The additional task knowledge from the policy encoder aids in enhancing the error detection performance. We have described PrObe's components and their usages in the main paper. Here, we introduce more details about the design of model architecture and training objectives.The pattern extractor aims to transform the policy features into observable patterns. To achieve this, we can leverage discrete encoding, as used in Hafner et al. (2022) . However, the required dimension of discrete embedding might vary and be sensitive to the evaluation tasks. To address this, we leverage an alternative approach that operates in a continuous but sparse space. Besides, for the pattern flow generator, we use a standard LSTM model instead of a modern transformer. This is motivated by the characteristics of the AED problem, where policy rollouts are collected sequentially, and adjacent frames contain crucial information when behavior errors occur. Thus, the LSTM model is better suited for addressing the AED problem. The contributions of PrObe's components are verified and shown in Figure 12 . From the results, our designed components significantly improve the performance.Objectives We leverage three objectives to optimize our PrObe: a BCE loss L cls for error classification, a L1 loss L pat for pattern enhancement, and a novel temporal-aware triplet loss L tem for logit discernibility. First, the PrObe's error prediction ŷ can be optimized by L cls since the ground-truth frame-level labels y are accessible during training. The L cls is calculated by,EQUATIONwhere N r and T h are the number and length of rollouts, respectively. Then, we leverage the L1 objective L pat to encourage the pattern extractor to learn a more sparse pattern embedding f p , which can be formulated by EQUATIONWith the objective L pat , the pattern embeddings are expected to be sparse and easily observable of changes, which benefits the model in distinguishing behavior errors. Next, as previously pointed out, when applying contrastive learning to temporal-sequence data, the time relationship should also be considered in addition to considering whether the frames are normal or errors for the task. Thus, L tem is a novel temporal-aware triplet loss (Schroff et al., 2015) , and the temporal-aware margin m t in L tem will have different effects depending on the relationship between the anchor and the positive/negative sample, as depicted in Figure 7 . The L tem can be calculated by:EQUATIONwherem t = m * (1.0 -α * (d ap -d an )), m is the margin in the original triplet loss, and d ap , d an are the clipped temporal distances between the anchor and positive sample and the anchor and negative sample, respectively. In addition, N is the number of sampled pairs. With L tem , PrObe can efficiently perform contrastive learning without getting confused by blindly pulling temporally distant anchors and positive samples closer. Lastly, the total loss L total is the combination of three objectives:EQUATIONwhere λ pat , λ tem , and λ pat are weights to balance different objectives.We compare our PrObe with several strong baselines that possesses different attributes, aiming to verify their effectiveness on addressing the adaptable error detection (AED) problem. In this section, we comprehensively present the attributes and training details associated with these baselines. Baseline attributes Table 3 presents the attributes of all error detection baselines (and our PrObe).All baselines have their own encoder and are independent of the policies, which offers the advantage that they only need to be trained once and can be used for various policies. However, as a result, they recognize errors only based on visual information. Now we describe the details of baselines:• SVDDED: A modified one-class SVM method (Ruff et al., 2018) trained with only successful rollouts. It relies on measuring the distance between rollout embeddings to the center embedding to detect behavior errors, without considering the temporal information.We average demonstration features as the center and minimize the embedding distance between rollout features and the center during training.• TaskEmbED: A single-frame baseline trained with successful and failed rollouts, which is a modification from the NaiveDC policy (James et al., 2018) . It concatenates and averages the first and last demonstration frames as the task-embedding. Then, it predicts the behavior errors conditioned on the current frame and task-embedding.• LSTMED: A standard LSTM model (Hochreiter & Schmidhuber, 1997) predicts errors based solely on the current rollout. It is trained with successful and failed rollouts with no demonstration knowledge provided. It is expected to excel in identifying errors that occur at similar timings to those observed during training. However, it may struggle to recognize errors that deviate from the demonstrations.• DCTED: A baseline derived from the DCT policy (Dance et al., 2021) with temporal information. It is trained with successful and failed rollouts and incorporates with demonstration information. One notable distinction between DCTED and our PrObe lies in their policy dependencies. DCTED detects errors using its own encoder, which solely leverages visual information obtained within the novel environment. Conversely, PrObe learns within the policy feature space, incorporating additional task-related knowledge.Training details Similar to optimizing the policies, we utilize a RMSProp optimizer with a learning rate of 1e-4 and a weight regularizer with a weight of 1e-2 to train the error detection models.During each iteration within an epoch, we sample five demonstrations, ten successful agent rollouts, and ten failed rollouts from the sampled base environment for all error detection models except SVDDED. For SVDDED, we sample five demonstrations and twenty successful agent rollouts since it is trained solely on normal samples. Notably, all error detection experiments are conducted on the same machine as the policy experiments, ensuring consistency between the two sets of experiments.To obtain accurate frame-level error labels in the base environments and to create a simulation environment that closely resembles real-world scenarios, we determined that existing robot manipulation benchmarks/tasks (Yeh et al., 2022; Yu et al., 2019b; James et al., 2020; Fu et al., 2020; Mandlekar et al., 2021) • Generalization: To present the characteristics of changeable visual signals in real scenes, we build dozens of novel environments for each task, comprising various targets and taskrelated or environmental distractors. Also, Dozens of base environments are built as training resources, enhancing AED methods' generalization ability. Letting them train in base environments with different domains can also be regarded as a domain randomization technique (Bonardi et al., 2020) , preprocessing for future sim2real experiments.• Challenging: Three challenges, including multi-stage tasks, misaligned demonstrations, and different appearances between expert and agent, are included in our designed FSI tasks. Moreover, unlike existing benchmarks (e.g., RLBench (James et al., 2020) ) attaching the objects to the robot while grasping, we turn on the gravity and friction during the whole process of simulation, so grasped objects may drop due to unsmooth movement.• Realistic: Our tasks support multi-source lighting, soft shadow, and complex object texture to make them closer to reality (see Figure 9 ). Existing benchmarks usually have no shadow (or only a hard shadow) and simple texture. The gravity and friction stated above also make our tasks more realistic. As stated in the main paper, the frame-level label is lacking during inference because novel environments only indicate whether the current rollout is a success or failure. Hence, we follow similar rules in Wong et al. (2021) to define the accuracy function A(•, •), i.e., how to determine the error prediction outcome, as elaborated in Figure 10 . The A(•, •) determines the prediction result at the sequence-level, which may not sufficiently reflect whether the timing to raise the error is accurate. Thus, we further conduct the timing accuracy experiment (shown in Figure 5 of the main paper).The policy performance is originally reported in Figure 4 of the main paper. Here, we summarize the comprehensive overview in Table 5 . It is important to note that the standard deviations (STD) are relatively large since they are calculated across novel environments rather than multiple experiment rounds. The difficulty variability of novel environments, such as variations in object size or challenging destination locations, leads to different performance outcomes for the policies. Besides, the policy will conduct 20 executions (rollouts) for each novel environment to get the success rate (SR). Then, the average success rate and its STD are calculated across those SRs. Furthermore, these rollouts are also used for later AED inference.In general, NaiveDC policy achieves the best result in the simplest task because the task-embedding from the concatenation of the first and last two frames is enough to provide effective information. However, its performance will drop rapidly as the task difficulty increases. On the other hand, DCT policy excels in tasks where the misalignment between demonstrations is less severe since it fuses the task-embeddings at the temporal dimension. At last, SCAN policy outperforms on the more difficult tasks because its attention mechanism filters out uninformative demonstration frames. Methods trained without RA (dark) and with RA(bright) and their performance gap are listed. RA is more beneficial for methods with time information (rightmost three columns). Also, the improvement from RA is more evident when the performance of FSI policy is higher (SCAN case).The main experiment result is originally reported in Figure 4 of the main paper. Here, Table 7 and Figure 15 present the main experiment's detailed values and ROC curves, which better demonstrate our PrObe's performance gap against other methods. To highlight again, our proposed PrObe obtains the best performance on both metrics and can handle the various behaviors from different policies.Rollout augmentation As stated before, the collected agent rollouts have limited rollout diversity.In this experiment, we aim to validate whether the rollout augmentation (RA) increases the rollout diversity so that error detectors can have better robustness. In Figure 11 , we present the results of all error detection methods (trained w/ and w/o RA) monitoring policies solving Pick & Place task, with the following findings: (1) RA provides a minor or even negative influence on SVDDED, which can be expected. Since SVDDED is only trained on (single) frames from successful rollouts, the frame dropped by RA decreases the frame diversity.(2) RA benefits methods with temporal information (LSTMED, DCTED, and PrObe), especially when FSI policy performs better (SCAN case) because error detectors cannot casually raise an error, or they will easily make a false-positive prediction. The results indicate that methods trained with RA raise fewer false-positive predictions and demonstrate improved robustness to different policy behaviors.We verify the component contributions of the proposed PrObe, as shown in Figure 12 , which results from monitoring the SCAN policy solving the Pick & Plcae task. The naive model (first column) removes the pattern extractor and uses a FC layer followed by an IN layer to transform history embeddings. Obviously, the designed components and objectives boost the performance, especially when adding the pattern extractor into the model. We attribute this to the extracted embedding (representation) patterns by our components are more informative and discernible. Performance stability Our work involves training and testing of all policies and AED methods to produce the results for an FSI task. In addition, each task consists of numerous base and novel environments. These factors make it time-consuming and impractical to report the main experiment (Figure 4 of the main paper) with error bars. Based on our experience, generating the results of the main experiment once would require over 150 hours.Therefore, we select the most representative Pick & Place task among our FSI tasks and conduct multiple training and inference iterations for all AED methods. We present the AUROC, AUPRC, and their standard deviations averaged over multiple experimental rounds (with five random seeds) in Figure 13 . From the results, our proposed PrObe not only achieves the best performance in detecting behavior errors across all policies but also exhibits the most stable performance among all AED methods, with the smallest standard deviation.Case study on PrObe's failure prediction We analyze the cases in which PrObe is poor at detecting behavior errors and visualize the results in Figure 14 . As stated in the main paper, our PrObe requires a short period after the error occurs to make the pattern flow causing enough changes. However, in the Figure 14 case, the mug had fallen off the table, causing this rollout to be stopped the mug is no longer on the table, which means the error detectors have only six frames to detect the error in this case. However, it took a while for our Probe's pattern flow to cause enough change. By the last frame, ProObe was just about to increase its predicted probability, but it was too late. immediately. Such time is insufficient for ProObe to reflect adequately, causing it to be unable to (substantially) increase the prediction probability of behavior errors at the last moment. Developing mechanisms for quick responsive pattern flow is one of our future research direction.To further examine the practicality of our AED problem, we conducted a pilot study on error correction. In this experiment, the FSI policy is paused after the AED methods detect an error. Then, a correction policy from Wong et al. (2021) , which resets the agent to a predefined safe pose, is applied. At last, the FSI policy continues to complete the task.We conducted the study on Press Button, which will most likely be corrected after the error happens. In this task, the correction policy is defined as moving to the top of the table center. We let the DCT policy cooperate with the correction policy and four AED methods (SVDDED excluded), whose results are summarized in Table 6 . We have two findings from the results: (1) Our PrObe is verified to be the most accurate method again. In contrast, other AED methods will raise the errors at the wrong timing, so the correction policy cannot benefit the performance (even negative influence for original success trials).(2) The performance gain from the correction policy is minor since it lacks precise guidance in unseen environments.We believe error correction in novel environments deserves a separate study due to its challenging nature, as we observed in the pilot study. One potential solution is the human-in-the-loop correction works through instructions (Sharma et al., 2022; Cui et al., 2023) or physical guidance (Li et al., 2021) . However, their generalization ability and cost when applying to our AED problem need further discussion and verification. We will leave it as our future work.Offline evaluation? Our ultimate goal is to terminate (or pause) the policy instantly when detecting a behavior error online. However, in our main experiment, only the trained policies performed in novel environments online, whose rollouts are collected. AED methods later use the offline rollouts to predict errors for a fair comparison. Nevertheless, we illustrate that PatObs can effectively address the online detection challenge through the time accuracy experiment. Also, the pilot study on error correction runs AED methods online to correct the error promptly.Not evaluated on benchmarks or in real-world environments? Due to the practical challenges inherent in FSI tasks and the AED problem, none of the existing robot manipulation benchmarks align with our specific requirements. For instance, FSI tasks necessitate multi-stage scenarios or variations in configurations and appearances between experts and agents. In the case of AED, it is crucial to gather failed rollouts in the base environments efficiently. As a result, we have to develop our own environments and tasks.Moreover, the lack of real-world experiments is caused by three factors: (1) Deploying the needed resources for the AED problem in the real world requires a high cost and long-term plan.(2) The FSI policies still fail to perform complex tasks in the real world, making assessments less than comprehensive.(3) Although our PrObe achieves the best performance, there is still significant room for improvement on the AED problem.We emphasize that rashly conducting AED and FSI research in real environments may cause irreparable damage. Even previous error correction work (Wong et al., 2021) or related safe exploration research (Sootla et al., 2022; Anderson et al., 2023; Huang et al., 2023) were conducted in simulation environments, which shows that it is reasonable and valuable for us to conduct detailed verification in the simulation first. As mentioned earlier, we develop seven challenging and realistic multi-stage FSI tasks containing dozens of base and novel environments. To our knowledge, the simulation environments we established are the closest to the real scenarios in the literature.Table 7 : AED methods' performance on our seven challenging FSI tasks. The value under the policy indicates the policy's success rate in that task. Our proposed PrObe demonstrates its superiority among all the methods on both metrics and is the most robust method for different policies. We expect the agent's optimal policy is known in base environments. Furthermore, successful agent rollouts are not demonstrations since the agent and expert have different configurations.Original name is TaskEmb, we renamed it to avoid confusion with error detection baselines.Original name is DCRL, we denote it as DCT (transformer) since we do not use RL objectives. 


Decision: This paper is about a new task, adaptable error detection, in context of few-shot imitation policy learning, where the task is to detect behavior errors made by the few shot imitation policy. from visual observations in unseen environments.

The paper proposes a new task (AED), an associated benchmark, and a novel model to solve the AED task called PrObe.

All reviewers agree that the paper proposes a novel task (AED) and approach (PrObe), the AED task is very relevant for few-shot imitation learning, it introduces new challenges, and PrObe contains novel components. The paper is well written, and evaluation seems correct. This AC agrees with these evaluations.

The paper needs some minor improvements, mainly to clarify benchmark details, and possibly shuffle some details between supplementary and main paper, to make sure the important details are in the main paper, and the mathematical descriptions of tasks and models. The rebuttal and discussion provide detailed feedback to improve the paper.

Overall this paper is considerably above the acceptance threshold, and should be accepted.

Reviews: review: paper_summary: paper_summary: This paper proposed a task Adaptable Error Detection (AED) that attempts to perform online behavior error detection for FSI policies, it advocate three main challenges comparing to FSAD: novel environment, no notable change for behavior error that AED tries to detect, and it has to be conduct simultaneously with policy execution to ensure timely error detection and damage control. The AED benchmark is provided, and a AED method, PrObe is proposed to solve AED through error prediction based on fusion of task embedding and flow pattern generated from history of rollout. The PrObe achieve good performance compare to baselines, and paper provided comprehensive studies analyzing PrObe error detection timing, it's embedding pattern, the importance of components, the stability of error detection, and provided pilot error correction to demonstrate practicality of the AED task.
strength_weakness: strength_weakness: The writing has improvement room:
1. line 161 - line 162: I am confused: if frame level label is not available, how come the measurement is averaging over actions?
2. The function of demonstration data is not clearly described in section 5.2, more explanation is helpful.

The AED benchmark is not well described in the writing.
questions: questions: See above.
limitations: limitations: The limitation in terms of proposed PrObe is discussed through analysis experiments, which make sense to me.
paper_summary: paper_summary: This paper introduces Adaptable Error Detection (**AED**) within Few-Shot Imitation (**FSI**) tasks, a critical yet underexplored area in robotics and AI. The authors establish a novel benchmark for assessing AED methods and present **PrObe**, designed specifically for this task. Through comprehensive evaluations, the paper demonstrates that PrObe outperforms baseline models and shows robustness across different FSI policies.
strength_weakness: strength_weakness: W1. While the textual descriptions in the Preliminaries section provide a solid understanding of the DC policy, incorporating mathematical notation would significantly enhance clarity and comprehension. Specifically, the paper could benefit from the following mathematical formulations:  
 1. The contents of history $h$, detailing what it encapsulates
 2. The feature encoder and task-embedding network computing the history, demonstration, and task-embedding features
 3. Task-embedding network padding the sequences
 4. The actor policy, e.g, $\pi(a|o_t, f_h, f_{\zeta})$
 5. The inverse dynamics module
 6. NLL and MSE objective in this particular setting

W2. The average difference metric presented in Figure 4, which compares each method's performance to the second-best performing method, may be inadequate. This approach is only sensible when comparing two methods. A more informative assessment might measure performance gains relative to the worst-performing method, or performance losses relative to the best-performing method. Alternatively, using a standardized baseline or oracle for comparisons could provide a clearer and more meaningful evaluation.

W3. PrObe is only shown to be effective for evaluating policies trained on image data. As a primary example, the authors motivate the FSI ED problem setting on robotic tasks. Although sensor data cannot always fully capture the properties and locations of separate objects, policies for robotic manipulation tasks are often trained on proprioceptive data—such as joint angles and torques—to achieve better precision. Image data, though rich in environmental context, can sometimes be unavailable or impractical due to factors like occlusions, lighting conditions, and higher training costs. The inherent differences between image-based and proprioceptive data mean that the latent space characteristics and the patterns critical for error detection would vary. Image-based models capture visual features like shapes and spatial relationships, whereas proprioceptive-focused models emphasize dynamics and kinematic features, including the robot’s mechanical interactions. While there's no inherent reason to doubt PrObe’s capability with different data modalities, the effectiveness of its pattern extractor when applied to latent features learned from proprioceptive data remains uncertain without empirical validation. Conducting experiments with proprioceptive sensor data and demonstrating successful results would further validate the robustness, generality, and applicability of PrObe.

### Minor Improvements

1. The second and third contributions—introducing a novel benchmark and PrObe—are blended together; these could be separate for clarity.
2. Please add a reference in the main paper that the limitations are in Appendix F.
3. Line 125 “semantically” refers to language or linguistics, which doesn’t seem to be the intended use of the word here
4. Line 133 “learns implicit mapping” → “learns an implicit mapping”
5. Line 136 “current history” → “the current history”
6. Line 154 “and few” → “and a few”
7. Line 168 “high controllable” → “highly controllable”
8. Line 174 “and few” → “and a few”
9. Line 192 “from agent’s” → “from the agent’s”
10. Line 205 “take as input the history features $f^h$” →“takes the history features $f^h$ from … as input”
11. Footnote on page 4 “thet” →”they” 
12. Line 614 “are are” → “are”
13. Line 661 “*Pick & Plcae*” → “*Pick & Place*”
questions: questions: Q1. Why did the authors choose the main conference track instead of the Datasets & Benchmarks track if they propose a benchmark?  
Q2. What is the intervention method employed at critical moments to the agent’s policy to collect failed rollouts?  
Q3. Why were TaskEmbED and LSTMED chosen among the 4 baselines for embedding visualization?  
Q4. Is AED only applicable for error detection on demonstration-conditioned policies or does it extend to more general policies?  
Q5. Is PrObe also able to evaluate policies that have been trained for robotic manipulation on proprioceptive data, not image data?
limitations: limitations: The authors have extensively discussed the limitations of their work.
paper_summary: paper_summary: The paper aims to address the adaptable error detection problem, i.e., detecting the error behavior of a robot policy in an unseen environment. Solving the problem ensures that the robot is stopped before performing behavior that causes disruptions to the surroundings.

The AED problem introduces three challenges: the abnormal behavior in the unseen environment is not observed in the training data and common anomaly detection method cannot be used; with complex background, there is no noticeable change between frames and this makes it difficult to indicate when the error occurs; AED is required to terminate the policy timely once the error occurs. The three challenges make it difficult to apply the current approaches on anormaly detection.

The paper creates a new benchmark specially designed for adaptable error detection, which consists of 322 base and 153 new environments encompassing six indoor tasks and one factory task.

The paper proposes a PrObe architecture consists of a pattern extractor and a flow generation equipped with a pattern loss, a temporal contrastive loss and a classification loss to predict the error probability.

The paper conducts experiments on the proposed benchmark and achieves the top 1 counts, average ranking, and average performance difference. Extensive ablative study justifies the effectiveness of the design choices.
strength_weakness: strength_weakness: One main contribution of the paper is that error detection could avoid the robots making disrupted behavior in the real-world environment. Thus, the authors may need a real-world environment that contains some disruption scenarios and test the method in such environments. Even the most realistic simulation may not simulate some real factors such as inaccurate control program of the robot arm.
questions: questions: On line 207-208, normalizing by L2 norm could make all the features have the same norm but how could this mitigate the biases? Even the unit vector may only learn the background information in the visual inputs.

On line 212-215, the authors need to note that the task embeddings are extracted using prior works, which is introduced only in the experiments.

Could the authors provide more details or some examples on how to collect the positive and negative pairs for L_{tem}?
limitations: limitations: None

Comments: comment: Thank you for your feedback! We are pleased that our responses have addressed your concerns. We will revise the paper following your valuable suggestions.

We are also grateful to the reviewers for their support and for increasing their ratings of our work. Thank you again.
Thanks for addressing my questions, i am happy to increase my score to 7.
Dear Reviewer qV6D,

Thank you once again for your valuable reviews and comments. We have provided additional details to address the concerns you raised.

Since the discussion period  **ends in two days**, we would appreciate knowing if there are any unresolved issues that require further clarification. If so, we would be happy to discuss them with you. Otherwise, if all your concerns have been addressed, we kindly ask you to consider adjusting your evaluation score accordingly.

Thank you again for your time and consideration.
Dear Reviewer SywQ,

Thank you once again for your valuable reviews and comments. We have provided additional details to address the concerns you raised.

Since the discussion period  **ends in two days**, we would appreciate knowing if there are any unresolved issues that require further clarification. If so, we would be happy to discuss them with you. Otherwise, if all your concerns have been addressed, we kindly ask you to consider adjusting your evaluation score accordingly.

Thank you again for your time and consideration.
We are grateful for the reviewer's prompt and detailed feedback, which provided valuable advice during our discussion.

We are also pleased that our responses have addressed all of the reviewer's concerns and questions. We would like to thank the reviewer once again for their high recognition of our work.
I appreciate the authors’ efforts in addressing my concerns and answering the questions.

**W2**. I do think the results that you provided in this format are more coherent. I suggest you select one of them and swap out the current one in Figure 4. I would say that including both is unnecessary as they infer the same thing and it would just take up more space. I think the comparison w/ the worst method is more informative, as the scale is larger.

**W3**. Thank you for the explanation. According to your reasoning, it indeed seems complicated to incorporate proprioceptive data. I do believe that camera footage for error detection is appropriate and sufficient, and image-based learning, after all, generally poses a greater challenge.

I thank the authors for their work. All my concerns and inquiries have been addressed. I will keep the current score. A higher score would necessitate a more high-impact paper 1) tackling a broader setting, 2) surpassing prior methods by a landslide, or 3) having experiments conducted in many simulation environments or even physical robots.