SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 14:  $\widehat{I}(a_t^I; a_{t-1}^E | a_t^E)$  evaluation results of regularization methods and baselines. Since the conditional MI can be decomposed into two MI terms as described in Section 4.1.1, we estimate two terms with MINE estimator respectively. We use the policy of the last iteration for each method to generate policy actions.  $(\downarrow)$ ,  $(\uparrow)$  indicates whether the averaged CMI score of each method is lower than the score of BC or not.
D.6 Details on the leakage-performance correlation
We first clarify the relationship between the number of training data and HSCIC scores in Section 5.1. To clarify, we plot the correlation into Figure 4. The correlation indicates that when the training data is insufficient, the problem of past action information leakage becomes more severe. This phenomenon can be attributed to the higher risk of overfitting in cases with smaller training instances, which can lead to the capture of false causal relationships within the training data. These findings align with similar results reported in [8] (see Figure 4 in [8]).
Figure 4: Relationship between the number of training data and the normalized HSCIC score.
Furthermore, to provide more insights for understanding Figure 1, we plot all points that indicate the normalized HSCIC and the normalized score of each BC policy in Figure 5. In addition, we calculate the sample Pearson's correlation coefficients using every point of each task (the first 4 columns) and all tasks (the last column) on Table 15.
Figure 5: Scatterplot that shows a correlation between HSCIC estimates and performance. Each small point indicates HSCIC and the performance of each policy, and the diamond point  $(\diamond)$  indicates the mean point of each group (which consists of policies trained with the same dataset size).
hopper	walker2d	halfcheetah	ant	Total
-0.8358	-0.8163	-0.8564	-0.9273	-0.8130
Table 15: Sample Pearson's correlation coefficients between the normalized HSCIC and the normalized score.
This work primarily addresses the reduction of the pervasive past action leakage effect in imitation learning with observation histories. Notably, our findings suggest potential vulnerabilities that may be exploited by adversarial actors. For instance, malicious attackers could introduce the conditional dependencies into the system, leading to potentially catastrophic outcomes, such as repeating the same actions. This will produce potentially irreversible consequences on the system, especially for security-critic tasks such as autonomous driving and medical devices, or industrial automation. Therefore, understanding and mitigating this risk is crucial for the safe and efficient operation of such systems.
While our approach is easily applicable to real-world tasks, the range of experiments in this paper is limited to robotics simulation. A broader scope of experiments, including more realistic observation settings like image-based or noise-inclusive observations, could provide a stronger argument for the effectiveness and real-world applicability of our method. Additionally, our method assumes that control-relevant information is sufficiently captured in the observation histories. However, this assumption might not hold in some real-world scenarios where key control information is missing and not retrievable from the observation histories. In such cases, the learning model requires additional information, inductive bias or expert knowledge to function effectively.
For our experiments, we used a cluster system with 20 nodes that have the following system specs:
- CPU: Intel i7-9700K CPU (3.60GHz)
- GPU: TITAN Xp (VRAM 12 GB)
Our code has been developed based on publicly available code repositories released under MIT licenses. The code used to generate our experimental results also follows to the MIT license. For more detailed information, see the README.md and LICENSE files included in our code files.