SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this section, we will present some additional plots to complement the main text and provide further insights.
Table 4: Result on V-D4RL distracting datasets. Embodiment datasets are marked on the left, and the demonstration datasets are chosen to be the expert dataset for each task in the origin environment. Values are averaged over 100 trajectories and reported as accumulated reward divided by 10, as suggested in the V-D4RL paper.
BCO(0)	AIME
walker-easy	2.10 ± 0.88	4.73 ± 2.54
walker-medium	2.15 ± 0.87	3.94 ± 0.99
walker-hard	2.15 ± 0.97	4.16 ± 1.98
walker-mix	2.12 ± 0.86	3.81 ± 2.07
walker-total_mix	2.15 ± 0.71	12.66 ± 4.51
cheetah-total_mix	16.61 ± 7.28	32.40 ± 14.52
Additional to Figure 2 and Figure 3, we also provide detailed profile plots in Figure 6 and Figure 7 as recommended in [52]. We can see that AIME is normally more stable w.r.t. the performance by having a smaller decay region. It is clearly shown on such tasks as walk  $\rightarrow$  walk and run  $\rightarrow$  run on Walker where BCO(0)-MDP has some trails with very low performance, while all variants of AIME maintain decent performance.
Figure 6: Performance distributions of each method on Walker tasks.
We also present some representative training curves of AIME's phase 2 from our experiments in Figure 8. The first three figures show the transfer from the mix dataset to the run task in the three settings which are the typical success cases of AIME. During the course of training, ELBO is maximised towards convergence and the MSE between the generated actions and the true actions decreases. We can also see that for the MDP and LPOMDP settings, the converged ELBO is lower than the ELBO when evaluated with the true action sequence, indicating there is still space for improvement. However, for the Visual setting, the converged ELBO exceeds the one with true actions, which should be attributed to the over-fitting of the world model from phase 1. The last three figures
Figure 7: Performance distributions of each method on Cheetah tasks.
show the transfer from the random dataset to the three tasks in the Visual settings which we consider as failure cases. For the stand and walk tasks, none of the metrics are converging. For the run task, we can observe a severe over-fitting starting from the beginning of the training, and the MSE keeps increasing. We conjecture these are all due to the less well-trained world models.
AIME-MDP mix -> run training curve
AIME-LPOMDP mix -> run training curve
AIME-Visual mix -> run training curve
AIME-Visual random -> stand training curve
AIME-Visual random -> walk training curve
AIME-Visual random -> run training curve
Figure 8: Samples of training curve in phase 2 of AIME. The first three showcase the typical successful training curves, while the remaining three demonstrate the failure cases. The true_action is referring to evaluating the trajectories with the true action sequence.