Before revision:  In this paper , we propose a hybrid offline meta-RL algorithm, which uses offline data with rewards to meta-train an adaptive policy, and then collects additional unsupervised online data, without any ground truth reward labels , to bridge this distribution shift problem. Our method uses the offline data to learn the distribution of reward functions, which is then sampled to self-supervise reward labels for the additional online data. By removing the need to provide reward labels for the online experience, our approach can be more practical to use in settings where reward supervision would otherwise be provided manually .
After revision:  In this paper , we propose a hybrid offline meta-RL algorithm, which uses offline data with rewards to meta-train an adaptive policy, and then collects additional unsupervised online data, without any ground truth reward labels , to bridge this distribution shift . By not requiring reward labels for the online experience, our approach can be more practical to use in settings where reward supervision would otherwise be provided manually .