SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this paper, we have proposed a doubly robust augmented estimator (DRaE) to tackle the mismatch of dynamics distributions in sample transfer under a more general sparse-reward meta-RL setting, where dynamics and reward functions can both vary across different tasks. While DRaE was established with an optimal dynamics importance weight by minimizing the MSE between estimated and true values of the value function, we proposed a tractable interval-based approximation that guaranteed to cover the optimum. We further developed a doubly robust augmented transfer (DRaT) algorithm for transferring informative samples across tasks in meta-RL. Experimental evaluation on several MuJoCo locomotion tasks demonstrated the effectiveness of our proposed DRaT algorithm.
The major limitation of our method is that it requires computation of the DR estimate and training of the dynamics prediction network for tasks, which may incur additional computational complexity. Besides, like other hindsight-based methods, the reward function is required for relabeling and selection of informative trajectories, which might be inaccessible in some extreme scenarios.
This work was supported in part by the National Natural Science Foundation of China under Grants 62250055, 61931023, T2122024, 62125109, 61932022, 61972256, 61971285, 61831018, 62120106007, 62320106003, and 62371288, in part by the Program of Shanghai Science and Technology Innovation Project under Grant 20511100100, and in part by the Shanghai Rising-Star Program under Grant 20QA1404600.