text: 187,154,1291565580827336705,2577596593,Chelsea Finn,"Want your robot to explore intelligently? We study how to learn to explore & introduce a *efficient* meta-learning method that can lead to optimal exploration. Paper: <LINK> w Evan Liu, Raghunathan, Liang @StanfordAILab Thread👇🏼(1/5) <LINK> Prior meta-RL methods either (a) optimize exploration & execution end-to-end w.r.t. reward (e.g. RL^2, VariBAD), or (b) leverage principled but suboptimal strategies (e.g. PEARL). The former is particularly hard, as it leads to a chicken-and-egg optimization problem. (2/5) <LINK> Turns out you can break this coupling by training a task-conditioned execution policy, and training the exploration policy to recover task-relevant information.  This is consistent with the end-to-end objective *and* substantially more efficient! (3/5) <LINK> With this approach, DREAM can learn an exploration strategy that navigates a 3D environment from pixels to go “read” a sign that carries info about the task. (and then execute the task using that info) (4/5) <LINK> In comparison with state-of-the-art meta-RL methods, this approach can better scale to challenging meta-RL problems such as 3D visual object navigation. See the paper for more experiments & theoretical analysis. (5/5) <LINK>",https://arxiv.org/abs/2008.02790,"The goal of meta-reinforcement learning (meta-RL) is to build agents that can quickly learn new tasks by leveraging prior experience on related tasks. Learning a new task often requires both exploring to gather task-relevant information and exploiting this information to solve the task. In principle, optimal exploration and exploitation can be learned end-to-end by simply maximizing task performance. However, such meta-RL approaches struggle with local optima due to a chicken-and-egg problem: learning to explore requires good exploitation to gauge the exploration's utility, but learning to exploit requires information gathered via exploration. Optimizing separate objectives for exploration and exploitation can avoid this problem, but prior meta-RL exploration objectives yield suboptimal policies that gather information irrelevant to the task. We alleviate both concerns by constructing an exploitation objective that automatically identifies task-relevant information and an exploration objective to recover only this information. This avoids local optima in end-to-end training, without sacrificing optimal exploration. Empirically, DREAM substantially outperforms existing approaches on complex meta-RL problems, such as sparse-reward 3D visual navigation. Videos of DREAM: this https URL ","Decoupling Exploration and Exploitation for Meta-Reinforcement Learning