Imitation learning has been commonly applied to solve different tasks in isolation. This usually requires either careful feature engineering, or a significant number of samples. This is far from what we desire: ideally, robots should be able to learn from very few demonstrations of any given task, and instantly generalize to new situations of the same task, without requiring task-specific engineering. In this paper, we propose a metalearning framework for achieving such capability, which we call one-shot imitation learning. Specifically, we consider the setting where there is a very large (maybe infinite) set of tasks, and each task has many instantiations. For example, a task could be to stack all blocks on a table into a single tower, another task could be to place all blocks on a table into two-block towers, etc. In each case, different instances of the task would consist of different sets of blocks with different initial states. At training time, our algorithm is presented with pairs of demonstrations for a subset of all tasks. A neural net is trained that takes as input one demonstration and the current state (which initially is the initial state of the other demonstration of the pair), and outputs an action with the goal that the resulting sequence of states and actions matches as closely as possible with the second demonstration. At test time, a demonstration of a single instance of a new task is presented, and the neural net is expected to perform well on new instances of this new task. Our experiments show that the use of soft attention allows the model to generalize to conditions and tasks unseen in the training data. We anticipate that by training this model on a much greater variety of tasks and settings, we will obtain a general system that can turn any demonstrations into robust policies that can accomplish an overwhelming variety of tasks. (left) Task-specific policy. This policy is trained to stack blocks into two towers, each of height 3. (right) A separate task-specific policy. This policy is trained to stack blocks into three towers, each of height 2. Figure 1 . Traditionally, policies are task-specific. For example, a policy might have been trained (through imitation or reinforcement learning) to stack blocks into towers of height 3, and then another policy would be trained to stack blocks into towers of height 2, etc. In this paper, we are interested in policies that are not specific to one task, but rather can be told (through a single demonstration) what the current new task is, and be successful at this new task. As illustrative examples, we would want to be able to provide a single demonstration of each task, and from that the one-shot policy would know what to do when faced with a new situation of the task, where the blocks are randomly rearranged. Videos of the illustrated tasks are available at http://bit.ly/one-shot-imitation.
Meta learning in inverse reinforcement learning focuses on imitation learning, like one-shot imitation learning REF .
One-Shot Imitation Learning
ArXiv
ArXiv
Computer Science