  Humans can leverage hierarchical structures to split a task into sub-tasks
and solve problems efficiently. Both imitation and reinforcement learning or a
combination of them with hierarchical structures have been proven to be an
efficient way for robots to learn complex tasks with sparse rewards. However,
in the previous work of hierarchical imitation and reinforcement learning, the
tested environments are in relatively simple 2D games, and the action spaces
are discrete. Furthermore, many imitation learning works focusing on improving
the policies learned from the expert polices that are hard-coded or trained by
reinforcement learning algorithms, rather than human experts. In the scenarios
of human-robot interaction, humans can be required to provide demonstrations to
teach the robot, so it is crucial to improve the learning efficiency to reduce
expert efforts, and know human's perception about the learning/training
process. In this project, we explored different imitation learning algorithms
and designed active learning algorithms upon the hierarchical imitation and
reinforcement learning framework we have developed. We performed an experiment
where five participants were asked to guide a randomly initialized agent to a
random goal in a maze. Our experimental results showed that using DAgger and
reward-based active learning method can achieve better performance while saving
more human efforts physically and mentally during the training process.
