Learning of Sophisticated Curriculums by viewing them as Graphs over Tasks
Authors: Lucas Willems, Yoshua Bengio
Keywords: learning, curriculum learning, reinforcement learning
TL;DR: We present a new algorithm for learning by curriculum based on the notion of mastering rate that outperforms previous algorithms.
Abstract: Curriculum learning consists in learning a difficult task by first training on an easy version of it, then on more and more difficult versions and finally on the difficult task. To make this learning efficient, given a curriculum and the current learning state of an agent, we need to find what are the good next tasks to train the agent on.
Teacher-Student algorithms assume that the good next tasks are the ones on which the agent is making the fastest progress or digress. We first simplify and improve them. However, two problematic situations where the agent is mainly trained on tasks it can't learn yet or it already learnt may occur.
Therefore, we introduce a new algorithm using min max ordered curriculums that assumes that the good next tasks are the ones that are learnable but not learnt yet. It outperforms Teacher-Student algorithms on small curriculums and significantly outperforms them on sophisticated ones with numerous tasks.