  Curriculum learning is a training strategy that sorts the training examples
by some measure of their difficulty and gradually exposes them to the learner
to improve the network performance. In this work, we propose two novel
curriculum learning algorithms, and empirically show their improvements in
performance with convolutional and fully-connected neural networks on multiple
real image datasets. Motivated by our insights from implicit curriculum
ordering, we introduce a simple curriculum learning strategy that uses
statistical measures such as standard deviation and entropy values to score the
difficulty of data points for real image classification tasks. We also propose
and study the performance of a dynamic curriculum learning algorithm. Our
dynamic curriculum algorithm tries to reduce the distance between the network
weight and an optimal weight at any training step by greedily sampling examples
with gradients that are directed towards the optimal weight. Further, we also
use our algorithms to discuss why curriculum learning is helpful.
