In Your Pace: Learning the Right Example at the Right Time

1 INTRODUCTION . Teaching complex tasks to humans and animals can be difficult . Often such tasks can not be grasped by the learners , or ” students ” , immediately , and need to be broken down into simpler problems . Therefore , in order to teach complex tasks , teachers are often required to create a curriculum . The curriculum imposes some order on the learning task ; it introduces different concepts at different times , hence exploiting previously learned concepts in order to ease the abstraction of new ones . Imposing a curriculum in order to speed up learning is widely used in the context of human learning , and also routinely used in animal training ( Skinner , 1958 ; Pavlov , 2010 ; Krueger & Dayan , 2009 ) . In many traditional machine learning approaches , known as supervised learning , a target function is estimated by using a set of labeled examples . The examples can be thought of as given by a teacher while the learning algorithm can be thought of as a student . The field of curriculum learning ( CL ) , which is motivated by the idea of a curriculum in human learning , attempts at imposing some structure on the labeled data . Such structure essentially relies on a notion of ” easy ” and ” hard ” examples and utilizes this distinction in order to teach the student how to generalize easier concepts before harder ones . Empirically , the use of CL has been shown to accelerate and improve the learning process ( e.g . Selfridge et al. , 1985 ; Bengio et al. , 2009 ) . In this work , we aim at extending the understanding of CL in the context of deep neural learning . More specifically , we wish to understand to what extent curriculum can improve the accuracy and convergence rate of deep neural networks . The main challenge in making CL practical is , arguably , finding a way to construct a good curriculum for a newly unseen database . In order to do so , we investigate two ideas motivated by transfer learning and bootstrapping respectively . When establishing a curriculum for human students , teachers need to arrange the material in a way that will present simple concepts before harder ones , so that the abstraction of simple ideas can help the student grasp more complex ones ( Hunkins & Ornstein , 2016 ) . However , sorting the concepts by difficulty is not sufficient . The teacher also needs to attend to the pace by which the material is presented – going over the simple ideas too fast may lead to more confusion than benefit , while moving along too slowly may lead to boredom and unproductive learning ( Hunkins & Ornstein , 2016 ) . These principles can also be beneficial when the learner is a neural network . Specifically , formalizing and generalizing what was implicitly done in Weinshall et al . ( 2018 ) , we decompose the problem of CL and define two separate - but closely related - functions . The first function , termed scoring function , determines the ” hardness ” or ” complexity ” of each example in the data . The scoring function enables us to sort the data by concept difficulty , allowing us to present to the network the easier ( and presumably simpler ) examples first . The underlying assumption is that generalization from the easier examples can simplify the learning of harder examples in the data . The second function , termed pacing function , determines the pace by which data is presented to the network . The pace depends on both the data itself and the learner . In our work , we analyze several scoring and pacing functions , investigating their inter-dependency and presenting ways to combine them in order to achieve faster learning and better generalization . The main challenge is , arguably , how to obtain an effective scoring function without additional human supervision . To this end we investigate two approaches , each providing a different estimator for the ideal scoring function : ( i ) Knowledge transfer . The first scoring function is based on transfer learning from networks trained on the large and versatile Imagenet dataset ( Deng et al. , 2009 ; Weinshall et al. , 2018 ) . ( ii ) Bootstrapping . The second scoring function is based on self-tutoring - we train the network once without curriculum , then use the resulting classifier to rank the training data in order to train the same network again from scratch . Both scoring functions are shown in Section 3 to speed up learning and improve the generalization of neural networks . In many approaches , including Self-Paced Learning ( SPL ) , Active-Learning and hard example mining ( Kumar et al. , 2010 ; Schein & Ungar , 2007 ; Shrivastava et al. , 2016 ) , the mini-batches which presented to the learner model are as sampled dynamically , based at each time point on the current hypothesis of the model . While in some contexts these approaches are beneficial ( Chang et al. , 2017 ; Zhang et al. , 2017 ) , they are based on the knowledge of the student at a specific time point . While a student can report what is easy/hard for it right now , it might be oblivious to some aspects of the bigger problem at hand , ignoring concepts which if learned early , could prove helpful in a later time . In the context of linear regression loss , Weinshall et al . ( 2018 ) showed that such distinction indeed holds : while it is beneficial to prefer points with lower loss with respect to the target hypothesis as suggested by CL , it is on the other hand beneficial to prefer points with higher loss with respect to the current hypothesis in agreement with hard data mining ( Shrivastava et al. , 2016 ) and boosting , contrary to SPL . To examine this somewhat confusing point , we have implemented a simplified version of the procedure described above , where the scoring function is based on the loss of the training points with respect to the current hypothesis , both in ascending and descending orders . These variants of SPL and hard example mining respectively learn slower and reach lower final accuracy when compared to self-taught , throughout all of our experiments . We have also investigated three pacing functions . ( i ) Fixed exponential pacing presents the learner initially with a small percentage of the data , increasing the amount exponentially every fixed number of learning iterations . ( ii ) Varied exponential pacing allows the number of iterations in each step to vary as well . ( iii ) Single-step pacing is a simplified version of the first protocol , where mini-batches are initially sampled from a fixed fraction of the data that includes the easiest examples , after which mini-batches are sampled from the whole data as usual . We show that the three functions have comparable performance , and analyze the complexity of their use . Previous work . While remaining in the fringes of machine learning , there has been some recent work on CL and its applications . Bengio et al . ( 2009 ) introduced the idea of CL for machine learning algorithms , showing simple examples where CL benefits learning . Weinshall et al . ( 2018 ) proved that CL boosts the speed of convergence in the convex case of linear regression . Otherwise most prior art is empirical , and almost always ranking by difficulty ( i.e. , the scoring function defined above ) is provided by the user based on prior knowledge ( in other words , supervision ) as in Jesson et al . ( 2017 ) . In a closely related line of works , a pair of teacher and student networks are trained simultaneously , where mini-batches for the student network are sampled dynamically by the teacher , based on the student ’ s output in each time point ( Jiang et al. , 2018 ; Fan et al. , 2018 ) . As opposed to our method , these works base the curriculum on the current hypothesis of the students , and achieve better performance for corrupted ( Jiang et al. , 2018 ) or smaller ( Fan et al. , 2018 ) datasets , instead of improved generalization on the original dataset . Our contribution , with respect to this previous work , is to provide a formal definition of CL algorithms by way of 2 functions for scoring and pacing , analyze and comparatively evaluate these functions , and show how CL can benefit learning in CNNs even without human supervision about the ranking of examples by difficulty and in a problem-free manner . 2 CURRICULUM LEARNING . Curriculum learning deals with the question of how to use prior knowledge about the difficulty of the training examples , in order to sample each mini-batch non-uniformly and thus boost the rate of learning and the accuracy of the final classifier . The paradigm of CL is based on the intuition that it helps the learning process when the learner is presented with simple concepts first . 2.1 NOTATIONS AND DEFINITIONS . Let X = { ( xi , yi ) } Ni=1 denote the data , where xi ∈ Rd denotes a single data point and yi ∈ [ K ] its corresponding label . Let Fθ : Rd → [ K ] denote the target classifier ( or learner ) , and mini-batch B ⊆ X denote a subset of X . In the most common training procedure , which is a robust variant of Stochastic Gradient Descent ( SGD ) , Fθ is trained sequentially when given as input a sequence of mini-batches [ B1 , ... , BM ] ( Shalev-Shwartz & Ben-David , 2014 ) . The common approach – denoted vanilla in the following sections – samples each mini-batch Bi uniformly from X . Both in the common approach and in our work , the size of each mini-batch remains constant , to be considered as a hyper-parameter defining the learner . We measure the difficulty of point xi by its minimal loss with respect to the set of optimal hypotheses under consideration . We define a scoring function ( or a ” difficulty ” function ) to be any function f : X → R , and say that example ( xi , yi ) is more ” difficult ” than example ( xj , yj ) if f ( xi , yi ) > f ( xj , yj ) . Choosing f is the main challenge of CL , as it encodes the prior knowledge of the teacher . We define a pacing function to be a function gFθ : [ M ] → [ N ] , which may depend on the learner Fθ . The pacing function is used to determine a sequence of subsets X ′ 1 , ... , X ′ M ⊆ X , of size |X′i| = gFθ ( i ) , from which { Bi } Mi=1 are sampled uniformly . In CL the i-th subset X ′ i includes the first gFθ ( i ) elements of the training data when sorted by the scoring function f in an ascending order . Although the choice of the subset can be encoded in the distribution from which each Bi is sampled , adding a pacing function simplifies the exposition and analysis . 2.2 CURRICULUM LEARNING METHOD . Together , each scoring function f and pacing function gFθ define a curriculum . Any learning algorithm which uses the ensuing sequence [ Bi ] Mi=1 is a curriculum learning algorithm . We note that in order to avoid bias when picking a subset of the N examples for some N , it is important to keep the sample balanced with the same number of examples from each class as in the training set . Pseudo-code for the CL algorithm is given in Alg . 1 . In order to narrow down the specific effects of using a scoring function based on ascending difficulty level , we examine two control conditions . Specifically , we define 2 additional scoring functions and corresponding algorithms : ( i ) The anti-curriculum algorithm uses the scoring function f ′ = −f , where the training examples are sorted in a descending order ; that results in presenting the harder examples before the easier ones . ( ii ) The random-curriculum algorithm ( henceforth denoted random ) uses a scoring function where the training examples are randomly sorted .

Summary: This problem of interest in this paper is Curriculum Learning (CL), in the context of deep learning in particular. CL refers to learning a non-random order of presenting the training examples to the learner, typically with easier examples presented before difficult ones, to guide learning more effectively. This has been shown to both speed up learning and lead to better generalization, especially for more challenging problems. In this paper, they claim that their contribution is to decompose the problem of CL into learning two functions: the scoring function and the pacing function, with the role of the former being to estimate the difficulty of each training example and the latter to moderate the schedule of presenting increasingly more challenging examples throughout training.