Curriculum Discovery through an Encompassing Curriculum Learning Framework

We describe a curriculum learning framework capable of discovering optimal curricula in addition to performing standard curriculum learning . We show that this framework encompasses existing curriculum learning approaches such as difficulty-based data sub-sampling , data pruning , and loss re-weighting . We employ the proposed framework to address the following key questions in curriculum learning : ( a ) what is the best curriculum to train a given model on a given dataset ? and ( b ) what are the characteristics of optimal curricula for different datasets and difficulty metrics ? We show that our framework outperforms competing stateof-the-art curriculum learning approaches in natural language inference and two other text classification tasks . Exhaustive experiments illustrate the generalizability of the discovered curricula across the datasets and difficulty metrics . 1 INTRODUCTION . Curriculum Learning ( CL ) is a technique in Machine Learning that mimics human education systems . To learn a complex subject , students must first learn the foundational and basic materials before learning more complex ones . Without a curriculum , learning may be intractable , inefficient and learners may never reach a full understanding of a topic due to lack of the required background knowledge . Machine learning optimization through stochastic gradient descent trains models by observing example data instances . Some data instances are harder than others and require background knowledge , which could be acquired by observing and being adept at easier examples before harder ones . CL techniques seek to order examples according to their difficulty for training to generate better models . It has been shown that CL improves performance in solving harder tasks , or in cases of limited or noisy data ( Wu et al. , 2021 ) . CL research has made significant progress in the last decade through the work of Bengio et al . ( 2009 ) , although the principle of ordering training samples from easier to harder was introduced in the 1990s ( Elman , 1993 ; Sanger , 1994 ; Rohde & Plaut , 1999 ) . A curriculum can be defined by ordering training samples based on their difficulty to learn . Given a measure of difficulty , there are different types of CL approaches for ordering the data : sub-sampling techniques , which sample the easiest or hardest data points at every iteration for training ( Zhou et al. , 2020 ; Bengio et al. , 2009 ; Xu et al. , 2020 ; Guo et al. , 2018 ; Platanios et al. , 2019 ) , sample weighting techniques , which use the complete data at every iteration but weight data points differently according to their difficulty ( Castells et al. , 2020 ; Kumar et al. , 2010 ; Jiang et al. , 2015 ; 2018 ; Zhou et al. , 2020 ; Yang et al. , 2019 ) , and pruning techniques , which prune the hard or noisy samples from the dataset prior to training ( Northcutt et al. , 2021 ; Guo et al. , 2018 ) . Sub-sampling methods can be cumulative , exclusive or a combination of both . Cumulative approaches add new data points to the ones that have previously been used for training . On the other hand , exclusive approaches create a new subset of data at every training stage . Such methods can introduce samples from easy to hard ( Bengio et al. , 2009 ; Kumar et al. , 2010 ) or hard to easy , which can be an effective learning strategy in some specific tasks ( Kocmi & Bojar , 2017 ; Zhang et al. , 2018 ; 2019 ) . In addition , CL methods may impose a curriculum by adjusting model ’ s capacity according to the difficulty of their inputs ( Karras et al. , 2018 ; Sinha et al. , 2020 ; Morerio et al. , 2017 ) or schedule the order of tasks in the context of multi-task learning ( Caubrière et al. , 2019 ; Sarafianos et al. , 2017 ; Florensa et al. , 2017 ) . Other CL approaches such as ( Zhou et al. , 2020 ; Saxena et al. , 2019 ) use O ( n ) , where n is the number of training samples , extra parameters for learning curricula . Current CL approaches calculate difficulty scores for training samples based on the loss of a trained model ( Xu et al. , 2020 ; Wu et al. , 2021 ) , trainable parameters that weight samples ( Kumar et al. , 2010 ; Jiang et al. , 2015 ; Castells et al. , 2020 ) , loss value during training ( Wu et al. , 2021 ) , moving average of loss during training ( Zhou et al. , 2020 ) , transformations of the loss during training ( Jiang et al. , 2018 ; Castells et al. , 2020 ) , and consistency in the correct classification of samples ( Amiri et al. , 2017 ; Xu et al. , 2020 ) . The difficulty scores will then be used to order samples for training . In this work , we focus on an alternative approach , where examples can be ordered based on prior knowledge about their difficulty , e.g. , object shape or orientation in image classification ( Bengio et al. , 2009 ) . We will demonstrate a CL framework that encompasses existing CL approaches through an effective and flexible data partitioning and weighting scheme . Our framework provides a new paradigm for selecting an ordering strategy . Instead of a pre-determined strategy , the framework allows searching over the curriculum space to identify the best curriculum for a particular dataset and model . It partitions training data into several groups of training samples , e.g . { easy , medium , hard } samples , according to a difficulty scoring function . Parameterized weighting functions will then be defined for each data group to specify the weight of its samples during training . Each weight function is controlled by two parameters , which can be set empirically or adjusted using Bayesian optimization . In addition , the framework discovers optimal curricula by optimizing the parameters of the weight functions ( only 2 parameters per function ) using a hyper-parameter optimization algorithm . Furthermore , the curricula identified through this search provide useful insight about the dataset , such as the relative importance of different samples or knowledge dependency between samples , e.g . which samples should be learned first . We begin by explaining our framework and showing how it is capable of approximating existing CL approaches . Then , in the context of the proposed framework , we investigate curriculum discovery , characteristics of discovered curricula , and generalizability of curricula with respect to their datasets . 2 CURRICULUM LEARNING FRAMEWORK . 2.1 WEIGHTING FUNCTIONS . We define the curriculum using generalized logistic functions ( Richards , 1959 ) of the form : w ( t ; r , s ) = 1 1 + exp ( −r ∗ ( t− s ) ) , ( 1 ) where r ∈ R is the rate-of-change parameter , which specifies how fast the weight can increase ( r > 0 ) or decrease ( r < 0 ) ; t ∈ [ 0 , 1 ] is the training progress ( iteration number divided by max iterations ) ; and s ∈ R shifts the pivot weight of the logistics function ( w ( . ) = .5 ) to the left or right such that at t = s the weight is 0.5 . Figure 1a illustrates the effect of varying these parameters . Greater absolute values for the rate parameter enforce greater slope and faster rate of changes in weights , while greater values of the shift parameter enforce longer delays in reaching the pivot weight of 0.5 . These parameters provide flexibility in controlling sample weights during training , which is critical for deriving effective curricula . Given the weighting function , we encode prior knowledge about sample difficulty into the training paradigm of our CL framework by splitting the data into several partitions of increasing difficulty according to the prior knowledge . Then , we define a weight function by a pair of parameters ( r , s ) for each partition , The proposed sample weighting framework provides flexibility in sample-ordering according to difficulty . It can be learned to approximate existing predetermind curricula . For example , it is possible to begin with only easy instances or only difficult instances ( as an anti-curriculum ) or a combination of both . Figure 1b shows a specific configuration for the logistic functions based on standard CL ( Bengio et al. , 2009 ; Kumar et al. , 2010 ) , where training starts with easier samples and gradually proceeds with harder ones . In addition , our approach enables discovering new data-driven curricula from data . Prior to training , the difficulty scores of samples are computed and each sample is assigned to a difficulty class c ∈ { easy , medium , hard } , see §2.2 . In addition , the hyper-parameters of our three weight functions are optimized prior to training and kept fixed throughout training { ( re , se ) , ( rm , sm ) , ( rh , sh ) } . During training , the weighted loss is computed as follows . l̂i = w ( t ; rc , sc ) ∗ li ( 2 ) Where li is the unweighted and instantaneous loss of instance i , l̂i is the weighted loss , t is the current training iteration divided by the maximum number of iterations , c is the difficulty class of instance i , and ( rc , sc ) are the corresponding rate and shift parameters for the difficulty class c . 2.2 SCORING FUNCTIONS . Ground-truth labels for many datasets are often obtained through human annotation and crowdsourcing . This is achieved by collecting multiple annotations per data sample and aggregating the results , typically by majority voting . We use sample-level annotator disagreement to define a difficulty score for each sample using Shannon entropy ( Shannon , 2001 ) , where higher disagreement among annotators corresponds to higher sample difficulty . Entropy is a natural measure of difficulty ( for human population ) and may serve as a reliable prior knowledge for partitioning data by difficulty . Entropy of each sample xi is calculated as H ( xi ) = − ∑ l pc log pc where c is a class category and pc is the fraction of annotators who chose label c for the sample . The use of entropy is supported by Nie et al . ( 2020 ) , who studied the correlation between human agreement and model performance and reported a consistent positive correlation between model accuracy and level of human agreement , showing that model performs better on samples with a high level of agreement . Furthermore , training loss contains valuable information about difficulty with respect to the model ( learner ) , which may be different among architectures and tasks , and indicative of the model ’ s specific needs . However , loss at a particular step ( e.g. , final loss ) is dictated by the stochastic gradient descent and mini-batching dynamics and therefore is not a good indicator of difficulty ( Zhou et al. , 2020 ; Wu et al. , 2021 ) . Using a baseline model , trained with no curriculum and with default hyperparameters , we collect the loss values of all training instances at intervals of 0.5 epochs and use the average loss to estimate sample difficulty . In our experiments , we obtain twenty observations of the loss and compute the average for each instance . Such an estimation is supported by Zhou et al . ( 2020 ) who showed that the moving average of a sample ’ s instantaneous loss is a good metric for difficulty . Partitioning the data into three groups of increasing difficulty can be done using difficulty score percentiles , or 1-dimensional k-means clustering of the scores . Examples of data partitions using entropy and loss are shown in Figure 2 .

Summary: This paper proposes a new parameterized data partitioning and weighing scheme, that partitions data into three groups {easy, medium, hard} and determines a curriculum based on relative importance of different samples. They evaluate on three datasets (full and balanced versions) and show improvements over other CL approaches. The curriculum also provides interesting insights about the datasets and scale from smaller datasets to larger datasets.