Original: M INIMAX C URRICULUM L EARNING : M ACHINE T EACHING WITH D ESIRABLE D IFFICULTIES AND S CHEDULED D IVERSITY
Revised: M INIMAX C URRICULUM L EARNING : M ACHINE T EACHING WITH D ESIRABLE D IFFICULTIES AND S CHEDULED D IVERSITY

Original: A BSTRACT
Revised: A BSTRACT

Original: We introduce and study minimax curriculum learning (MCL), a new method for adaptively selecting a sequence of training subsets for a succession of stages in machine learning.
Revised: We introduce and study minimax curriculum learning (MCL), a new method for adaptively selecting a sequence of training subsets for a succession of stages in machine learning.

Original: The subsets are encouraged to be small and diverse early on, and then larger, harder, and allowably more homogeneous in later stages.
Revised: The subsets are encouraged to be small and diverse early on, and then larger, harder, and allowably more homogeneous in later stages.

Original: At each stage, model weights and training sets are chosen by solving a joint continuous-discrete minimax optimization, whose objective is composed of a continuous loss (reﬂecting training set hardness) and a discrete submodular promoter of diversity for the chosen subset.
Revised: At each stage, model weights and training sets are chosen by solving a joint continuousdiscrete minimax optimization, whose objective is composed of a continuous loss (reﬂecting training set hardness) and a discrete submodular promoter of diversity for the chosen subset.

Original: MCL repeatedly solves a sequence of such optimizations with a schedule of increasing training set size and decreasing pressure on diversity encouragement.
Revised: MCL repeatedly solves a sequence of such optimizations with a schedule of increasing training set size and decreasing pressure on diversity encouragement.

Original: We reduce MCL to the minimization of a surrogate function handled by submodular maximization and continuous gradient methods.
Revised: We reduce MCL to the minimization of a surrogate function handled by submodular maximization and continuous gradient methods.

Original: We show that MCL achieves better performance and, with a clustering trick, uses fewer labeled samples for both shallow and deep models while achieving the same performance.
Revised: We show that MCL achieves better performance and, with a clustering trick, uses fewer labeled samples for both shallow and deep models.

Original: Our method involves repeatedly solving constrained submodular maximization of an only slowly varying function on the same ground set.
Revised: Our method involves repeatedly solving constrained submodular maximization of an only slowly varying function on the same ground set.

Original: Therefore, we develop a heuristic method that utilizes the previous submodular maximization solution as a warm start for the current submodular maximization process to reduce computation while still yielding a guarantee.
Revised: Therefore, we develop a heuristic method that utilizes the previous submodular maximization solution as a warm start for the current submodular maximization process to reduce computation while still yielding a guarantee.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Inspired by the human interaction between teacher and student, recent studies (Khan et al., 2011b;
Revised: Inspired by the human interaction between teacher and student, recent studies (Khan et al., 2011b;

Original: Basu & Christensen, 2013; Spitkovsky et al., 2009) support that learning algorithms can be improved by updating a model on a designed sequence of training sets, i.e., a curriculum.
Revised: Basu & Christensen, 2013; Spitkovsky et al., 2009) support that learning algorithms can be improved by updating a model on a designed sequence of training sets, i.e., a curriculum.

Original: This problem is addressed in curriculum learning (CL) (Bengio et al., 2009) where the sequence is designed by a human expert or heuristic before training begins.
Revised: This problem is addressed in curriculum learning (CL) (Bengio et al., 2009) where the sequence is designed by a human expert or heuristic before training begins.

Original: Instead of relying on a teacher to provide the curriculum, self-paced learning (SPL) (Kumar et al., 2010; Tang et al., 2012a; Supancic III &
Revised: Instead of relying on a teacher to provide the curriculum, self-paced learning (SPL) (Kumar et al., 2010; Tang et al., 2012a; Supancic III &

Original: Ramanan, 2013; Tang et al., 2012b) chooses the curriculum during the training process.
Revised: Ramanan, 2013; Tang et al., 2012b) chooses the curriculum during the training process.

Original: It does so by letting the student (i.e., the algorithm) determine which samples to learn from based on their hardness.
Revised: It does so by letting the student (i.e., the algorithm) determine which samples to learn from based on their hardness.

Original: Given a training set D = { ( x 1 , y 1 ) , . . .
Revised: Given a training set D = { ( x 1 , y 1 ) , . . .

Original: , ( x n , y n ) } of n samples and loss function L ( y i , f ( x i , w )) , where x i ∈ R m represents the feature vector for the i th sample, y i is its label, and f ( x i , w ) is the predicted label provided by a model with weight w , SPL performs the following:
Revised: , ( x n , y n ) } of n samples and loss function L ( y i , f ( x i , w )) , where x i ∈ R m represents the feature vector for the i th sample, y i is its label, and f ( x i , w ) is the predicted label provided by a model with weight w , SPL performs the following:

Original: [Equation]
Revised: [Equation]

Original: SPL jointly learns the model weights w and sample weights ν , which end up being 0 - 1 indicators of selected samples, and it does so via alternating minimization.
Revised: SPL jointly learns the model weights w and sample weights ν , which end up being 0 - 1 indicators of selected samples, and it does so via alternating minimization.

Original: Fixing w , minimization w.r.t.
Revised: Fixing w , minimization w.r.t.

Original: ν selects samples with loss L ( y i , f ( x i , w )) <
Revised: ν selects samples with loss L ( y i , f ( x i , w )) <

Original: λ , where λ is a “hardness parameter” as it corresponds to the hardness as measure by the current loss (since with large λ , samples with greater loss are allowed in).
Revised: λ , where λ is a “hardness parameter” as it corresponds to the hardness as measure by the current loss (since with large λ , samples with greater loss are allowed in).

Original: Self-paced curriculum learning (Jiang et al., 2015) introduces a blending of “teacher mode” in CL and “student mode” in SPL, where the teacher can deﬁne a region of ν by attaching a linear constraint a T ν ≤ c to Eq.
Revised: Self-paced curriculum learning (Jiang et al., 2015) introduces a blending of “teacher mode” in CL and “student mode” in SPL, where the teacher can deﬁne a region of ν by attaching a linear constraint a T ν ≤ c to Eq.

Original: SPL with diversity (SPLD) (Jiang et al., 2014), adds to Eq.
Revised: SPL with diversity (SPLD) (Jiang et al., 2014), adds to Eq.

Original: (1) a negative group sparse regularization term − γ (cid:107) ν (cid:107) 2 , 1 (cid:44) − γ (cid:80) bj =1 (cid:107)
Revised: (1) a negative group sparse regularization term − γ (cid:107) ν (cid:107) 2 , 1 (cid:44) − γ (cid:80) bj =1 (cid:107)

Original: ν
Revised: ν

Original: ( j ) (cid:107) 2 , where the samples are divided into b groups beforehand and ν ( j ) is the weight vector for the j th group.
Revised: ( j ) (cid:107) 2 , where the samples are divided into b groups beforehand and ν ( j ) is the weight vector for the j th group.

Original: Samples coming from different groups are thus preferred, to the extent that γ > 0 is large.
Revised: Samples coming from different groups are thus preferred, to the extent that γ > 0 is large.

Original: CL, SPL, and SPLD can be seen as a form of continuation scheme (Allgower & Georg, 2003) that handles a hard task by solving a sequence of tasks moving from easy to hard; the solution to each task is the warm start for the next slightly harder task.
Revised: CL, SPL, and SPLD can be seen as a form of continuation scheme (Allgower & Georg, 2003) that handles a hard task by solving a sequence of tasks moving from easy to hard; the solution to each task is the warm start for the next slightly harder task.

Original: That is, each task, in the present case, is determined by the training data subset and other training hyperparameters, and the resulting parameters at the end of a training round are used as the initial parameters for the next training round.
Revised: That is, each task, in the present case, is determined by the training data subset and other training hyperparameters, and the resulting parameters at the end of a training round are used as the initial parameters for the next training round.

Original: Such continuation schemes can reduce the impact of local minima within neural networks (Bengio et al., 2013; Bengio, 2014).
Revised: Such continuation schemes can reduce the impact of local minima within neural networks (Bengio et al., 2013; Bengio, 2014).

Original: With SPL, after each round of alternating minimization to optimize Eq.
Revised: With SPL, after each round of alternating minimization to optimize Eq.

Original: (1), λ is increased so that the next round selects samples that have a larger loss, a process (Khan et al., 2011b; Tang et al., 2012b; Basu & Christensen, 2013) that can both help avoid local minima and reduce generalization error.
Revised: (1), λ is increased so that the next round selects samples that have a larger loss, a process (Khan et al., 2011b; Tang et al., 2012b; Basu & Christensen, 2013) that can both help avoid local minima and reduce generalization error.

Original: In SPLD, γ is also increased between training rounds, increasingly preferring diversity.
Revised: In SPLD, γ is also increased between training rounds, increasingly preferring diversity.

Original: In each case, each round results in a fully trained model for the currently selected training samples.
Revised: In each case, each round results in a fully trained model for the currently selected training samples.

Original: Selection of training samples has been studied in other settings as well, often with a different motivation.
Revised: Selection of training samples has been studied in other settings as well, often with a different motivation.

Original: In active learning (AL) (Settles, 2010) and experimental design (Montgomery, 2006), the learner can actively query labels of samples from an unlabeled pool during the training process, and the goal is to reduce annotation costs.
Revised: In active learning (AL) (Settles, 2010) and experimental design (Montgomery, 2006), the learner can actively query labels of samples from an unlabeled pool during the training process, and the goal is to reduce annotation costs.

Original: The aim is to achieve the same or better performance using fewer labeled samples by ruling out uninformative ones.
Revised: The aim is to achieve the same or better performance using fewer labeled samples by ruling out uninformative ones.

Original: Diversity modeling was introduced to AL in (Wei et al., 2015).
Revised: Diversity modeling was introduced to AL in (Wei et al., 2015).

Original: It uses submodular maximization to select diverse training batches from the most uncertain samples.
Revised: It uses submodular maximization to select diverse training batches from the most uncertain samples.

Original: However, changing the diversity during the learning process has not been investigated as far as we know.
Revised: However, changing the diversity during the learning process has not been investigated as far as we know.

Original: In boosting (Schapire, 1990; Freund & Schapire, 1997), the goal is to learn an ensemble of weak classiﬁers sequentially; it does this by assigning weights to all samples, with larger weights given to samples having larger loss measured by an aggregation of previously trained models.
Revised: In boosting (Schapire, 1990; Freund & Schapire, 1997), the goal is to learn an ensemble of weak classiﬁers sequentially; it does this by assigning weights to all samples, with larger weights given to samples having larger loss measured by an aggregation of previously trained models.

Original: Both active learning and boosting favor samples that are difﬁcult to predict, since they are the most informative to learn.
Revised: Both active learning and boosting favor samples that are difﬁcult to predict, since they are the most informative to learn.

Original: For example, uncertainty sampling (Culotta & McCallum, 2005; Scheffer et al., 2001; Dagan & Engelson, 1995; Dasgupta & Hsu, 2008) selects samples that are most uncertain, while query by committee (Seung et al., 1992; Dagan & Engelson, 1995; Abe &
Revised: For example, uncertainty sampling (Culotta & McCallum, 2005; Scheffer et al., 2001; Dagan & Engelson, 1995; Dasgupta & Hsu, 2008) selects samples that are most uncertain, while query by committee (Seung et al., 1992; Dagan & Engelson, 1995; Abe &

Original: Mamitsuka, 1998) selects the ones that multiple models most disagree on.
Revised: Mamitsuka, 1998) selects the ones that multiple models most disagree on.

Original: With machine teaching (Khan et al., 2011a; Zhu, 2015; Patil et al., 2014; Zhu et al., 2018), a separate teacher helps the training procedure ﬁnd a good model.
Revised: With machine teaching (Khan et al., 2011a; Zhu, 2015; Patil et al., 2014; Zhu et al., 2018), a separate teacher helps the training procedure ﬁnd a good model.

Original: The SPL approach starts with a smaller set of easy samples and gradually increases the difﬁculty of the chosen samples as measured by the sample loss of the model produced by previous round’s training.
Revised: The SPL approach starts with a smaller set of easy samples and gradually increases the difﬁculty of the chosen samples as measured by the sample loss of the model produced by previous round’s training.

Original: One of the difﬁculties of this approach is the following: since for any given value of λ the relatively easiest samples are chosen, there is a good chance that the process can repeatedly select a similar training set over multiple rounds and therefore can learn slowly.
Revised: One of the difﬁculties of this approach is the following: since for any given value of λ the relatively easiest samples are chosen, there is a good chance that the process can repeatedly select a similar training set over multiple rounds and therefore can learn slowly.

Original: This is precisely the problem that SPLD address — by concomitantly increasing the desired diversity over rounds, the sample selection procedure chooses from an increasingly diverse set of different groups, as measured by (cid:107) ν (cid:107) 2 , 1 .
Revised: This is precisely the problem that SPLD address — by concomitantly increasing the desired diversity over rounds, the sample selection procedure chooses from an increasingly diverse set of different groups, as measured by (cid:107) ν (cid:107) 2 , 1 .

Original: Therefore, in SPLD, early stages train on easier not necessarily diverse samples and later stages train on harder more diverse samples.
Revised: Therefore, in SPLD, early stages train on easier not necessarily diverse samples and later stages train on harder more diverse samples.

Original: There are several challenges remaining with SPLD, however.
Revised: There are several challenges remaining with SPLD, however.

Original: One is that in early stages, it is still possible to repeatedly select a similar training set over multiple rounds since diversity might not increase dramatically between successive rounds.
Revised: One is that in early stages, it is still possible to repeatedly select a similar training set over multiple rounds since diversity might not increase dramatically between successive rounds.

Original: Potentially more problematically, it is not clear that having a large diversity selection weight in late stages is desirable.
Revised: Potentially more problematically, it is not clear that having a large diversity selection weight in late stages is desirable.

Original: For example, with a reasonably trained model, it might be best to select primarily the hardest samples in the part of the space near the difﬁcult regions of the decision boundaries.
Revised: For example, with a reasonably trained model, it might be best to select primarily the hardest samples in the part of the space near the difﬁcult regions of the decision boundaries.

Original: With a high diversity weight, samples in these difﬁcult decision boundary regions might be avoided in favor of other samples perhaps already well learnt and having a large margin only because they are diverse, thereby leading to wasted effort.
Revised: With a high diversity weight, samples in these difﬁcult decision boundary regions might be avoided in favor of other samples perhaps already well learnt and having a large margin only because they are diverse, thereby leading to wasted effort.

Original: At such point, it would be beneﬁcial to choose points having small margin from the same region but that might not have the greatest diversity, especially when using only a simple notion of diversity such as the group sparse norm (cid:107) v (cid:107) 2 , 1 .
Revised: At such point, it would be beneﬁcial to choose points having small margin from the same region but that might not have the greatest diversity, especially when using only a simple notion of diversity such as the group sparse norm (cid:107) v (cid:107) 2 , 1 .

Original: Also, it is possible that late stages of learning can select outliers only because they are both hard and diverse.
Revised: Also, it is possible that late stages of learning can select outliers only because they are both hard and diverse.

Original: Lastly, the SPL/SPLD min-min optimization involves minimizing a lower bound of the loss, while normally one would, if anything, wish to minimize the loss directly or at least an upper bound.
Revised: Lastly, the SPL/SPLD min-min optimization involves minimizing a lower bound of the loss, while normally one would, if anything, wish to minimize the loss directly or at least an upper bound.

Original: Motivated by these issues, we introduce a new form of CL that chooses the hardest diverse samples in early rounds of training and then actually decreases , rather than increases, diversity as training rounds proceed.
Revised: Motivated by these issues, we introduce a new form of CL that chooses the hardest diverse samples in early rounds of training and then actually decreases , rather than increases, diversity as training rounds proceed.

Original: Our contention is that diversity is more important during the early phases of training when only relatively few samples are selected.
Revised: Our contention is that diversity is more important during the early phases of training when only relatively few samples are selected.

Original: Later rounds of training will naturally have more diversity opportunity simply because the size of the selected samples is much larger.
Revised: Later rounds of training will naturally have more diversity opportunity simply because the size of the selected samples is much larger.

Original: Also, to avoid successive rounds selecting similar sets of samples, our approach selects the hardest, rather than the easiest, samples at each round.
Revised: Also, to avoid successive rounds selecting similar sets of samples, our approach selects the hardest, rather than the easiest, samples at each round.

Original: Hence, if a set of samples is learnt well during one training round, those samples will tend to be ill-favored in the next round because they become easier.
Revised: Hence, if a set of samples is learnt well during one training round, those samples will tend to be ill-favored in the next round because they become easier.

Original: We also measure hardness via the loss function, but the selection is always based on the hardest and most diverse samples of a given size k , where the degree of diversity is controlled by a parameter λ , and where diversity is measured by an arbitrary non-monotone submodular function.
Revised: We also measure hardness via the loss function, but the selection is always based on the hardest and most diverse samples of a given size k , where the degree of diversity is controlled by a parameter λ , and where diversity is measured by an arbitrary non-monotone submodular function.

Original: In fact, for binary variables the group sparse norm is also submodular where (cid:107)
Revised: In fact, for binary variables the group sparse norm is also submodular where (cid:107)

Original: ν
Revised: ν

Original: (cid:107) 2 , 1 = (cid:80) bj =1 (cid:112)
Revised: (cid:107) 2 , 1 = (cid:80) bj =1 (cid:112)

Original: | C j ∩ A | = F ( A ) where A is the set for which ν is the characteristic vector, and C j is the set of samples in the j th group.
Revised: | C j ∩ A | = F ( A ) where A is the set for which ν is the characteristic vector, and C j is the set of samples in the j th group.

Original: Our approach allows the full expressive class of submodular functions to be used to measure diversity since the selection phases is based on submodular optimization.
Revised: Our approach allows the full expressive class of submodular functions to be used to measure diversity since the selection phases is based on submodular optimization.

Original: Evidence for the naturalness of such hardness and diversity adjustment in a curriculum can also be found in human education.
Revised: Evidence for the naturalness of such hardness and diversity adjustment in a curriculum can also be found in human education.

Original: For example, courses in primary school usually cover a broad, small, and relatively easy range of topics, in order to expose the young learner to a diversity of knowledge early on.
Revised: For example, courses in primary school usually cover a broad, small, and relatively easy range of topics, in order to expose the young learner to a diversity of knowledge early on.

Original: In college and graduate school, by contrast, students focus on advanced deeper knowledge within their majors.
Revised: In college and graduate school, by contrast, students focus on advanced deeper knowledge within their majors.

Original: As another example, studies of bilingualism (Bialystok et al., 2012; Li et al., 2014;
Revised: As another example, studies of bilingualism (Bialystok et al., 2012; Li et al., 2014;

Original: Mechelli et al., 2004; Kov ´ acs & Mehler, 2009) show that learning multiple languages in childhood is beneﬁcial for future brain development, but early-age multi-lingual learning is usually not advanced or concentrated linguistically for any of the languages involved.
Revised: Mechelli et al., 2004; Kov ´ acs & Mehler, 2009) show that learning multiple languages in childhood is beneﬁcial for future brain development, but early-age multi-lingual learning is usually not advanced or concentrated linguistically for any of the languages involved.

Original: Still other studies argue that difﬁculty can be desired at early human learning stages (Bjork & Bjork, 1992; McDaniel & Butler, 2011).
Revised: Still other studies argue that difﬁculty can be desired at early human learning stages (Bjork & Bjork, 1992; McDaniel & Butler, 2011).

Original: 1. O UR A PPROACH : M INIMAX C URRICULUM L EARNING
Revised: 1. O UR A PPROACH : M INIMAX C URRICULUM L EARNING

Original: We introduce a new form of curriculum learning called minimax curriculum learning (MCL).
Revised: We introduce a new form of curriculum learning called minimax curriculum learning (MCL).

Original: MCL increases desired hardness and reduces diversity encouragement over rounds of training.
Revised: MCL increases desired hardness and reduces diversity encouragement over rounds of training.

Original: This is accomplished by solving a sequence of minimax optimizations, each of which having the form:
Revised: This is accomplished by solving a sequence of minimax optimizations, each of which having the form:

Original: [Equation]
Revised: [Equation]

Original: The objective is composed of the loss on a subset A of samples evaluating their hardness and a normalized monotone non-decreasing submodular function F : 2 V → R + measuring A ’s diversity, where V is the ground set of all available samples.
Revised: The objective is composed of the loss on a subset A of samples evaluating their hardness and a normalized monotone non-decreasing submodular function F : 2 V → R + measuring A ’s diversity, where V is the ground set of all available samples.

Original: A larger loss implies that the subset A has been found harder to learn, while a larger F ( A ) indicates greater diversity.
Revised: A larger loss implies that the subset A has been found harder to learn, while a larger F ( A ) indicates greater diversity.

Original: The weight λ controls the trade-off between hardness and diversity, while k , the size of the resulting A , determines the number of samples to simultaneously learn and hence is a hardness parameter.
Revised: The weight λ controls the trade-off between hardness and diversity, while k , the size of the resulting A , determines the number of samples to simultaneously learn and hence is a hardness parameter.

Original: It is important to realize that F ( A ) is not a parameter regularizer (e.g., (cid:96) 1 or (cid:96) 2 regularization on the parameters w ) but rather an expression of preference for a diversity of training samples.
Revised: It is important to realize that F ( A ) is not a parameter regularizer (e.g., (cid:96) 1 or (cid:96) 2 regularization on the parameters w ) but rather an expression of preference for a diversity of training samples.

Original: In practice, one would add to Eq.
Revised: In practice, one would add to Eq.

Original: (2) an appropriate parameter regularizer as we do in our experiments (Section 3).
Revised: (2) an appropriate parameter regularizer as we do in our experiments (Section 3).

Original: Like SPL/SPLD, learning rounds are scheduled, here each round with increasing k and decreasing λ .
Revised: Like SPL/SPLD, learning rounds are scheduled, here each round with increasing k and decreasing λ .

Original: Unlike SPL/SPLD, we explicitly schedule the number of selected samples via k rather than indirectly via a hardness parameter.
Revised: Unlike SPL/SPLD, we explicitly schedule the number of selected samples via k rather than indirectly via a hardness parameter.

Original: This makes sense since we are always choosing the hardest k samples at a given λ diversity preference, so there is no need for an explicit real-valued hardness parameter as in
Revised: This makes sense since we are always choosing the hardest k samples at a given λ diversity preference, so there is no need for an explicit real-valued hardness parameter as in

Original: SPL/SPLD.
Revised: SPL/SPLD.

Original: Also, the MCL optimization minimizes an upper bound of the loss on any size k subset of training samples.
Revised: Also, the MCL optimization minimizes an upper bound of the loss on any size k subset of training samples.

Original: The function F ( · ) may be chosen from the large expressive family of submodular functions, all of which are natural for measuring diversity, and all having the following diminishing returns property:
Revised: The function F ( · ) may be chosen from the large expressive family of submodular functions, all of which are natural for measuring diversity, and all having the following diminishing returns property:

Original: given a ﬁnite ground set V , and any A ⊆ B ⊆ V and a v / ∈ B
Revised: given a ﬁnite ground set V , and any A ⊆ B ⊆ V and a v / ∈ B

Original: [Equation]
Revised: [Equation]

Original: This implies v is no less valuable to the smaller set A than to the larger set B .
Revised: This implies v is no less valuable to the smaller set A than to the larger set B .

Original: The marginal gain of v conditioned on A is denoted f ( v | A ) (cid:44)
Revised: The marginal gain of v conditioned on A is denoted f ( v | A ) (cid:44)

Original: f
Revised: f

Original: ( v ∪ A )
Revised: ( v ∪ A )

Original: − f
Revised: − f

Original: ( A ) and reﬂects the importance of v to A .
Revised: ( A ) and reﬂects the importance of v to A .

Original: Submodular functions (Fujishige, 2005) have been widely used for diversity models (Lin et al., 2009; Lin & Bilmes, 2011; Batra et al., 2012;
Revised: Submodular functions (Fujishige, 2005) have been widely used for diversity models (Lin et al., 2009; Lin & Bilmes, 2011; Batra et al., 2012;

Original: Prasad et al., 2014; Gillenwater et al., 2012; Iyer & Bilmes, 2015).
Revised: Prasad et al., 2014; Gillenwater et al., 2012; Iyer & Bilmes, 2015; Bilmes & Bai, 2017).

Original: Although Eq.
Revised: Although Eq.

Original: (2) is a hybrid optimization involving both continuous variables w and discrete variables
Revised: (2) is a hybrid optimization involving both continuous variables w and discrete variables

Original: A , it can be reduced to the minimization of a piecewise function, where each piece is deﬁned by a subset A achieving the maximum in a region around w .
Revised: A , it can be reduced to the minimization of a piecewise function, where each piece is deﬁned by a subset A achieving the maximum in a region around w .

Original: Each piece is convex when the loss is convex, so various off-the-shelf algorithms can be applied once A has been computed.
Revised: Each piece is convex when the loss is convex, so various off-the-shelf algorithms can be applied once A has been computed.

Original: However, the number of possible sets A is (cid:0) nk (cid:1) , and enumerating them all to ﬁnd the maximum is intractable.
Revised: However, the number of possible sets A is (cid:0) nk (cid:1) , and enumerating them all to ﬁnd the maximum is intractable.

Original: Thanks to submodularity, fast approximate algorithms (Nemhauser et al., 1978; Minoux, 1978; Mirzasoleiman et al., 2015) exist to ﬁnd an approximately optimal A .
Revised: Thanks to submodularity, fast approximate algorithms (Nemhauser et al., 1978; Minoux, 1978; Mirzasoleiman et al., 2015) exist to ﬁnd an approximately optimal A .

Original: Therefore, the outer optimization over w will need to minimize an approximation of the piecewise function deﬁned by an approximate A computed via submodular maximization.
Revised: Therefore, the outer optimization over w will need to minimize an approximation of the piecewise function deﬁned by an approximate A computed via submodular maximization.

Original:  M INIMAX C URRICULUM L EARNING AND M ACHINE T EACHING
Revised:  M INIMAX C URRICULUM L EARNING AND M ACHINE T EACHING

Original: The minimax problem in Eq.
Revised: The minimax problem in Eq.

Original: (2) can be seen as a two-person zero-sum game between a teacher (the maximizer) and a student (the minimizer): the teacher chooses training set A based on the student’s feedback about the hardness (i.e., the loss achieved by current model w ) and how diverse according to the teacher ( λF ( A ) ), while the student updates w to reduce the loss on training set A (i.e., learn
Revised: (2) can be seen as a two-person zero-sum game between a teacher (the maximizer) and a student (the minimizer): the teacher chooses training set A based on the student’s feedback about the hardness (i.e., the loss achieved by current model w ) and how diverse according to the teacher ( λF ( A ) ), while the student updates w to reduce the loss on training set A (i.e., learn

Original: A ) given by the teacher.
Revised: A ) given by the teacher.

Original: Similar teacher-student interaction also exist in real life.
Revised: Similar teacher-student interaction also exist in real life.

Original: In addition, the teacher usually introduces concepts at the beginning and asks a small number of easy questions from a diverse range of topics and receives feedback from the student, and then further trains the student on the topics the student ﬁnds difﬁcult while eschewing topics the student has mastered.
Revised: In addition, the teacher usually introduces concepts at the beginning and asks a small number of easy questions from a diverse range of topics and receives feedback from the student, and then further trains the student on the topics the student ﬁnds difﬁcult while eschewing topics the student has mastered.

Original: MCL’s minimax formulation is different from the min-min formulation used in SPL/SPLD.
Revised: MCL’s minimax formulation is different from the min-min formulation used in SPL/SPLD.

Original: For certain losses and models, L ( y i , f ( x i , w )) is convex in w .
Revised: For certain losses and models, L ( y i , f ( x i , w )) is convex in w .

Original: The min-min formulation, however, is only bi-convex and requires procedures such as alternative convex search (ACS) as in (Bazaraa et al., 1993).
Revised: The min-min formulation, however, is only bi-convex and requires procedures such as alternative convex search (ACS) as in (Bazaraa et al., 1993).

Original: Furthermore, diversity regularization of ν in SPLD leads to the loss of bi-convexity altogether.
Revised: Furthermore, diversity regularization of ν in SPLD leads to the loss of bi-convexity altogether.

Original: Minimizing the worst case loss, as in MCL, is a widely used strategy in machine learning (Lanckriet et al., 2003; Farnia & Tse, 2016) to achieve better generalization performance and model robustness, especially when strong assumptions cannot be made about the data distribution.
Revised: Minimizing the worst case loss, as in MCL, is a widely used strategy in machine learning (Lanckriet et al., 2003; Farnia & Tse, 2016) to achieve better generalization performance and model robustness, especially when strong assumptions cannot be made about the data distribution.

Original: Compared to
Revised: Compared to

Original: SPL/SPLD, MCL is also better in that the outer minimization over w in Eq.
Revised: SPL/SPLD, MCL is also better in that the outer minimization over w in Eq.

Original: (2) is a convex program, and corresponds to minimizing the objective g ( w ) in Eq.(4).
Revised: (2) is a convex program, and corresponds to minimizing the objective g ( w ) in Eq.(4).

Original: On the other hand, querying g ( w ) requires submodular maximization which can only be solved approximately.
Revised: On the other hand, querying g ( w ) requires submodular maximization which can only be solved approximately.

Original: The goal of this section, therefore, is to address the minimax problem in Eq.
Revised: The goal of this section, therefore, is to address the minimax problem in Eq.

Original: (2), i.e., the minimization min w ∈ R m g ( w ) of the following objective g ( w ) .
Revised: (2), i.e., the minimization min w ∈ R m g ( w ) of the following objective g ( w ) .

Original: [Equation]
Revised: [Equation]

Original: If the loss function L ( y i , f ( x i , w )) is convex w.r.t.
Revised: If the loss function L ( y i , f ( x i , w )) is convex w.r.t.

Original: w , then g ( w ) is convex but, as mentioned above, enumerating all subsets is intractable.
Revised: w , then g ( w ) is convex but, as mentioned above, enumerating all subsets is intractable.

Original: Deﬁning the discrete objective G w : 2 V → R + where
Revised: Deﬁning the discrete objective G w : 2 V → R + where

Original: [Equation]
Revised: [Equation]

Original:  shows that computing g ( w ) in involves a discrete optimization over G w ( A ) , a problem that is submodular since G w ( A ) is weighted sum of a non-negative (since loss is non-negative) modular and a submodular function, and thus G w is monotone non-decreasing submodular.
Revised: [Equation] shows that computing g ( w ) in involves a discrete optimization over G w ( A ) , a problem that is submodular since G w ( A ) is weighted sum of a non-negative (since loss is non-negative) modular and a submodular function, and thus G w is monotone non-decreasing submodular.

Original: Thus, the fast greedy procedure mentioned earlier can be used to approximately optimizes G w ( A ) for any w .
Revised: Thus, the fast greedy procedure mentioned earlier can be used to approximately optimizes G w ( A ) for any w .

Original: Let ˆ A w ⊆
Revised: Let ˆ A w ⊆

Original: V be the k -constrained greedy approximation to maximizing G w ( A ) .
Revised: V be the k -constrained greedy approximation to maximizing G w ( A ) .

Original: We deﬁne the following approximate objective:
Revised: We deﬁne the following approximate objective:

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: and note that it satisﬁes αg ( w ) ≤ ˆ g ( w ) ≤ g ( w ) where α is the approximation factor of submodular optimization.
Revised: and note that it satisﬁes αg ( w ) ≤ ˆ g ( w ) ≤ g ( w ) where α is the approximation factor of submodular optimization.

Original: For ˜ w within a region around w , ˆ g ( ˜ w ) will utilize the same set ˆ A w .
Revised: For ˜ w within a region around w , ˆ g ( ˜ w ) will utilize the same set ˆ A w .

Original: Therefore, ˆ g ( w ) is piecewise convex, if the loss function L ( y i , f ( x i , w )) is convex w.r.t.
Revised: Therefore, ˆ g ( w ) is piecewise convex, if the loss function L ( y i , f ( x i , w )) is convex w.r.t.

Original: w , and different regions of within R m are associated with different ˆ A although not necessarily the same regions or sets that deﬁne g ( w ) .
Revised: w , and different regions of within R m are associated with different ˆ A although not necessarily the same regions or sets that deﬁne g ( w ) .

Original: We show in Section 2.2 that minimizing ˆ g ( w ) offers an approximate solution to Eq.
Revised: We show in Section 2.2 that minimizing ˆ g ( w ) offers an approximate solution to Eq.

Original: With ˆ g ( w ) given, our algorithm is simply gradient descent for minimizing ˆ g ( w ) , where many offthe-shelf methods can be invoked, e.g., SGD, momentum methods, Nesterov’s accelerated gradient (Nesterov, 2005), Adagrad (Duchi et al., 2011), etc.
Revised: With ˆ g ( w ) given, our algorithm is simply gradient descent for minimizing ˆ g ( w ) , where many offthe-shelf methods can be invoked, e.g., SGD, momentum methods, Nesterov’s accelerated gradient (Nesterov, 2005), Adagrad (Duchi et al., 2011), etc.

Original: The key problem is how to obtain ˆ g ( w ) , which depends on suboptimal solutions in different regions of w .
Revised: The key problem is how to obtain ˆ g ( w ) , which depends on suboptimal solutions in different regions of w .

Original: It is not necessary, however, to run submodular maximization for every region of w .
Revised: It is not necessary, however, to run submodular maximization for every region of w .

Original: Since we use gradient descent, we only need to know ˆ g ( w ) for w on the optimization path.
Revised: Since we use gradient descent, we only need to know ˆ g ( w ) for w on the optimization path.

Original: At the beginning of each iteration, we ﬁx w and use submodular maximization to achieve the ˆ A w that deﬁnes ˆ g ( w ) .
Revised: At the beginning of each iteration, we ﬁx w and use submodular maximization to achieve the ˆ A w that deﬁnes ˆ g ( w ) .

Original: Then a gradient update step is applied to ˆ g ( w ) .
Revised: Then a gradient update step is applied to ˆ g ( w ) .

Original: Let A ∗ w represent the optimal solution to Eq.
Revised: Let A ∗ w represent the optimal solution to Eq.

Original: (5), then ˆ A w satisﬁes G ( ˆ A ) ≥ αG ( A ∗ ) .
Revised: (5), then ˆ A w satisﬁes G ( ˆ A ) ≥ αG ( A ∗ ) .

Original: 1: input: π ( · , η ) , γ , p , ∆ 2: output: w 0 T 3: initialize: τ ← 1 , w 0 τ , λ , k , 4: while not “converged” do 5: for t ∈ { 0 , · · · , p } do 6: 
Revised: do 6: g tτ )

Original: G ( A) ← (cid:80) i ∈
Revised: ∂∂w i A ← w

Original: A L ( y i , f ( x i , w tτ ))
Revised: [Equation]

Original: + λF ( A ) ;
Revised: + π [Equation]

Original: 7: ˆ A ← WS-S UBMODULAR M AX ( G, k ) ; 8: ∇ ˆ g ( w tτ ) =
Revised: { w 1: t τ } , {∇ ˆ g ( w 1: t τ ) } , η (cid:1) ; 10: end for 11: w [Equation] ←

Original: ∂∂w (cid:80) i ∈
Revised: [Equation]

Original: ˆ A L ( y i , f ( x i , w tτ )) ; 9: w t +1 τ ←w tτ
Revised: ←

Original: [-+ π (cid:0) { w 1: t τ } , {∇ ˆ g ( w 1: t τ ) } , η (cid:1) ; 10: end for 11: w 
-]
Revised: [-+ π (cid:0) { w 1: t τ } , {∇ ˆ g ( w 1: t τ ) } , η (cid:1) ; 10: end for 11: w 
-][Equation]

Original: τ +1 ← w pτ , λ ← (1 − γ ) · λ , k ← k +∆ ,τ ← τ +1 ; 12: end while
Revised: [Equation] ← [Equation] ∆ ,

Revised: [Equation]

Revised: ←

Revised: [Equation]

Revised: +1 ; 12: end while

Original: Algorithm 1 details MCL.
Revised: Algorithm 1 details MCL.

Original: Lines 5solve the optimization in Eq.
Revised: Lines 5solve the optimization in Eq.

Original: (2) with λ and k scheduled in line 11.
Revised: (2) with λ and k scheduled in line 11.

Original: Lines 6-7 ﬁnds an approximate ˆ A via submodular maximization, discussed further in Section 2.1.
Revised: Lines 6-7 ﬁnds an approximate ˆ A via submodular maximization, discussed further in Section 2.1.

Original: Lines 8-9 update w for the current ˆ
Revised: Lines 8-9 update w for the current ˆ

Original: A by gradient descent π ( · , η ) with learning rate η .
Revised: A by gradient descent π ( · , η ) with learning rate η .

Original: The inner optimization stops after p steps and then λ is reduced by factor 1 − γ where γ ∈ [0 , 1] and k is increased by ∆ .
Revised: The inner optimization stops after p steps and then λ is reduced by factor 1 − γ where γ ∈ [0 , 1] and k is increased by ∆ .

Original: The outer optimization stops after T steps when a form of “convergence”, described below, is achieved.
Revised: The outer optimization stops after T steps when a form of “convergence”, described below, is achieved.

Original: Given ˆ A w , ˆ g ( w ) has gradient
Revised: Given ˆ A w , ˆ g ( w ) has gradient

Original: [Equation]
Revised: [Equation]

Original: and thus gradient descent method can update w .
Revised: and thus gradient descent method can update w .

Original: For example, we can treat ˆ A as a batch if k is small, and update w by w ← w − η ∇
Revised: For example, we can treat ˆ A as a batch if k is small, and update w by w ← w − η ∇

Original: ˆ g ( w ) with learning rate η .
Revised: ˆ g ( w ) with learning rate η .

Original: For large ˆ A w , we can use SGD that applies an update rule to mini-batches within ˆ A w .
Revised: For large ˆ A w , we can use SGD that applies an update rule to mini-batches within ˆ A w .

Original: More complex gradient descent rules π ( · , η ) can take historical gradients and w tτ ’s into account leading to w t +1 ← w t
Revised: More complex gradient descent rules π ( · , η ) can take historical gradients and w tτ ’s into account leading to w t +1 ← w t

Original: + π (cid:0) { w 1: t } , {∇ ˆ g ( w 1: t ) } , η (cid:1) .
Revised: + π (cid:0) { w 1: t } , {∇ ˆ g ( w 1: t ) } , η (cid:1) .

Original: Considering the outer loop as well, the algorithm approximately solves a sequence of Eq.
Revised: Considering the outer loop as well, the algorithm approximately solves a sequence of Eq.

Original: (2)s with decreasing λ and increasing k , where the previous solutions act as a warm start for the next iterations.
Revised: (2)s with decreasing λ and increasing k , where the previous solutions act as a warm start for the next iterations.

Original: This corresponds to repeatedly updating the model w on a sequence of training sets ˆ A that changes from small, diverse, and hard to large.
Revised: This corresponds to repeatedly updating the model w on a sequence of training sets ˆ A that changes from small, diverse, and hard to large.

Original: 2. S UBMODULAR M AXIMIZATION
Revised: 2. S UBMODULAR M AXIMIZATION

Original: Although solving Eq.
Revised: Although solving Eq.

Original: (5) exactly is NP-hard, a near-optimal solution can be achieved by the greedy algorithm, which offers a worst-case approximation factor of α = 1 − e − 1 (Nemhauser et al., 1978).
Revised: (5) exactly is NP-hard, a near-optimal solution can be achieved by the greedy algorithm, which offers a worst-case approximation factor of α = 1 − e − 1 (Nemhauser et al., 1978).

Original: The algorithm starts with A ← ∅ , and selects next the element with the largest marginal gain f ( v | A ) from V \ A , i.e., A ← A ∪ { v ∗ } where v ∗ ∈ argmax v ∈ V \
Revised: The algorithm starts with A ← ∅ , and selects next the element with the largest marginal gain f ( v | A ) from V \ A , i.e., A ← A ∪ { v ∗ } where v ∗ ∈ argmax v ∈ V \

Original: A f ( v | A ) , and this repeats until | A | = k .
Revised: A f ( v | A ) , and this repeats until | A | = k .

Original: It is simple to implement, fast, and usually outperforms other methods, e.g., those based on integer linear programming.
Revised: It is simple to implement, fast, and usually outperforms other methods, e.g., those based on integer linear programming.

Original: It requires O ( nk ) function evaluations for ground set size | V | = n .
Revised: It requires O ( nk ) function evaluations for ground set size | V | = n .

Original: Since
Revised: Since

Original: Algorithm 1 runs greedy
Revised: Algorithm 1 runs greedy

Original: Tp times, it is useful for the greedy procedure to be as fast as possible.
Revised: Tp times, it is useful for the greedy procedure to be as fast as possible.

Original: The accelerated, or lazy, greedy algorithm (Minoux, 1978) reduces the number of evaluations per step by updating a priority queue of marginal gains, while having the same output and guarantee as the original (thanks to submodularity) and offers signiﬁcant speedups.
Revised: The accelerated, or lazy, greedy algorithm (Minoux, 1978) reduces the number of evaluations per step by updating a priority queue of marginal gains, while having the same output and guarantee as the original (thanks to submodularity) and offers signiﬁcant speedups.

Original: Still faster variants are also available Mirzasoleiman et al.
Revised: Still faster variants are also available Mirzasoleiman et al.

Original: Our own implementation takes advantage of the fact that line 7 of Algorithm 1 repeatedly solves submodular maximization over a sequence of submodular functions that are changing only slowly, and hence the previous set solution can be used as a warm start for the current algorithm, a process we call WS-S UBMODULAR M AX outlined in Algorithm 2.
Revised: Our own implementation takes advantage of the fact that line 7 of Algorithm 1 repeatedly solves submodular maximization over a sequence of submodular functions that are changing only slowly, and hence the previous set solution can be used as a warm start for the current algorithm, a process we call WS-S UBMODULAR M AX outlined in Algorithm 2.

Original: The greedy procedure offers much better approximation factors than 1 − e − 1 when the objective
Revised: The greedy procedure offers much better approximation factors than 1 − e − 1 when the objective

Original: G ( A ) is close to modular.
Revised: G ( A ) is close to modular.

Original: Speciﬁcally, the approximation factor becomes α = (1 − e − κ G ) /κ
Revised: Speciﬁcally, the approximation factor becomes α = (1 − e − κ G ) /κ

Original: G (Conforti & Cornuejols, 1984), which depends on the curvature κ G ∈ [0 , 1] of G ( A ) deﬁned as
Revised: G (Conforti & Cornuejols, 1984), which depends on the curvature κ G ∈ [0 , 1] of G ( A ) deﬁned as

Original: [Equation]
Revised: [Equation]

Original: When κ G = 0 , G is modular, and when κ G = 1 , G is fully curved and the above bound recovers 1 − e − 1 .
Revised: When κ G = 0 , G is modular, and when κ G = 1 , G is fully curved and the above bound recovers 1 − e − 1 .

Original: G ( A ) becomes more modular as the outer loop proceeds since λ decreases.
Revised: G ( A ) becomes more modular as the outer loop proceeds since λ decreases.

Original: Therefore, the approximation improves with the number of outer loops.
Revised: Therefore, the approximation improves with the number of outer loops.

Original: In fact, we have:
Revised: In fact, we have:

Original: Lemma 1.
Revised: Lemma 1.

Original: Let G ( A ) =
Revised: Let G ( A ) =

Original: L ( A )+ λF ( A ) where F is a monotone non-decreasing submodular function with curvature κ F , L is a non-negative modular function, and λ ≥ 0 .
Revised: L ( A )+ λF ( A ) where F is a monotone non-decreasing submodular function with curvature κ F , L is a non-negative modular function, and λ ≥ 0 .

Original: Then κ G ≤ κ F / ( c 1 /λ + 1) where c 1 = min j ∈ V L ( j ) /F
Revised: Then κ G ≤ κ F / ( c 1 /λ + 1) where c 1 = min j ∈ V L ( j ) /F

Original: ( j ) .
Revised: ( j ) .

Original: The proof is given in Appendix 4.1.
Revised: The proof is given in Appendix 4.1.

Original: In MCL, therefore, the submodular approximation improves ( α → 1 ) as λ grows, and the surrogate function ˆ g ( w ) correspondingly approaches the true convex objective g ( w ) .
Revised: In MCL, therefore, the submodular approximation improves ( α → 1 ) as λ grows, and the surrogate function ˆ g ( w ) correspondingly approaches the true convex objective g ( w ) .

Original: 2. C ONDITIONS AT C ONVERGENCE
Revised: 2. C ONDITIONS AT C ONVERGENCE

Original: In this section, we study how close the solution ˆ w is of applying gradient descent to ˆ g ( w ) , where we assume p is large enough so that a form of convergence occurs.
Revised: In this section, we study how close the solution ˆ w is of applying gradient descent to ˆ g ( w ) , where we assume p is large enough so that a form of convergence occurs.

Original: Speciﬁcally, in Theorem 1, we analyze the upper bound on (cid:107) ˆ w − w ∗ (cid:107) 22 based on two assumptions: 1) the loss L ( y i , f ( x i , w )) being β -strongly convex w.r.t.
Revised: Speciﬁcally, in Theorem 1, we analyze the upper bound on (cid:107) ˆ w − w ∗ (cid:107) 22 based on two assumptions: 1) the loss L ( y i , f ( x i , w )) being β -strongly convex w.r.t.

Original: w ; and 2) ˆ w is achieved by running gradient descent in lines 6-9 of
Revised: w ; and 2) ˆ w is achieved by running gradient descent in lines 6-9 of

Original: Algorithm 1 until convergence, deﬁned as the gradient reaching zero.
Revised: Algorithm 1 until convergence, deﬁned as the gradient reaching zero.

Original: In case the loss L ( y i , f ( x i , w )) is convex but not β -strongly convex, a commonly used trick to modify it to β -strongly convex is to add an (cid:96) 2 regularization ( β/ 2) (cid:107) w (cid:107) 22 .
Revised: In case the loss L ( y i , f ( x i , w )) is convex but not β -strongly convex, a commonly used trick to modify it to β -strongly convex is to add an (cid:96) 2 regularization ( β/ 2) (cid:107) w (cid:107) 22 .

Original: In addition, for non-convex L ( y i , f ( x i , w )) , it is possible to prove that with high probability, a noise perturbed SGD on ˆ g ( w ) can hit an (cid:15) -optimal local solution of g ( w ) in polynomial time — we leave this for future work.
Revised: In addition, for non-convex L ( y i , f ( x i , w )) , it is possible to prove that with high probability, a noise perturbed SGD on ˆ g ( w ) can hit an (cid:15) -optimal local solution of g ( w ) in polynomial time — we leave this for future work.

Original: In our empirical study (Section 3),
Revised: In our empirical study (Section 3),

Original: MCL achieves good performance even when applied to non-convex deep neural networks.
Revised: MCL achieves good performance even when applied to non-convex deep neural networks.

Original: The following theorem relies on the fact that the maximum of multiple β -strongly convex functions is also β -strongly convex, shown in Appendix 4.2.
Revised: The following theorem relies on the fact that the maximum of multiple β -strongly convex functions is also β -strongly convex, shown in Appendix 4.2.

Original: Theorem 1 (Inner-loop convergence) .
Revised: Theorem 1 (Inner-loop convergence) .

Original: For the minimax problem in Eq.
Revised: For the minimax problem in Eq.

Original: (2) with ground set of samples
Revised: (2) with ground set of samples

Original: V and λ ≥ 0 , if the loss function L ( y i , f ( x i , w )) is β -strongly convex and | V | ≥ k , running lines 6-9 of Algorithm 1 until convergence (deﬁned as the gradient reaching zero) yields a solution ˆ w satisfying
Revised: V and λ ≥ 0 , if the loss function L ( y i , f ( x i , w )) is β -strongly convex and | V | ≥ k , running lines 6-9 of Algorithm 1 until convergence (deﬁned as the gradient reaching zero) yields a solution ˆ w satisfying

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: ) ˆ w is the solution achieved at convergence, w ∗ is the optimal solution of the minimax problem in Eq.
Revised: ) ˆ w is the solution achieved at convergence, w ∗ is the optimal solution of the minimax problem in Eq.

Original: (2) , g ( w ∗ ) is the objective value achieved on w ∗ , and α is the approximation factor that submodular maximization can guarantee for G ( A ) .
Revised: (2) , g ( w ∗ ) is the objective value achieved on w ∗ , and α is the approximation factor that submodular maximization can guarantee for G ( A ) .

Original: The proof is given in Appendix 4.3.
Revised: The proof is given in Appendix 4.3.

Original: It is interesting to note that the bound depends both on the strong convexity parameter β and on the submodular maximization approximation α .
Revised: It is interesting to note that the bound depends both on the strong convexity parameter β and on the submodular maximization approximation α .

Original: As mentioned in Lemma 1, as λ gets smaller, the approximation factor α approaches 1 meaning that the bound in Equation (9) improves.
Revised: As mentioned in Lemma 1, as λ gets smaller, the approximation factor α approaches 1 meaning that the bound in Equation (9) improves.

Original: We mention the convergence criteria where the gradient reaches zero.
Revised: We mention the convergence criteria where the gradient reaches zero.

Original: While it is possible, in theory, for lines 6-9 of Algorithm 1 to oscillate amongst the non-differentiable boundaries between the convex pieces, with most damped learning rates, this will eventually subside and the algorithm will remain within one convex piece.
Revised: While it is possible, in theory, for lines 6-9 of Algorithm 1 to oscillate amongst the non-differentiable boundaries between the convex pieces, with most damped learning rates, this will eventually subside and the algorithm will remain within one convex piece.

Original: The reason for this is line 7 of the algorithm always chooses one ˆ
Revised: The reason for this is line 7 of the algorithm always chooses one ˆ

Original: A thereby selecting one convex piece associated with the region around w tτ , and with only small subsequent adjustments to w tτ , the same ˆ A will continue to be selected.
Revised: A thereby selecting one convex piece associated with the region around w tτ , and with only small subsequent adjustments to w tτ , the same ˆ A will continue to be selected.

Original: Hence, the algorithm will, in such case, reach the minimum of that convex piece where the gradient is zero.
Revised: Hence, the algorithm will, in such case, reach the minimum of that convex piece where the gradient is zero.

Original: We can restate and then simplify the above bound in terms of the resulting parameters, and corresponding λ, k values, used at a particular iteration τ of the outer loop.
Revised: We can restate and then simplify the above bound in terms of the resulting parameters, and corresponding λ, k values, used at a particular iteration τ of the outer loop.

Original: In the following, ˆ w τ is the solution achieved by Algorithm 1 at the iteration τ of the outer loop, and the optimal solution of the minimax problem in Eq.(2) with λ, k set as in iteration τ is denoted w ∗ T .
Revised: In the following, ˆ w τ is the solution achieved by Algorithm 1 at the iteration τ of the outer loop, and the optimal solution of the minimax problem in Eq.(2) with λ, k set as in iteration τ is denoted w ∗ T .

Original: Corollary 1.
Revised: Corollary 1.

Original: If the loss function L ( y i , f ( x i , w )) is β -strongly convex, the submodular function F ( · ) has curvature κ F , and if each inner-loop in Algorithm 1 runs until convergence, then the solution ˆ w τ at the end of the τ th iteration of the outer-loop fulﬁlls:
Revised: If the loss function L ( y i , f ( x i , w )) is β -strongly convex, the submodular function F ( · ) has curvature κ F , and if each inner-loop in Algorithm 1 runs until convergence, then the solution ˆ w τ at the end of the τ th iteration of the outer-loop fulﬁlls:

Original: [Equation]
Revised: [Equation]

Original: where w ∗ τ is the optimal solution of the minimax problem in Eq.
Revised: where w ∗ τ is the optimal solution of the minimax problem in Eq.

Original: (2) with λ set as in the τ th outer loop iteration.
Revised: (2) with λ set as in the τ th outer loop iteration.

Original: Thus, if k starts from k 0 and linearly increases via k ← k + ∆ (as in line 11 of Algorithm alg:mcl),
Revised: Thus, if k starts from k 0 and linearly increases via k ← k + ∆ (as in line 11 of Algorithm alg:mcl),

Original: [Equation]
Revised: [Equation]

Original: Otherwise, if k increases exponentially, i.e., k
Revised: Otherwise, if k increases exponentially, i.e., k

Original: [Equation]
Revised: [Equation]

Original: In the above, λand kare the initial values for λ and k
Revised: In the above, λand kare the initial values for λ and k

Original: [Equation]
Revised: [Equation]

Original: The proof can be found in Appendix 4.5.
Revised: The proof can be found in Appendix 4.5.

Original: On the one hand, the upper bound above is in terms of the ratio λ/k which improves with larger subset sizes.
Revised: On the one hand, the upper bound above is in terms of the ratio λ/k which improves with larger subset sizes.

Original: On the other hand, submodular maximization becomes more expensive with k .
Revised: On the other hand, submodular maximization becomes more expensive with k .

Original: Hence, Algorithm 1 chooses a schedule to decrease λ exponentially and increase k only linearly.
Revised: Hence, Algorithm 1 chooses a schedule to decrease λ exponentially and increase k only linearly.

Original: Also, we see that the bound is dependent on the submodular curvature κ F , the strongly-convex constant β , and c 1 which relates the submodular and modular terms (similar to as in Lemma 1).
Revised: Also, we see that the bound is dependent on the submodular curvature κ F , the strongly-convex constant β , and c 1 which relates the submodular and modular terms (similar to as in Lemma 1).

Original: These quantities ( κ F /β and c 1 ) might be relevant for other convex-submodular optimization schemes.
Revised: These quantities ( κ F /β and c 1 ) might be relevant for other convex-submodular optimization schemes.

Original: 2. H EURISTIC I MPROVEMENTS
Revised: 2. H EURISTIC I MPROVEMENTS

Original: There are several heuristic improvements we employ that are described next.
Revised: There are several heuristic improvements we employ that are described next.

Original: Algorithm 1 stops gradient descent after p steps.
Revised: Algorithm 1 stops gradient descent after p steps.

Original: A reason for doing this is that ˆ w p can be sufﬁcient as a warm-start for the next iteration if p is large enough.
Revised: A reason for doing this is that ˆ w p can be sufﬁcient as a warm-start for the next iteration if p is large enough.

Original: We also have not observed any beneﬁt for larger p , although we do eventually observe convergence empirically when the average loss no longer change appreciably between stages.
Revised: We also have not observed any beneﬁt for larger p , although we do eventually observe convergence empirically when the average loss no longer change appreciably between stages.

Original: Also, lines 6-7 of Algorithm 1 require computing the loss on all the samples, and each step of the greedy algorithm needs to, in the worst case, evaluate the marginal gains of all of the unselected samples.
Revised: Also, lines 6-7 of Algorithm 1 require computing the loss on all the samples, and each step of the greedy algorithm needs to, in the worst case, evaluate the marginal gains of all of the unselected samples.

Original: Moreover, this is done repeatedly in the inner-most block of two nested loops.
Revised: Moreover, this is done repeatedly in the inner-most block of two nested loops.

Original: Therefore, we use two heuristic tricks to improve efﬁciency.
Revised: Therefore, we use two heuristic tricks to improve efﬁciency.

Original: Fist, rather than selecting individual samples, we ﬁrst cluster the data and then select clusters, thereby reducing the ground set size from the number of samples to the number of clusters.
Revised: Fist, rather than selecting individual samples, we ﬁrst cluster the data and then select clusters, thereby reducing the ground set size from the number of samples to the number of clusters.

Original: We replace the per-sample loss L ( y i , f ( x i , w )) with a per-cluster loss L (cid:0) Y ( i ) , f ( X ( i ) , w ) (cid:1) that we approximate by the loss of the sample closest to the centroid within each cluster:
Revised: We replace the per-sample loss L ( y i , f ( x i , w )) with a per-cluster loss L (cid:0) Y ( i ) , f ( X ( i ) , w ) (cid:1) that we approximate by the loss of the sample closest to the centroid within each cluster:

Original: [Equation]
Revised: [Equation]

Original: (13) where C ( i ) is the set of indices of the samples in the i th cluster, and x ( i ) with label y
Revised: (13) where C ( i ) is the set of indices of the samples in the i th cluster, and x ( i ) with label y

Original: ( i ) is the sample closest to the cluster centroid.
Revised: ( i ) is the sample closest to the cluster centroid.

Original: We ﬁnd that the loss on x ( i ) is sufﬁciently representative to approximately indicate the hardness of the cluster.
Revised: We ﬁnd that the loss on x ( i ) is sufﬁciently representative to approximately indicate the hardness of the cluster.

Original: The set V becomes the set of clusters and A ⊆ V is a set of clusters, and hence the ground set size is reduced speeding up the greedy algorithm.
Revised: The set V becomes the set of clusters and A ⊆ V is a set of clusters, and hence the ground set size is reduced speeding up the greedy algorithm.

Original: When computing F ( A ) , the diversity of selected clusters, cluster centroids again represent the cluster.
Revised: When computing F ( A ) , the diversity of selected clusters, cluster centroids again represent the cluster.

Original: In line 8, the gradient is computed on all the samples in the selected clusters rather than on only x ( i ) at which point the labels of all the samples in the selected clusters are used.
Revised: In line 8, the gradient is computed on all the samples in the selected clusters rather than on only x ( i ) at which point the labels of all the samples in the selected clusters are used.

Original: Otherwise, when selecting clusters via submodular maximization, the labels of only the centroid samples are needed.
Revised: Otherwise, when selecting clusters via submodular maximization, the labels of only the centroid samples are needed.

Original: Thus, we need only annotate and compute the loss for samples in the selected clusters and the representative centroid samples x
Revised: Thus, we need only annotate and compute the loss for samples in the selected clusters and the representative centroid samples x

Original: ( i ) of other clusters.
Revised: ( i ) of other clusters.

Original: This also reduces the need to label all samples up front as only the labels of the selected clusters, and centroid samples of each cluster, are used (i.e., the clustering process itself does not use the labels).
Revised: This also reduces the need to label all samples up front as only the labels of the selected clusters, and centroid samples of each cluster, are used (i.e., the clustering process itself does not use the labels).

Original: We can further reduce the ground set to save computation during submodular maximization via preﬁltering methods that lead either to no (Wei et al., 2014a) or little (Zhou et al., 2017; Mirzasoleiman et al., 2015) reduction in approximation quality.
Revised: We can further reduce the ground set to save computation during submodular maximization via preﬁltering methods that lead either to no (Wei et al., 2014a) or little (Zhou et al., 2017; Mirzasoleiman et al., 2015) reduction in approximation quality.

Original: Moreover, as λ decreases in the MCL objective and
Revised: Moreover, as λ decreases in the MCL objective and

Original: G ( A ) becomes more modular, pruning method become more effective.
Revised: G ( A ) becomes more modular, pruning method become more effective.

Original: More details are given in
Revised: More details are given in

Original: Section 4.6.
Revised: Section 4.6.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: [Table]
Revised: [Table]

Original: In this section, we apply different curriculum learning methods to train logistic regression models on 20newsgroups (Lang, 1995), LeNet5 models on MNIST (Lecun et al., 1998), convolutional neural nets (CNNs) with three convolutional layers 1 on CIFAR10 (Krizhevsky & Hinton, 2009), CNNs with two convolutional layers 2 on Fashion-MNIST (“Fashion” in all tables) (Xiao et al., 2017), CNNs with six convolutional layers on STL10 (Coates et al., 2011), and CNNs with seven convolutional layers on SVHN (Netzer et al., 2011) 3 .
Revised: In this section, we apply different curriculum learning methods to train logistic regression models on 20newsgroups (Lang, 1995), LeNet5 models on MNIST (Lecun et al., 1998), convolutional neural nets (CNNs) with three convolutional layers 1 on CIFAR10 (Krizhevsky & Hinton, 2009), CNNs with two convolutional layers 2 on Fashion-MNIST (“Fashion” in all tables) (Xiao et al., 2017), CNNs with six convolutional layers on STL10 (Coates et al., 2011), and CNNs with seven convolutional layers on SVHN (Netzer et al., 2011) 3 .

Original: Details on the datasets can be found in Table 3 of the appendix.
Revised: Details on the datasets can be found in Table 3 of the appendix.

Original: In all cases, we also use (cid:96) 2 parameter regularization on w I used l2 regularization.
Revised: In all cases, we also use (cid:96) 2 parameter regularization on w I used l2 regularization.

Original: Its weight is the weight decay factor of the optimizer, and I always set it as 1e-4 for all experiments
Revised: Its weight is the weight decay factor of the optimizer, and I always set it as 1e-4 for all experiments

Original: We compare MCL and its variants to SPL (Kumar et al., 2010), SPLD (Jiang et al., 2014) and SGD with a random curriculum (i.e., with random batches).
Revised: We compare MCL and its variants to SPL (Kumar et al., 2010), SPLD (Jiang et al., 2014) and SGD with a random curriculum (i.e., with random batches).

Original: Each method uses mini-batch SGD for π ( · , η ) with the same learning rate strategy to update w .
Revised: Each method uses mini-batch SGD for π ( · , η ) with the same learning rate strategy to update w .

Original: The methods, therefore, differ only in the curriculum (i.e., the sequence of training sets).
Revised: The methods, therefore, differ only in the curriculum (i.e., the sequence of training sets).

Original: For SGD, in each iteration, we randomly select 4000 samples (20newsgroups) or 5000 samples (other datasets) and apply mini-batch SGD to the selected samples.
Revised: For SGD, in each iteration, we randomly select 4000 samples (20newsgroups) or 5000 samples (other datasets) and apply mini-batch SGD to the selected samples.

Original: In SPL and SPLD, the training set starts from a ﬁxed size k ( 4000 samples for 20newsgroups, 5000 samples for other datasets), and increases by a factor of 1 + µ (where µ = 0 . 1 ) per round of alternating minimization (i.e., per iteration of the outer loop) 4 .
Revised: In SPL and SPLD, the training set starts from a ﬁxed size k ( 4000 samples for 20newsgroups, 5000 samples for other datasets), and increases by a factor of 1 + µ (where µ = 0 . 1 ) per round of alternating minimization (i.e., per iteration of the outer loop) 4 .

Original: We use ρ to denote the number of iterations of the inner loop, which aims to minimize the loss w.r.t.
Revised: We use ρ to denote the number of iterations of the inner loop, which aims to minimize the loss w.r.t.

Original: the model w on the selected training set.
Revised: the model w on the selected training set.

Original: In SPLD, we also have a weight for the negative group sparsity: it starts from ξ and increases by a factor of 1 . 1 at each round of alternating minimization (i.e., per iteration of the outer loop).
Revised: In SPLD, we also have a weight for the negative group sparsity: it starts from ξ and increases by a factor of 1 . 1 at each round of alternating minimization (i.e., per iteration of the outer loop).

Original: We test ﬁve different combinations of { ρ, µ } and { ρ, ξ } for SPL and SPLD respectively.
Revised: We test ﬁve different combinations of { ρ, µ } and { ρ, ξ } for SPL and SPLD respectively.

Original: The best combination with the smallest test error rate is what we report.
Revised: The best combination with the smallest test error rate is what we report.

Original: Neither SPL nor SPLD uses the clustering trick we applied to MCL: they compute the exact loss on each sample in each iteration.
Revised: Neither SPL nor SPLD uses the clustering trick we applied to MCL: they compute the exact loss on each sample in each iteration.

Original: Hence, they have more accurate estimation of the hardness on each sample, and require knowing the labels of all samples (selected and unselected) and cannot reduce annotation costs.
Revised: Hence, they have more accurate estimation of the hardness on each sample, and require knowing the labels of all samples (selected and unselected) and cannot reduce annotation costs.

Original: Note SPLD still needs to run clustering and use the resulted clusters as groups in the group sparsity (which measures diversity in SPLD).
Revised: Note SPLD still needs to run clustering and use the resulted clusters as groups in the group sparsity (which measures diversity in SPLD).

Original: We did not select samples with SPL/SPLD as we do with MCL since we wanted to test SPL/SPLD as originally presented — intuitively, SPL/SPLD should if anything only do better without such clustering due to the more accurate sample-speciﬁc hardness estimation.
Revised: We did not select samples with SPL/SPLD as we do with MCL since we wanted to test SPL/SPLD as originally presented — intuitively, SPL/SPLD should if anything only do better without such clustering due to the more accurate sample-speciﬁc hardness estimation.

Original: The actual clustering, however, used for
Revised: The actual clustering, however, used for

Original: SPLD’s diversity term is the same as that used for MCL’s cluster samples.
Revised: SPLD’s diversity term is the same as that used for MCL’s cluster samples.

Original: We apply the mini-batch k-means algorithm to the features detailed in the next paragraph to get the clusters used in MCL and
Revised: We apply the mini-batch k-means algorithm to the features detailed in the next paragraph to get the clusters used in MCL and

Original: SPLD.
Revised: SPLD.

Original: Although both SPL and SPLD can be reduced to SGD when λ → ∞ (i.e., all samples always selected), we do not include this special case because SGD is already a baseline.
Revised: Although both SPL and SPLD can be reduced to SGD when λ → ∞ (i.e., all samples always selected), we do not include this special case because SGD is already a baseline.

Original: For SGD with a random curriculum, results of 10 independent trials are reported.
Revised: For SGD with a random curriculum, results of 10 independent trials are reported.

Original: In our MCL experiments, we use a simple “feature based” submodular function (Wei et al., 2014b) where F ( A )
Revised: In our MCL experiments, we use a simple “feature based” submodular function (Wei et al., 2014b) where F ( A )

Original: =
Revised: =

Original: (cid:80) u
Revised: (cid:80) u

Original: ∈U ω u (cid:112) c u ( A ) and where U is a set of features.
Revised: ∈U ω u (cid:112) c u ( A ) and where U is a set of features.

Original: For a subset A of clusters,
Revised: For a subset A of clusters,

Original: [Table]
Revised: [Table]

Original: c u ( A ) =
Revised: c u ( A ) =

Original: (cid:80)
Revised: (cid:80)

Original: i ∈
Revised: i ∈

Original: A c u ( i ) , where c u ( i ) is the nonnegative feature u of the centroid for cluster i , and can be interpreted as a nonnegative score for cluster i .
Revised: A c u ( i ) , where c u ( i ) is the nonnegative feature u of the centroid for cluster i , and can be interpreted as a nonnegative score for cluster i .

Original: We use TF-IDF features for 20newsgroup.
Revised: We use TF-IDF features for 20newsgroup.

Original: For the other datasets, we train a corresponding neural networks on a small random subset of training data (e.g., hundreds of samples) for one epoch, and use the inputs to the last fully connected layer (whose outputs are processed by softmax to generate class probabilities) as features.
Revised: For the other datasets, we train a corresponding neural networks on a small random subset of training data (e.g., hundreds of samples) for one epoch, and use the inputs to the last fully connected layer (whose outputs are processed by softmax to generate class probabilities) as features.

Original: Because we always use
Revised: Because we always use

Original: ReLU activations between layers, the features are all nonnegative and the submodularity of F ( A ) follows as a consequence.
Revised: ReLU activations between layers, the features are all nonnegative and the submodularity of F ( A ) follows as a consequence.

Original: These features are also used by mini-batch k -means to generate clusters for MCL and SPLD.
Revised: These features are also used by mini-batch k -means to generate clusters for MCL and SPLD.

Original: For MCL, we set the number of inner loop iterations to p ≤ 50 .
Revised: For MCL, we set the number of inner loop iterations to p ≤ 50 .

Original: For each dataset, we choose p as the number among { 10 , 20 , 50 } that reduces the training loss the most in the ﬁrst few iterations of the outer loop, and then use that p for the remaining iterations.
Revised: For each dataset, we choose p as the number among { 10 , 20 , 50 } that reduces the training loss the most in the ﬁrst few iterations of the outer loop, and then use that p for the remaining iterations.

Original: As shown in Table 4, we use p = 50 for 20newsgroups, MNIST and Fashion-MNIST, and p = 20 for the other three datasets.
Revised: As shown in Table 4, we use p = 50 for 20newsgroups, MNIST and Fashion-MNIST, and p = 20 for the other three datasets.

Original: We consider ﬁve variants of MCL: 1) MCL( ∆ = 0 , λ = 0 , γ = 0 ) having neither submodular regularization that promotes diversity nor scheduling of k that increases hardness; 2) MCL( ∆ = 0 , λ > 0 , γ > 0 ), which decreases diversity by exponentially reducing the weight λ of the submodular regularization, but does not have any scheduling of k , i.e., k is ﬁxed during the algorithm; 3)
Revised: We consider ﬁve variants of MCL: 1) MCL( ∆ = 0 , λ = 0 , γ = 0 ) having neither submodular regularization that promotes diversity nor scheduling of k that increases hardness; 2) MCL( ∆ = 0 , λ > 0 , γ > 0 ), which decreases diversity by exponentially reducing the weight λ of the submodular regularization, but does not have any scheduling of k , i.e., k is ﬁxed during the algorithm; 3)

Original: MCL( ∆ > 0 , λ > 0 , γ = 0 ), which only uses the scheduling of k shown in Algorithm 1, but the diversity weight λ is positive and ﬁxed during the algorithm, i.e., with γ = 0 ; 4) MCL-RAND( r , q ), which randomly samples r clusters as a training set ˆ A after every q rounds of the outer loop in
Revised: MCL( ∆ > 0 , λ > 0 , γ = 0 ), which only uses the scheduling of k shown in Algorithm 1, but the diversity weight λ is positive and ﬁxed during the algorithm, i.e., with γ = 0 ; 4) MCL-RAND( r , q ), which randomly samples r clusters as a training set ˆ A after every q rounds of the outer loop in

Original: Algorithm 1, and thus combines both MCL and SGD; 5) MCL( ∆ > 0 , λ > 0 , γ > 0 ), which uses the scheduling of both λ and k shown in Algorithm 1.
Revised: Algorithm 1, and thus combines both MCL and SGD; 5) MCL( ∆ > 0 , λ > 0 , γ > 0 ), which uses the scheduling of both λ and k shown in Algorithm 1.

Original: We tried ﬁve different combinations of { q, r } for MCL-RAND( r , q ) and ﬁve different ∆ values for MCL( ∆ > 0 , λ > 0 , γ > 0 ), and report the one with the smallest test error.
Revised: We tried ﬁve different combinations of { q, r } for MCL-RAND( r , q ) and ﬁve different ∆ values for MCL( ∆ > 0 , λ > 0 , γ > 0 ), and report the one with the smallest test error.

Original: Other parameters, such as the initial values for λ and k , the values for γ and p , and the total number of clusters are the same for different variants (the exact values of these quantities are given in Table 4 of the Appendix).
Revised: Other parameters, such as the initial values for λ and k , the values for γ and p , and the total number of clusters are the same for different variants (the exact values of these quantities are given in Table 4 of the Appendix).

Original: [Figure]
Revised: [Figure]

Original: a t e ( % )
Revised: a t e ( % )

Original: SPL( ρ =25, μ =0.1) SPLD( ρ =25, ξ =0.4) MCL(Δ>0, λ >0, γ =0)
Revised: SPL( ρ =25, μ =0.1) SPLD( ρ =25, ξ =0.4) MCL(Δ>0, λ >0, γ =0)

Original: MCL(Δ=0, λ >0, γ >0)
Revised: MCL(Δ=0, λ >0, γ >0)

Original: MCLΔRAND( μ =3, r =12) MCL(Δ=0, λ =0, γ =0) MCL(Δ>0, λ >0, γ >0)
Revised: MCLΔRAND( μ =3, r =12) MCL(Δ=0, λ =0, γ =0) MCL(Δ>0, λ >0, γ >0)

Original: In MCL, running greedy is the only extra computation comparing to normal SGD.
Revised: In MCL, running greedy is the only extra computation comparing to normal SGD.

Original: To show that in our implementation (see Section 4.6) its additional time cost is negligible, we report in Tablethe total time cost for MCL( ∆ > 0 , λ > 0 , γ > 0 ) and the time spent on our implementation
Revised: To show that in our implementation (see Section 4.6) its additional time cost is negligible, we report in Tablethe total time cost for MCL( ∆ > 0 , λ > 0 , γ > 0 ) and the time spent on our implementation

Original: WS-S UBMODULAR M AX .
Revised: WS-S UBMODULAR M AX .

Original: We summarize the main results in Figure 1-8.
Revised: We summarize the main results in Figure 1-8.

Original: More results are given at the end of the appendix (Section 4.7).
Revised: More results are given at the end of the appendix (Section 4.7).

Original: In all ﬁgures, grey curves correspond to the ten trials of SGD under a random curriculum.
Revised: In all ﬁgures, grey curves correspond to the ten trials of SGD under a random curriculum.

Original: The legend in all ﬁgures gives the parameters used for the different methods using the following labels: 1) SPL ( ρ, µ ); 2) SPLD( ρ, ξ ); and 3) MCL-RAND( q, r ).
Revised: The legend in all ﬁgures gives the parameters used for the different methods using the following labels: 1) SPL ( ρ, µ ); 2) SPLD( ρ, ξ ); and 3) MCL-RAND( q, r ).

Original: Figures 1-6 show how the test error changes with (on the left) the number of distinct labeled samples ever needing a loss gradient calculation, and (on the right) the number of training batches
Revised: Figures 1-6 show how the test error changes with (on the left) the number of distinct labeled samples ever needing a loss gradient calculation, and (on the right) the number of training batches

Original: [Figure]
Revised: [Figure]

Original: corresponding to training time.
Revised: corresponding to training time.

Original: Note only MCL and its variants use the clustering trick, while
Revised: Note only MCL and its variants use the clustering trick, while

Original: SPL/SPLD need to compute loss on every sample and thus require knowledge of the labels of all samples.
Revised: SPL/SPLD need to compute loss on every sample and thus require knowledge of the labels of all samples.

Original: The left plot shows only the number of loss gradient calculations needed — 1) in MCL, for those clusters never selected in the curriculum, the loss (and hence the label) of only the centroid sample is needed; 2) in SPL/SPLD, for those samples never selected in the curriculum, their labels are needed only to compute the loss but not the gradient, so they are not reﬂected in the left plots of all ﬁgures because their labels are not used to compute a gradient.
Revised: The left plot shows only the number of loss gradient calculations needed — 1) in MCL, for those clusters never selected in the curriculum, the loss (and hence the label) of only the centroid sample is needed; 2) in SPL/SPLD, for those samples never selected in the curriculum, their labels are needed only to compute the loss but not the gradient, so they are not reﬂected in the left plots of all ﬁgures because their labels are not used to compute a gradient.

Original: Therefore, thanks to the clustering trick, MCL and its variants can train without needing all labels, similar to semi-supervised learning methods.
Revised: Therefore, thanks to the clustering trick, MCL and its variants can train without needing all labels, similar to semi-supervised learning methods.

Original: This can help to reduce the annotation costs, if an MCL process is done in tandem with a labeling procedure analogous to active learning.
Revised: This can help to reduce the annotation costs, if an MCL process is done in tandem with a labeling procedure analogous to active learning.

Original: The right plots very roughly indicate convergence rate, namely how the test error decreases as a function of the amount of training.
Revised: The right plots very roughly indicate convergence rate, namely how the test error decreases as a function of the amount of training.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: On all datasets, MCL and most of its variants outperform SPL and SPLD in terms of ﬁnal test accuracy (shown in Table 1) with comparable efﬁciency (shown in the right plots of all ﬁgures).
Revised: On all datasets, MCL and most of its variants outperform SPL and SPLD in terms of ﬁnal test accuracy (shown in Table 1) with comparable efﬁciency (shown in the right plots of all ﬁgures).

Original: MCL is slightly slower than SGD to converge in early stages but it can achieve a much smaller error when using the same number of labeled samples for loss gradients.
Revised: MCL is slightly slower than SGD to converge in early stages but it can achieve a much smaller error when using the same number of labeled samples for loss gradients.

Original: Moreover, when using the same learning rate strategy, they can be more robust to overﬁtting, as shown in Figure 2.
Revised: Moreover, when using the same learning rate strategy, they can be more robust to overﬁtting, as shown in Figure 2.

Original: Comparing Figure 1 with
Revised: Comparing Figure 1 with

Original: Figure 2-6, MCL has the advantage when applied to deep models [Figure] t [Figure]
Revised: Figure 2-6, MCL has the advantage when applied to deep models.

Original: rr

Original: [Figure]
Revised: [Figure]

Original: r r a t
Revised: t e s t

Original: [Figure]

Original: [Figure]

Original: a b e l e d s a m
Revised: e rr o r r a t e

Original: [Figure]
Revised: [Figure]

Original: l

Original: [Figure]
Revised: [Figure]

Original: s e v e r n

Original: [Figure]
Revised: [Figure]

Original:  d i n g l o ss g r a d i e n t c a l c u l a t i o n
Revised: a b e l e d s a m p l e s e v e r n ee d i n g l o ss g r a d i e n t c a l c u l a t i o n

Original: [Figure]
Revised: [Figure]

Original: [Equation] Among the ﬁve variants of MCL, MCL( λ > 0 , γ > 0 , ∆ > 0 ) achieves the fastest convergence speed in later stages and the smallest ﬁnal test error, while MCL( ∆ = 0 , λ = 0 , ∆ = 0 )
Revised: Among the ﬁve variants of MCL, MCL( λ > 0 , γ > 0 , ∆ > 0 ) achieves the fastest convergence speed in later stages and the smallest ﬁnal test error, while MCL( ∆ = 0 , λ = 0 , ∆ = 0 )

Original: usually achieves the worst performance (the only exception is on News20).
Revised: usually achieves the worst performance (the only exception is on News20).

Original: Comparison between MCL( ∆ = 0 , λ > 0 , γ > 0 ) and MCL( λ = 0 , γ = 0 , ∆ = 0 ) shows that decreasing diversity improves the performance.
Revised: Comparison between MCL( ∆ = 0 , λ > 0 , γ > 0 ) and MCL( λ = 0 , γ = 0 , ∆ = 0 ) shows that decreasing diversity improves the performance.

Original: MCL( λ > 0 , γ > 0 , ∆ > 0 ) always outperforms MCL( ∆ = 0 , λ > 0 , γ > 0 ).
Revised: MCL( λ > 0 , γ > 0 , ∆ > 0 ) always outperforms MCL( ∆ = 0 , λ > 0 , γ > 0 ).

Original: This indicates that increasing k can bring advantages, e.g., more improvements in later stages.
Revised: This indicates that increasing k can bring advantages, e.g., more improvements in later stages.

Original: [Figure]
Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Original: e v e r n ee d i n g l o ss g r a d i e n t c a l c u l a t
Revised: e v e r n ee d i n g l o ss g r a d i e n t c a l c u l a t

Original: i o n
Revised: i o n

Original: SPL( ρ =10, μ =0.1) SPLD( ρ =10, ξ =0.2) MCL(.=0, λ =0, γ =0) MCL(.>0, λ >0, γ =0)
Revised: SPL(ρ=10, μ=0.1) SPLD(ρ=10, ξ=0.2) MCL(.=0,λ=0,γ=0)MCL(.>0,λ>0,γ=0)

Original: MCL(.=0, λ >0, γ >0) MCLΔRAND( μ =5, r =10) MCL(.>0, λ >0, γ >0)
Revised: MCL(.=0,λ>0,γ>0)MCLΔRAND(μ=5, r=10) MCL(.>0,λ>0,γ>0)

Original: MCL( λ > 0 , γ > 0 , ∆ > 0 ) always outperforms MCL( ∆ > 0 , λ > 0 , γ = 0 ), which supports our claim that it is better to decrease the diversity as training proceeds rather than keeping it ﬁxed.
Revised: MCL( λ > 0 , γ > 0 , ∆ > 0 ) always outperforms MCL( ∆ > 0 , λ > 0 , γ = 0 ), which supports our claim that it is better to decrease the diversity as training proceeds rather than keeping it ﬁxed.

Original: In particular, MCL( ∆ > 0 , λ > 0 , γ = 0 ) shows slower convergence than other MCL variants in later stages.
Revised: In particular, MCL( ∆ > 0 , λ > 0 , γ = 0 ) shows slower convergence than other MCL variants in later stages.

Original: In our experiments in the MCL( ∆ > 0 , λ > 0 , γ = 0 ) case, we needed to carefully choose λ and use a relatively large ∆ for it to work at all, as otherwise it would repeatedly choose the same subset (with small ∆ , the loss term decreases as training proceeds, so with ﬁxed λ the diversity term comes to dominate the objective).
Revised: In our experiments in the MCL( ∆ > 0 , λ > 0 , γ = 0 ) case, we needed to carefully choose λ and use a relatively large ∆ for it to work at all, as otherwise it would repeatedly choose the same subset (with small ∆ , the loss term decreases as training proceeds, so with ﬁxed λ the diversity term comes to dominate the objective).

Original: This suggests that a large diversity encouragement is neither necessary nor beneﬁcial when the model matures, possibly since k is large at that point and there is ample opportunity for a diversity of samples to be selected just because k is large, and also since encouraging too much loss-unspeciﬁc diversity at that point might only select outliers.
Revised: This suggests that a large diversity encouragement is neither necessary nor beneﬁcial when the model matures, possibly since k is large at that point and there is ample opportunity for a diversity of samples to be selected just because k is large, and also since encouraging too much loss-unspeciﬁc diversity at that point might only select outliers.

Original: The combination of MCL and random curriculum (MCL-RAND) speeds up convergence, and sometimes (e.g., on MNIST, SVHN and Fashion-MNIST) leads to a good ﬁnal test accuracy, but requires more labeled samples for gradient computation and still cannot outperform MCL( λ > 0 , γ > 0 , ∆ > 0 ).
Revised: The combination of MCL and random curriculum (MCL-RAND) speeds up convergence, and sometimes (e.g., on MNIST, SVHN and Fashion-MNIST) leads to a good ﬁnal test accuracy, but requires more labeled samples for gradient computation and still cannot outperform MCL( λ > 0 , γ > 0 , ∆ > 0 ).

Original: These results indicate that the diversity introduced by submodular regularization does yield improvements, and changing both hardness and diversity improves performance.
Revised: These results indicate that the diversity introduced by submodular regularization does yield improvements, and changing both hardness and diversity improves performance.

Original: Figure 7 and Figure 8 shows how the “number of distinct labeled samples ever needing loss gradient calculation” changes as training proceeds.
Revised: Figure 7 and Figure 8 shows how the “number of distinct labeled samples ever needing loss gradient calculation” changes as training proceeds.

Original: It shows how the different methods trade-off between “training on more new samples” vs. “training on fewer distinct samples more often.”
Revised: It shows how the different methods trade-off between “training on more new samples” vs. “training on fewer distinct samples more often.”

Original: Thanks to the clustering trick, MCL and its variants usually require fewer labeled samples for model training than
Revised: Thanks to the clustering trick, MCL and its variants usually require fewer labeled samples for model training than

Original: SGD but more than SPL and SPLD.
Revised: SGD but more than SPL and SPLD.

Original:  A PPENDIX 4. P ROOF OF L EMMA
Revised:  A PPENDIX 4. P ROOF OF L EMMA

Original: Proof.
Revised: Proof.

Original: We have
Revised: We have

Original: [Equation]
Revised: [Equation]

Original: Where c 1 (cid:44) min j ∈ V L ( j ) F ( j ) .
Revised: Where c 1 (cid:44) min j ∈ V L ( j ) F ( j ) .

Original: 4. P ROOF OF P ROPOSITION
Revised: 4. P ROOF OF P ROPOSITION

Original: Proposition 1.
Revised: Proposition 1.

Original: The maximum of multiple β -strongly convex functions is β -strongly convex as well.
Revised: The maximum of multiple β -strongly convex functions is β -strongly convex as well.

Original: Proof.
Revised: Proof.

Original: Let g ( x ) = max
Revised: Let g ( x ) = max

Original: i g i ( x ) , where g i ( x ) is β -strongly convex for any i .
Revised: i g i ( x ) , where g i ( x ) is β -strongly convex for any i .

Original: According to a deﬁnition of strongly convex function given in Theorem 2.1.9 (page 64) of (Nesterov, 2004)
Revised: According to a deﬁnition of strongly convex function given in Theorem 2.1.9 (page 64) of (Nesterov, 2004)

Original: [Equation]
Revised: [Equation]

Original: ∀ λ ∈ [0 , 1] , we have
Revised: ∀ λ ∈ [0 , 1] , we have

Original: [Equation]
Revised: [Equation]

Original: The following proves that g ( x ) is also β -strongly convex:
Revised: The following proves that g ( x ) is also β -strongly convex:

Original: [Equation]
Revised: [Equation]

Original: 4. P ROOF OF T HEOREM
Revised: 4. P ROOF OF T HEOREM

Original: Proof.
Revised: Proof.

Original: The objective g ( w ) of the minimax problem in Eq.
Revised: The objective g ( w ) of the minimax problem in Eq.

Original: (2) after eliminating A is given in Eq.
Revised: (2) after eliminating A is given in Eq.

Original: Since G ( A ) in Eq.
Revised: Since G ( A ) in Eq.

Original: (5) is monotone non-decreasing submodular, the optimal subset A when deﬁning g ( w ) in Eq.
Revised: (5) is monotone non-decreasing submodular, the optimal subset A when deﬁning g ( w ) in Eq.

Original: (4) always has size k if | V | ≥ k .
Revised: (4) always has size k if | V | ≥ k .

Original: In addition, because the loss function L ( y i , f ( x i , w )) is β -strongly convex, g ( w ) in Eq.
Revised: In addition, because the loss function L ( y i , f ( x i , w )) is β -strongly convex, g ( w ) in Eq.

Original: (4) is the maximum over multiple kβ -strongly convex functions with different A .
Revised: (4) is the maximum over multiple kβ -strongly convex functions with different A .

Original: According to Proposition 1, g ( w ) is also kβ -strongly convex, i.e.,
Revised: According to Proposition 1, g ( w ) is also kβ -strongly convex, i.e.,

Original: [Equation]
Revised: [Equation]

Original: Since the convex function g ( w ) achieves minimum on w ∗ , it is valid to substitute ∇ g ( w ∗ ) = 0 ∈ ∂g ( w ∗ ) into Eq.
Revised: Since the convex function g ( w ) achieves minimum on w ∗ , it is valid to substitute ∇ g ( w ∗ ) = 0 ∈ ∂g ( w ∗ ) into Eq.

Original: After rearrangement, we have
Revised: After rearrangement, we have

Original: [Equation]
Revised: [Equation]

Original: In the following, we will prove g ( w ∗ ) ≥ α · g ( ˆ w ) , which together with Eq.
Revised: In the following, we will prove g ( w ∗ ) ≥ α · g ( ˆ w ) , which together with Eq.

Original: (15) will lead to the ﬁnal bound showing how close ˆ w is to w ∗ .
Revised: (15) will lead to the ﬁnal bound showing how close ˆ w is to w ∗ .

Original: Note ˆ g ( w ) (Eq.
Revised: Note ˆ g ( w ) (Eq.

Original: (6)) is a piecewise function, each piece of which is convex and associated with different ˆ A achieved by a submodular maximization algorithm of approximation factor α .
Revised: (6)) is a piecewise function, each piece of which is convex and associated with different ˆ A achieved by a submodular maximization algorithm of approximation factor α .

Original: Since ˆ A is not guaranteed to be a global maxima, unlike g ( w ) , the whole ˆ g ( w ) cannot be written as the maximum of multiple convex functions and thus can be non-convex.
Revised: Since ˆ A is not guaranteed to be a global maxima, unlike g ( w ) , the whole ˆ g ( w ) cannot be written as the maximum of multiple convex functions and thus can be non-convex.

Original: Therefore, gradient descent in lines 6-9 of Algorithm 1 can lead to either: 1) ˆ w is a global minima of ˆ g ( w ) ; or 2) ˆ w is a local minima of ˆ g ( w ) .
Revised: Therefore, gradient descent in lines 6-9 of Algorithm 1 can lead to either: 1) ˆ w is a global minima of ˆ g ( w ) ; or 2) ˆ w is a local minima of ˆ g ( w ) .

Original: Saddle points do not exist on ˆ g ( w ) because each piece of it is convex.
Revised: Saddle points do not exist on ˆ g ( w ) because each piece of it is convex.

Original: We are also assuming other issues associated with the boundaries between convex pieces do not repeatedly occur.
Revised: We are also assuming other issues associated with the boundaries between convex pieces do not repeatedly occur.

Original: 1) When ˆ w is a global minima of ˆ g ( w ) , we have
Revised: 1) When ˆ w is a global minima of ˆ g ( w ) , we have

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst inequality is due to g ( · ) ≥ ˆ g ( · ) .
Revised: The ﬁrst inequality is due to g ( · ) ≥ ˆ g ( · ) .

Original: The second inequality is due to the global optimality of ˆ w .
Revised: The second inequality is due to the global optimality of ˆ w .

Original: The third inequality is due to the approximation bound ˆ g ( · ) ≥ α · g ( · ) guaranteed by the submodular maximization in Step 7 of Algorithm 1.
Revised: The third inequality is due to the approximation bound ˆ g ( · ) ≥ α · g ( · ) guaranteed by the submodular maximization in Step 7 of Algorithm 1.

Original: 2) When ˆ w is a local minima of ˆ g ( w ) , we have ∇ ˆ g ( ˆ w ) = 0 .
Revised: 2) When ˆ w is a local minima of ˆ g ( w ) , we have ∇ ˆ g ( ˆ w ) = 0 .

Original: Let h ( w ) be the piece of ˆ g ( w ) where ˆ w is located, then ˆ w has to be a global minima of h ( w ) due to the convexity of h ( w ) .
Revised: Let h ( w ) be the piece of ˆ g ( w ) where ˆ w is located, then ˆ w has to be a global minima of h ( w ) due to the convexity of h ( w ) .

Original: Let A denote the ground set of ˆ A on all pieces of ˆ g ( w ) , we deﬁne an auxiliary convex function ˜ g ( w ) as
Revised: Let A denote the ground set of ˆ A on all pieces of ˆ g ( w ) , we deﬁne an auxiliary convex function ˜ g ( w ) as

Original: [Equation]
Revised: [Equation]

Original: It is convex because it is deﬁned as the maximum of multiple convex function.
Revised: It is convex because it is deﬁned as the maximum of multiple convex function.

Original: So we have
Revised: So we have

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst inequality is due to the deﬁnition of A , and the second inequality is a result of A ⊆ 2 V by comparing g ( w ) in Eq.
Revised: The ﬁrst inequality is due to the deﬁnition of A , and the second inequality is a result of A ⊆ 2 V by comparing g ( w ) in Eq.

Original: (4) with ˜ g
Revised: (4) with ˜ g

Original: ( w ) in Eq.
Revised: ( w ) in Eq.

Original: Let ˜ w denote a global minima of ˜ g ( w ) , we have
Revised: Let ˜ w denote a global minima of ˜ g ( w ) , we have

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst inequality is due to Eq.
Revised: The ﬁrst inequality is due to Eq.

Original: (18), the second inequality is due to the global optimality of ˜ w on ˜ g
Revised: (18), the second inequality is due to the global optimality of ˜ w on ˜ g

Original: ( w ) , the third inequality is due to the deﬁnition of ˜ g
Revised: ( w ) , the third inequality is due to the deﬁnition of ˜ g

Original: ( w ) in Eq.
Revised: ( w ) in Eq.

Original: (17) ( ˜ g ( w ) is the maximum of all pieces of ˆ g ( w ) and h ( w ) is one piece of them), the fourth inequality is due to the global optimality of ˆ w on h ( w ) , the last inequality is due to the approximation bound ˆ g ( · ) ≥ α · g ( · ) guaranteed by the submodular maximization in Step 7 of Algorithm 1.
Revised: (17) ( ˜ g ( w ) is the maximum of all pieces of ˆ g ( w ) and h ( w ) is one piece of them), the fourth inequality is due to the global optimality of ˆ w on h ( w ) , the last inequality is due to the approximation bound ˆ g ( · ) ≥ α · g ( · ) guaranteed by the submodular maximization in Step 7 of Algorithm 1.

Original: Therefore, in both cases we have g ( w ∗ ) ≥ α · g ( ˆ w ) .
Revised: Therefore, in both cases we have g ( w ∗ ) ≥ α · g ( ˆ w ) .

Original: Applying it to Eq.
Revised: Applying it to Eq.

Original: (15) results in
Revised: (15) results in

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: 4. P ROPOSITION
Revised: 4. P ROPOSITION

Original: Proposition 2.
Revised: Proposition 2.

Original: If x ∈ [0 , 1] , the following inequality holds true.
Revised: If x ∈ [0 , 1] , the following inequality holds true.

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Due to two inequalities e x ≤ 1 + x + x 2 / 2 for x ≤ 0 and 1 − e − x ≥ x/ 2 for x ∈ [0 , 1] ,
Revised: Due to two inequalities e x ≤ 1 + x + x 2 / 2 for x ≤ 0 and 1 − e − x ≥ x/ 2 for x ∈ [0 , 1] ,

Original: [Equation]
Revised: [Equation]

Original: 4. P ROOF OF C OROLLARY
Revised: 4. P ROOF OF C OROLLARY

Original: Proof.
Revised: Proof.

Original: Applying the inequality in Proposition 2 and the approximation factor of lazy greedy
Revised: Applying the inequality in Proposition 2 and the approximation factor of lazy greedy

Original: α = (1 − e − κ G ) /κ
Revised: α = (1 − e − κ G ) /κ

Original: G to the right hand side of Eq.
Revised: G to the right hand side of Eq.

Original: (9) from Theorem 1 yields
Revised: (9) from Theorem 1 yields

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where κ G is the curvature of submodular function G ( · ) deﬁned in Eq.
Revised: where κ G is the curvature of submodular function G ( · ) deﬁned in Eq.

Original: Substituting the inequality about κ G from Lemma 1 into Eq.
Revised: Substituting the inequality about κ G from Lemma 1 into Eq.

Original: (23) results in
Revised: (23) results in

Original: [Equation]
Revised: [Equation]

Original: We use subscript as the index for iterations in the outer-loop, e.g., ˆ w T is the model weights w after the T
Revised: We use subscript as the index for iterations in the outer-loop, e.g., ˆ w T is the model weights w after the T

Original: th iteration of outer-loop.
Revised: th iteration of outer-loop.

Original: If we decrease λ exponentially from λ = λ 0 and increase k linearly from k = k 0 , as Step 11 in Algorithm 1, we have
Revised: If we decrease λ exponentially from λ = λ 0 and increase k linearly from k = k 0 , as Step 11 in Algorithm 1, we have

Original: [Equation]
Revised: [Equation]

Original: According to the deﬁnition of g ( · ) in Eq.
Revised: According to the deﬁnition of g ( · ) in Eq.

Original: (4), for g ( w ∗ T )
Revised: (4), for g ( w ∗ T )

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: Substituting Eq.
Revised: Substituting Eq.

Original: (26) to Eq.
Revised: (26) to Eq.

Original: (25) yields
Revised: (25) yields

Original: [Equation]
Revised: [Equation]

Original: If we can tolerate more expensive computational cost for running submodular maximization with larger budget k
Revised: If we can tolerate more expensive computational cost for running submodular maximization with larger budget k

Original: [Equation]
Revised: [Equation]

Original: and increase k exponentially
Revised: and increase k exponentially

Original: [Equation]
Revised: [Equation]

Original: i
Revised: i

Original: [Equation]
Revised: [Equation]

Original: e., k ← (1 + ∆) · k , we have
Revised: e., k ← (1 + ∆) · k , we have

Original: [Equation]
Revised: [Equation]

Original: This completes the proof.
Revised: This completes the proof.

Original: 4. S UBMODULAR M
Revised: 4. S UBMODULAR M

Original: Algorithm 1 repeatedly runs a greedy procedure to solve submodular maximization, and this occurs two nested loops deep.
Revised: Algorithm 1 repeatedly runs a greedy procedure to solve submodular maximization, and this occurs two nested loops deep.

Original: In this section we describe how we speed this process up.
Revised: In this section we describe how we speed this process up.

Original: Our ﬁrst strategy reduces the size of the ground set before starting a more expensive submodular maximization procedure.
Revised: Our ﬁrst strategy reduces the size of the ground set before starting a more expensive submodular maximization procedure.

Original: We use a method described in (Wei et al., 2014a) where we sort the elements of V non-increasingly by G ( i | V \ i ) and then remove any element i from V having
Revised: We use a method described in (Wei et al., 2014a) where we sort the elements of V non-increasingly by G ( i | V \ i ) and then remove any element i from V having

Original: G ( i ) <
Revised: G ( i ) <

Original: G ( δ ( k ) | V \ δ ( k )) where δ ( k ) is k th element in the sorted permutation.
Revised: G ( δ ( k ) | V \ δ ( k )) where δ ( k ) is k th element in the sorted permutation.

Original: Any such element will never be chosen by the k -cardinality constrained greedy procedure because for any (cid:96) ∈ { 1 , 2 , . . .
Revised: Any such element will never be chosen by the k -cardinality constrained greedy procedure because for any (cid:96) ∈ { 1 , 2 , . . .

Original: , k } , and any set A , we have G ( δ ( (cid:96) )
Revised: , k } , and any set A , we have G ( δ ( (cid:96) )

Original: | A ) ≥ G ( δ ( (cid:96) )
Revised: | A ) ≥ G ( δ ( (cid:96) )

Original: | V \ δ ( (cid:96) ))
Revised: | V \ δ ( (cid:96) ))

Original: ≥ G ( δ ( k ) | V \ δ ( k )) >
Revised: ≥ G ( δ ( k ) | V \ δ ( k )) >

Original: G ( i ) ≥ G ( i | A ) and thus greedy would always be able to choose an element better than i .
Revised: G ( i ) ≥ G ( i | A ) and thus greedy would always be able to choose an element better than i .

Original: This method results in no reduction in approximation quality, although it might not yield any speedup at all.
Revised: This method results in no reduction in approximation quality, although it might not yield any speedup at all.

Original: But with a decreasing λ , G ( A ) becomes more modular, and the ﬁltering method can become more effective.
Revised: But with a decreasing λ , G ( A ) becomes more modular, and the ﬁltering method can become more effective.

Original: Other methods we can employ are those such as (Zhou et al., 2017; Mirzasoleiman et al., 2015), resulting in small reduction in approximation quality, but we do not describe these further.
Revised: Other methods we can employ are those such as (Zhou et al., 2017; Mirzasoleiman et al., 2015), resulting in small reduction in approximation quality, but we do not describe these further.

Original: The key contribution of this section is a method exploiting a potential warm start set that might already achieve a sufﬁcient approximation quality.
Revised: The key contribution of this section is a method exploiting a potential warm start set that might already achieve a sufﬁcient approximation quality.

Original: Normally, the greedy procedure starts with the empty set and adds elements greedily until a set of size k is reached.
Revised: Normally, the greedy procedure starts with the empty set and adds elements greedily until a set of size k is reached.

Original: In Algorithm 1, by contrast, a previous iteration has already solved a size- k constrained submodular maximization problem for the previous submodular function, the solution to which is one that could very nearly already satisfy a desired approximation bound for the current submodular function.
Revised: In Algorithm 1, by contrast, a previous iteration has already solved a size- k constrained submodular maximization problem for the previous submodular function, the solution to which is one that could very nearly already satisfy a desired approximation bound for the current submodular function.

Original: The reason for this is that, depending on the weight update method in line 9 of Algorithm 1 between inner loop iterations, and the changes to parameters ∆ and γ between outer iterations, the succession of submodular functions might not change very quickly.
Revised: The reason for this is that, depending on the weight update method in line 9 of Algorithm 1 between inner loop iterations, and the changes to parameters ∆ and γ between outer iterations, the succession of submodular functions might not change very quickly.

Original: For example, when the learning rate η is small, the ˆ A from the previous iteration could still be valued highly by the current iteration’s function, so running a greedy procedure from scratch is unnecessary.
Revised: For example, when the learning rate η is small, the ˆ A from the previous iteration could still be valued highly by the current iteration’s function, so running a greedy procedure from scratch is unnecessary.

Original: Our method warm-starts a submodular maximization process with a previously computed set, and offers a bound that trades off speed and approximation quality.
Revised: Our method warm-starts a submodular maximization process with a previously computed set, and offers a bound that trades off speed and approximation quality.

Original: The approach is given in Algorithm 2, which (after the aforementioned ﬁltering in line 3 (Wei et al., 2014a)) tests in linear time if the warm start set ˆ A already achieves a sufﬁcient approximation quality, and if so, possibly improves it further with an additional linear or quasilinear time computation.
Revised: The approach is given in Algorithm 2, which (after the aforementioned ﬁltering in line 3 (Wei et al., 2014a)) tests in linear time if the warm start set ˆ A already achieves a sufﬁcient approximation quality, and if so, possibly improves it further with an additional linear or quasilinear time computation.

Original: To test approximation quality of ˆ A , our approach uses a simple modular function upper bound, in line 4, to compute an upper bound on the global maximum value.
Revised: To test approximation quality of ˆ A , our approach uses a simple modular function upper bound, in line 4, to compute an upper bound on the global maximum value.

Original: For the subsequent improvement of ˆ A , our approach utilizes a submodular semigradient approach (Iyer et al., 2013) (speciﬁcally subgradients (Fujishige, 2005) in this case).
Revised: For the subsequent improvement of ˆ A , our approach utilizes a submodular semigradient approach (Iyer et al., 2013) (speciﬁcally subgradients (Fujishige, 2005) in this case).

Original: If the warm-start set ˆ A does not achieve sufﬁcient approximation quality in line 5, the algorithm backs off to standard submodular maximization in line
Revised: If the warm-start set ˆ A does not achieve sufﬁcient approximation quality in line 5, the algorithm backs off to standard submodular maximization in line

Original: 11 (we use the accelerated/lazy greedy procedure (Minoux, 1978) here although other methods, e.g., (Mirzasoleiman et al., 2015), can be used as well).
Revised: 11 (we use the accelerated/lazy greedy procedure (Minoux, 1978) here although other methods, e.g., (Mirzasoleiman et al., 2015), can be used as well).

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 1: Input: G ( ·
Revised: 1: Input: G ( ·

Original: , k , ˆ A , ˜
Revised: , k , ˆ A , ˜

Original: α 2: Output: ˜
Revised: α 2: Output: ˜

Original: A 3: Reduce ground set size: arrange V non-increasingly in terms of G ( i | V \ i ) in a permutation δ where δ ( k ) is the k th element, set V ← { i ∈ V | G ( i ) ≥ G ( δ ( k ) | V \ δ ( k )) } ; 4: Compute upper bound to maximum of Eq.
Revised: A 3: Reduce ground set size: arrange V non-increasingly in terms of G ( i | V \ i ) in a permutation δ where δ ( k ) is the k th element, set V ← { i ∈ V | G ( i ) ≥ G ( δ ( k ) | V \ δ ( k )) } ; 4: Compute upper bound to maximum of Eq.

Original: [Equation]
Revised: [Equation]

Original: 5: if G ( ˆ A ) ≥ ˜ α · τ then 6:
Revised: 5: if G ( ˆ A ) ≥ ˜ α · τ then 6:

Original: Permutation σ of V : the ﬁrst k elements have S σk =
Revised: Permutation σ of V : the ﬁrst k elements have S σk =

Original: ˆ A and are chosen ordered non-increasing by κ G ( v ) ; the remaining n − k elements V \
Revised: ˆ A and are chosen ordered non-increasing by κ G ( v ) ; the remaining n − k elements V \

Original: ˆ A for σ are chosen non-increasing by κ G ( v ) .
Revised: ˆ A for σ are chosen non-increasing by κ G ( v ) .

Original: Deﬁne modular function
Revised: Deﬁne modular function

Original: h ˆ A ( A ) (cid:44) (cid:80) i ∈
Revised: h ˆ A ( A ) (cid:44) (cid:80) i ∈

Original: A h ˆ
Revised: A h ˆ

Original: A
Revised: A

Original: ( i ) with h ˆ A ( σ ( i ))
Revised: ( i ) with h ˆ A ( σ ( i ))

Original: =
Revised: =

Original: G ( S σ i ) − G ( S σ i − 1 ) ; 8:
Revised: G ( S σ i ) − G ( S σ i − 1 ) ; 8:

Original: Compute tight, at ˆ A , lower bound L ( A ) of G ( A ) :
Revised: Compute tight, at ˆ A , lower bound L ( A ) of G ( A ) :

Original: [Equation]
Revised: [Equation]

Original: A
Revised: A

Original: [Equation]
Revised: [Equation]

Original: A ← argmax
Revised: A ← argmax

Original: ; 10: else 11:
Revised: ; 10: else 11:

Original: A
Revised: A

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ) ; 12: end if
Revised: ) ; 12: end if

Original: Line 4 computes the upper bound τ ≥ max A ∈ V, | A |≤ k G ( A ) which holds due to submodularity, requiring only a modular maximization problem (which can be done in O ( | V | ) time, independent of k , to select the top k elements).
Revised: Line 4 computes the upper bound τ ≥ max A ∈ V, | A |≤ k G ( A ) which holds due to submodularity, requiring only a modular maximization problem (which can be done in O ( | V | ) time, independent of k , to select the top k elements).

Original: Line 5 checks if an ˜ α approximation to this upper bound is achieved by the warm-start set ˆ A , and if not we back off to a standard submodular maximization procedure in line 11.
Revised: Line 5 checks if an ˜ α approximation to this upper bound is achieved by the warm-start set ˆ A , and if not we back off to a standard submodular maximization procedure in line 11.

Original: If ˆ A is an ˜ α approximation to the upper bound τ , then lines 6-9 runs a subgradient optimization procedure, a process that can potentially improve it further.
Revised: If ˆ A is an ˜ α approximation to the upper bound τ , then lines 6-9 runs a subgradient optimization procedure, a process that can potentially improve it further.

Original: The approach selects a subgradient deﬁned by a permutation σ = ( σ (1) , σ (2) , . . .
Revised: The approach selects a subgradient deﬁned by a permutation σ = ( σ (1) , σ (2) , . . .

Original: , σ ( n )) of the elements.
Revised: , σ ( n )) of the elements.

Original: The algorithm then deﬁnes a modular function L ( A ) , tight at ˆ A and a lower bound everywhere else, i.e., L ( ˆ A ) =
Revised: The algorithm then deﬁnes a modular function L ( A ) , tight at ˆ A and a lower bound everywhere else, i.e., L ( ˆ A ) =

Original: G ( ˆ A ) , and ∀ A, L ( A ) ≤ G ( A ) .
Revised: G ( ˆ A ) , and ∀ A, L ( A ) ≤ G ( A ) .

Original: Any permutation will achieve this as long as ˆ A
Revised: Any permutation will achieve this as long as ˆ A

Original: = { σ (1) , σ (2) , . . .
Revised: = { σ (1) , σ (2) , . . .

Original: , σ ( k ) } .
Revised: , σ ( k ) } .

Original: The speciﬁc permutation we use is described below.
Revised: The speciﬁc permutation we use is described below.

Original: Once we have the modular lower bound, we can do simple and fast modular maximization.
Revised: Once we have the modular lower bound, we can do simple and fast modular maximization.

Original: Lines 6-9 of Algorithm 2 offer a heuristic that can only improve the objective — letting ˜ A be the solution after line 9, we have
Revised: Lines 6-9 of Algorithm 2 offer a heuristic that can only improve the objective — letting ˜ A be the solution after line 9, we have

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst inequality follows since L ( · ) is a lower bound of G ( · ) ; the second inequality follows from the optimality of ˆ A + ; the equality follows since L is tight at ˆ A .
Revised: The ﬁrst inequality follows since L ( · ) is a lower bound of G ( · ) ; the second inequality follows from the optimality of ˆ A + ; the equality follows since L is tight at ˆ A .

Original: The approximation factor ˜ α is distinct from the submodular maximization approximation factor α achieved by the greedy algorithm.
Revised: The approximation factor ˜ α is distinct from the submodular maximization approximation factor α achieved by the greedy algorithm.

Original: Setting, for example ˜ α
Revised: Setting, for example ˜ α

Original: = 1 − 1 /e would ask for the previous solution to be this good relative to τ , the upper bound on the global maximum, and the algorithm would almost always immediately jump to line 11 since achieving such approximation quality might not even be possible in polynomial time (Feige, 1998).
Revised: = 1 − 1 /e would ask for the previous solution to be this good relative to τ , the upper bound on the global maximum, and the algorithm would almost always immediately jump to line 11 since achieving such approximation quality might not even be possible in polynomial time (Feige, 1998).

Original: With ˜ α large, we recover the approximation factor of the greedy algorithm but ignore the warm start.
Revised: With ˜ α large, we recover the approximation factor of the greedy algorithm but ignore the warm start.

Original: If ˜ α is small, many iterations might use the warm start from the previous iteration, updating it only via one step of subgradient optimization, but with a worse approximation factor.
Revised: If ˜ α is small, many iterations might use the warm start from the previous iteration, updating it only via one step of subgradient optimization, but with a worse approximation factor.

Original: In practice, therefore, we use a more lenient bound (often we set ˜ α
Revised: In practice, therefore, we use a more lenient bound (often we set ˜ α

Original: = 1 / 2 ) which is a good practical tradeoff between approximation accuracy and speed (meaning lines 6-9 execute a reasonable fraction of the time leading to a good speedup, i.e., in our experiments, the time cost for WS-S UBMODULAR M AX increases if α = 1 by a factor ranging from about 3 to 5).
Revised: = 1 / 2 ) which is a good practical tradeoff between approximation accuracy and speed (meaning lines 6-9 execute a reasonable fraction of the time leading to a good speedup, i.e., in our experiments, the time cost for WS-S UBMODULAR M AX increases if α = 1 by a factor ranging from about 3 to 5).

Original: In general, we have the following ﬁnal bound based on the smaller of ˜ α and α .
Revised: In general, we have the following ﬁnal bound based on the smaller of ˜ α and α .

Original: Lemma 2.
Revised: Lemma 2.

Original: Algorithm 2 outputs a solution ˆ A such that G ( ˆ A ) ≥ min { ˜ α, α }× max A ∈ V, | A |≤ k G ( A ) , where α is the approximation factor of the greedy procedure (typically α = (1 − e − κ G ) /κ G ).
Revised: Algorithm 2 outputs a solution ˆ A such that G ( ˆ A ) ≥ min { ˜ α, α }× max A ∈ V, | A |≤ k G ( A ) , where α is the approximation factor of the greedy procedure (typically α = (1 − e − κ G ) /κ G ).

Original: Proof.
Revised: Proof.

Original: Let A ∗ denote an optimal solution to Eq.
Revised: Let A ∗ denote an optimal solution to Eq.

Original: [Equation]
Revised: [Equation]

Original: τ computed in line 4 is an upper bound to G ( A ∗ ) since:
Revised: τ computed in line 4 is an upper bound to G ( A ∗ ) since:

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst inequality follows by the deﬁnition of τ ; the last inequality is due to submodularity, guaranteeing F ( i ) ≥ F ( i | B ) for any B ⊆ V .
Revised: The ﬁrst inequality follows by the deﬁnition of τ ; the last inequality is due to submodularity, guaranteeing F ( i ) ≥ F ( i | B ) for any B ⊆ V .

Original: When G ( ˆ A ) ≥ ˜ α · τ (line 5), the subgradient ascent can only improve the objective.
Revised: When G ( ˆ A ) ≥ ˜ α · τ (line 5), the subgradient ascent can only improve the objective.

Original: Thus, we have
Revised: Thus, we have

Original: G ( ˆ A ) ≥ ˜ α · max A ∈ V, | A |≤ k G ( A ) for ˆ A obtained in line 9.
Revised: G ( ˆ A ) ≥ ˜ α · max A ∈ V, | A |≤ k G ( A ) for ˆ A obtained in line 9.

Original: Otherwise, we run the greedy algorithm on the reduced ground set V .
Revised: Otherwise, we run the greedy algorithm on the reduced ground set V .

Original: Thus, we have G ( ˆ A ) ≥ α · max A ∈ V, | A |≤ k G ( A ) for ˆ A obtained in line 11.
Revised: Thus, we have G ( ˆ A ) ≥ α · max A ∈ V, | A |≤ k G ( A ) for ˆ A obtained in line 11.

Original: The heuristic in lines 6-9 is identical to one step of the semigradient-based minorization-maximization (MM) scheme used in, for example, (Narasimhan & Bilmes, 2005; Jegelka & Bilmes, 2011; Iyer &
Revised: The heuristic in lines 6-9 is identical to one step of the semigradient-based minorization-maximization (MM) scheme used in, for example, (Narasimhan & Bilmes, 2005; Jegelka & Bilmes, 2011; Iyer &

Original: Bilmes, 2012; Iyer et al., 2013).
Revised: Bilmes, 2012; Iyer et al., 2013).

Original: Which permutation to use for the subgradient in order to tighten the gap has been an issue discussed as far back as (Narasimhan & Bilmes, 2005).
Revised: Which permutation to use for the subgradient in order to tighten the gap has been an issue discussed as far back as (Narasimhan & Bilmes, 2005).

Original: In the present work, we offer a new heuristic for this problem.
Revised: In the present work, we offer a new heuristic for this problem.

Original: Let the ﬁrst i elements in the permutation σ be denoted
Revised: Let the ﬁrst i elements in the permutation σ be denoted

Original: S σi = { σ (1) , σ (2) , . . .
Revised: S σi = { σ (1) , σ (2) , . . .

Original: , σ ( i ) } , and let A σi − 1 (cid:44) { σ ( j ) ∈
Revised: , σ ( i ) } , and let A σi − 1 (cid:44) { σ ( j ) ∈

Original: A | j < i } =
Revised: A | j < i } =

Original: S σi − 1 ∩ A ⊆ S σi − 1 for any i ∈ A .
Revised: S σi − 1 ∩ A ⊆ S σi − 1 for any i ∈ A .

Original: The gap we wish to reduce is
Revised: The gap we wish to reduce is

Original: [Equation]
Revised: [Equation]

Original: Which follows since h ˆ A ( σ ( i ))
Revised: Which follows since h ˆ A ( σ ( i ))

Original: =
Revised: =

Original: G ( σ ( i ) | S σi − 1 ) by deﬁnition.
Revised: G ( σ ( i ) | S σi − 1 ) by deﬁnition.

Original: Line 6 chooses a particular permutation in an attempt to reduce this gap.
Revised: Line 6 chooses a particular permutation in an attempt to reduce this gap.

Original: Deﬁne an element-wise form of curvature as κ G ( v ) = 1 −
Revised: Deﬁne an element-wise form of curvature as κ G ( v ) = 1 −

Original: G ( v | V \ v ) /G
Revised: G ( v | V \ v ) /G

Original: ( v ) ∈
Revised: ( v ) ∈

Original: [0 , 1] for all v ∈ V .
Revised: [0 , 1] for all v ∈ V .

Original: Note that κ G = max v ∈ V κ G ( v ) .
Revised: Note that κ G = max v ∈ V κ G ( v ) .

Original: If κ G ( v ) ≈ 0 then G is practically modular at v and so G ( v | A )
Revised: If κ G ( v ) ≈ 0 then G is practically modular at v and so G ( v | A )

Original: ≈ G ( v ) for any set A ; in other words, G ( v | A ) is close to v ’s maximum possible gain even if v is ranked with index very late in the permutation σ where
Revised: ≈ G ( v ) for any set A ; in other words, G ( v | A ) is close to v ’s maximum possible gain even if v is ranked with index very late in the permutation σ where

Original: A is very large.
Revised: A is very large.

Original: If κ G ( v ) ≈ 1 , on the other hand, then there is some set A ⊆ V \ { v } that can appreciably reduce G ( v | A ) relative to the maximum possible gain G ( v ) , and so it is best to rank v very early in the order σ where A must be a small set.
Revised: If κ G ( v ) ≈ 1 , on the other hand, then there is some set A ⊆ V \ { v } that can appreciably reduce G ( v | A ) relative to the maximum possible gain G ( v ) , and so it is best to rank v very early in the order σ where A must be a small set.

Original: One heuristic to achieve these goals is to choose a permutation σ that arranges the elements in an order non-increasing according to κ G ( v ) , meaning κ G ( σ (1)) ≥ κ G ( σ (2)) ≥ . . . .
Revised: One heuristic to achieve these goals is to choose a permutation σ that arranges the elements in an order non-increasing according to κ G ( v ) , meaning κ G ( σ (1)) ≥ κ G ( σ (2)) ≥ . . . .

Original: Choosing this order is therefore an attempt to keep each of the conditional gains G ( σ ( i ) | S σi − 1 ) as close as possible to σ ( i ) ’s maximum possible gain, G ( σ ( i )) .
Revised: Choosing this order is therefore an attempt to keep each of the conditional gains G ( σ ( i ) | S σi − 1 ) as close as possible to σ ( i ) ’s maximum possible gain, G ( σ ( i )) .

Original: This corresponds to an attempt to reduce Eq.
Revised: This corresponds to an attempt to reduce Eq.

Original: (34) (and correspondingly close the G ( A ) − L ( A ) gap) as much as possible.
Revised: (34) (and correspondingly close the G ( A ) − L ( A ) gap) as much as possible.

Original: Line 6 of Algorithm 2 does this, subject to the requirement that the ﬁrst k elements of the permutation must correspond to ˆ
Revised: Line 6 of Algorithm 2 does this, subject to the requirement that the ﬁrst k elements of the permutation must correspond to ˆ

Original: A
Revised: A

Original: in order to be a subgradient.
Revised: in order to be a subgradient.

Original: These tricks all help lines 6-9 produce a better updated approximate maximizer but at appreciably increased speed.
Revised: These tricks all help lines 6-9 produce a better updated approximate maximizer but at appreciably increased speed.

Original: 4. A DDITIONAL R ESULTS
Revised: 4. A DDITIONAL R ESULTS

Original: This section concludes by, in the form of tables and plots, providing more information about our experiments and experimental results for the algorithms mentioned above.
Revised: This section concludes by, in the form of tables and plots, providing more information about our experiments and experimental results for the algorithms mentioned above.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Figure]
Revised: [Figure]

Original: t r a i n i n g e rr o r r a t e ( % )
Revised: t r a i n i n g e rr o r r a t e ( % )

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]