Original: T RAINING WITH G ROWING S ETS : A S IMPLE A LTERNATIVE TO C URRICULUM L EARNING AND S ELF P ACED L EARNING
Revised: T RAINING WITH G ROWING S ETS : A S IMPLE A LTERNATIVE TO C URRICULUM L EARNING AND S ELF P ACED L EARNING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Curriculum learning and Self paced learning are popular topics in the machine learning that suggest to put the training samples in order by considering their difﬁculty levels.
Revised: Curriculum learning and Self paced learning are popular topics in the machine learning that suggest to put the training samples in order by considering their difﬁculty levels.

Original: Studies in these topics show that starting with a small training set and adding new samples according to difﬁculty levels improves the learning performance.
Revised: Studies in these topics show that starting with a small training set and adding new samples according to difﬁculty levels improves the learning performance.

Original: In this paper we experimented that we can also obtain good results by adding the samples randomly without a meaningful order.
Revised: In this paper we experimented that we can also obtain good results by adding the samples randomly without a meaningful order.

Original: We compared our method with classical training, Curriculum learning, Self paced learning and their reverse ordered versions.
Revised: We compared our method with classical training, Curriculum learning, Self paced learning and their reverse ordered versions.

Original: Results of the statistical tests show that the proposed method is better than classical method and similar with the others.
Revised: Results of the statistical tests show that the proposed method is better than classical method and similar with the others.

Original: These results point a new training regime that removes the process of difﬁculty level determination in Curriculum and Self paced learning and as successful as these methods
Revised: These results point a new training regime that removes the process of difﬁculty level determination in Curriculum and Self paced learning and as successful as these methods

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Bengio et al.
Revised: Bengio et al.

Original: (2009) named Curriculum learning the idea of following an order related with difﬁculty of the samples during training which provides an optimization for non convex objectives.
Revised: (2009) named Curriculum learning the idea of following an order related with difﬁculty of the samples during training which provides an optimization for non convex objectives.

Original: After this many researchers tried to ﬁnd the most efﬁcient curriculum to get the best yield with this approach.
Revised: After this many researchers tried to ﬁnd the most efﬁcient curriculum to get the best yield with this approach.

Original: In Zaremba & Sutskever (2014)’s study conventional curriculum learning did not work so well and they developed a new version.
Revised: In Zaremba & Sutskever (2014)’s study conventional curriculum learning did not work so well and they developed a new version.

Original: Shi et al.
Revised: Shi et al.

Original: (2015) proposed three different curriculum strategies for language model adaptation of recurrent neural networks.
Revised: (2015) proposed three different curriculum strategies for language model adaptation of recurrent neural networks.

Original: In the ﬁeld of computer vision Pentina et al.
Revised: In the ﬁeld of computer vision Pentina et al.

Original: (2015) looked for the best order of tasks to learn.
Revised: (2015) looked for the best order of tasks to learn.

Original: Although these models have better generalization performance with the proposed curriculum methods it is not known whether the tried methods ensures the best curriculum.
Revised: Although these models have better generalization performance with the proposed curriculum methods it is not known whether the tried methods ensures the best curriculum.

Original: A curriculum is work-speciﬁc so could not be applicable for another work.
Revised: A curriculum is work-speciﬁc so could not be applicable for another work.

Original: In order to use the curriculum logic in different applications Kumar et al. (2010) suggested a method that the learner decides itself which samples are easy or difﬁcult at every stage.
Revised: In order to use the curriculum logic in different applications Kumar et al. (2010) suggested a method that the learner decides itself which samples are easy or difﬁcult at every stage.

Original: This method called Self paced learning was combined with Curriculum learning which provides prior information by Jiang et al.
Revised: This method called Self paced learning was combined with Curriculum learning which provides prior information by Jiang et al.

Original: In another work Graves et al.
Revised: In another work Graves et al.

Original: (2017) introduced a method to automatically select the syllabus to follow for the neural networks.
Revised: (2017) introduced a method to automatically select the syllabus to follow for the neural networks.

Original: Matiisen et al.
Revised: Matiisen et al.

Original: (2017) also proposed a way to learn simple subtasks before the complex tasks and achieved better results than using manually designed curriculum.
Revised: (2017) also proposed a way to learn simple subtasks before the complex tasks and achieved better results than using manually designed curriculum.

Original: In some cases higher learning performance could be obtainable by adding some noises to easy-tohard ordering of the samples.
Revised: In some cases higher learning performance could be obtainable by adding some noises to easy-tohard ordering of the samples.

Original: Jiang et al.
Revised: Jiang et al.

Original: (2014) gave preference to both easy and diverse samples and outperform the conventional Self paced learning (Kumar et al.
Revised: (2014) gave preference to both easy and diverse samples and outperform the conventional Self paced learning (Kumar et al.

Original: algorithm.
Revised: algorithm.

Original: Emphasizing the uncertain samples suggested by Chang et al.
Revised: Emphasizing the uncertain samples suggested by Chang et al.

Original: (2017) lead to more accurate and robust SGD training.
Revised: (2017) lead to more accurate and robust SGD training.

Original: Avramova (2015) explored the inversed versions of the Self paced learning and Self paced learning with diversity (Jiang et al.
Revised: Avramova (2015) explored the inversed versions of the Self paced learning and Self paced learning with diversity (Jiang et al.

Original: and demonstrated that these methods performed slightly better than their standard variants.
Revised: and demonstrated that these methods performed slightly better than their standard variants.

Original: Consistent with the literature we have showed in our previous work () that using both curriculum and anti-curriculum strategies improving generalization performance in a wide application area.
Revised: Consistent with the literature we have showed in our previous work () that using both curriculum and anti-curriculum strategies improving generalization performance in a wide application area.

Original: These researches brings a question to minds: While it is natural and logical to obtain better results by sorting the samples from easy-to-hard why it is also better to sort the samples from hard-to-easy?
Revised: These researches brings a question to minds: While it is natural and logical to obtain better results by sorting the samples from easy-to-hard why it is also better to sort the samples from hard-to-easy?

Original: In this study we point that to start with a small training set and add new samples in both curriculum and anti-curriculum learning makes these methods better.
Revised: In this study we point that to start with a small training set and add new samples in both curriculum and anti-curriculum learning makes these methods better.

Original: So we claim that it is possible to have
Revised: So we claim that it is possible to have

Original: [Table]
Revised: [Table]

Original: better results only by adding new samples stage-by-stage without a meaningful order.
Revised: better results only by adding new samples stage-by-stage without a meaningful order.

Original: We experimented two ordering types related with difﬁculty (easy-to-hard and hard-to-easy) and our method without a meaningful order.
Revised: We experimented two ordering types related with difﬁculty (easy-to-hard and hard-to-easy) and our method without a meaningful order.

Original: Training was carried out by adding a new group to the training set at every stage.
Revised: Training was carried out by adding a new group to the training set at every stage.

Original: We compared the proposed method with two strategies.
Revised: We compared the proposed method with two strategies.

Original: First one is Curriculum learning which we give the difﬁculty levels of the samples as pre-information.
Revised: First one is Curriculum learning which we give the difﬁculty levels of the samples as pre-information.

Original: Second one is Self paced learning which the trained network determines the difﬁculty levels of the samples at each stage.
Revised: Second one is Self paced learning which the trained network determines the difﬁculty levels of the samples at each stage.

Original: All methods including usual baseline training have been compared by using paired T-test and the results are examined.
Revised: All methods including usual baseline training have been compared by using paired T-test and the results are examined.

Original:  M ETHODS
Revised:  M ETHODS

Original: We explored 2 ordering types determined by 2 different strategies and our method without ordering.
Revised: We explored 2 ordering types determined by 2 different strategies and our method without ordering.

Original: 2. C URRICULUM LEARNING
Revised: 2. C URRICULUM LEARNING

Original: In the Curriculum learning(CL) learner follows a pre-determined curriculum during training.
Revised: In the Curriculum learning(CL) learner follows a pre-determined curriculum during training.

Original: For this reason it is necessary to know the difﬁculty of the samples.
Revised: For this reason it is necessary to know the difﬁculty of the samples.

Original: This information can be given by deﬁning the speciﬁc curriculum of the task or labeling the samples at the beginning.
Revised: This information can be given by deﬁning the speciﬁc curriculum of the task or labeling the samples at the beginning.

Original: Even though labeling with difﬁculty levels is easy for artiﬁcial data sets, it is costly and demanding for real world data sets.
Revised: Even though labeling with difﬁculty levels is easy for artiﬁcial data sets, it is costly and demanding for real world data sets.

Original: Besides that, a sample which is easy for humans may not be easy for machines.
Revised: Besides that, a sample which is easy for humans may not be easy for machines.

Original: In this study, we used an ensemble method to automatically determine the difﬁculty of the samples.
Revised: In this study, we used an ensemble method to automatically determine the difﬁculty of the samples.

Original: We created an ensemble with Bagging method(Breiman (1996)) and calculated decision consistency for each training sample accordingly the difﬁculty levels are determined.
Revised: We created an ensemble with Bagging method(Breiman (1996)) and calculated decision consistency for each training sample accordingly the difﬁculty levels are determined.

Original: Then, the ordered training set which is easy-to-hard was grouped for Curriculum learning and hard-to-easy ordered training set was grouped for Anti-curriculum learning(ACL).
Revised: Then, the ordered training set which is easy-to-hard was grouped for Curriculum learning and hard-to-easy ordered training set was grouped for Anti-curriculum learning(ACL).

Original: For CL, the samples are sorted ﬁrstly then separated into same sized groups by dividing the number of training samples into the number of stages.
Revised: For CL, the samples are sorted ﬁrstly then separated into same sized groups by dividing the number of training samples into the number of stages.

Original: Remainings are added to the ﬁrst stage so that the number of new samples to add at each stage is equal.
Revised: Remainings are added to the ﬁrst stage so that the number of new samples to add at each stage is equal.

Original: Training starts from the easiest group and continues by adding the groups one-by-one according to the easiness.
Revised: Training starts from the easiest group and continues by adding the groups one-by-one according to the easiness.

Original: Hardest group is given with the entire training set at the last stage and training is completed in stages same number as groups.
Revised: Hardest group is given with the entire training set at the last stage and training is completed in stages same number as groups.

Original: For ACL, the training set is sorted by difﬁculty in descending order and then separated into groups.
Revised: For ACL, the training set is sorted by difﬁculty in descending order and then separated into groups.

Original: Training starts with the group that is the most difﬁcult.
Revised: Training starts with the group that is the most difﬁcult.

Original: At the last stage of the training, the group includes the easiest samples with the entire training set.
Revised: At the last stage of the training, the group includes the easiest samples with the entire training set.

Original: The groups given at each stage are shown in Table 1 when the training set considered as sorted according to the difﬁculty levels.
Revised: The groups given at each stage are shown in Table 1 when the training set considered as sorted according to the difﬁculty levels.

Original: At each step a new group of samples are given together with the previous samples for both methods.
Revised: At each step a new group of samples are given together with the previous samples for both methods.

Original: In the last stage the entire training set is given like the baseline training.
Revised: In the last stage the entire training set is given like the baseline training.

Original: The path for CL is given in
Revised: The path for CL is given in

Original: Algorithm 1.
Revised: Algorithm 1.

Original: The same algorithm is applied for ACL with reverse order training set.
Revised: The same algorithm is applied for ACL with reverse order training set.

Original: Curriculum learning starts with random initialized weights in the ﬁrst stage.
Revised: Curriculum learning starts with random initialized weights in the ﬁrst stage.

Original: The optimum weights found at each stage taken as the initial weights of the next stage.
Revised: The optimum weights found at each stage taken as the initial weights of the next stage.

Original: In other words, next stage of the optimization starts from the minimum found in the previous stage.
Revised: In other words, next stage of the optimization starts from the minimum found in the previous stage.

Original: 2. S ELF PACED LEARNING
Revised: 2. S ELF PACED LEARNING

Original: Self paced learning(SPL) is a solution for ﬁnding the difﬁculty levels in Curriculum learning.
Revised: Self paced learning(SPL) is a solution for ﬁnding the difﬁculty levels in Curriculum learning.

Original: In this method learner determines the instances to learn according to the current situation of the objective
Revised: In this method learner determines the instances to learn according to the current situation of the objective

Original: 1: D ← training set sorted by easiness 2: n ← number of stages 3: s ← number of samples to add at each stage 4: f ← non-convex objective function of neural network with parameters θ 5: procedure CL( D, n, s, f ) 6: Randomly initialize the parameters θ7: D 

Revised: 1: D ← training set sorted by easiness 2: n ← number of stages 3: s ← number of samples to add at each stage 4: f ← non-convex objective function of neural network with parameters θ 5: procedure CL( D, n, s, f ) 6: Randomly initialize the parameters θ7: D

Original: = ﬁrst s sample of D 8: θ 1 = argmin θ f
Revised: = ﬁrst s sample of D 8: θ 1 = argmin θ f

Original: ( D 0 , θ 0 ) 9: for t = 1 to n − 1 do 10:
Revised: ( D 0 , θ 0 ) 9: for t = 1 to n − 1 do 10:

Original: D t =
Revised: D t =

Original: ﬁrst ( t + 1) ∗ s sample of D 11: θ t +1 =
Revised: ﬁrst ( t + 1) ∗ s sample of D 11: θ t +1 =

Original: argmin θ f
Revised: argmin θ f

Original: ( D t , θ t ) 12: end for 13: return θ n 14: end procedure function.
Revised: ( D t , θ t ) 12: end for 13: return θ n 14: end procedure function.

Original: Training starts with a group of random samples and the samples that best ﬁt the current model space are labeled as easy at each step.
Revised: Training starts with a group of random samples and the samples that best ﬁt the current model space are labeled as easy at each step.

Original: Weights are updated by easy labeled examples at the next step.
Revised: Weights are updated by easy labeled examples at the next step.

Original: The network is trained with more examples in the next step and all training set is given in the ﬁnal step.
Revised: The network is trained with more examples in the next step and all training set is given in the ﬁnal step.

Original: This is realized by annealing the self pace parameter.
Revised: This is realized by annealing the self pace parameter.

Original: Same number of samples at each stage can be added to the training set to make the training progress homogeneous.
Revised: Same number of samples at each stage can be added to the training set to make the training progress homogeneous.

Original: In this work the only parameter to set for SPL is the number of stages.
Revised: In this work the only parameter to set for SPL is the number of stages.

Original: The number of samples to add at each stage is equal like as CL.
Revised: The number of samples to add at each stage is equal like as CL.

Original: Training starts with randomly selected samples.
Revised: Training starts with randomly selected samples.

Original: At the end of each stage, the samples are sorted by ascending order according to the losses calculated with MSE.
Revised: At the end of each stage, the samples are sorted by ascending order according to the losses calculated with MSE.

Original: The samples to give in the next stage are selected from this order.
Revised: The samples to give in the next stage are selected from this order.

Original: The most consistent samples with the model space are selected for the next stage.
Revised: The most consistent samples with the model space are selected for the next stage.

Original: For the Self paced learning-Inversed(SPLI), the most outlying samples have priority.
Revised: For the Self paced learning-Inversed(SPLI), the most outlying samples have priority.

Original: Same as SPL, training starts with random samples.
Revised: Same as SPL, training starts with random samples.

Original: Training samples in the next stage are selected by considering the incompatibility with the model space.
Revised: Training samples in the next stage are selected by considering the incompatibility with the model space.

Original: All training set is sorted by loss in descending order, samples with the highest loss are selected for the next stage.
Revised: All training set is sorted by loss in descending order, samples with the highest loss are selected for the next stage.

Original: The path for SPL is given in Algorithm 2.
Revised: The path for SPL is given in Algorithm 2.

Original: The same algorithm is applied for SPLI, only training set is sorted by reverse order at each stage.
Revised: The same algorithm is applied for SPLI, only training set is sorted by reverse order at each stage.

Original: Training in the next stage is started with using previous solutions in SPL also as in CL.
Revised: Training in the next stage is started with using previous solutions in SPL also as in CL.

Original: SPL does not guarantee that the all samples from previous stage will be taken again in the next stage.
Revised: SPL does not guarantee that the all samples from previous stage will be taken again in the next stage.

Original: The samples best adapted with the objective function are taken among all training samples and some of the samples used in the previous stage may not be inside these samples.
Revised: The samples best adapted with the objective function are taken among all training samples and some of the samples used in the previous stage may not be inside these samples.

Original: For example, if there is noisy samples in the ﬁrst stage these probably will not be in the training set at the second stage.
Revised: For example, if there is noisy samples in the ﬁrst stage these probably will not be in the training set at the second stage.

Original: 1: T ← random ordered training set 2: n ← number of stages 3: s ← number of samples to add at each stage 4: f ← non-convex objective function of neural network with parameters θ 5: procedure SPL( T, n, s, f ) 6: Randomly initialize the parameters θ7: T 

Revised: 1: T ← random ordered training set 2: n ← number of stages 3: s ← number of samples to add at each stage 4: f ← non-convex objective function of neural network with parameters θ 5: procedure SPL( T, n, s, f ) 6: Randomly initialize the parameters θ7: T

Original: = randomly chosen s sample from T 8: θ 1 = argmin θ f ( T 0 , θ 0 ) 9: for t = 1 to n − 1 do 10:
Revised: = randomly chosen s sample from T 8: θ 1 = argmin θ f ( T 0 , θ 0 ) 9: for t = 1 to n − 1 do 10:

Original: D t = training set sorted by loss of f ( T, θ t ) ascending order 11:
Revised: D t = training set sorted by loss of f ( T, θ t ) ascending order 11:

Original: T t =
Revised: T t =

Original: ﬁrst ( t + 1) ∗ s sample of D t 12: θ t +1 =
Revised: ﬁrst ( t + 1) ∗ s sample of D t 12: θ t +1 =

Original: argmin θ f ( T t , θ t ) 13: end for 14: return θ n 15: end procedure
Revised: argmin θ f ( T t , θ t ) 13: end for 14: return θ n 15: end procedure

Original: 2. R ANDOM ORDERED GROWING SETS
Revised: 2. R ANDOM ORDERED GROWING SETS

Original: CL, SPL and their inverse versions have a common point: At each next step the number of training samples is increasing with addition of a new group.
Revised: CL, SPL and their inverse versions have a common point: At each next step the number of training samples is increasing with addition of a new group.

Original: Due to the both versions (easy-to-hard and hard-to-easy) have better performance than the classical method it is considered that this common feature may provide an optimization.
Revised: Due to the both versions (easy-to-hard and hard-to-easy) have better performance than the classical method it is considered that this common feature may provide an optimization.

Original: We investigated the total weight change in each iteration for all methods in the Appendix A.
Revised: We investigated the total weight change in each iteration for all methods in the Appendix A.

Original: In this case, it is important to train with accumulating groups rather than giving the samples with a meaningful order.
Revised: In this case, it is important to train with accumulating groups rather than giving the samples with a meaningful order.

Original: Therefore, dividing the unordered training set into the groups and adding a new group at each stage could also raise the generalization performance.
Revised: Therefore, dividing the unordered training set into the groups and adding a new group at each stage could also raise the generalization performance.

Original: In the Random ordered growing sets(ROGS) method unsorted training set is divided into groups with equal number of samples.
Revised: In the Random ordered growing sets(ROGS) method unsorted training set is divided into groups with equal number of samples.

Original: Training is carried out in the same fashion as starting with the ﬁrst group then adding a new group at each stage without a meaningful order.
Revised: Training is carried out in the same fashion as starting with the ﬁrst group then adding a new group at each stage without a meaningful order.

Original: Algorithm 3 shows the way of training with Random ordered growing sets.
Revised: Algorithm 3 shows the way of training with Random ordered growing sets.

Original: Same as the other methods, ROGS uses the found solutions in the previous stages as initial weights in the next stage.
Revised: Same as the other methods, ROGS uses the found solutions in the previous stages as initial weights in the next stage.

Original: end end
Revised: end end

Original: Examined 5 methods ﬁnd the optimum weights with the same objective:
Revised: Examined 5 methods ﬁnd the optimum weights with the same objective:

Original: [Equation]
Revised: [Equation]

Original: where s denotes the number of samples to add at each stage, parameters at each stage determined by same amount of samples for all methods.
Revised: where s denotes the number of samples to add at each stage, parameters at each stage determined by same amount of samples for all methods.

Original: x i , y i are taken from pre-sorted easy-to-hard training set for CL, from pre-sorted hard-to-easy training set for ACL, from the training set that sorted easy-tohard at each stage for SPL and from the training set that sorted hard-to-easy at each stage for SPLI.
Revised: x i , y i are taken from pre-sorted easy-to-hard training set for CL, from pre-sorted hard-to-easy training set for ACL, from the training set that sorted easy-tohard at each stage for SPL and from the training set that sorted hard-to-easy at each stage for SPLI.

Original: In ROGS method samples are selected from non-ordered training set.
Revised: In ROGS method samples are selected from non-ordered training set.

Original:  A THEORETICAL EXPLANATION ABOUT TRAINING WITH GROWING SETS
Revised:  A THEORETICAL EXPLANATION ABOUT TRAINING WITH GROWING SETS

Original: Deﬁnitions: (a) The loss function (cid:96) ( θ ) with one parameter θ ∈ R is given as in (2) where N is the number of training instances and the loss function for each individual instance (cid:96) i ( θ ) is given as in (3).
Revised: Deﬁnitions: (a) The loss function (cid:96) ( θ ) with one parameter θ ∈ R is given as in (2) where N is the number of training instances and the loss function for each individual instance (cid:96) i ( θ ) is given as in (3).

Original: [Equation]
Revised: [Equation]

Original: (b) The point θ B is a deﬁnite local minimum of (cid:96) ( θ ) .
Revised: (b) The point θ B is a deﬁnite local minimum of (cid:96) ( θ ) .

Original: Geometric representation of a deﬁnite local minimum point is shown in Figure 1(a) with the loss functions.
Revised: Geometric representation of a deﬁnite local minimum point is shown in Figure 1(a) with the loss functions.

Original: We show the loss functions for instances with thin lines and their average with the bold line.
Revised: We show the loss functions for instances with thin lines and their average with the bold line.

Original: If we denote the expected value of (cid:96) ( θ ) as E [ (cid:96) ( θ )] , expected values of the loss function in the given points can be ordered as in (4).
Revised: If we denote the expected value of (cid:96) ( θ ) as E [ (cid:96) ( θ )] , expected values of the loss function in the given points can be ordered as in (4).

Original: [Equation]
Revised: [Equation]

Original: Expected values of the derivatives of the loss function in the given points can be written as in (5) and the derivatives are given in Figure 1(b).
Revised: Expected values of the derivatives of the loss function in the given points can be written as in (5) and the derivatives are given in Figure 1(b).

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation] 
Revised: [Equation] [Figure] [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: ) We deﬁne the complexity as the number of critical points and denote with κ .
Revised: ) We deﬁne the complexity as the number of critical points and denote with κ .

Original: Complexity of the
Revised: Complexity of the

Original: Assumptions: (a) Complexity of the loss functions of each individual instance can be as much as the complexity of the loss function of all instances.
Revised: Assumptions: (a) Complexity of the loss functions of each individual instance can be as much as the complexity of the loss function of all instances.

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: (b) More than half of the individual instances are less complex than average of all instances.
Revised: (b) More than half of the individual instances are less complex than average of all instances.

Original: [Equation]
Revised: [Equation]

Original: Lemma 1: Probability density function of the derivatives in the θ B point has a skewed distribution with zero mean.
Revised: Lemma 1: Probability density function of the derivatives in the θ B point has a skewed distribution with zero mean.

Original: Proof: Derivatives of the loss functions is under the point θ B in Figure 1(b) for more than half of the individual instances.
Revised: Proof: Derivatives of the loss functions is under the point θ B in Figure 1(b) for more than half of the individual instances.

Original: However (cid:96) (cid:48)
Revised: However (cid:96) (cid:48)

Original: i ( θ B ) values are high for the above instances.
Revised: i ( θ B ) values are high for the above instances.

Original: If we denote the probability density function of the derivatives in the θ B as Pr ( (cid:96) (cid:48) i ( θ B )) ,
Revised: If we denote the probability density function of the derivatives in the θ B as Pr ( (cid:96) (cid:48) i ( θ B )) ,

Original: 1) Deﬁnition (c) gives that θ B is a local minimum therefore Pr ( (cid:96) (cid:48)
Revised: 1) Deﬁnition (c) gives that θ B is a local minimum therefore Pr ( (cid:96) (cid:48)

Original: i ( θ B )) has zero mean.
Revised: i ( θ B )) has zero mean.

Original: 2) If κ ( (cid:96) ( θ )) = 2 as in Figure 1(b) then κ ( (cid:96) i ( θ )) <
Revised: 2) If κ ( (cid:96) ( θ )) = 2 as in Figure 1(b) then κ ( (cid:96) i ( θ )) <

Original: = 2 from Assumption (a) and κ ( (cid:96) i ( θ )) <for more than half of the instances from Assumption (b).
Revised: = 2 from Assumption (a) and κ ( (cid:96) i ( θ )) <for more than half of the instances from Assumption (b).

Original: (cid:96) (cid:48)
Revised: (cid:96) (cid:48)

Original: i ( θ B ) < 0 for the instances that has κ ( (cid:96) i ( θ )) < 2 .
Revised: i ( θ B ) < 0 for the instances that has κ ( (cid:96) i ( θ )) < 2 .

Original: (cid:96) (cid:48)
Revised: (cid:96) (cid:48)

Original: i ( θ B ) < 0 for more than half of the instances therefore Pr ( (cid:96) (cid:48)
Revised: i ( θ B ) < 0 for more than half of the instances therefore Pr ( (cid:96) (cid:48)

Original: i ( θ B )) has a skewed distribution.
Revised: i ( θ B )) has a skewed distribution.

Original: Corollary 1: If we take a subset with k instances from the training set we can denote:
Revised: Corollary 1: If we take a subset with k instances from the training set we can denote:

Original: [Equation]
Revised: [Equation]

Original: Expected value for the derivative of the loss function of the subset at the point θ B is less than zero.
Revised: Expected value for the derivative of the loss function of the subset at the point θ B is less than zero.

Original: [Equation]
Revised: [Equation]

Original: Proof: Probability of being negative for E [ (cid:96) (cid:48) s ( θ B )] depends on k and always high from Lemma 1.
Revised: Proof: Probability of being negative for E [ (cid:96) (cid:48) s ( θ B )] depends on k and always high from Lemma 1.

Original: Theorem 1: If we take a subset with k instances and train with batch gradient descent, we obtain a better local minimum for this subset.
Revised: Theorem 1: If we take a subset with k instances and train with batch gradient descent, we obtain a better local minimum for this subset.

Original: Proof: Optimization in the (cid:96) ( θ ) surface which starts from θ A will probably end at θ B .
Revised: Proof: Optimization in the (cid:96) ( θ ) surface which starts from θ A will probably end at θ B .

Original: However in the (cid:96) s ( θ ) surface optimization will be continue to ﬁnd a better minimum for the current set according to Corollary 1.
Revised: However in the (cid:96) s ( θ ) surface optimization will be continue to ﬁnd a better minimum for the current set according to Corollary 1.

Original: Theorem 2: To continue the optimization with all samples provides a better local minimum for all training set than stopping at the minimum of the subset.
Revised: Theorem 2: To continue the optimization with all samples provides a better local minimum for all training set than stopping at the minimum of the subset.

Original: Proof: Error surface for all instances ( (cid:96) ( θ )) and some subsample instances ( (cid:96) sx ( θ ) , (cid:96) sy ( θ ) , (cid:96) sz ( θ ) , (cid:96) sw ( θ )) are given in Figure 2.
Revised: Proof: Error surface for all instances ( (cid:96) ( θ )) and some subsample instances ( (cid:96) sx ( θ ) , (cid:96) sy ( θ ) , (cid:96) sz ( θ ) , (cid:96) sw ( θ )) are given in Figure 2.

Original: These examples shows all possible situations for the stopping points by considering θ A , θ B and θ C .
Revised: These examples shows all possible situations for the stopping points by considering θ A , θ B and θ C .

Original: We suppose to start the optimization from θ A
Revised: We suppose to start the optimization from θ A

Original: If we use (cid:96) ( θ ) it will be stop at θ B . -
Revised: If we use (cid:96) ( θ ) it will be stop at θ B . -

Original: If we use (cid:96) sx ( θ ) it will be stop at θ sx in the range θ B < θ sx < θ C .
Revised: If we use (cid:96) sx ( θ ) it will be stop at θ sx in the range θ B < θ sx < θ C .

Original: If we ﬁnish the optimization here, error on the surface for all instances will be high (  (cid:96) ( θ sx ) <
Revised: If we ﬁnish the optimization here, error on the surface for all instances will be high ( E [ (cid:96) ( θ sx )] >

Original:  (cid:96) ( θ B )) .
Revised: E [ (cid:96) ( θ B )]) .

Original: Therefore optimizing in the (cid:96) ( θ ) subsequently prevents to stay at a worse point.
Revised: Therefore optimizing in the (cid:96) ( θ ) subsequently prevents to stay at a worse point.

Original: - If we use (cid:96) ( θ sy ) it will be stop at θ sy in the range θ A < θ sy < θ B .
Revised: - If we use (cid:96) ( θ sy ) it will be stop at θ sy in the range θ A < θ sy < θ B .

Original: Here  (cid:96) ( θ sy ) <
Revised: Here E [ (cid:96) ( θ sy )] >

Original:  (cid:96) ( θ B ) therefore optimization for all instances is necessary.
Revised: E [ (cid:96) ( θ B )] therefore optimization for all instances is necessary.

Original: - If we use (cid:96) ( θ sz ) it will be stop at θ sz in the range θ C < θ sz and it is possible to obtain a better local minimum with optimization on (cid:96) ( θ ) surface by starting from the local minimum of the subset.
Revised: - If we use (cid:96) ( θ sz ) it will be stop at θ sz in the range θ C < θ sz and it is possible to obtain a better local minimum with optimization on (cid:96) ( θ ) surface by starting from the local minimum of the subset.

Original: - If we use (cid:96) ( θ sw ) it will be stop at θ sw in the range θ A > θ sw .
Revised: - If we use (cid:96) ( θ sw ) it will be stop at θ sw in the range θ A > θ sw .

Original: Here the stopping point can be worse for all instances.
Revised: Here the stopping point can be worse for all instances.

Original: Therefore it is possible to avoid from worse point by optimization on the (cid:96) ( θ ) surface.
Revised: Therefore it is possible to avoid from worse point by optimization on the (cid:96) ( θ ) surface.

Original: [Table]
Revised: [Table]

Original: Theorem 3: We can obtain a better local minimum with training with growing sets.
Revised: Theorem 3: We can obtain a better local minimum with training with growing sets.

Original: Proof:
Revised: Proof:

Original: Subset of the training set provides to get a better local minimum as in Theorem 1.
Revised: Subset of the training set provides to get a better local minimum as in Theorem 1.

Original: By the same way subset of the subset surface ( (cid:96) ss ( θ )) provides a better local minimum for the subset ( (cid:96) s ( θ )) .
Revised: By the same way subset of the subset surface ( (cid:96) ss ( θ )) provides a better local minimum for the subset ( (cid:96) s ( θ )) .

Original: To optimize the surface of the whole training set ﬁrst optimize the following surfaces respectively:
Revised: To optimize the surface of the whole training set ﬁrst optimize the following surfaces respectively:

Original: [Equation]
Revised: [Equation]

Original:  I MPLEMENTATION
Revised:  I MPLEMENTATION

Original: We trained 3-layer artiﬁcial neural network which has 10 neurons in the hidden layer with stochastic gradient descent with momentum for all methods.
Revised: We trained 3-layer artiﬁcial neural network which has 10 neurons in the hidden layer with stochastic gradient descent with momentum for all methods.

Original: Stop condition is the rising validation error series for 6 times during the training.
Revised: Stop condition is the rising validation error series for 6 times during the training.

Original: Baseline and all other methods applied with incremental training.
Revised: Baseline and all other methods applied with incremental training.

Original: We set the number of stages as 25 for all methods with growing sets.
Revised: We set the number of stages as 25 for all methods with growing sets.

Original: The network must provide the stopping condition at each stage to pass the next stage.
Revised: The network must provide the stopping condition at each stage to pass the next stage.

Original: Optimized weights in the previous stage are given as initial weights in the next stage.
Revised: Optimized weights in the previous stage are given as initial weights in the next stage.

Original: We have tested the suggested methods in 36 data sets retrieved from UCI repository 1 with the sample number range from 57 to 20000.
Revised: We have tested the suggested methods in 36 data sets retrieved from UCI repository 1 with the sample number range from 57 to 20000.

Original: We divide each data set 5 folds, 3 for training, 1 for validation and 1 for testing at each experiment.
Revised: We divide each data set 5 folds, 3 for training, 1 for validation and 1 for testing at each experiment.

Original: For each data set 20 error rates (MSE) obtained with 4x5 fold cross validation are compared by 0.95 signiﬁcance level paired T-test.
Revised: For each data set 20 error rates (MSE) obtained with 4x5 fold cross validation are compared by 0.95 signiﬁcance level paired T-test.

Original: Results of the comparisons are in Table 2.
Revised: Results of the comparisons are in Table 2.

Original: (Abbreviations are as follows:
Revised: (Abbreviations are as follows:

Original: Curriculum learning=CL, Anti-curriculum learning=
Revised: Curriculum learning=CL, Anti-curriculum learning=

Original: ACL, Self paced Learning=SPL, Self paced learning-Inversed
Revised: ACL, Self paced Learning=SPL, Self paced learning-Inversed

Original: =SPLI, Random ordered growing sets=ROGS)
Revised: =SPLI, Random ordered growing sets=ROGS)

Original: Each cell contains the win/tie/loss information for corresponding row against corresponding column.
Revised: Each cell contains the win/tie/loss information for corresponding row against corresponding column.

Original: For example, Curriculum learning wins versus Baseline in 12 data sets, ties indata sets, not loses in any data set.
Revised: For example, Curriculum learning wins versus Baseline in 12 data sets, ties indata sets, not loses in any data set.

Original: Results of the comparisons with Baseline for all data sets are given in the Appendix B with the number of samples, number of features, number of classes and average Baseline MSE.
Revised: Results of the comparisons with Baseline for all data sets are given in the Appendix B with the number of samples, number of features, number of classes and average Baseline MSE.

Original: If the comparison result of the corresponding method against Baseline is win it is marked with W, tie marked with T and loss marked with L.
Revised: If the comparison result of the corresponding method against Baseline is win it is marked with W, tie marked with T and loss marked with L.

Original: It is seen that training with ROGS is better than Baseline method in 17 data sets.
Revised: It is seen that training with ROGS is better than Baseline method in 17 data sets.

Original: Considering the comparisons with the Baseline, ROGS method wins in more data sets than CL, ACL and SPL.
Revised: Considering the comparisons with the Baseline, ROGS method wins in more data sets than CL, ACL and SPL.

Original: Obtaining good results without a meaningful order shows that only giving the training set as growing subsets provides an optimization.
Revised: Obtaining good results without a meaningful order shows that only giving the training set as growing subsets provides an optimization.

Original: Additionally, SPLI method wins against Baseline in more than half of the data sets.
Revised: Additionally, SPLI method wins against Baseline in more than half of the data sets.

Original: This method seems as the best method in terms of total number of wins.
Revised: This method seems as the best method in terms of total number of wins.

Original: It is striking that CL and SPL methods did not lose against Baseline in any data set.
Revised: It is striking that CL and SPL methods did not lose against Baseline in any data set.

Original: This shows the robustness of these methods in noisy data sets.
Revised: This shows the robustness of these methods in noisy data sets.

Original: It is possible to give priority to noises with hard-toeasy and random ordered methods so training may be misdirected in the data sets with high error rates.
Revised: It is possible to give priority to noises with hard-toeasy and random ordered methods so training may be misdirected in the data sets with high error rates.

Original: In such data sets giving priority to easy examples provides a safer training.
Revised: In such data sets giving priority to easy examples provides a safer training.

Original:  D ISCUSSION AND FUTURE WORK
Revised:  D ISCUSSION AND FUTURE WORK

Original: We drew our attention that both versions of training with easy-to-hard ordered and hard-to easy ordered samples have better performance.
Revised: We drew our attention that both versions of training with easy-to-hard ordered and hard-to easy ordered samples have better performance.

Original: That led us to investigate what common issues they have.
Revised: That led us to investigate what common issues they have.

Original: We considered that their common point is growing the training sets during training.
Revised: We considered that their common point is growing the training sets during training.

Original: Therefore, instead of ordering the samples according to difﬁculty we only added some samples randomly at each stage.
Revised: Therefore, instead of ordering the samples according to difﬁculty we only added some samples randomly at each stage.

Original: In these experiments we obtained similar results with Curriculum, Anti-curriculum, Self
Revised: In these experiments we obtained similar results with Curriculum, Anti-curriculum, Self

Original: Paced and Self Paced-Inversed methods which are related to difﬁculty levels.
Revised: Paced and Self Paced-Inversed methods which are related to difﬁculty levels.

Original: According to these results, we can claim that the success of Curriculum learning and Self paced learning approaches not comes from the fact that they follow a meaningful order but trained by growing training sets.
Revised: According to these results, we can claim that the success of Curriculum learning and Self paced learning approaches not comes from the fact that they follow a meaningful order but trained by growing training sets.

Original: In Figure 1(a) we showed some examples for the individual instances.
Revised: In Figure 1(a) we showed some examples for the individual instances.

Original: We started the optimization from θ A , instances under this point are considered as easy and above are difﬁcult.
Revised: We started the optimization from θ A , instances under this point are considered as easy and above are difﬁcult.

Original: If we take an easy instance it is possible or not to guide the optimization to a better minimum.
Revised: If we take an easy instance it is possible or not to guide the optimization to a better minimum.

Original: It will be stop at the local minimum θ B in the worst case.
Revised: It will be stop at the local minimum θ B in the worst case.

Original: Similarly if we take a difﬁcult instance it is possible or not to obtain a better result.
Revised: Similarly if we take a difﬁcult instance it is possible or not to obtain a better result.

Original: Implementation results also showed that both easy-to-hard and hard-to-easy ordered methods can be successful.
Revised: Implementation results also showed that both easy-to-hard and hard-to-easy ordered methods can be successful.

Original: Therefore ordering of the samples are not so important to guide the optimization.
Revised: Therefore ordering of the samples are not so important to guide the optimization.

Original: It is a better situation to shorten the distance between θ B and θ C in Figure 2 to bypass the local minimum.
Revised: It is a better situation to shorten the distance between θ B and θ C in Figure 2 to bypass the local minimum.

Original: When the points are same for a saddle point, training with growing sets will probably overcome this point and ﬁnd a better minimum.
Revised: When the points are same for a saddle point, training with growing sets will probably overcome this point and ﬁnd a better minimum.

Original: This is a good condition when considering saddle points are so much than local minimums in high dimensional functions as mentioned in Dauphin et al.
Revised: This is a good condition when considering saddle points are so much than local minimums in high dimensional functions as mentioned in Dauphin et al.

Original: On many data sets with different distributions we used ensemble method to automatically determine the difﬁculty of the samples for curriculum learning.
Revised: On many data sets with different distributions we used ensemble method to automatically determine the difﬁculty of the samples for curriculum learning.

Original: Pre-processing for difﬁculty level determination can be thought to caused slowdown.
Revised: Pre-processing for difﬁculty level determination can be thought to caused slowdown.

Original: However it has provided a faster neural network training than SPL.
Revised: However it has provided a faster neural network training than SPL.

Original: Also it could be said that ensemble method set a better ordering than SPL by considering their number of wins against Baseline.
Revised: Also it could be said that ensemble method set a better ordering than SPL by considering their number of wins against Baseline.

Original: ACL and SPLI, which are the inverse versions of the CL and SPL methods, has performed poorly in some high error rated data sets.
Revised: ACL and SPLI, which are the inverse versions of the CL and SPL methods, has performed poorly in some high error rated data sets.

Original: The effect of giving the samples at different points during the training has been studied in Erhan et al.
Revised: The effect of giving the samples at different points during the training has been studied in Erhan et al.

Original: In these methods, noisy examples may be effecting the output more because of giving at the beginning.
Revised: In these methods, noisy examples may be effecting the output more because of giving at the beginning.

Original: Nevertheless, the inverse versions of the approaches have better performance than their standard versions.
Revised: Nevertheless, the inverse versions of the approaches have better performance than their standard versions.

Original: However, CL and SPL methods did not lose in any data set so this shows they have a robust aspect.
Revised: However, CL and SPL methods did not lose in any data set so this shows they have a robust aspect.

Original: It is thought that these methods must have a theoretical explanation about ensuring resistance to noises.
Revised: It is thought that these methods must have a theoretical explanation about ensuring resistance to noises.

Original: Gong et al.
Revised: Gong et al.

Original: (2016) studied on why these methods are effectiveness especially on big and noisy data.
Revised: (2016) studied on why these methods are effectiveness especially on big and noisy data.

Original: SPLI method has the most winning against Baseline.
Revised: SPLI method has the most winning against Baseline.

Original: In this method strategy of selecting the samples to learn at each stage reminds pool-based active learning(Lewis & Gale (1994)) in which the learner wants to learn the uncertain samples of the unlabeled data pool.
Revised: In this method strategy of selecting the samples to learn at each stage reminds pool-based active learning(Lewis & Gale (1994)) in which the learner wants to learn the uncertain samples of the unlabeled data pool.

Original: Also non-loss of CL and SPL, and more wins of ACL and SPLI shows the necessity of determining the valuable-example-based curriculum instead of easiness-based-curriculum for the future work.
Revised: Also non-loss of CL and SPL, and more wins of ACL and SPLI shows the necessity of determining the valuable-example-based curriculum instead of easiness-based-curriculum for the future work.

Original: Our contributions:
Revised: Our contributions:

Original: • Previous work on CL and SPL has made experiments on speciﬁc domains.
Revised: • Previous work on CL and SPL has made experiments on speciﬁc domains.

Original: In this study, the performance of these methods on various data sets has been shown.
Revised: In this study, the performance of these methods on various data sets has been shown.

Original: • We proposed a new method that need not to determine the difﬁculty levels as in CL and SPL, so faster and as successful as these methods.
Revised: • We proposed a new method that need not to determine the difﬁculty levels as in CL and SPL, so faster and as successful as these methods.

Original: • We explained the theoretical perspective of the training with growing sets.
Revised: • We explained the theoretical perspective of the training with growing sets.

Original: • We recommend to use the ROGS method as a benchmark in subsequent CL and SPL studies.
Revised: • We recommend to use the ROGS method as a benchmark in subsequent CL and SPL studies.

Original: Only if there is a success up beyond the growing training sets, it can be said that this is the effect of the right ordering.
Revised: Only if there is a success up beyond the growing training sets, it can be said that this is the effect of the right ordering.

Original: A E XPLORING THE TOTAL WEIGHT CHANGES AT EACH ITERATION
Revised: A E XPLORING THE TOTAL WEIGHT CHANGES AT EACH ITERATION

Original: In order to ﬁnd the common feature of CL, SPL and their reverse versions, we examined the total weight change in each iteration during training.
Revised: In order to ﬁnd the common feature of CL, SPL and their reverse versions, we examined the total weight change in each iteration during training.

Original: Graphs of each method for vehicle data set are given in the Figure 3.
Revised: Graphs of each method for vehicle data set are given in the Figure 3.

Original: In these graphs, total weight change is increasing and decreasing during stages.
Revised: In these graphs, total weight change is increasing and decreasing during stages.

Original: This may be indicates a feature that provides avoidance from the local minimum.
Revised: This may be indicates a feature that provides avoidance from the local minimum.

Original: By adding a new group at each stage, it is possible to continue with a larger step from the minimum of the previous stage.
Revised: By adding a new group at each stage, it is possible to continue with a larger step from the minimum of the previous stage.

Original: Iterations of the last stage which we give the all training set is marked with orange.
Revised: Iterations of the last stage which we give the all training set is marked with orange.

Original: When the whole training set is given it reaches the minimum in a shorter time than baseline in all methods.
Revised: When the whole training set is given it reaches the minimum in a shorter time than baseline in all methods.

Original: This implies the whole training set has been started from a better minimum in the methods with growing sets than random initializing in the baseline.
Revised: This implies the whole training set has been started from a better minimum in the methods with growing sets than random initializing in the baseline.

Original: The common feature of the methods are not related to the ordering of the samples.
Revised: The common feature of the methods are not related to the ordering of the samples.

Original: Only the growth of the training set at each stage with the addition of a new group made to continue the optimization with avoiding local minimum.
Revised: Only the growth of the training set at each stage with the addition of a new group made to continue the optimization with avoiding local minimum.

Original: In this case training with growing sets without ordering can be also provide a better minimum.
Revised: In this case training with growing sets without ordering can be also provide a better minimum.

Original: B C OMPARISONS WITH B ASELINE
Revised: B C OMPARISONS WITH B ASELINE

Original: ID
Revised: ID

Original: Data set # of
Revised: Data set # of

Original: Samp.
Revised: Samp.

Original: # of
Revised: # of

Original: Feat.
Revised: Feat.

Original: # of
Revised: # of

Original: Clas.
Revised: Clas.

Original: Avg.
Revised: Avg.

Original: MSE
Revised: MSE

Original: CL
Revised: CL

Original: ACL
Revised: ACL

Original: SPL
Revised: SPL

Original: SPLI
Revised: SPLI

Original: ROGS labor   0,
Revised: ROGS labor   0,

Original: W
Revised: W

Original: T
Revised: T

Original: W W
Revised: W W

Original: W zoo   0,
Revised: W zoo   0,

Original: T
Revised: T

Original: T T
Revised: T T

Original: W
Revised: W

Original: T lymph   0,
Revised: T lymph   0,

Original: T T
Revised: T T

Original: T T
Revised: T T

Original: T iris   0,
Revised: T iris   0,

Original: T
Revised: T

Original: W
Revised: W

Original: T
Revised: T

Original: W
Revised: W

Original: W hepatitis   0,
Revised: W hepatitis   0,

Original: T T
Revised: T T

Original: T T
Revised: T T

Original: T audiology   0,
Revised: T audiology   0,

Original: T
Revised: T

Original: W
Revised: W

Original: T
Revised: T

Original: W
Revised: W

Original: W autos   0,
Revised: W autos   0,

Original: W W
Revised: W W

Original: T
Revised: T

Original: W
Revised: W

Original: W glass   0,
Revised: W glass   0,

Original: T T
Revised: T T

Original: T T
Revised: T T

Original: T sonar   0,
Revised: T sonar   0,

Original: T
Revised: T

Original: T T
Revised: T T

Original: W
Revised: W

Original: T heart-statlog   0,
Revised: T heart-statlog   0,

Original: T
Revised: T

Original: L
Revised: L

Original: T T
Revised: T T

Original: T breast-cancer   0,
Revised: T breast-cancer   0,

Original: T
Revised: T

Original: L
Revised: L

Original: T
Revised: T

Original: L
Revised: L

Original: L primary-tumor   0,
Revised: L primary-tumor   0,

Original: W
Revised: W

Original: T
Revised: T

Original: W W
Revised: W W

Original: W ionosphere   0,
Revised: W ionosphere   0,

Original: T T
Revised: T T

Original: T T
Revised: T T

Original: T colic   0,
Revised: T colic   0,

Original: T
Revised: T

Original: L
Revised: L

Original: T
Revised: T

Original: L
Revised: L

Original: L vote   0,
Revised: L vote   0,

Original: W
Revised: W

Original: T T
Revised: T T

Original: T
Revised: T

Original: T balance-scale   0,
Revised: T balance-scale   0,

Original: T
Revised: T

Original: T T
Revised: T T

Original: W
Revised: W

Original: T soybean   0,
Revised: T soybean   0,

Original: W
Revised: W

Original: T
Revised: T

Original: W W
Revised: W W

Original: W credit-a   0,
Revised: W credit-a   0,

Original: T T
Revised: T T

Original: T T
Revised: T T

Original: T breast-w   0,
Revised: T breast-w   0,

Original: T T
Revised: T T

Original: T T
Revised: T T

Original: T diabetes   0,
Revised: T diabetes   0,

Original: T T
Revised: T T

Original: T T
Revised: T T

Original: T vehicle   0,
Revised: T vehicle   0,

Original: W W
Revised: W W

Original: W W
Revised: W W

Original: W anneal   0,
Revised: W anneal   0,

Original: T
Revised: T

Original: W
Revised: W

Original: T
Revised: T

Original: W
Revised: W

Original: W vowel   0,
Revised: W vowel   0,

Original: W W
Revised: W W

Original: W W
Revised: W W

Original: W credit-g   0,
Revised: W credit-g   0,

Original: T
Revised: T

Original: L
Revised: L

Original: T
Revised: T

Original: L
Revised: L

Original: L col    0,
Revised: L col    0,

Original: W
Revised: W

Original: T T
Revised: T T

Original: W
Revised: W

Original: W segment   0,
Revised: W segment   0,

Original: W W
Revised: W W

Original: T
Revised: T

Original: W
Revised: W

Original: W splice   0,
Revised: W splice   0,

Original: T
Revised: T

Original: L
Revised: L

Original: T T
Revised: T T

Original: T kr-vs-kp   0,
Revised: T kr-vs-kp   0,

Original: T
Revised: T

Original: W
Revised: W

Original: T
Revised: T

Original: W
Revised: W

Original: W hypothyroid   0,
Revised: W hypothyroid   0,

Original: T
Revised: T

Original: W
Revised: W

Original: T T
Revised: T T

Original: W sick   0,
Revised: W sick   0,

Original: T
Revised: T

Original: W
Revised: W

Original: T
Revised: T

Original: W
Revised: W

Original: W abalone   0,
Revised: W abalone   0,

Original: W W
Revised: W W

Original: W W
Revised: W W

Original: W waveform   0,
Revised: W waveform   0,

Original: T
Revised: T

Original: L
Revised: L

Original: T T
Revised: T T

Original: T d    3eW
Revised: T d    3eW

Original: W
Revised: W

Original: W W
Revised: W W

Original: W ringnorm   0,
Revised: W ringnorm   0,

Original: T
Revised: T

Original: T T
Revised: T T

Original: T
Revised: T

Original: T mushroom   3eT
Revised: T mushroom   3eT

Original: W
Revised: W

Original: T
Revised: T

Original: W
Revised: W

Original: T letter   0,
Revised: T letter   0,

Original: W
Revised: W

Original: T T
Revised: T T

Original: W W
Revised: W W