Original: L EARNING TO T EACH
Revised: L EARNING TO T EACH

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Teaching plays a very important role in our society, by spreading human knowledge and educating our next generations.
Revised: Teaching plays a very important role in our society, by spreading human knowledge and educating our next generations.

Original: A good teacher will select appropriate teaching materials, impact suitable methodologies, and set up targeted examinations, according to the learning behaviors of the students.
Revised: A good teacher will select appropriate teaching materials, impact suitable methodologies, and set up targeted examinations, according to the learning behaviors of the students.

Original: In the ﬁeld of artiﬁcial intelligence, however, one has largely overlooked the role of teaching, and pays most attention to machine learning .
Revised: In the ﬁeld of artiﬁcial intelligence, however, one has largely overlooked the role of teaching, and pays most attention to machine learning .

Original: In this paper, we argue that equal attention, if not more, should be paid to teaching, and furthermore, an optimization framework (instead of heuristics) should be used to obtain good teaching strategies.
Revised: In this paper, we argue that equal attention, if not more, should be paid to teaching, and furthermore, an optimization framework (instead of heuristics) should be used to obtain good teaching strategies.

Original: We call this approach “learning to teach”.
Revised: We call this approach “learning to teach”.

Original: In the approach, two intelligent agents interact with each other: a student model (which corresponds to the learner in traditional machine learning algorithms), and a teacher model (which determines the appropriate data, loss function, and hypothesis space to facilitate the training of the student model).
Revised: In the approach, two intelligent agents interact with each other: a student model (which corresponds to the learner in traditional machine learning algorithms), and a teacher model (which determines the appropriate data, loss function, and hypothesis space to facilitate the training of the student model).

Original: The teacher model leverages the feedback from the student model to optimize its own teaching strategies by means of reinforcement learning, so as to achieve teacher-student co-evolution.
Revised: The teacher model leverages the feedback from the student model to optimize its own teaching strategies by means of reinforcement learning, so as to achieve teacher-student co-evolution.

Original: To demonstrate the practical value of our proposed approach, we take the training of deep neural networks (DNN) as an example, and show that by using the learning to teach techniques, we are able to use much less training data and fewer iterations to achieve almost the same accuracy for different kinds of DNN models (e.g., multi-layer perceptron, convolutional neural networks and recurrent neural networks) under various machine learning tasks (e.g., image classiﬁcation and text understanding).
Revised: To demonstrate the practical value of our proposed approach, we take the training of deep neural networks (DNN) as an example, and show that by using the learning to teach techniques, we are able to use much less training data and fewer iterations to achieve almost the same accuracy for different kinds of DNN models (e.g., multi-layer perceptron, convolutional neural networks and recurrent neural networks) under various machine learning tasks (e.g., image classiﬁcation and text understanding).

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: The evolution of modern human society heavily depends on its advanced education system.
Revised: The evolution of modern human society heavily depends on its advanced education system.

Original: The goal of education is to equip the students with necessary knowledge and skills, so as to empower them to further deepen the understanding of the world, and push the frontier of our humanity.
Revised: The goal of education is to equip the students with necessary knowledge and skills, so as to empower them to further deepen the understanding of the world, and push the frontier of our humanity.

Original: In general, the growth of a student will be inﬂuenced by two factors: his/her own learning ability and the teaching ability of his/her teacher.
Revised: In general, the growth of a student will be inﬂuenced by two factors: his/her own learning ability and the teaching ability of his/her teacher.

Original: Among these two, the teacher plays a critical role: an experienced teacher enables faster learning of a student through elaborated strategies such as selecting appropriate teaching materials, imparting suitable methodologies, and setting up targeted examinations.
Revised: Among these two, the teacher plays a critical role: an experienced teacher enables faster learning of a student through elaborated strategies such as selecting appropriate teaching materials, imparting suitable methodologies, and setting up targeted examinations.

Original: The training of an agent in artiﬁcial intelligence (e.g., an image classiﬁcation model) is very similar to the growth of a student in human society.
Revised: The training of an agent in artiﬁcial intelligence (e.g., an image classiﬁcation model) is very similar to the growth of a student in human society.

Original: However, after carefully revisiting the literature of artiﬁcial intelligence (AI), we ﬁnd that the importance role of the teacher has been largely neglected.
Revised: However, after carefully revisiting the literature of artiﬁcial intelligence (AI), we ﬁnd that the importance role of the teacher has been largely neglected.

Original: Researchers put most of their efforts on the student , e.g., designing various optimization algorithms to enhance the learning ability of intelligent agents.
Revised: Researchers put most of their efforts on the student , e.g., designing various optimization algorithms to enhance the learning ability of intelligent agents.

Original: In contrast, there are very limited attempts on building good teaching strategies, as brieﬂy summarized below.
Revised: In contrast, there are very limited attempts on building good teaching strategies, as brieﬂy summarized below.

Original: Machine teaching (Zhu, 2013; 2015;
Revised: Machine teaching (Zhu, 2013; 2015;

Original: Liu & Zhu, 2016; Liu et al., 2017) studies the problem of how to identify the smallest training set to push the machine learning model towards a pre-deﬁned oracle model.
Revised: Liu & Zhu, 2016; Liu et al., 2017) studies the problem of how to identify the smallest training set to push the machine learning model towards a pre-deﬁned oracle model.

Original: Curriculum learning (CL) (Bengio et al., 2009; Spitkovsky et al., 2010; Graves et al., 2017) and self-paced learning (SPL)
Revised: Curriculum learning (CL) (Bengio et al., 2009; Spitkovsky et al., 2010; Graves et al., 2017) and self-paced learning (SPL)

Original: (Kumar et al., 2010; Lee & Grauman, 2011; Jiang et al., 2014b) heuristically deﬁne the scheduling of training data in a from-easy-to-hard order.
Revised: (Kumar et al., 2010; Lee & Grauman, 2011; Jiang et al., 2014b) heuristically deﬁne the scheduling of training data in a from-easy-to-hard order.

Original: Graduated optimization (Hazan et al., 2016) heuristically reﬁnes the non-convex loss function in a from-smooth-to-sharp manner, in order to make the machine learning process more robust.
Revised: Graduated optimization (Hazan et al., 2016) heuristically reﬁnes the non-convex loss function in a from-smooth-to-sharp manner, in order to make the machine learning process more robust.

Original: These attempts are either based on task-speciﬁc heuristic rules, or the strong assumption of a pre-known oracle model.
Revised: These attempts are either based on task-speciﬁc heuristic rules, or the strong assumption of a pre-known oracle model.

Original: In this regard, these works have not reﬂected the nature of education and the best practices in human society, where a good teacher is able to adaptively adopt different teaching strategies for different students under different circumstances, and is good at constantly improving his/her own teaching skills based on the feedback from the students.
Revised: In this regard, these works have not reﬂected the nature of education and the best practices in human society, where a good teacher is able to adaptively adopt different teaching strategies for different students under different circumstances, and is good at constantly improving his/her own teaching skills based on the feedback from the students.

Original: In this paper, we argue that a formal study on the role of ‘teaching’ in artiﬁcial intelligence is sorely needed.
Revised: In this paper, we argue that a formal study on the role of ‘teaching’ in artiﬁcial intelligence is sorely needed.

Original: Actually, there could be a natural analogy between teaching in artiﬁcial intelligence and teaching in human society.
Revised: Actually, there could be a natural analogy between teaching in artiﬁcial intelligence and teaching in human society.

Original: For example, selecting training data corresponds to choosing right teaching materials (e.g. textbooks); designing the loss functions corresponds to setting up targeted examinations; deﬁning the hypothesis space corresponds to imparting the proper methodologies.
Revised: For example, selecting training data corresponds to choosing right teaching materials (e.g. textbooks); designing the loss functions corresponds to setting up targeted examinations; deﬁning the hypothesis space corresponds to imparting the proper methodologies.

Original: Furthermore, an optimization framework (instead of heuristics) should be used to update the teaching skills based on the feedback from the students, so as to achieve teacher-student co-evolution.
Revised: Furthermore, an optimization framework (instead of heuristics) should be used to update the teaching skills based on the feedback from the students, so as to achieve teacher-student co-evolution.

Original: Just as French essayist Joseph Joubert said – “To teach is to learn twice”, we call this new approach “learning to teach” (L2T).
Revised: Just as French essayist Joseph Joubert said – “To teach is to learn twice”, we call this new approach “learning to teach” (L2T).

Original: In the L2T framework, there are two intelligent agents: a student model/agent, corresponding to the learner in traditional machine learning algorithms, and a teacher model/agent, determining the appropriate data, loss function, and hypothesis space to facilitate the learning of the student model.
Revised: In the L2T framework, there are two intelligent agents: a student model/agent, corresponding to the learner in traditional machine learning algorithms, and a teacher model/agent, determining the appropriate data, loss function, and hypothesis space to facilitate the learning of the student model.

Original: The training phase of L2T contains several episodes of sequential interactions between the teacher model and the student model.
Revised: The training phase of L2T contains several episodes of sequential interactions between the teacher model and the student model.

Original: Based on the state information in each step, the teacher model updates the teaching actions so as to reﬁne the machine learning problem of the student model.
Revised: Based on the state information in each step, the teacher model updates the teaching actions so as to reﬁne the machine learning problem of the student model.

Original: The student model then performs its learning process based on the inputs from the teacher model, and provides reward signals (e.g., the accuracy on the held-out development set) back to the teacher afterwards.
Revised: The student model then performs its learning process based on the inputs from the teacher model, and provides reward signals (e.g., the accuracy on the held-out development set) back to the teacher afterwards.

Original: The teacher model then utilizes such rewards to update its parameters via policy gradient methods (e.g., REINFORCE (Williams, 1992)).
Revised: The teacher model then utilizes such rewards to update its parameters via policy gradient methods (e.g., REINFORCE (Williams, 1992)).

Original: This interactive process is end-to-end trainable, exempt from the limitations of human-deﬁned heuristics.
Revised: This interactive process is end-to-end trainable, exempt from the limitations of human-deﬁned heuristics.

Original: Once converged, the teacher model could be applied to new learning scenarios and even new students, without extra efforts on re-training.
Revised: Once converged, the teacher model could be applied to new learning scenarios and even new students, without extra efforts on re-training.

Original: To demonstrate the practical value of our proposed approach, we take a speciﬁc problem, training data scheduling, as an example.
Revised: To demonstrate the practical value of our proposed approach, we take a speciﬁc problem, training data scheduling, as an example.

Original: We show that by using our method to adaptively select the most suitable training data, we can signiﬁcantly improve the accuracy and convergence speed of various neural networks including multi-layer perceptron (MLP), convolutional neural networks (CNNs) and recurrent neural networks (RNNs), for different applications including image classiﬁcation and text understanding.
Revised: We show that by using our method to adaptively select the most suitable training data, we can signiﬁcantly improve the accuracy and convergence speed of various neural networks including multi-layer perceptron (MLP), convolutional neural networks (CNNs) and recurrent neural networks (RNNs), for different applications including image classiﬁcation and text understanding.

Original: Furthermore, the teacher model obtained by our method from one task can be smoothly transferred to other tasks.
Revised: Furthermore, the teacher model obtained by our method from one task can be smoothly transferred to other tasks.

Original: For example, with the teacher model trained on MNIST with the MLP learner, one can achieve a satisfactory performance on CIFAR-10 only using roughly half of the training data to train a ResNet model as the student.
Revised: For example, with the teacher model trained on MNIST with the MLP learner, one can achieve a satisfactory performance on CIFAR-10 only using roughly half of the training data to train a ResNet model as the student.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Our work connects two recently emerged trends of machine learning.
Revised: Our work connects two recently emerged trends of machine learning.

Original: First, machine learning has evolved from simple learning to advanced learning.
Revised: First, machine learning has evolved from simple learning to advanced learning.

Original: Representative works include learning to learn (Schmidhuber, 1987; Thrun & Pratt, 2012), or meta learning, which explores the possibility of automatic learning via transferring generic knowledge learnt from meta tasks.
Revised: Representative works include learning to learn (Schmidhuber, 1987; Thrun & Pratt, 2012), or meta learning, which explores the possibility of automatic learning via transferring generic knowledge learnt from meta tasks.

Original: The two-level setup including meta-level model evolves slowly and task-level model progresses quickly is regarded to be important in improving AI.
Revised: The two-level setup including meta-level model evolves slowly and task-level model progresses quickly is regarded to be important in improving AI.

Original: Recently meta learning has been widely adopted in quite a few machine learning scenarios.
Revised: Recently meta learning has been widely adopted in quite a few machine learning scenarios.

Original: Several researchers try to design general optimizers or neural network architectures based on meta learning (Hochreiter et al., 2001; Andrychowicz et al., 2016; Li & Malik, 2016; Zoph & Le, 2017).
Revised: Several researchers try to design general optimizers or neural network architectures based on meta learning (Hochreiter et al., 2001; Andrychowicz et al., 2016; Li & Malik, 2016; Zoph & Le, 2017).

Original: Meta learning has also been studied in few-shot learning scenarios (Santoro et al., 2016; Munkhdalai & Yu, 2017;
Revised: Meta learning has also been studied in few-shot learning scenarios (Santoro et al., 2016; Munkhdalai & Yu, 2017;

Original: Finn et al., 2017).
Revised: Finn et al., 2017).

Original: Second, teaching has gradually attracted attention from researchers and been evolved as a new research direction in recent years.
Revised: Second, teaching has gradually attracted attention from researchers and been evolved as a new research direction in recent years.

Original: The limited efforts on teaching can be classiﬁed into two categories: machine-teaching and hardness based methods.
Revised: The limited efforts on teaching can be classiﬁed into two categories: machine-teaching and hardness based methods.

Original: The goal of machine teaching (Zhu, 2015; 2013) is to construct a minimal training set for the student model to learn a target model (i.e., an oracle).
Revised: The goal of machine teaching (Zhu, 2015; 2013) is to construct a minimal training set for the student model to learn a target model (i.e., an oracle).

Original: (Liu & Zhu, 2016)) deﬁne the teaching dimension of several learners.
Revised: (Liu & Zhu, 2016)) deﬁne the teaching dimension of several learners.

Original: (Liu et al., 2017) extend machine teaching from batch settings to iterative setting.
Revised: (Liu et al., 2017) extend machine teaching from batch settings to iterative setting.

Original: But with the strong assumption of oracle existence, machine teaching is applied in limited areas such as security (Scott Alfeld, 2017) and human-computer interaction
Revised: But with the strong assumption of oracle existence, machine teaching is applied in limited areas such as security (Scott Alfeld, 2017) and human-computer interaction

Original: (Suh et al., 2016).
Revised: (Suh et al., 2016).

Original: Without the assumption of the existence of the oracle model, hardness based methods assume that a data order from easy instances to hard ones beneﬁts learning process.
Revised: Without the assumption of the existence of the oracle model, hardness based methods assume that a data order from easy instances to hard ones beneﬁts learning process.

Original: The measure of hardness in curriculum learning (CL) (Bengio et al., 2009; Spitkovsky et al., 2010; Tsvetkov et al., 2016; Graves et al., 2017) is typically determined by heuristic understandings of data.
Revised: The measure of hardness in curriculum learning (CL) (Bengio et al., 2009; Spitkovsky et al., 2010; Tsvetkov et al., 2016; Graves et al., 2017) is typically determined by heuristic understandings of data.

Original: As a comparison, self-paced learning (SPL) (Kumar et al., 2010;
Revised: As a comparison, self-paced learning (SPL) (Kumar et al., 2010;

Original: Lee & Grauman, 2011; Jiang et al., 2014a;b; Supancic & Ramanan, 2013) quantiﬁes the hardness by the loss on data.
Revised: Lee & Grauman, 2011; Jiang et al., 2014a;b; Supancic & Ramanan, 2013) quantiﬁes the hardness by the loss on data.

Original: There are parallel related work (Graves et al., 2017) exploring several reward signals for automatically adapting data distributions along LSTM training.
Revised: There are parallel related work (Graves et al., 2017) exploring several reward signals for automatically adapting data distributions along LSTM training.

Original: The teaching strategies in (Graves et al., 2017) are on per-task basis without any generalization ability to other learners.
Revised: The teaching strategies in (Graves et al., 2017) are on per-task basis without any generalization ability to other learners.

Original: The above works related to teaching have certain limitations.
Revised: The above works related to teaching have certain limitations.

Original: First, while a learning problem (e.g., the mathematic deﬁnition of binary classiﬁcation (Mohri et al., 2012)) has been formally deﬁned and studied, the teaching problem is not formally deﬁned and thus it is difﬁcult to differentiate a teaching problem from a learning problem.
Revised: First, while a learning problem (e.g., the mathematic deﬁnition of binary classiﬁcation (Mohri et al., 2012)) has been formally deﬁned and studied, the teaching problem is not formally deﬁned and thus it is difﬁcult to differentiate a teaching problem from a learning problem.

Original: Second, most works rely on heuristic and ﬁxed rules for teaching, which are task speciﬁc and not easy to apply to general teaching tasks.
Revised: Second, most works rely on heuristic and ﬁxed rules for teaching, which are task speciﬁc and not easy to apply to general teaching tasks.

Original:  L EARNING TO T EACH
Revised:  L EARNING TO T EACH

Original: In this section, we will formally deﬁne the framework of learning to teach.
Revised: In this section, we will formally deﬁne the framework of learning to teach.

Original: For simplicity and without loss of generality, we consider the setting of supervised learning in this section.
Revised: For simplicity and without loss of generality, we consider the setting of supervised learning in this section.

Original: 3. P ROBLEM D EFINITION
Revised: 3. P ROBLEM D EFINITION

Original: In supervised learning, we are given an input (feature) space X and an output (label) space Y ; for any sample x drawn from the input space according to a ﬁxed but unknown distribution P ( x ) , a supervisor returns a label y according to a ﬁxed but unknown conditional distribution P ( y | x ) ; the goal of supervised learning is to choose a function f ω ( x ) with parameter vector ω that can predict the supervisor’s label in the best possible way.
Revised: In supervised learning, we are given an input (feature) space X and an output (label) space Y ; for any sample x drawn from the input space according to a ﬁxed but unknown distribution P ( x ) , a supervisor returns a label y according to a ﬁxed but unknown conditional distribution P ( y | x ) ; the goal of supervised learning is to choose a function f ω ( x ) with parameter vector ω that can predict the supervisor’s label in the best possible way.

Original: The goodness of a function f with parameter ω is evaluated by the risk
Revised: The goodness of a function f with parameter ω is evaluated by the risk

Original: [Equation]
Revised: [Equation]

Original: where M ( , ) is the metric to evaluate the gap between the label and the prediction of the function.
Revised: where M ( , ) is the metric to evaluate the gap between the label and the prediction of the function.

Original: One needs to consider several practical issues when training a machine learning model.
Revised: One needs to consider several practical issues when training a machine learning model.

Original: First, as the joint distribution P ( x, y ) =
Revised: First, as the joint distribution P ( x, y ) =

Original: P ( x ) P ( y | x ) is unknown, the selection of a good function f is based on a set of training data D
Revised: P ( x ) P ( y | x ) is unknown, the selection of a good function f is based on a set of training data D

Original: = { x i , y i } ni =1 .
Revised: = { x i , y i } ni =1 .

Original: Second, since the metric M ( , ) is usually discrete and difﬁcult to optimize, in training one usually employs a surrogate loss L .
Revised: Second, since the metric M ( , ) is usually discrete and difﬁcult to optimize, in training one usually employs a surrogate loss L .

Original: Third, to search for a good function f , a space of hypothesis functions should be given in advance, and one uses Ω to denote the set of parameters corresponding to the hypothesis space.
Revised: Third, to search for a good function f , a space of hypothesis functions should be given in advance, and one uses Ω to denote the set of parameters corresponding to the hypothesis space.

Original: Thus, the training process actually corresponds to the following optimization problem:
Revised: Thus, the training process actually corresponds to the following optimization problem:

Original: [Equation]
Revised: [Equation]

Original: As a summary, in conventional machine learning, a learning algorithm takes the set of training data
Revised: As a summary, in conventional machine learning, a learning algorithm takes the set of training data

Original: D , the function class speciﬁed by Ω , and the loss function L as inputs, and outputs a function with parameter ω ∗ by minimizing the empirical risk min
Revised: D , the function class speciﬁed by Ω , and the loss function L as inputs, and outputs a function with parameter ω ∗ by minimizing the empirical risk min

Original: ω ∈ Ω (cid:80) ( x,y ) ∈
Revised: ω ∈ Ω (cid:80) ( x,y ) ∈

Original: D L ( y, f ω ( x )) .
Revised: D L ( y, f ω ( x )) .

Original: We use µ () to denote a learning algorithm, and we call it the student model to differentiate from the teaching algorithm deﬁned as below.
Revised: We use µ () to denote a learning algorithm, and we call it the student model to differentiate from the teaching algorithm deﬁned as below.

Original: In contrast to traditional machine learning, which is only concerned with the student model, in the learning to teach framework, we are also concerned with a teacher model, which tries to provide appropriate inputs to the student model so that it can achieve low risk functional R ( ω ) as efﬁciently as possible:
Revised: In contrast to traditional machine learning, which is only concerned with the student model, in the learning to teach framework, we are also concerned with a teacher model, which tries to provide appropriate inputs to the student model so that it can achieve low risk functional R ( ω ) as efﬁciently as possible:

Original: • Training data .
Revised: • Training data .

Original: The teacher model outputs a good training set D ∈ D to facilitate the training of the student model, where D is the Borel set on ( X , Y ) (i
Revised: The teacher model outputs a good training set D ∈ D to facilitate the training of the student model, where D is the Borel set on ( X , Y ) (i

Original: e., the set of all possible training set).
Revised: e., the set of all possible training set).

Original: Data plays a similar role to the teaching materials such as textbooks in human teaching.
Revised: Data plays a similar role to the teaching materials such as textbooks in human teaching.

Original: • Loss function .
Revised: • Loss function .

Original: The teacher model designs a good loss function L ∈ L to guide the training process of the student model, where L is the set of all possible loss functions.
Revised: The teacher model designs a good loss function L ∈ L to guide the training process of the student model, where L is the set of all possible loss functions.

Original: As an analogy, the loss corresponds to the examination criteria for the student in human teaching.
Revised: As an analogy, the loss corresponds to the examination criteria for the student in human teaching.

Original: • Hypothesis space .
Revised: • Hypothesis space .

Original: The teacher model deﬁnes a good function class Ω ∈ W , such as linear function class and polynomial function class, for the student model to search from,
Revised: The teacher model deﬁnes a good function class Ω ∈ W , such as linear function class and polynomial function class, for the student model to search from,

Original: where W is the set of all possible hypothesis spaces.
Revised: where W is the set of all possible hypothesis spaces.

Original: This also has a good analogy in human teaching: in order to solve a mathematical problem, middle school students are only taught with basic algebraic skills whereas undergraduate students are taught with calculus.
Revised: This also has a good analogy in human teaching: in order to solve a mathematical problem, middle school students are only taught with basic algebraic skills whereas undergraduate students are taught with calculus.

Original: The choice of different hypothesis spaces Ω will lead to different optimization difﬁculty, approximation errors, and generalization errors (Mohri et al., 2012).
Revised: The choice of different hypothesis spaces Ω will lead to different optimization difﬁculty, approximation errors, and generalization errors (Mohri et al., 2012).

Original: The goal of the teacher model is to provide D , L and Ω (or any combination of them) to the student model such that the student model either achieves lower risk R ( ω ) or progresses as fast as possible.
Revised: The goal of the teacher model is to provide D , L and Ω (or any combination of them) to the student model such that the student model either achieves lower risk R ( ω ) or progresses as fast as possible.

Original: Taking the ﬁrst case as an example, the goal of the teacher model, denoted as φ , is:
Revised: Taking the ﬁrst case as an example, the goal of the teacher model, denoted as φ , is:

Original: [Equation]
Revised: [Equation]

Original: For ease of reference, we use A to represent the output space of the teacher model.
Revised: For ease of reference, we use A to represent the output space of the teacher model.

Original: It can be any combination of D , L and W .
Revised: It can be any combination of D , L and W .

Original: When A only contains D , we call the special case “data teaching”.
Revised: When A only contains D , we call the special case “data teaching”.

Original: 3. F RAMEWORK
Revised: 3. F RAMEWORK

Original: As reviewed in Section 2, existing works that also consider the teaching strategies simply employ some heuristic rules and are task speciﬁc.
Revised: As reviewed in Section 2, existing works that also consider the teaching strategies simply employ some heuristic rules and are task speciﬁc.

Original: In this subsection, we propose to model the learning and teaching strategies in L2T as a sequential decision process, as elaborated below.
Revised: In this subsection, we propose to model the learning and teaching strategies in L2T as a sequential decision process, as elaborated below.

Original: • S is a set of states.
Revised: • S is a set of states.

Original: The state s t ∈ S at each time step t represents the information available to the teacher model.
Revised: The state s t ∈ S at each time step t represents the information available to the teacher model.

Original: s t is typically constructed from the current student model f t − 1 and the past teaching history of the teacher model.
Revised: s t is typically constructed from the current student model f t − 1 and the past teaching history of the teacher model.

Original: • At the t -th step, given the state s t , the teacher model takes an action a t ∈ A .
Revised: • At the t -th step, given the state s t , the teacher model takes an action a t ∈ A .

Original: Depending on speciﬁc teaching tasks, a t can be (1) a set of training data, (2) a loss function, or (3) a hypothesis space.
Revised: Depending on speciﬁc teaching tasks, a t can be (1) a set of training data, (2) a loss function, or (3) a hypothesis space.

Original: • φ θ : S
Revised: • φ θ : S

Original: → A is the policy with parameter θ employed by the teacher model to generate its action: φ θ ( s t ) = a t .
Revised: → A is the policy with parameter θ employed by the teacher model to generate its action: φ θ ( s t ) = a t .

Original: When without confusion, we also call φ θ the teacher model.
Revised: When without confusion, we also call φ θ the teacher model.

Original: • The student model takes a t as input and outputs a function f t , by using conventional machine learning technologies.
Revised: • The student model takes a t as input and outputs a function f t , by using conventional machine learning technologies.

Original: [Figure]
Revised: [Figure]

Original: During the training phase of the teacher model, the teacher model keeps interacting with the student model.
Revised: During the training phase of the teacher model, the teacher model keeps interacting with the student model.

Original: In particular, it provides the student model with a subset A train from A and takes the performance of the learned student model as a feedback to update its own parameter.
Revised: In particular, it provides the student model with a subset A train from A and takes the performance of the learned student model as a feedback to update its own parameter.

Original: After the convergence of the training process, the teacher model can be used to teach either new student models, or the same student models in new learning scenarios such as another subset A test is provided.
Revised: After the convergence of the training process, the teacher model can be used to teach either new student models, or the same student models in new learning scenarios such as another subset A test is provided.

Original: Such a generalization is feasible as long as the state representations S are the same across different student models and different scenarios.
Revised: Such a generalization is feasible as long as the state representations S are the same across different student models and different scenarios.

Original: As an example case, in the case of data teaching where
Revised: As an example case, in the case of data teaching where

Original: A = D , in the training process teacher model φ θ could be optimized via the interaction with an MLP learner by selecting data from the MNIST dataset (acted as A train ), and then the learned teacher model can be applied to teach a CNN student model on the CIFAR-10 dataset (acted as
Revised: A = D , in the training process teacher model φ θ could be optimized via the interaction with an MLP learner by selecting data from the MNIST dataset (acted as A train ), and then the learned teacher model can be applied to teach a CNN student model on the CIFAR-10 dataset (acted as

Original: A test ).
Revised: A test ).

Original: While one can choose different approaches to train the teacher model, in this paper, we employ reinforcement learning (RL) for this purpose.
Revised: While one can choose different approaches to train the teacher model, in this paper, we employ reinforcement learning (RL) for this purpose.

Original: In this case, the teacher model φ θ acts as the policy interacting with the environment , which is represented by S .
Revised: In this case, the teacher model φ θ acts as the policy interacting with the environment , which is represented by S .

Original: After seeing the teaching action a t , the student model updates itself based on a t , changes the environment to s t +1 and then provides a reward r t to the teacher model.
Revised: After seeing the teaching action a t , the student model updates itself based on a t , changes the environment to s t +1 and then provides a reward r t to the teacher model.

Original: The reward indicates how good the current student model f t is, e.g., measured by the evaluation measure M on a held-out validation set.
Revised: The reward indicates how good the current student model f t is, e.g., measured by the evaluation measure M on a held-out validation set.

Original: The teacher model then updates its own parameters in φ θ to maximize the accumulated reward.
Revised: The teacher model then updates its own parameters in φ θ to maximize the accumulated reward.

Original: Such an interactive process between the teacher model and the student model is illustrated in Fig.
Revised: Such an interactive process between the teacher model and the student model is illustrated in Fig.

Original: The interaction process stops when the student model get converged, forming one episode of the teacher model training.
Revised: The interaction process stops when the student model get converged, forming one episode of the teacher model training.

Original: Mathematically speaking, taking data teaching as an example in which L and Ω are ﬁxed, the objective of the teacher model in the L2T framework is:
Revised: Mathematically speaking, taking data teaching as an example in which L and Ω are ﬁxed, the objective of the teacher model in the L2T framework is:

Original: [Equation]
Revised: [Equation]

Original: where s t is the t -th step state in the interaction of student model µ and teacher model φ .
Revised: where s t is the t -th step state in the interaction of student model µ and teacher model φ .

Original:  A PPLICATION TO D ATA T EACHING FOR N EURAL N ETWORKS
Revised:  A PPLICATION TO D ATA T EACHING FOR N EURAL N ETWORKS

Original: In this section, taking data scheduling as an example, we show how to fully leverage the proposed learning to teach framework to help deep neural network training.
Revised: In this section, taking data scheduling as an example, we show how to fully leverage the proposed learning to teach framework to help deep neural network training.

Original: 4. S TUDENT AND T EACHER S ETUP
Revised: 4. S TUDENT AND T EACHER S ETUP

Original: The student model f is the deep neural network model for several real-world classiﬁcation tasks.
Revised: The student model f is the deep neural network model for several real-world classiﬁcation tasks.

Original: The evaluation measure M is therefore the accuracy.
Revised: The evaluation measure M is therefore the accuracy.

Original: The student model obeys mini-batch stochastic gradient descent (SGD) as its learning rule (i.e., the arg min part in Eqn. 1).
Revised: The student model obeys mini-batch stochastic gradient descent (SGD) as its learning rule (i.e., the arg min part in Eqn. 1).

Original: Mini-batch SGD is a sequential process, in which mini-batches of data { D 1 , · · · D t , . . .
Revised: Mini-batch SGD is a sequential process, in which mini-batches of data { D 1 , · · · D t , . . .

Original: } arrive sequentially in a random order.
Revised: } arrive sequentially in a random order.

Original: Here D t = ( d 1 , · · · , d M ) is the mini-batch of data arriving at the t -th time step and consisting of M training instances.
Revised: Here D t = ( d 1 , · · · , d M ) is the mini-batch of data arriving at the t -th time step and consisting of M training instances.

Original: The teacher model is responsible to provide training data to the student, i.e, A
Revised: The teacher model is responsible to provide training data to the student, i.e, A

Original: = D .
Revised: = D .

Original: Considering the sequential nature of SGD, essentially the teacher model wants to actively determine what is the next mini-batch data D t for the student.
Revised: Considering the sequential nature of SGD, essentially the teacher model wants to actively determine what is the next mini-batch data D t for the student.

Original: Furthermore, in reality it is computationally prohibitive to scan over all the remaining training data to select out D t at each step.
Revised: Furthermore, in reality it is computationally prohibitive to scan over all the remaining training data to select out D t at each step.

Original: To overcome this, after receiving the randomly arrived mini-batch D t of M training instances, our teacher model A dynamically determine which instances in D t are used for training and the others are abandoned.
Revised: To overcome this, after receiving the randomly arrived mini-batch D t of M training instances, our teacher model A dynamically determine which instances in D t are used for training and the others are abandoned.

Original: By teaching with appropriate data, the teacher aims to help the student model f make faster progress, as reﬂected by the rapid improvement of M ( f, D test ) .
Revised: By teaching with appropriate data, the teacher aims to help the student model f make faster progress, as reﬂected by the rapid improvement of M ( f, D test ) .

Original: 4. M ODELLING THE I NTERACTION OF T EACHER AND S TUDENT VIA R EINFORCEMENT L EARNING
Revised: 4. M ODELLING THE I NTERACTION OF T EACHER AND S TUDENT VIA R EINFORCEMENT L EARNING

Original: We introduce in details on how to leverage reinforcement learning to model the interaction between student and teacher.
Revised: We introduce in details on how to leverage reinforcement learning to model the interaction between student and teacher.

Original: That is, the concrete concepts for s t , a t and r t introduced in Subsection 3.2.
Revised: That is, the concrete concepts for s t , a t and r t introduced in Subsection 3.2.

Original: For the state representation S , it corresponds to the mini-batch data arrived and current state of the deep neural network (i.e., the student): s t =
Revised: For the state representation S , it corresponds to the mini-batch data arrived and current state of the deep neural network (i.e., the student): s t =

Original: ( D t , f t ) .
Revised: ( D t , f t ) .

Original: The teacher’s actions are denoted via a = { a m } Mm =1 ∈ { 0 , 1 } M , where M is the batch size and a m ∈ { 1 , 0 } denotes whether to keep the m -th data instance in D t or not 2 .
Revised: The teacher’s actions are denoted via a = { a m } Mm =1 ∈ { 0 , 1 } M , where M is the batch size and a m ∈ { 1 , 0 } denotes whether to keep the m -th data instance in D t or not 2 .

Original: Those ﬁltered instances will have no effects to student training.
Revised: Those ﬁltered instances will have no effects to student training.

Original: To encourage fast teaching convergence, we set the reward to be related with how fast the student model learns.
Revised: To encourage fast teaching convergence, we set the reward to be related with how fast the student model learns.

Original: Concretely speaking, r is set as the terminal reward, with r t = 0 , ∀ t < T , and r T is computed in the following way: we set an accuracy threshold τ ∈ [0 , 1] and record the ﬁrst mini-batch index i τ in which the accuracy on a held-out dev set D (cid:48) dev exceeds τ , then set r T as r T = − log( i τ /T (cid:48) ) , where T (cid:48) is a pre-deﬁned maximum iteration number.
Revised: Concretely speaking, r is set as the terminal reward, with r t = 0 , ∀ t < T , and r T is computed in the following way: we set an accuracy threshold τ ∈ [0 , 1] and record the ﬁrst mini-batch index i τ in which the accuracy on a held-out dev set D (cid:48) dev exceeds τ , then set r T as r T = − log( i τ /T (cid:48) ) , where T (cid:48) is a pre-deﬁned maximum iteration number.

Original: The teacher model sample its action a t per step by its policy φ θ ( a | s ) with parameters θ to be learnt.
Revised: The teacher model sample its action a t per step by its policy φ θ ( a | s ) with parameters θ to be learnt.

Original: The policy φ θ can be any binary classiﬁcation model, such as logistic regression and deep neural network.
Revised: The policy φ θ can be any binary classiﬁcation model, such as logistic regression and deep neural network.

Original: For example, φ θ ( a | s ) = aσ ( w · g ( s ) + b ) + (1 − a )(1 − σ ( θg ( s ) + b )) , where σ ( · ) is the sigmoid function, θ = { w, b } and g ( s ) is the feature vector to effectively represent state s , discussed as below.
Revised: For example, φ θ ( a | s ) = aσ ( w · g ( s ) + b ) + (1 − a )(1 − σ ( θg ( s ) + b )) , where σ ( · ) is the sigmoid function, θ = { w, b } and g ( s ) is the feature vector to effectively represent state s , discussed as below.

Original: State Features : The aim of designing state feature vector g ( s ) is to effectively and efﬁciently represent state s (Graves et al., 2017).
Revised: State Features : The aim of designing state feature vector g ( s ) is to effectively and efﬁciently represent state s (Graves et al., 2017).

Original: Since state s includes both arrived training data and student model, we adopt three categories features to compose g ( s ) :
Revised: Since state s includes both arrived training data and student model, we adopt three categories features to compose g ( s ) :

Original: • Data features, contain information for data instance, such as its label category (we useof | Y | representations), (for texts) the length of sentence, linguistic features for text segments (Tsvetkov et al., 2016), or (for images) gradients histogram features (Dalal & Triggs, 2005).
Revised: • Data features, contain information for data instance, such as its label category (we useof | Y | representations), (for texts) the length of sentence, linguistic features for text segments (Tsvetkov et al., 2016), or (for images) gradients histogram features (Dalal & Triggs, 2005).

Original: Such data features are commonly used in curriculum learning (Bengio et al., 2009; Tsvetkov et al., 2016).
Revised: Such data features are commonly used in curriculum learning (Bengio et al., 2009; Tsvetkov et al., 2016).

Original: • Student model features, include the signals reﬂecting how well current neural network is trained.
Revised: • Student model features, include the signals reﬂecting how well current neural network is trained.

Original: We collect several simple features, such as passed mini-batch number (i
Revised: We collect several simple features, such as passed mini-batch number (i

Original: e., iteration), the average historical training loss and historical validation accuracy.
Revised: e., iteration), the average historical training loss and historical validation accuracy.

Original: They are proven to be effective enough to represent the status of current student model.
Revised: They are proven to be effective enough to represent the status of current student model.

Original: • Features to represent the combination of both data and learner model.
Revised: • Features to represent the combination of both data and learner model.

Original: By using these features, we target to represent how important the arrived training data is for current leaner.
Revised: By using these features, we target to represent how important the arrived training data is for current leaner.

Original: We mainly use three parts of such signals in our classiﬁcation tasks: 1) the predicted probabilities of each class; 2) the loss value on that data, which appears frequently in self-paced learning (Kumar et al., 2010; Jiang et al., 2014a; Sachan & Xing, 2016); 3) the margin value.
Revised: We mainly use three parts of such signals in our classiﬁcation tasks: 1) the predicted probabilities of each class; 2) the loss value on that data, which appears frequently in self-paced learning (Kumar et al., 2010; Jiang et al., 2014a; Sachan & Xing, 2016); 3) the margin value.

Original: The state features g ( s ) are computed after the arrival of each mini-batch of training data.
Revised: The state features g ( s ) are computed after the arrival of each mini-batch of training data.

Original: For a concrete feature list, as well as an analysis of different importance of each set of features, the readers may further refer to Appendix Subsection 7.3.
Revised: For a concrete feature list, as well as an analysis of different importance of each set of features, the readers may further refer to Appendix Subsection 7.3.

Original: 4. O PTIMIZATION B Y P OLICY G RADIENT
Revised: 4. O PTIMIZATION B Y P OLICY G RADIENT

Original: The teacher model is trained by maximizing the expected reward: J ( θ ) =
Revised: The teacher model is trained by maximizing the expected reward: J ( θ ) =

Original: E φ θ ( a | s )
Revised: E φ θ ( a | s )

Original: [ R ( s, a )] , where
Revised: [ R ( s, a )] , where

Original: R ( s, a ) is the state-action value function.
Revised: R ( s, a ) is the state-action value function.

Original: Since R ( s, a ) is non-differentiable w.r.t.
Revised: Since R ( s, a ) is non-differentiable w.r.t.

Original: θ , we use REINFORCE (Williams, 1992), a likelihood ratio policy gradient algorithm to optimize J ( θ ) based on the gradient: ∇ θ = (cid:80)
Revised: θ , we use REINFORCE (Williams, 1992), a likelihood ratio policy gradient algorithm to optimize J ( θ ) based on the gradient: ∇ θ = (cid:80)

Original: Tt =1 E φ θ ( a t | s t )
Revised: Tt =1 E φ θ ( a t | s t )

Original: [ ∇ θ log φ θ ( a t | s t ) R
Revised: [ ∇ θ log φ θ ( a t | s t ) R

Original: ( s t , a t )] , which is empirically estimated as ∇ θ ≈ (cid:80) Tt =1 ∇ θ log φ ( a t | s t ) v t .
Revised: ( s t , a t )] , which is empirically estimated as ∇ θ ≈ (cid:80) Tt =1 ∇ θ log φ ( a t | s t ) v t .

Original: Here v t is the sampled estimation of reward R ( s t , a t ) from one episode execution of the teaching policy φ θ ( a | s ) .
Revised: Here v t is the sampled estimation of reward R ( s t , a t ) from one episode execution of the teaching policy φ θ ( a | s ) .

Original: Given the reward is terminal reward, we ﬁnally have ∇ θ ≈ (cid:80)
Revised: Given the reward is terminal reward, we ﬁnally have ∇ θ ≈ (cid:80)

Original: Tt =1 ∇ θ log φ θ ( a t | s t ) r T .
Revised: Tt =1 ∇ θ log φ θ ( a t | s t ) r T .

Original:  E XPERIMENTS 5. E XPERIMENTS S ETUP 5.1. T ASKS AND S TUDENT M ODELS
Revised:  E XPERIMENTS 5. E XPERIMENTS S ETUP 5.1. T ASKS AND S TUDENT M ODELS

Original: We conduct comprehensive experiments to test the effectiveness of the L2T framework: we consider three most widely used neural network architectures as the student models: multi-layer perceptron (MLP), convolutional neural networks (CNNs) and recurrent neural networks (RNNs), and adopt three popular deep learning tasks: image classiﬁcation for MNIST, for CIFAR-10 (Krizhevsky, 2009), and sentiment classiﬁcation for IMDB movie review dataset (Maas et al., 2011).
Revised: We conduct comprehensive experiments to test the effectiveness of the L2T framework: we consider three most widely used neural network architectures as the student models: multi-layer perceptron (MLP), convolutional neural networks (CNNs) and recurrent neural networks (RNNs), and adopt three popular deep learning tasks: image classiﬁcation for MNIST, for CIFAR-10 (Krizhevsky, 2009), and sentiment classiﬁcation for IMDB movie review dataset (Maas et al., 2011).

Original: We use ResNet (He et al., 2015) as the CNN student model and Long-Short-Term-Memory network (Hochreiter & Schmidhuber, 1997) as the RNN student model.
Revised: We use ResNet (He et al., 2015) as the CNN student model and Long-Short-Term-Memory network (Hochreiter & Schmidhuber, 1997) as the RNN student model.

Original: Adam (Kingma & Ba, 2014) is used to train the MLP and RNN student models and Momentum-SGD (Sutskever et al., 2013) is used for the CNN student model.
Revised: Adam (Kingma & Ba, 2014) is used to train the MLP and RNN student models and Momentum-SGD (Sutskever et al., 2013) is used for the CNN student model.

Original: We guarantee that the ﬁnal performance of each student model without teaching matches with previous public reported results.
Revised: We guarantee that the ﬁnal performance of each student model without teaching matches with previous public reported results.

Original: Please refer to Appendix Subsection 7.1 for more details about student models/tasks setup.
Revised: Please refer to Appendix Subsection 7.1 for more details about student models/tasks setup.

Original: 5.1. D IFFERENT TEACHING STRATEGIES
Revised: 5.1. D IFFERENT TEACHING STRATEGIES

Original: • NoTeach .
Revised: • NoTeach .

Original: It
Revised: It

Original: means training the student model without any teaching strategy, i.e, the conventional machine learning process.
Revised: means training the student model without any teaching strategy, i.e, the conventional machine learning process.

Original: • Self-Paced Learning (SPL) (Kumar et al., 2010).
Revised: • Self-Paced Learning (SPL) (Kumar et al., 2010).

Original: It refers to teaching by the hardness of data, as reﬂected by loss value.
Revised: It refers to teaching by the hardness of data, as reﬂected by loss value.

Original: Mathematically speaking, those training data d satisfying loss value l ( d ) > η will be ﬁltered out, where the threshold η grows from smaller to larger during the training process.
Revised: Mathematically speaking, those training data d satisfying loss value l ( d ) > η will be ﬁltered out, where the threshold η grows from smaller to larger during the training process.

Original: To improve the robustness of SPL, following the widely used trick in common SPL implementation (Jiang et al., 2014b), we ﬁlter training data using its loss rank in one mini-batch rather than the absolute loss value: we ﬁlter data instances with top K largest training loss values within a M -sized mini-batch, where K linearly drops from M − 1 to 0 during training.
Revised: To improve the robustness of SPL, following the widely used trick in common SPL implementation (Jiang et al., 2014b), we ﬁlter training data using its loss rank in one mini-batch rather than the absolute loss value: we ﬁlter data instances with top K largest training loss values within a M -sized mini-batch, where K linearly drops from M − 1 to 0 during training.

Original: • Learning to Teach (L2T) , i.e., the teacher model in L2T framework.
Revised: • Learning to Teach (L2T) , i.e., the teacher model in L2T framework.

Original: The state features g ( s ) are constructed according to the principles described in Subsection 4.2.
Revised: The state features g ( s ) are constructed according to the principles described in Subsection 4.2.

Original: We use a three-layer neural network as the policy function φ for the teacher model.
Revised: We use a three-layer neural network as the policy function φ for the teacher model.

Original: Appendix Subsection 7.2 lists more details of teacher model training.
Revised: Appendix Subsection 7.2 lists more details of teacher model training.

Original: • RandTeach.
Revised: • RandTeach.

Original: To conduct comprehensive comparison, for the L2T model we obtained, we record the ratio of ﬁltered data instances per epoch, and then randomly ﬁlter data in each epoch according to the logged ratio.
Revised: To conduct comprehensive comparison, for the L2T model we obtained, we record the ratio of ﬁltered data instances per epoch, and then randomly ﬁlter data in each epoch according to the logged ratio.

Original: In this way we form one more baseline, referred to as RandTeach.
Revised: In this way we form one more baseline, referred to as RandTeach.

Original: For all teaching strategies, we make sure that the base neural network model will not be updated until M un-trained, yet selected data instances are accumulated .
Revised: For all teaching strategies, we make sure that the base neural network model will not be updated until M un-trained, yet selected data instances are accumulated .

Original: That
Revised: That

Original: is to guarantee that the convergence speed is only determined by the quality of taught data, not by different model updating frequencies.
Revised: is to guarantee that the convergence speed is only determined by the quality of taught data, not by different model updating frequencies.

Original: The model is implemented with Theano and run on one NVIDIA Tesla K40 GPU for each training/testing process.
Revised: The model is implemented with Theano and run on one NVIDIA Tesla K40 GPU for each training/testing process.

Original: 5.1. E VALUATION P ROTOCOL
Revised: 5.1. E VALUATION P ROTOCOL

Original: For each teaching strategy in every task, we report the test accuracy with respect to the number of effective training instances.
Revised: For each teaching strategy in every task, we report the test accuracy with respect to the number of effective training instances.

Original: To demonstrate the robustness of L2T, we set different hyper-parameters for both L2T and SPL, and then plot the curve for each hyper-parameter conﬁguration.
Revised: To demonstrate the robustness of L2T, we set different hyper-parameters for both L2T and SPL, and then plot the curve for each hyper-parameter conﬁguration.

Original: For L2T, we vary the validation threshold τ in reward computation.
Revised: For L2T, we vary the validation threshold τ in reward computation.

Original: For SPL, we test different speeds to include all the training data during training process.
Revised: For SPL, we test different speeds to include all the training data during training process.

Original: Such a speed is characterized by a pre-deﬁned epoch number E , which means all the training data will gradually be included (i.e., K linearly drops from
Revised: Such a speed is characterized by a pre-deﬁned epoch number E , which means all the training data will gradually be included (i.e., K linearly drops from

Original: M − 1 to 0 ) among the ﬁrst E epochs.
Revised: M − 1 to 0 ) among the ﬁrst E epochs.

Original: All the experimental curves reported below are the average results of 5 repeated runs.
Revised: All the experimental curves reported below are the average results of 5 repeated runs.

Original: To test the generalization ability of the teacher model learnt in the L2T framework, we consider two test settings:
Revised: To test the generalization ability of the teacher model learnt in the L2T framework, we consider two test settings:

Original: • Teaching a new student with the same model architecture (see Subsection 5.2).
Revised: • Teaching a new student with the same model architecture (see Subsection 5.2).

Original: It refers to train the teacher model using a student model, and then ﬁxed the teacher model to train a new student model with the same architecture.
Revised: It refers to train the teacher model using a student model, and then ﬁxed the teacher model to train a new student model with the same architecture.

Original: That is, the student model used in the training phase of the teacher model and the student model used in the test phase of the teacher model share the same architecture.
Revised: That is, the student model used in the training phase of the teacher model and the student model used in the test phase of the teacher model share the same architecture.

Original: The difference between the two student models is that they use different datasets for training.
Revised: The difference between the two student models is that they use different datasets for training.

Original: For example, we use the ﬁrst half of MNIST dataset to train the teacher model for a CNN learner, and apply the teacher to train the same CNN student model on the second half.
Revised: For example, we use the ﬁrst half of MNIST dataset to train the teacher model for a CNN learner, and apply the teacher to train the same CNN student model on the second half.

Original: • Teaching a new student with different model architecture (see Subsection 5.3).
Revised: • Teaching a new student with different model architecture (see Subsection 5.3).

Original: Different from the ﬁrst setting, the two student models in the training and test phases of the teacher model are of different architectures.
Revised: Different from the ﬁrst setting, the two student models in the training and test phases of the teacher model are of different architectures.

Original: For example, we use MNIST to train the teacher model for a MLP student, but ﬁx the teacher model to teach a CNN model on CIFAR-10.
Revised: For example, we use MNIST to train the teacher model for a MLP student, but ﬁx the teacher model to teach a CNN model on CIFAR-10.

Original: 5. T EACHING A NEW STUDENT WITH THE SAME MODEL ARCHITECTURE
Revised: 5. T EACHING A NEW STUDENT WITH THE SAME MODEL ARCHITECTURE

Original: In this setting, we have a training set D train and a test set D test for each task.
Revised: In this setting, we have a training set D train and a test set D test for each task.

Original: We evenly split the training data D train in each task into two folds: D teachertrain and D studenttrain .
Revised: We evenly split the training data D train in each task into two folds: D teachertrain and D studenttrain .

Original: We conduct experiments as follows.
Revised: We conduct experiments as follows.

Original: Step 1 :
Revised: Step 1 :

Original: The ﬁrst fold D teachertrain is used to train the teacher model, with 5% of D teachertrain acting as a held-out set D (cid:48) dev used to compute reward for the teacher model during training.
Revised: The ﬁrst fold D teachertrain is used to train the teacher model, with 5% of D teachertrain acting as a held-out set D (cid:48) dev used to compute reward for the teacher model during training.

Original: Step 2 : After the teacher model is well trained using D teachertrain , it is ﬁxed to teach and train the student model using the second fold D studenttrain .
Revised: Step 2 : After the teacher model is well trained using D teachertrain , it is ﬁxed to teach and train the student model using the second fold D studenttrain .

Original: The other teaching strategies listed in Subsection 5.1.2 are also used to teach the student model on D studenttrain .
Revised: The other teaching strategies listed in Subsection 5.1.2 are also used to teach the student model on D studenttrain .

Original: Step 3 :
Revised: Step 3 :

Original: The student model is tested on the test set D test .
Revised: The student model is tested on the test set D test .

Original: The accuracy curve of the student model accompany with different teaching strategies on D test is plotted in Fig.
Revised: The accuracy curve of the student model accompany with different teaching strategies on D test is plotted in Fig.

Original: [Figure]
Revised: [Figure]

Original: introduced in Subsection 5.1.3.
Revised: introduced in Subsection 5.1.3.

Original: We can observe that L2T achieves the best convergence speed, signiﬁcantly better than other teaching strategies in all the three tasks.
Revised: We can observe that L2T achieves the best convergence speed, signiﬁcantly better than other teaching strategies in all the three tasks.

Original: For example, in MNIST experiments 2(a), L2T achieves a fairly good classiﬁcation accuracy (e.g, 0 . 96 ) with roughly 45% training data of the student model without any data teaching strategy, i.e., the baseline NoTeach.
Revised: For example, in MNIST experiments 2(a), L2T achieves a fairly good classiﬁcation accuracy (e.g, 0 . 96 ) with roughly 45% training data of the student model without any data teaching strategy, i.e., the baseline NoTeach.

Original: Such a reduction ratio of training data for
Revised: Such a reduction ratio of training data for

Original: CIFAR-10 and IMDB is about 50% and 75% respectively.
Revised: CIFAR-10 and IMDB is about 50% and 75% respectively.

Original: Therefore, we conclude that L2T performs quite well when its learnt teacher model is used to teach a new student model with the same architecture.
Revised: Therefore, we conclude that L2T performs quite well when its learnt teacher model is used to teach a new student model with the same architecture.

Original: 5.2. F ILTRATION N UMBER A NALYSIS
Revised: 5.2. F ILTRATION N UMBER A NALYSIS

Original: To further investigate the learnt teacher model in L2T, in Fig.
Revised: To further investigate the learnt teacher model in L2T, in Fig.

Original: 3 we show the number of training data it decides to ﬁlter in each epoch in Step 2 of the student model training.
Revised: 3 we show the number of training data it decides to ﬁlter in each epoch in Step 2 of the student model training.

Original: [Figure]
Revised: [Figure]

Original: speaking, we evenly partition all the rank values { 1 , 2 , · · · , M } , where M is the batch size, into ﬁve buckets.
Revised: speaking, we evenly partition all the rank values { 1 , 2 , · · · , M } , where M is the batch size, into ﬁve buckets.

Original: Bucket 1 denotes the hardest data whose loss values are largest among the instances in each mini-batch, while bucket 5 is the easiest.
Revised: Bucket 1 denotes the hardest data whose loss values are largest among the instances in each mini-batch, while bucket 5 is the easiest.

Original: There are several interesting observations: (1) For the two image recognition tasks L2T acts quite differently from CL/SPL: as training goes on, more and more data will be ﬁltered.
Revised: There are several interesting observations: (1) For the two image recognition tasks L2T acts quite differently from CL/SPL: as training goes on, more and more data will be ﬁltered.

Original: Meanwhile, hard data (the purple curve) tend to be kept as teaching materials, while easy ones (the green and blue lines) will probably be ﬁltered.
Revised: Meanwhile, hard data (the purple curve) tend to be kept as teaching materials, while easy ones (the green and blue lines) will probably be ﬁltered.

Original: Such a result suggests that the student models for MNIST and CIFAR-10 favor harder data as training goes on, whereas those less informative data instances
Revised: Such a result suggests that the student models for MNIST and CIFAR-10 favor harder data as training goes on, whereas those less informative data instances

Original: [Figure]
Revised: [Figure]

Original: with smaller loss values are comparatively redundant and negligible.
Revised: with smaller loss values are comparatively redundant and negligible.

Original: In contrast, L2T behaves similarly to CL/SPL for the LSTM student model on IMDB by teaching from easy to hard order.
Revised: In contrast, L2T behaves similarly to CL/SPL for the LSTM student model on IMDB by teaching from easy to hard order.

Original: This observation is consistent with previous ﬁndings (Zaremba & Sutskever, 2014).
Revised: This observation is consistent with previous ﬁndings (Zaremba & Sutskever, 2014).

Original: Our intuitive explanation is that harder instances on one aspect may affect the initialization of LSTM (Dai &
Revised: Our intuitive explanation is that harder instances on one aspect may affect the initialization of LSTM (Dai &

Original: Le, 2015), and on the other aspect are likely to contain noises.
Revised: Le, 2015), and on the other aspect are likely to contain noises.

Original: Comparatively speaking, MLP and
Revised: Comparatively speaking, MLP and

Original: CNN student models are relatively easier to initialize and image data instances contain less noise.
Revised: CNN student models are relatively easier to initialize and image data instances contain less noise.

Original: Thus, for the two image tasks, the teacher model can provide hard instances to the student model for training from the very beginning, while for the natural language task, the student model needs to start from easy instances.
Revised: Thus, for the two image tasks, the teacher model can provide hard instances to the student model for training from the very beginning, while for the natural language task, the student model needs to start from easy instances.

Original: The different teaching behaviors of L2T in image and language tasks demonstrate its adaptivity and applicability to different learning tasks, and seems to suggest the advantage of learning to teach over ﬁxed/heuristic teaching rules.
Revised: The different teaching behaviors of L2T in image and language tasks demonstrate its adaptivity and applicability to different learning tasks, and seems to suggest the advantage of learning to teach over ﬁxed/heuristic teaching rules.

Original: 5. T EACHING A NEW STUDENT WITH DIFFERENT ARCHITECTURE
Revised: 5. T EACHING A NEW STUDENT WITH DIFFERENT ARCHITECTURE

Original: In this subsection, we consider more difﬁcult, yet practical scenarios, in which the teacher model is trained through the interaction with a student model and then used to teach another student model with different model architecture.
Revised: In this subsection, we consider more difﬁcult, yet practical scenarios, in which the teacher model is trained through the interaction with a student model and then used to teach another student model with different model architecture.

Original: 5.3. R EST N ET 32 → R ES N ET 110 ON CIFAR- 
Revised: 5.3. R EST N ET 32 → R ES N ET 110 ON CIFAR-

Original: The ﬁrst scenario is using the teacher model trained with ResNet32 as student on the ﬁrst half of CIFAR-10 training set, to teach a much deeper student model, ResNet110, on the second half of CIFAR-10 training set.
Revised: The ﬁrst scenario is using the teacher model trained with ResNet32 as student on the ﬁrst half of CIFAR-10 training set, to teach a much deeper student model, ResNet110, on the second half of CIFAR-10 training set.

Original: The accuracy curve on the test set is shown in Fig. 4(a).
Revised: The accuracy curve on the test set is shown in Fig. 4(a).

Original: Apparently,
Revised: Apparently,

Original: L2T effectively collects the knowledge in teaching the student with smaller model, and successfully transfers it to the student with much bigger model capacity.
Revised: L2T effectively collects the knowledge in teaching the student with smaller model, and successfully transfers it to the student with much bigger model capacity.

Original: 5.3. MLP ON MNIST ↔ CNN ON CIFAR- 
Revised: 5.3. MLP ON MNIST ↔ CNN ON CIFAR-

Original: The second scenario is even more aggressive:
Revised: The second scenario is even more aggressive:

Original: We ﬁrst train the teacher model based on the interaction with a MLP student model using the MNIST dataset, and then apply it to teach a ResNet student model on the CIFAR-10 dataset.
Revised: We ﬁrst train the teacher model based on the interaction with a MLP student model using the MNIST dataset, and then apply it to teach a ResNet student model on the CIFAR-10 dataset.

Original: The accuracy curve of the ResNet32 model on the CIAR10 test set is shown in Fig. 4(b).
Revised: The accuracy curve of the ResNet32 model on the CIAR10 test set is shown in Fig. 4(b).

Original: Similarly, we conduct experiments in the reverse direction, and the results are shown in Fig. 4(c).
Revised: Similarly, we conduct experiments in the reverse direction, and the results are shown in Fig. 4(c).

Original: Again, L2T succeeds in such difﬁcult scenarios, demonstrating its powerful generalization ability.
Revised: Again, L2T succeeds in such difﬁcult scenarios, demonstrating its powerful generalization ability.

Original: In particular, the teacher model trained on CIFAR-10 signiﬁcantly boosts the convergence of the MLP student model trained with MNIST (show in Fig. 4(c)).
Revised: In particular, the teacher model trained on CIFAR-10 signiﬁcantly boosts the convergence of the MLP student model trained with MNIST (show in Fig. 4(c)).

Original: 5.3. W ALL -C LOCK T IME A NALYSIS
Revised: 5.3. W ALL -C LOCK T IME A NALYSIS

Original: Different from previous curves showing the performance w.r.t.
Revised: Different from previous curves showing the performance w.r.t.

Original: the number of effective training data, we in Fig.
Revised: the number of effective training data, we in Fig.

Original: 5 show the learning curves of training a ResNet32 model on CIFAR-10 using different teaching strategies, but varying with wall-clock time.
Revised: 5 show the learning curves of training a ResNet32 model on CIFAR-10 using different teaching strategies, but varying with wall-clock time.

Original: The teacher model in L2T is trained on MNIST with MLP student models, i.e., the same one with Fig. 4(b).
Revised: The teacher model in L2T is trained on MNIST with MLP student models, i.e., the same one with Fig. 4(b).

Original: Apparently, even with the process of obtaining all the state features, L2T also achieves training time reduction for the student model through providing high-quality training data.
Revised: Apparently, even with the process of obtaining all the state features, L2T also achieves training time reduction for the student model through providing high-quality training data.

Original: [Figure]
Revised: [Figure]

Original: C ONCLUSION
Revised: SPLRandTeachNoTeach 5. T EACHING FOR I MPROVING A CCURACY In this subsection, apart from boosting the convergence speed, we show that the teacher model in L2T also helps to improve the ﬁnal accuracy.

Revised: The student model is the LSTM network trained on

Revised: IMDB.

Revised: We ﬁrst train the teacher model on half of the training data of IMDB dataset.

Revised: The terminal reward is deﬁned as the dev set accuracy after the student model is trained for 15 epochs.

Revised: Then the teacher model is applied to train the student model on the full dataset till its convergence (the dev set accuracy stops to increase).

Revised: The state features are kept the same as previous experiments.

Revised: The other settings in student model training such as LSTM model sizes are the same as previous work (Dai &

Revised: Le, 2015).

Revised: The results are shown in Table 1.

Revised: Note that the baseline accuracy of NoTeach is comparable to the result reported in (Dai & Le, 2015).

Revised: We can see that L2T achieves better classiﬁcation accuracy for training LSTM network, surpassing the SPL baseline by more than 0 .

Revised: 5 point (with p value < 0 . 001 ).

Revised: [Table]

Revised:  C ONCLUSION

Original: Inspired by the education systems in human society, we have proposed the framework of learning to teach, an end-to-end trainable method to automate the teaching process.
Revised: Inspired by the education systems in human society, we have proposed the framework of learning to teach, an end-to-end trainable method to automate the teaching process.

Original: Comprehensive experiments on several real-world tasks have demonstrated the effectiveness of the framework.
Revised: Comprehensive experiments on several real-world tasks have demonstrated the effectiveness of the framework.

Original: There are many directions to explore for learning to teach in future.
Revised: There are many directions to explore for learning to teach in future.

Original: First, we have studied the application of L2T to image classiﬁcation and sentiment analysis.
Revised: First, we have studied the application of L2T to image classiﬁcation and sentiment analysis.

Original: We will study more applications such as machine translation and speech recognition.
Revised: We will study more applications such as machine translation and speech recognition.

Original: Second, we have focused on data teaching in this work.
Revised: Second, we have focused on data teaching in this work.

Original: As stated in Subsection 3.1, we plan to investigate other teaching problems such as loss function teaching and hypothesis space teaching.
Revised: As stated in Subsection 3.1, we plan to investigate other teaching problems such as loss function teaching and hypothesis space teaching.

Original: Third, we have empirically veriﬁed the L2T framework through experiments.
Revised: Third, we have empirically veriﬁed the L2T framework through experiments.

Original: It is interesting to build theoretical foundations for learning to teach, such as the consistence and generalization of the teacher model
Revised: It is interesting to build theoretical foundations for learning to teach, such as the consistence and generalization of the teacher model.

Original:  A PPENDIX 7. T ASKS AND S TUDENTS S ETUP 7.1. S ETTINGS FOR T RAINING MLP ON MNIST
Revised:  A PPENDIX 7. T ASKS AND S TUDENTS S ETUP 7.1. S ETTINGS FOR T RAINING MLP ON MNIST

Original: MNIST dataset consists of 60 k training and 10 k testing images of handwritten digits from 10 categories (i.e., 0, · · · , 9).
Revised: MNIST dataset consists of 60 k training and 10 k testing images of handwritten digits from 10 categories (i.e., 0, · · · , 9).

Original: SGD with momentum is used to perform MLP model training, and mini-batch size is set as 20 .
Revised: SGD with momentum is used to perform MLP model training, and mini-batch size is set as 20 .

Original: The base model is a three-layer feedforward neural network with 784 / 500 / 10 neurons in its input/hidden/output layers.
Revised: The base model is a three-layer feedforward neural network with 784 / 500 / 10 neurons in its input/hidden/output layers.

Original: tanh acts as the activation function for the hidden layer.
Revised: tanh acts as the activation function for the hidden layer.

Original: Cross-entropy loss is used for training.
Revised: Cross-entropy loss is used for training.

Original: 7.1. S ETTINGS FOR T RAINING CNN ON CIFAR- 
Revised: 7.1. S ETTINGS FOR T RAINING CNN ON CIFAR-

Original: Cifar10 is a widely used dataset for image classiﬁcation, which contains 60 k RGB images of size 32 × 32 categorized into 10 classes.
Revised: Cifar10 is a widely used dataset for image classiﬁcation, which contains 60 k RGB images of size 32 × 32 categorized into 10 classes.

Original: The dataset is partitioned into a training set with 50 k images and a test set with 10 k images.
Revised: The dataset is partitioned into a training set with 50 k images and a test set with 10 k images.

Original: Furthermore, data augmentation is applied to every training image, with padding 4 pixels to each side and randomly sampling a 32 × 32 crop.
Revised: Furthermore, data augmentation is applied to every training image, with padding 4 pixels to each side and randomly sampling a 32 × 32 crop.

Original: ResNet (He et al., 2015), a well-known effective CNN model for image recognition, is adopted to perform classiﬁcation on
Revised: ResNet (He et al., 2015), a well-known effective CNN model for image recognition, is adopted to perform classiﬁcation on

Original: CIFAR-10.
Revised: CIFAR-10.

Original: Concretely speaking, we use ResNet32 and ResNet110 models, respectively containing 32 and 110 layers.
Revised: Concretely speaking, we use ResNet32 and ResNet110 models, respectively containing 32 and 110 layers.

Original: The code is based on a public Lasagne implementation 3 .
Revised: The code is based on a public Lasagne implementation 3 .

Original: The mini-batch size is set as M = 128 and Momentum-SGD Sutskever et al.
Revised: The mini-batch size is set as M = 128 and Momentum-SGD Sutskever et al.

Original: (2013) is used as the optimization algorithm.
Revised: (2013) is used as the optimization algorithm.

Original: Following the learning rate scheduling strategy in the original paper (He et al., 2015), we set the initial learning rate as 0 .
Revised: Following the learning rate scheduling strategy in the original paper (He et al., 2015), we set the initial learning rate as 0 .

Original: 1 and multiply it by a factor of 0 .
Revised: 1 and multiply it by a factor of 0 .

Original: 1 after the 32 k -th and 48 k -th model update.
Revised: 1 after the 32 k -th and 48 k -th model update.

Original: Training in this way the test accuracy reaches about 92 .
Revised: Training in this way the test accuracy reaches about 92 .

Original: 4% and 93 .
Revised: 4% and 93 .

Original: 2% , respectively for
Revised: 2% , respectively for

Original: ResNet32 and ResNet110.
Revised: ResNet32 and ResNet110.

Original: 7.1. S ETTINGS FOR T RAINING LSTM ON IMDB
Revised: 7.1. S ETTINGS FOR T RAINING LSTM ON IMDB

Original: IMDB 4 is a binary sentiment classiﬁcation dataset consisting of 50 k movie review comments with positive/negative sentiment labels (Maas et al., 2011), which are evenly separated (i.e., 25 k / 25 k ) as train/test set.
Revised: IMDB 4 is a binary sentiment classiﬁcation dataset consisting of 50 k movie review comments with positive/negative sentiment labels (Maas et al., 2011), which are evenly separated (i.e., 25 k / 25 k ) as train/test set.

Original: The sentences in IMDB dataset are signiﬁcantly long, with average word token number as 281 .
Revised: The sentences in IMDB dataset are signiﬁcantly long, with average word token number as 281 .

Original: Top 10 k most frequent words are selected as the dictionary while the others are replaced with a special token UNK.
Revised: Top 10 k most frequent words are selected as the dictionary while the others are replaced with a special token UNK.

Original: We apply LSTM (Hochreiter & Schmidhuber, 1997)
Revised: We apply LSTM (Hochreiter & Schmidhuber, 1997)

Original: RNN to each sentence, taking randomly initialized word embedding vectors as input, and the last hidden state of LSTM is fed into a logistic regression classiﬁer to predict the sentiment label (Dai & Le, 2015).
Revised: RNN to each sentence, taking randomly initialized word embedding vectors as input, and the last hidden state of LSTM is fed into a logistic regression classiﬁer to predict the sentiment label (Dai & Le, 2015).

Original: The size of word embedding in RNN is 256 , the size of hidden state of RNN is 512 , and the mini-batch size is set as M = 16 .
Revised: The size of word embedding in RNN is 256 , the size of hidden state of RNN is 512 , and the mini-batch size is set as M = 16 .

Original: Adam (Kingma & Ba, 2014) is used to perform LSTM model training with early stopping based on validation set accuracy.
Revised: Adam (Kingma & Ba, 2014) is used to perform LSTM model training with early stopping based on validation set accuracy.

Original: The test accuracy is roughly 88 .
Revised: The test accuracy is roughly 88 .

Original: 5% , matching the public result in previous work (Dai & Le, 2015).
Revised: 5% , matching the public result in previous work (Dai & Le, 2015).

Original: 7. D ETAILS FOR THE TEACHER MODELS IN L2T
Revised: 7. D ETAILS FOR THE TEACHER MODELS IN L2T

Original: In L2T, we use a three-layer neural network, with layer sizes d × 12 × 1 , as the teacher model φ θ ( a | s ) .
Revised: In L2T, we use a three-layer neural network, with layer sizes d × 12 × 1 , as the teacher model φ θ ( a | s ) .

Original: d is the dimension of g ( s ) and tanh is the activation function for the middle layer.
Revised: d is the dimension of g ( s ) and tanh is the activation function for the middle layer.

Original: All the weight values in this network are uniformly initialized between ( − 0 .
Revised: All the weight values in this network are uniformly initialized between ( − 0 .

Original: The bias terms are all set as 0 except for the bias in the last-layer which is initialized as 2 , with the goal of not ﬁltering too much data in the early age.
Revised: The bias terms are all set as 0 except for the bias in the last-layer which is initialized as 2 , with the goal of not ﬁltering too much data in the early age.

Original: Adam (Kingma & Ba, 2014) is leveraged to optimize the policy.
Revised: Adam (Kingma & Ba, 2014) is leveraged to optimize the policy.

Original: To reduce estimation variance, a moving average of the historical reward values in previous episodes is set as a reward baseline for the current episode (Weaver & Tao, 2001).
Revised: To reduce estimation variance, a moving average of the historical reward values in previous episodes is set as a reward baseline for the current episode (Weaver & Tao, 2001).

Original: We train the teacher model till convergence, i.e., the terminal reward r T stops improving for several episodes.
Revised: We train the teacher model till convergence, i.e., the terminal reward r T stops improving for several episodes

Original: 7. S TATE F EATURE A NALYSIS
Revised: 7. S TATE F EATURE A NALYSIS

Original: In this section, we give a detailed list for all the features used to construct the state feature vector g ( s ) (Subsection 7.3.1), as well as their different importance in making a qualiﬁed L2T policy (Subsection 7.3.2).
Revised: In this section, we give a detailed list for all the features used to construct the state feature vector g ( s ) (Subsection 7.3.1), as well as their different importance in making a qualiﬁed L2T policy (Subsection 7.3.2).

Original: 7.3. S TATE F EATURES g ( s )
Revised: 7.3. S TATE F EATURES g ( s )

Original: Corresponding to the feature description in Section 4.2 of the paper, we list details of the aforementioned three categories of the features:
Revised: Corresponding to the feature description in Section 4.2 of the paper, we list details of the aforementioned three categories of the features:

Original: • Data features, mainly containing the label information of the training data.
Revised: • Data features, mainly containing the label information of the training data.

Original: For all the three tasks, we use 1 of | Y | representations to characterize the label.
Revised: For all the three tasks, we use 1 of | Y | representations to characterize the label.

Original: Additionally, the sequence length (i.e., word token number), divided by a pre-deﬁne maximum token number 500 and truncated to maximum value 1 . 

Revised: Additionally, the sequence length (i.e., word token number), divided by a pre-deﬁne maximum token number 500 and truncated to maximum value 1 .

Original: if exceeded, is set as an additional data feature for IMDB dataset.
Revised: if exceeded, is set as an additional data feature for IMDB dataset.

Original: • Model features.
Revised: • Model features.

Original: We use three signals to represent the status of current model W t : 1) current iteration number; 2) the averaged training loss over past iterations; 3) the best validation loss so far.
Revised: We use three signals to represent the status of current model W t : 1) current iteration number; 2) the averaged training loss over past iterations; 3) the best validation loss so far.

Original: All the three signals are respectively divided by pre-deﬁned maximum number to constrain their values in the interval [0 , 1] .
Revised: All the three signals are respectively divided by pre-deﬁned maximum number to constrain their values in the interval [0 , 1] .

Original: • The combined features.
Revised: • The combined features.

Original: Three parts of signals are used in our classiﬁcation tasks: 1) the predicted probabilities of each class; 2) the loss value on that data, i.e, − log P y , which appears frequently in self-paced learning algorithms (Kumar et al., 2010; Jiang et al., 2014a; Sachan & Xing, 2016); 3) the margin value on the training instance ( x, y ) , deﬁned as P ( y | x ) − max y (cid:48) (cid:54) =
Revised: Three parts of signals are used in our classiﬁcation tasks: 1) the predicted probabilities of each class; 2) the loss value on that data, i e, − log P y , which appears frequently in self-paced learning algorithms (Kumar et al., 2010; Jiang et al., 2014a; Sachan & Xing, 2016); 3) the margin value on the training instance ( x, y ) , deﬁned as P ( y | x ) − max y (cid:48) (cid:54) =

Original: y P ( y (cid:48) | x ) (Cortes et al., 2013).
Revised: y P ( y (cid:48) | x ) (Cortes et al., 2013).

Original: For the loss and margin features, to improve stability, we use their (normalized) ranks in the mini-batch, rather than the original values.
Revised: For the loss and margin features, to improve stability, we use their (normalized) ranks in the mini-batch, rather than the original values.

Original: Based on the above designs, the dimensions of the state feature vector g ( s ) for the three tasks are respectively: a) 25 = 10 (Data features) +3 (Model features)
Revised: Based on the above designs, the dimensions of the state feature vector g ( s ) for the three tasks are respectively: a) 25 = 10 (Data features) +3 (Model features)

Original: +12 (Combined features) for MNIST; b) 25 = 10 + 3 + 12 for CIFAR-10; c) 10 = 3 ( 1 of | Y | = 2 representation + sequence length)
Revised: +12 (Combined features) for MNIST; b) 25 = 10 + 3 + 12 for CIFAR-10; c) 10 = 3 ( 1 of | Y | = 2 representation + sequence length)

Original: + (Model features)+ 4 (Combined features) for IMDB.
Revised: + (Model features)+ 4 (Combined features) for IMDB.

Original: The feature vector g ( s ) is further normalized to satisfy ||
Revised: The feature vector g ( s ) is further normalized to satisfy ||

Original: g ( s ) ||
Revised: g ( s ) ||

Original: 2 = 1 .
Revised: 2 = 1 .

Original: 7.3. I MPORTANCE A NALYSIS FOR D IFFERENT F EATURES
Revised: 7.3. I MPORTANCE A NALYSIS FOR D IFFERENT F EATURES

Original: To better understand how different features play effects in constructing a good L2T policy, we conduct a systematic studies on the importance of different features.
Revised: To better understand how different features play effects in constructing a good L2T policy, we conduct a systematic studies on the importance of different features.

Original: Concretely speaking, for all the three categories of features, we respectively remove each of them, and use the remaining two parts as state features g ( s ) to re-train/test the L2T policy.
Revised: Concretely speaking, for all the three categories of features, we respectively remove each of them, and use the remaining two parts as state features g ( s ) to re-train/test the L2T policy.

Original: The base task is training MLP on MNIST dataset since it takes shortest time among all the three tasks in our experiments.
Revised: The base task is training MLP on MNIST dataset since it takes shortest time among all the three tasks in our experiments.

Original: The experimental results are shown in Fig.
Revised: The experimental results are shown in Fig.

Original: [Figure]

Original: We have the following observations from Fig.
Revised: We have the following observations from Fig.

Original: • The model features and the combined features are critical to the success of L2T policy, as shown by the poor convergence when either of the two are removed.
Revised: • The model features and the combined features are critical to the success of L2T policy, as shown by the poor convergence when either of the two are removed.

Original: Actually without any category of the two subset of features, the performance of SGD with L2T decreases to that of SGD without data scheduling.
Revised: Actually without any category of the two subset of features, the performance of SGD with L2T decreases to that of SGD without data scheduling.

Original: • The data features are relatively less important to L2T. By removing the data features,
Revised: • The data features are relatively less important to L2T. By removing the data features,

Original: i
Revised: i

Original: [Figure] e., the label information of the data, the performance of SGD with L2T drops but not by much. 
Revised: e., the label information of the data, the performance of SGD with L2T drops but not by much. [Figure]