L2T-DLN: Learning to Teach with Dynamic Loss Network
Authors: Zhaoyang Hai
Liyuan Pan
Xiabi Liu
Zhengzheng Liu
Mirna Yunita
Venue: NeurIPS 2023 poster (2023)
Keywords: Learning to teach
dynamic loss function
optimization

Abstract: With the concept of teaching being introduced to the machine learning community, a teacher model start using dynamic loss functions to teach the training of a student model. The dynamic intends to set adaptive loss functions to different phases of student model learning. In existing works, the teacher model 1) merely determines the loss function based on the present states of the student model, e.g., disregards the experience of the teacher; 2) only utilizes the states of the student model, e.g., training iteration number and loss/accuracy from training/validation sets, while ignoring the states of the loss function. In this paper, we first formulate the loss adjustment as a temporal task by designing a teacher model with memory units, and, therefore, enables the student learning to be guided by the experience of the teacher model. Then, with a Dynamic Loss Network, we can additionally use the states of the loss to assist the teacher learning in enhancing the interactions between the teacher and the student model. 
  Extensive experiments demonstrate our approach can enhance student learning and improve the performance of various deep models on real-world tasks, including classification, objective detection, and semantic segmentation scenario.

L2T-DLN: Learning to Teach with Dynamic Loss Network
Zhaoyang Hai 1,,, Liyuan Pan 1.2.t.t, Xiabi Liu li+,, Zhengzheng Liu', Mirna Yunita'
School of Computer Science and Technology. 2 BIT Special Zone
Beijing Institute of Technology
Beijing, China, 100081
(haizhaoyang, liyuan. pan, liuxiabi, liuzhengzheng, mirnayunita)@bit. edu, cn
Abstract
With the concept of teaching being introduced to the machine learning community, teacher model start using dynamic loss functions teach the training student model. The dynamic intends to set adaptive loss functions to different phases of student model learning. In existing works, the teacher model 1) merely determines the loss function based on the present states of the student model, i.e., disregards the experience of the teacher; 2) only utilizes the states of the student model, e.g., training iteration number and loss/accuracy from training/validatton ignoring the states of the loss function. In this paper, we first formulate the loss adjustment as temporal task by designing teacher model with memory units, and, therefore, enables the student learning to guided by the experience the teacher model. Then, with dynamic loss network, we can additionally use the states of the loss to assist the teacher learning in enhancing between the teacher and the student model. Extensive experiments demonstrate approach can enhance student learning and improve the performance of various deep models real- world tasks, including classification, objective detection, and semantic segmentation scenarios.
sets, while
interactions
Introduction
pedagogy study, teachers refine their teaching ability based on student feedback, e,g. exam scores.
Students benefited from the enhanced ability of teachers and then achieved high scores on the exam. Both teachers and students are developed this interaction iteratively and constantly (27 [7].
The phenomenon is known (eaching-learning transaction [5], learning to teach (L2T) in machine learning [7].
In I.2T, a teacher model uses dynamic loss function, which acts exam paper, to train and optimize the student model (as shown in Figure Fi) However, existing approaches adjust the loss functions by only employing simple feedforward network the teacher model, and neglecting the temporal nature of loss function adjustment. The disregarding of the experience accumulation for teachers (e.g., the ability to analyze all previous exam scores for a student), and, therefore, limits the potential L2T. In addition, in previous works, the teacher model only focuses on the state of the student model, i.e., training iteration number [37], training/validation accuracy [37] [6], training/validatee loss [). and the output of the student model 16, 23). However, the states of loss neglected, which dilutes the benefit of
functions (e.g., the gradients concerning loss functions) improving the exam paper. In other words, the teacher needs to consider that the question changes of exam paper also influence the performance of student.
Equul contribution.
Corresponding author.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

memory unit
VΦ
T
x Sg 1. - 1
X Sg
m++
y
(a)
(b)
Figure I: Illustration of the common L2T and our L2T-DLN framework. (a) The common framework existing L2T works. (b) The framework of our L2T-DLN. Both (a) and (b) contain three models: a student model Sn with the parameter 0, dynamic loss model L and a teacher model T.p with the parameter sp. Here, m denotes the mh iteration, I and denote the input data of the student and corresponding label, respectively. Different from existing works that only use feedforward networks, employ network with memory unit to enhance the temporal analyzing ability of the teacher. Then, we use the gradient Vø concerning L% by designing a dynamic loss network to provide more information the teacher model, compared to the state of the student T.
In this paper, we propose an L2T framework with a Dynamic Loss Network (L2T-DLN), address the above-mentioned issue (in Figure [b). First, we adopt a Long-Short Term Memory (LSTM) model the teacher and design differentiable three-stage asynchronous optimization strategy. Then, to ensure the teacher model can be optimized with the state of loss functions, we design a Dynamic Loss Network (DLN) instead of using the dynamic loss function (DLF). Specifically, we start by optimizing the student model through backpropagation in the first step with a fixed DLN as the loss function. Then, compute the gradient of the validation errоr of the student model with respect the DIN. Next, we input this gradient into the teacher model, and the output of the teacher model is used to update the DLN. To achieve the updating of the teacher model, we perform another round of student learning with the updated DLN and obtain the gradient of the validation error of the updated student with respect to the teacher model. Moreover, we analyze how L2T-DLN exploits the negative curvature by using special alternating gradient descent (AGD) sequence, achieving differentiable asynchronous optimization.
In summary, the usage of the gradient concerning DLN and the LSTM teacher model both ensure the teacher model captures and maintains short- and long-term temporal information, which can further
improve the performance of loss function teaching, compared to feedforward teachers [7 [6 23].
Our main contributions teacher model learn optimized by the gradient of DLN; 2) use LSTM as the teacher model to update the DLN with the temporal information of the teacher model; and 3) a convergence analysis of the approach, which is treated as a special AGD sequence and has the potential to escape strict saddle points. We conduct extensive experiments on wide range of loss functions and tasks to demonstrate the effectiveness of our approach.
1) design a dynamic loss network-based teaching strategy let the
Related work
Recent work by L2T 7 provides a comprehensive view of teaching for machine learning, encom- passing aspects such training data teaching. loss function teaching, and hypothesis teaching. In contrast to previous literature on machine teaching [1 25 10 38), L2T breaks the strong assump- tion regarding the existence of an optimal off-the-shelf student model [37]. Instead, L.2T employs automatic techniques to reduce the reliance prior human knowledge, aligning with principles such as learning to learn and meta-learning
The recent focus of L.2T has been mainly on
loss function teaching
337 16. 23 2
and training data teaching 7 25 35 32 g.

---

During the training of a student model, there is variation in the distribution of predictions where earlier in the training the distribution tends to differ from that at convergence. Consequently, an adaptive loss function is crucial. Existing works 37 some independent tasks by performing a multi-layer perceptron (MLP) as the teacher model. The differences lie in the representation of dynamic loss functions and the input information of teacher models. The representation of the dynamic loss function includes the variation of handcrafted loss functions [37 23 and neural network [6 2]. The input information contains the training iteration number 17. training/validation accuracy 137 [6], training/validation loss 22, and the output of the student model I6, 2 23). In detail, Wu et al. [77 trains neural network with attention mechanism to generate a coefficient matrix between the prediction of the student and the ground truth. Huang et al. [16] constructs the teaching-leamin
16 23 2
formulate the loss adjustment as
framework with reinforcement learning. Their teacher also employs an MI.P and generates the policy gradient for a loss network. Liu and Lai (3 utilizes teacher model to guide the selection and combination of handcrafted loss functions. Baik et al. 2 performs a teacher to generate two weights for each layer of a loss network, and then updates the parameters of the loss network by affine transformation with the two weights.
Assigning weights to different data points has been widely investigated in the literature, where the weights can be either continuous [8] or binary 77]. In detail, Fan et al. 77] proposed a learning paradigm where a teacher model guides the training of the student model. Based on the collected information, the teacher model provides signals to the student model, which can be the weights of training data. Liu et al. [25] leveraged a teaching way to speed up the training, where the teacher model selects the training data balancing the trade-off between the difficulty and usefulness of the data, Fan et al. [8] inputs internal states of the student model, egg., feature maps of the student model, to the teacher model and obtains the output of the teacher the weight for corresponding training
data.
3 Methodology
In this section, we overview our L2T-DLN in Section 3.1 introduce the corresponding framework for student learning in Section 3.2 describe the DLN learning framework in Section 3.3 and discuss
teacher learning in Section 3.4
3.1 Overview
Our L2T-DLN is a differentiable teaching framework that enhances the performance of a student model. The L2T-DLN contains stages: (I) student learning, which optimizes student model So with parameter 0; (II) DLN learning, which optimizes the DLN Lo with parameter Φ; (III) teacher learning, which optimizes the teacher model T' with parameter 42
Starting from S9. in stage (I), we optimize the student model S - S. on training data Ttrain by leveraging the DLN L as the loss function, where N denotes the number of iterations in student learning stage. In stage (II), we compute the error Geat of the student model SEY on validation data Teat, and then determine the gradient VΦ" = 00 This gradient is then given to the teacher model, and the DLN is updated as o² = 80 + n where g indicates the output of the teacher model To. In stage (III), we first train the student model S - >S with the updated DLN L.- Then, we obtain the validation error Geat of the student model 53 and optimize the teacher model by backpropagation (BP) based on Evel. The objective function of our L2T-DLN is:
K
(Q(M+K)N G.M, (p)) 000 (0) V (0.00.))
eval (02(K+1)N).
(I)
Our goal is to achieve ((M+K)N M and K where M and K denote the number of iterations of DLN and teacher learning, respectively (details are shown in Figure 2).
3.2 Student learning
For given task, we define the input and output space as and Y, respectively. The student model is then denoted by Sa X + Y. Our student learning involves minimizing the output value of the
3

---

Sg
S8
Student
BP
KM
learning
e
ON
02N
0
+b
up
DLN
Φ M
learning
Φ
db
VΦ
Vd 1
T
T1
ho
h
Teacher learning op
k
BP
Figure 2: The pipeline of L2T-DLN. Grey, black, and red lines represent the optimization of the student parameter 0, the DLN parameter é, and the teacher parameter yo, respectively. We aim at obtaining ((M++))N d and øK where M and K denote the number of iterations for DLN and the teacher and M = K. There are three stages: student, DLN, and teacher learning. During student learning, student model is optimized by backpropagation with L During DLN learning, the teacher TØ accept the gradient V and output gd to update Lg During teacher learning, we perform another student learning with L! and optimize the teacher 60 with the validation error of the updated student.
y), under a hypothesis space using training data Etrain- Note, w is weight parameter of LT is the loss function, and m is the m iteration for the DLN, During each stage of student learning, we iteratively train the student model N times with m "
DLN, i.e., arg min E (z.g)) BEQ
The optimization of the student model during the current stage is:
= 1= (St (z),
L
2,, ,N,
where 7 denotes the learning rate of the student model. In our framework, the L that with learnable parameters are optimized in different student learning stages for providing seemly guidance.
3.3 DLN learning
After the student learning stage (e.g., Sg + S with Lg), we use the teacher model TV to adjust the DLN parameters é (e.g., Φ + x1)). To enable the temporal property of DLN, an LSTM to transform ó dynamically:
07 h1
0' 0'=0"+79"
To TVO" h2),
where 7 denotes the learning rate of DLN and V represents the gradient of the validation error Eval of S, with respect to σ²
Considering the gap between training data and validation data, we employ the Reverse-Mer Mode Dif- ferentiation (RMD) to calculate V The RMD involves performing the SGD process in reverse order from N to 1. as depicted by the black lines in Figure 2 According to Eq. (2). the gradient of the validation error emat (S ( with respect to @N can be calculated as follows:
N (evar(()) \
(4)

---

Then looping backward from N. At each step i = {N - 1,, 1}. the calculation of gradient is shown as:
1 (S) (x)
V0'-1
V0 Tpw (30
V0
(5)
At the same time, the gradient of eval (5 with respect to d is accumulated as:
0LD0 (S (z) V
V
(6)
a 0Φ0
Reverting backward from N to 1, we get VΦD.
Taking N we rewrite Eq. (6) by using the gradient, e.g., Vo", as:
i BTg (S
VΦL 17W dSh (Zeat)
Depal 0Sg (Xina)
002
(
05%
(7)
00
aL (s) v)
060
O00
(55(2)
where V contains the information of both training and validation data, (0S) (a /001 and as% (00%) As these pieces of information are dependent on each other, they are integrated into temporal changes of A. The gradient concerning 6d achieves holistic information integration throughout the learning process, facilitated by prior knowledge (chain rule). Employing the gradient concerning the loss allows the teacher model to concentrate on capturing and preserving crucial information from gradients, negating the need for supplementary handling of dispersed states. There- fore, compared to the state of the student, egg. training/validation error, prediction, and numbers of iteration, V の provides more information promote deep interaction between DLN, the teacher, and the student.
3.4 Teacher learning
To retain the teaching experience, we utilize an LSTM to play the role of teacher. The LSTM utilizes long-term memory unit, called the cell, allow itself to maintain effective and long-term dependencies to make predictions, both in the current and future [9. We employ the coordinate-wist manner I process each value of the input information independently. Different behavior on each coordinate is achieved by using separate activations for each input value. This property leads the student to converge
good solution.
To evaluate the teacher model, we perform another student learning (e.g., S - SZN) with the new DLN (e.g., (.)). The RMD is also utilized to calculate the gradient of validation error evat (Sg with respect to the teacher model parameters sp. We represent the computation using red lines in Figure To obtain Vy=0, we loop the SGD process backward from 2N to N + with the updated DLN l according to Eq. (4, and 6). At each step i (2N - 1, N + 1}, the gradient V o is updated as:
8² (S) (Eivain), 4
Vip
wyw
(8)
00 10 1000000
The process of L2T-DLN is summarized in Algorithm
4 Convergence Analysis
Since our L2T-DLN is updated asynchronously, we can only access partial second-order information at each training stage. For example, given quadratic objective function, while fixing part of L2T-DLN, the problem is strongly convex with respect to the other part, but the entire problem is nonconvex. Even if the iterates converge for each part to the corresponding minimum points, the stationary point could still be a saddle point for the overall objective function (26]. Therefore, the analysis of how L.2T-DLN exploits the negative curvature is necessary.
The Alternating Gradient Descent (AGD) algorithm only updates partial variables of vector, egg. (0, i + Φ). which belongs to subset of the feasible set. From the mean value theorem, can
5

---

Algorithm Obtaining the optimal student, DLN and teacher in L2T-DLN.
1: Hyperparameter: length of the student training N, total number of iterations for DLN learning M, total number iterations for teacher learning K.
2: Input: Random initialization parameter 8 o0, and yp.
3: m =
4: for (1, K) do
5: for i= ((m + N)N + 1, 1, (m + I) do
A student learning stage
6:
Conduct student model training step via Eq. ( (Sp end for
St).
7:
8:
VΦTT = 00 compute V0(m=R+1)W via Eq.
9:
for = ((m + k + I).N.... (m K)N do Reversely calculating the gradient V
10:
Update V via Eq.
(6)
Compute V via Eq.
12: end for
update 2Th -> 1 via Eq. 3
14:
m m +1
D. Updating m
for = ((m + K)N + 1, (m + 1)N) do
student learning stage
Conduct student model training step via Eq. (2) (S) (S Si). end for
17:
VΦ V 0, compute V0(m+*+12N via Eq.
19:
for i= ((m + k + I)N, ...... (m + K)N + do Reversely calculating the gradient V
20:
update Vøn via Eq.
21:
Update Vit via Eq.
22:
Compute V via Eq. end for
23:
24: Update g - g using Vi via gradient based optimization algorithm.
25:
end for
26:
output: ((M+K)N GM sk
express the AGD rule of updating variables by assuming v(() = O as follows, with denoting the validation error.
Vie(EPKN wht N+1 = b - (2ee(2 2(E+1)N unt))
x 11 /'' Hi H1 - I/ Hido
and H [VieloTTN 2kN o
Vize(oReN
where H/
V e(v) 2(++
0
Vize(0u
2(++) Out))
The right-hand side of Eq. 9 not only contains the second order information of the previous point,
i.., [u?KN, v.] but also the one of the most recently updated point, i.e., [o] (kk++1)) wg].
Different from traditional AGD, the dynamic system L2T-DLN takes the first-order informa- tion to update the student and the second-order information update the teacher. Specifically,
Vie(u w²) = ve(ukk, w.) and (2e ((++1)) uu)) = " ((+(()) .u)). These represent the main challenges in understanding the behavior of the sequence generated by the AGD algorithm.
Although the higher-order information divided into two parts, we can still characterize the recursion of the iterates around strict saddle points v", We can also split H. as two parts, which are
H. Viie("))
Vige(vr)] [22e(1")
Hi (Zle(0)) 0
(10)
and obviously, we have H. = H. + H4. Then recursion Eq. g can be written
y 2(K+1)N + 2 = at HH.u 4 th AF 2((+1))
(II)

---

where 4 & [o (H) de, 4 (v) - Mi However, it is still unclear from Eq. how the iteration evolves around the strict saddle point. To highlight ideas, let us define
M I 17H4, G -
(12)
can be observed that M is a lower triangular matrix where the diagonal entries are all 1s; therefore it is invertible. After taking the inverse of matrix M on both sides of Eq. [11. we can obtain
nM-1 A t* n A (((++))
(13)
Our goal of analyzing the recursion of u becomes to find the maximum eigenvalue of M M With the help of the matrix perturbation theory, we can quantify the difference between the eigenvalues of matrix H that contains the negative curvature and matrix M- that we interested in analyzing. With the gradient Lipschitz constants following conclusion.
{C*]. , we set Lmoz - max Ckk, Ck Vke S C and give the
Conclusion Let H A e(x(z) denote the Hessian matrix at an secoond-ordeer stationary solution (SS2) V" where Xmii (H) -y and 7 > O. We have
Amaz (M -1 G) > + C/Cmaa
777
(14)
The proof of Conclusion contains the following steps:
Step 1. (Lemma 1 (26)) Giving generic sequence generated by AGD (v* E u). As long as the initial point of u is close saddle point ج.= the distance between u' and j can be upper bounded by using the p-Hessian Lipschitz continuity property.
Step 2. Leveraging the negative curvature around the strict saddle point, we can project the u² onto the two subspaces, where the first subspace is spanned by the eigenvector of M- and the other one is spanned by the remaining eigenvectors. We use two steps show Xmaz (M-+G) 1: we show that all eigenvalues of Q()) = [G AM] are real; 2) YE > 1, (Q((() = 0.
Conclusion 1 illustrates that there exists subspace spanned by the eigenvector of M whose cigenvalue greater than 1, indicating that the sequence generated by AGD can still potentially escape from the strict saddle point by leveraging such negative curvature information (more can be found in supplementary matcrials).
Experiments
5.1 Experimental setup
Datasets. We evaluate our method on three tasks, i.e., image classification, objective detection, three datasets: CIFAR-10 (20).
and semantic segmentation. For the image classification, CIFAR-100 [21], and ImageNet [33]. CIFAR- 10 and CIFAR-I00 contain 50000 training and 10000 testing images with 10-class and 100-class separately. ImageNet is 1000-class task that contains 1281167 training and 50000 testing pairs. For the objective detection, we use MS-COCO dataset (22),
which contains 82783, 40504, and 81434 pairs in the training, validation, and testing set separately. For the semantic segmentation, we choose PASCAL VOC 2012 [6]. Following the procedure of Zhao et al. [40], 1449, and 1456 images for training, validation, and testing.
augmented data with the annotation of Hariharan et al. [11]. resulting in 10582,
Evaluation metries. In the classification, we use the accuracy on the testing set of each dataset Ri 23 In the objective detection, we use the mean of Average Precision (mAP) [31] to evaluate the student model on the testing set of MS-COCO 222 In the semantic segmentation, Intersection over Union (mloU) 40 to evaluate the student model on the testing set of VOC [6]
use Mean
Baseline methods. For the classification, we employ several popular loss functions, including fixed loss functions such as Cross Entropy loss (CE), the large-margin softmax loss (L-M softmax) 24], and the smooth 0-1 1 loss function (Smooth) 30 as well as dynamic loss functions, namely the adaptive robust loss function (ARLF) 3 the L.2TDDFF loss function [37], stochastic loss function (SLF) 23 and ALA [6]. For objective detection, we compare our approach with the objective function set by YOLO-v3 BI. For the semantic segmentation, we compare our approach with the
objective function set by PSPNet [40].
7

---

Table 1: Results on datasets CIFAR-I0 (0). CIFAR-IOR 100 21 and ImageNet [33 for the classification task. All experiments are implemented with the same settings. The best results are highlighted in bold.
CIFAR-10
CIFAR-100 ResNer20
ImageNet A3 NASNet-A
length
Method
ResNet8
ResNer20
ResNer32
WRN
ResNet8
ResNer32
87.6
91.3
92.5
67.7
73.5
Smooth
R0
57.9
91.5
92.6
96.2
60.5
68.0
69.9
I.M Softmax
68.7
92.0
90.0
96.3
61.1
68.4
70.4
L.TT-DLI
37
89.2
92.4
93.1
96.8
61.7
89.0
70.8
ARLF
8
89.5
91.3
92.2
95.9
60.2
67.8
69.9
SLF
23
89.8
93.0
93.6
97.1
62.7
69.9
71.3
ALA
16
93.2
96.7
62.2
60.5
70.9
74.6
200
[5]
Ours
93.4
0.20 96.7
0.00
63.5
0.07
72.0
0.11
Implementation details. In all experiments, we optimize student models using standard stochastic gradient descent (SGD) with a learning rate of 0.1. The teacher model is trained with Adam, utilizing a learning rate of 0.001. The learning rate of DLN is set 0.001. The teacher model is trained for 10 epochs, with redividing the training and validation data after each epoch. The validation errors in each task are explicitly reported. Our teacher model comprises four-layer LSTM [14 with 64 neurons in the first three layers and neuron in the final layer. We utilize a 1-vs-1 approach (details in supplementary materials) to process the student model's output in both classification and segmentation. We present DLN architecture for each task and ensure reliable evaluation by conducting random restarts, using average results for comprehensive comparisons.
5.2 Results
Image classification. For CIFAR-10 and CIFAR 100, we follow the SLF (2 and use architectures that include ResNet [12, and (ide-ResNee(WRN) [39 as the student model. For ImageNet, we follow the ALA [16] and use the identical NASNet-A 43 In each experiment, the batch sizes for training and validation are set to 25 and 100, respectively. We perform five-layer fully connected network, which contains 40 neurons in each hidden layer and 1 neuron in the output layer, as the DLN. The activation function for each hidden layer is set to Leaky-ReLU. The validation error is computed by CE.
Table reports the performance of each loss function. Our approach achieves the best results on CIFAR-10 with ResNet8, ResNer20, and ResNet32, achieving 90%70, 93.40%, and 93.81% %. respectively. On CIFAR- 100, our method also outperforms baselines with an overall accuracy of 63.50% 70.47%, and 72.06% for ResNet8, ResNet20, and ResNet32, respectively. For WRN, our approach achieves the second-best performance, following the SLF method. The results on ImageNet illustrate that L2T-DLN improves the accuracy of the baseline by 0.7%. On ImageNet, our DLN demonstrates the second-best performance. The performance of ALA benefits from the larger length of student learning stage ALA set (200) compared with ours (25). The ablation showed that the size of the length is positively correlated with the test accuracy and computational consumption (see Table
Objective detection. In the task of objective detection, the YOLO-v3 model with backbone of darknet-53 (31 is used in this experiment. The traditional loss in the YOLO model is multi-pan -art + Atocetoe. Eets f and floe are detailed in supplementary materials. Redmon and Farhadi (31] set Xete = Aaonn Atoc 1. In our experiment, our L2T-DLN learns to set these weights dynamically with a single-layer perceptron as DLN. The backbone of the YOLO is pre-trained on ImageNet, and we finetune the header of the YOLO.. Specifically, the objective function of the student model is set to DLN ([Cttt loons (toe)). The validation error is computed by Late + Lcons + lioc- The batch sizes for training and validation are set to 2 and 8, respectively. The length of student learning is set to 2. We take the training set and 35000 images of the validation set to train our L2T-DLN with an input size of 416*416. From Table 2 our L2T-DLN has more than 1.6% improvement with the baseline on mAP.
loss function, i.e. Actelels - +
Semantic segmentation. The objective function of PSPNet 40 is set to CE(p,)) y) + 0.4 C'Eauu y), where p. autp, and y denote the output the master branch, the auxiliary branch of PSPNet, and the ground truth, respectively. For PSPNet with I.2T-DLN, the objective function is set to DLN(p, y) +0.4 * DLN (ausp, y), where the architecture DLN the one used in classification tasks. The validation error is computed by CE(p, v) + 0.4 CE(aux, 3). The batch sizes for
8

---

Table 2: Objective detection on COCO 22 DLN and original losses in YOLOV3.
Table 3: Segmentation on VOC 6
DLN
and original losses in PSPNet.
Detectors
Size
mAP
FPS
Method
mloU
YOLOV3 31
416
55.3
35
PSPNet 40
82.6
YOLOV3-ours
416
56.9
35
PSPNet-ours
82.9
training and validation are set to 2 and 8, respectively. The length of student learning is set to 2. Table 3 shows that our L2T-DLN improves 0.3% compared with the bascline on mloU.
5.3 Ablations
In this subsection, we conduct ablation studies on CIFAR-IO 20 using ResNet8 to analyze the L2T-DLN synthetically. We specifically examine the proportion of training and validation data, the length of student learning stage (N), the wrong learning rate setting, and the influence of the LSTM teacher. Furthermore,
provide the visualization of DLN different learning stages in MNIST and CIFARIO tasks. We assess the impact cach component by computing the test accuracy of the student model after optimizing the teacher model for 10 epochs.
The proportion of training and validation data. In L.2T-DLN, the training dataset is divided into two sets: validation data and training data, with validation data serving as an unbiased estimator for model generalization. After each epoch, the dataset redivided, allowing samples used in the validation data to be included in the training data, and vice versa. The validation ratio represents the fraction of training dataset samples exclusively used for validation. This study explores different wraiiing-validation training- validation data separations. Table results indicate our performance remains stable across varying ratios due to the teacher's ability to capture short- and long-term dependencies. To make a trade-off between computational cost and accuracy, set the ratio= 50% for all our experiments.
Table 4: Results on different validation ratios ranging from 10% to 90% to show the impact of ratios.
Ratio
10%
25%
50%
75%
90%
Accuracy 90.41
90.53
90.70
90.68
90.35
The length of student learning. The computation of higher-order gradients in L2T-DLN (Eq. O and (8) is computationally intensive and should be highlighted. Thus, this study explores the influence of the length of student learning (N) the test accuracy and computational load in CIFAR-I0 experiments using ResNet8. As shown in Table the findings reveal that the test accuracy increases with the length of student learning. To make a trade-off between performance and computational cost, we suggest that a maximum length 25 should be set for student learning. Overall, the study concludes that L2T-DLN has the potential to further improves the performance of student model with sufficient computing resources.
Table 5: Results on different lengths ranging from 1 to 75 to show the impact of the length. Time denotes the time consumption of a round of teacher learning.
Length
5
10
25
50
75
Accuracy
81.40 1s
87.07
89.95
90.18
90.70
90.73
90.74
Time
3s
5.8s
7.9s
13.4s
32s
76.7s
The influence of an LSTM teacher. As introduced above, the teacher model is similar to an optimiza- tion algorithm. Then we perform various optimizers, including Adam [19], SGD, RMSProp [13] and the LSTM teacher, to optimize the DLN and present the results in T Compared with ADAM. SGD. and RMSProp, our teacher can improve the performance of the student by 0.48%, 1.6%, and 0.53%. We can conclude that 1) algorithms that can use the historical information, e.g., momentum, perform well; 2) the adaptability to capture and maintain short- and long-term dependencies can further enhance the loss function teaching, compared to handcrafted methods, egg., exponentially weighted moving average [3] and moment estimation [9]

---

Table 6: Results on different optimizers to show the effectiveness of the LSTM teacher.
Optimizer
Adam
19
SGD
RMSProp 13
LSTM
Accuracy
90.17
89.05
90.12
90.65
(a)
(c)
(d)
g
(h)
Figure 3: Visualization of DLN loss value at different training stages in the MNIST-LeNet task (a-d) and CIFARIOS (e-ResNet8 task (e-h), (a) & (e) initialized DLN, (b) & (f) DLN finished second teacher learning epoch, (c) & (g) DLN finished fifth teacher learning epoch, (d) (h) DLN finished tenth teacher learning epoch (final). The X-, Y-, and Z-axis are the prediction for O-category (denoted as predictionO), the prediction for 1-category (denoted as predictionl), and the loss value of DLN, respectively. We set the O-category the correct category and the 1-category the wrong category. We can observe that the output value range of DLN initially expands and subsequently contracts. Specifically, the range shifts from (-0.3, 0.4) to (-2.25, -0.25) (-0.3, -0.05) in MNIST, and from (0.8, 1.8) to (-2, 1) to (-0.175, 0) in CIFAR10.
Visualization. We visualize the loss value DLN on MNIST and CIFAR-10 separately in Figure 3 which illustrates the capacity of L.2T-DIN to adapt the evolving states of students attain improved performance. The DLN is initialized with the Kaiming normal initialization with LeakyRel.U activations.
6 Conclusions
This paper introduces 1.2T-DLN, an adaptive model for various stages of student learning. Technically, We propose a differentiable three-stage teaching framework, asynchronously optimizing the student, DLN, and teacher. An LSTM teacher dynamically captures and retains experiences during DLN learning. Additionally,
assess L2T-DLN's ability to navigate strict saddle points using the negative curvature of their Hessian matrix. Experiments demonstrate our DLN outperforming specially designed loss functions. Nevertheless, our approach demands significant computational resources for high-order derivatives, which we aim to mitigate in future work.
7 Acknowledgment
This work was supported in part by National Natural Science Foundation of China (62302045, 82171965), Clinical and Translational Medical Research Fund of the Chinese Academy of Medical Sciences 20200-2M-C&&T-B-0077 and Beijing Institute of Technology Research Fund Program for Young Scholars.
10

---

References
[1] Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W Hoffman, David Pfau, Tom Schaul, Brendan Shillingfor,, and Nando De Freitas. Learning learn by gradient descent by gradient descent. Advances in neural information processing systems, 29, 2016.
[2] Sungyong Baik, Janghoon Choi, Heewon Kim, Dohee Cho, Jaesik Min, and Kyoung Mu Lee. Meta-learning with task-adaptive loss function for few-shot learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9465-9474, 2021.
[3] Jonathan Barron. general and adaptive robust loss function. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4331 44399, 2019.
[4] Dimitri P Bertsekas. Nonlincar programming. Journal of the Operational Research Society, 48(3):334-334, 1997.
[5] Leland Bradford. The teaching-learning transaction. Adult Education, 8(3):135-145, 1958.
[6] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88:303 338, 2010.
[7] Yang Fan, Fei Tian, Tao Qin, Xiang-Yang Li, and Tic- Yan Liu. Learning to teach. In International Conference on Learning Representations, 2018.
[8] Yang Fan, Yingce Xia, Lijun Wu, Shufang Xie, Weiqing Liu, Jiang Bian, Tao Qin, and Xiang-Yaue Yang Li. Learning to reweight with deep interactions. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pages 7385-7393 2021.
[9] Felix A Gers, Jürgen Schmidhuber, and Fred Cummins. Learning to forget: Continual prediction with Istm. Neural computation, 12(10):2451-244 2000.
[10] Mengqiao Han, Liyuan Pan, and Xiabi Liu. Astronet: When astrocyte meets artificial neural network. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 20258 20268,,028 June 2023.
[11] Bharath Hariharan, Pablo Arbeláez, Lubomir Bourdev, Subhransu Maji, and Jitendra Malik. Semantic contours from inverse detectors. In 2011 international conference computer vision, pages 991-998. IEEE, 2011.
[12] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[13] Geoffrey Hinton, Nitish Srivastava, and Kevin Swersky. Neural networks for machine Icarning lecture 6a overview of mini-batch gradient descent. Cited on, 14(8):2, 2012.
[14] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural compulation. 173-:173517780 1997.
[15] Chen Huang, Shuangfei Zhai, Walter Talbott, Miguel Angel Bautista, Shih- Yu Sun, Carlos Guestrin, and Josh Susskind. Addressing the loss-metric mismatch with adaptive loss alignment supplementary material.
[16] Chen Huang, Shuangfei Zhai, Walter Talbott, Miguel Bautista Martin, Shih-Y Yu Sun, Carlos Guestrin, and Josh Susskind. Addressing the loss-metrie mismatch with adaptive loss alignment. In International conference on machine learning, pages 2891-2900. PMLR, 2019.
[17] JC lhejirika. Teaching strategies for adult learners: Implications of learning characteristics for effective leaching-learning transaction. Academic Research International, 4(2):310, 2013.
[18] Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. Mentomet: Learning data-driven curriculum for very deep neural networks corrupted labels. In International conference on machine learning, pages 2304-2313. PMIR, 2018.
[19] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint urXiv:1412.600 2014.
[20] A Krizhevsky. Learning multiple layers of features from tiny images. Master's thesis, University of Tront, 2009.
[21] Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar 10 and cifar- 100 datasets. URl: https://www. cs. toronto. edu/kriz/cffar. huml, 6(1):1, 2009.

---

[22] Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part 13, pages 740-755 Springer, 2014.
[23] Qingliang Liu and Jinmei Lai. Stochastic loss function. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34. pages 4884-4894 2020.
[24] Weiyang Liu, Yandong Wen. Zhiding Yu, and Meng Yang. Large- margin softmax loss for convolutional neural networks. International Conference on Machine Learning, pages 507-516, PMLR, 2016.
[25] Weiyang Liu, Bo Dai, Ahmad Humayun, Charlene Tay. Chen Yu, Linda B Smith, James M Rehg. and Le Song. Iterative machine teaching. In International Conference Machine Learning, pages 2149-2158.
PMLR, 2017.
[26] Songtao Lu, Mingyi Hong, and Zhengdao Wang. On the sublinear convergence of randomly perturbed alternating gradient deseent to second order stationary solutions. arXiv preprint arXiv:1802.1000 10418, 2018.
[27] Payal Mehra and Monika Mital. Integrating technology into the teaching- -earning transaction: Pedagogical and technological perceptions of management faculty. International Journal of Education and Development using ICT, 3(1), 2007.
[28] Thomas Howard Morris. Vocational education of young adults in england: A systemic analysis of teaching- learning transactions that facilitate self-directed learning. Journal of Vocational Education Training, 70 (4):619 643, 2018.
[29] Baraka Manjale Ngussa and Lazarus Ndiku Makewa. Constructivism experiences in seaching-lcarninn transaction among adventist secondary schools in south nyanza, tanzania. American Journal of Educational Research, 2(11 A):177, 2014.
[30] Tan Nguyen and Scott Sanner. Algorithms for direct 0-1 loss optimization in binary classification. In International Conference on Machine Learning, pages 1085-1093. PMLR, 2013.
[31] Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. arXiv preprint arXiv: 1804,02767, 2018.
[32] Mengye Ren, Wenyuan Zeng, Bin Yang. and Raquel Urtasun. Learning to reweight examples for robust deep learning. In International conference on machine learning, pages 4334 4343. PMLR, 2018.
[33] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International jourhin journal of computer vision, 115:211-2521 2015.
(34] Jürgen Schmidhuber. Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-... hook. PhD thesis, Technische Universität München, 1987.
[35] Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Zongben Xu, and Deyu Meng. Meta deta-weight--ne net: Learning an explicit mapping for sample weighting. Advances in neural information processing systems, 32, 2019.
[36] Sehastian Thrun and Lorien Pratt. Learning to leamn Springer Science & Business Media, 2012.
[37] Lijun Wu, Fei Tian, Yingee Xia, Yang Fan, Tao Qin, Lai HunnHHang, and Tic- Tie-Yan Liu. Learnnng teach with dynamic loss functions. Advances in neural information processing systems, 31, 2018
[38] Yan Yang, Liyuan Pan, and Liu Liu, Event camera data pre-training. arXiv preprint a7X.922.30..01128 2023.
[39] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In British Machine Vision Conference 2016. British Machine Vision Association, 2016.
[40] Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiyy Jia. Pyramid scene parsing network. In Proceedings the IEEE conference on computer vision and pattern recognition, pages 2881 2890, 2017.
[41] Xiaojin Zhu. Machine teaching: An inverse problem to machine learning and approach toward optimal education. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 29, 2015.
[42] Barret Zoph and Quoe Le. Neural architecture search with reinforcement learning. In International Conference on Learning Representations, 2016.
12

---

[43] Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V Le. Learning transferable architectures for scalable image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 8697-8710, 2018.
13


Reviews:
[{"review_id": "tAbqGnNW39", "reviewer": "Reviewer_wSe9", "summary": "This paper presents a three-stage framework that dynamically adjusts the learning process of student model (i.e., target model). The loss value is calculated via the proposed Dynamic Loss Network (DLN), parameterized as neural network. And the DLN is updated by the teacher network implemented as LSTM. The authors also provide a convergence analysis. The model is evaluated on an array of tasks including image classification, object detection and semantic segmentation.", "questions": "1. The output layer (e.g., activation function) of the DLN is not clearly described.\n2. It would be better to understand the DLN if the loss value/weights can be visualized.\n3. Figure 1 in the supplementary material indicates the DLN (blue line) would produces lower gradients applied on the student model. Is this because the baseline model uses a large learning rate (0.1 line 219)?\n4. The claimed effectiveness would be more convincing if an ablation study of learning rates is provided.\n5. Regarding the initial gradients produced by the DLN, which greatly exceeds the baseline, it may be related to the network initialization of the DLN. Since this detail is missing, I would suggest the author clarifying the initialization. \n6. For the DLN of object detection, is it initialized as identical mapping? If not, it’s helpful to investigate this manner.\n7. What’s the training epoch for baseline model and the L2T-DLN for image classification?", "limitations": "1. Some details (e.g., initialization and activation of the output layer) of the proposed DLN are missing. \n2. The used learning rate for image classification may not be optimal, an ablation study would help understand the framework.\n3. It would be helpful to visualize the output of DLN.  \n4. The framework may consume lots of GPU memory.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. Compared to previous works, L2T-DLN leverages the state of loss function and the temporal information of student learning experience  to update the teacher.\n2. Experiments on various application including image classification, object detection and semantic segmentation showcase the effectiveness.\n3. The provided convergence analysis is helpful to understand the framework.", "weaknesses": "1. The proposed DLN is not unified. For image classification, it produces the loss value. For object detection, it generates the loss weights.\n2. Some technical details of the DLN are missed, as will discussed in next session.\n3. To access the temporal information of student learning, L2T-DLN would store the historical models and consume lots of memory.\n4. Since the DLN adjusts the loss weights of the objectives of YOLO-v3, it’s reminiscent of multi-task learning. In this regard, the authors should discuss the relationship with MTL.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments. \n\nWeaknesses:\n\n1. Objective detection involves two sub-tasks, i.e., regression and classification. It is challenging to handle two tasks simultaneously with one dynamic loss. An alternative way commonly used in existing dynamic loss-based works is to dynamically combine the objective functions of different tasks. Therefore, we follow their setting to perform our L2T-DLN. \n\n2. We provide point-to-point responses in the next sessions.\n\n3. Our memory unit stores the long- and short-term dependencies captured during the teaching process, disregarding the historical models. The size of the memory unit is fixed at 2.55MB and not consumes lots of memory.\n\n4. L2T-DLN employs a dynamic loss set by a teacher to train a student model on a specific task, whereas MTL leverages potential correlations between diverse tasks to train a shared representation network. The L2T-DLN adjusts loss weights based on the temporal relationship between student performance and losses, for the same task, instead of using inter-task correlations as in MTL.\n\nQuestions:\n\n1. Our DLN is a fully connected network (kindly refer to Lines 233-235, Lines 246-247, and Line 257, main paper, and Lines 184-186, supplementary material), as well as the output layer. Following existing works, the output layer is not processed by any activation function.\n\n2. We visualize the loss value of DLN on MNIST and CIFAR-10 separately in Figure 1 and 2, rebuttal. The DLN is initialized with the kaiming normal initialization with LeakyReLU activations.\n\n3. No. Both our L2T-DLN and baseline follow the same learning rate settings of related work (references [3, 7, 15], supplementary material) about noise-label classification.\n\n4. We evaluate the student model (ResNet8) concerning different learning rates on CIFAR-10. The learning rates were set to 1, 0.5, 0.1, 0.05, 0.01, 0.005, and 0.001, corresponding to accuracies of 18.8%, 49.0%, 90.7%, 87.6%, 88.1%, 87.3%, and 84.2%, respectively. Therefore, we set the learning rate as 0.1 in our experiment.\n\n5. The DLN experiences random initialization through Kaiming normal initialization coupled with LeakyReLU activations for classification and segmentation tasks. For the objective detection task, all parameters of the DLN are initialized to a value of 1. There is no significant correlation between the substantial gradient and initialization of DLN during the initial training phases. In fact, in the early training stages, the significant DLN gradient prompts substantial changes in the student model, thereby broadening L2T's exploratory reach within the solution space. This expanded exploration facilitates the investigation of numerous local optimal solutions, enabling the selection of the most favorable among them. \n\n6. Yes, it is initialized as identical mapping. \n\n7. The baseline is trained with 200 epochs. In our approach, we set the teacher learning phase for 10 epochs (Line 222, main paper). "}, {"review_id": "HxilSFY2CZ", "reviewer": "Reviewer_9Vkw", "summary": "In this paper, the authors state that existing methods only employ a simple feedforward network as the teacher model, which limits the potential of L2T. This issue motivates authors to propose a network with a memory unit to enhance the temporal analyzing ability of the teacher in the learning to teach (L2T) tasks. This proposed new network is combined with a dynamic loss network to address the issue of the existing works above. The experiment shows the superiority of the proposed method.", "questions": "See weakness.", "limitations": "No limitations", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 1, "strengths": "(+) This paper theoretically analyzes the convergence of the proposed method.\n\n(+) The proposed method shows the state-of-the-art performance compared with other methods.\n", "weaknesses": "(-) The novelty of this paper is not clear. In the title, abstract and Introduction, the authors repeatedly emphasize that they propose an L2T framework with a Dynamic Loss Network (L2T-DLN). However, these emphases can cause misleading since Dynamic Loss has already been introduced in previous work, as stated by the authors themselves in Lines 2, 23, and the caption of Figure 1. Consequently, the application of Dynamic Loss in L2T cannot be considered as the novelty or the main contribution of the paper. The continued highlighting of DLN in these sections adds to the confusion and may hinder the understanding of the paper. In my understanding, the primary contribution of this paper is a proposed network with a memory unit to enhance the temporal analyzing ability of the teacher in the learning to teach (L2T) tasks. By emphasizing this aspect, rather than the Dynamic Loss network, the paper will become more coherent and easier to comprehend.\n\n(-) The authors have not adequately demonstrated the effectiveness of their proposed method in both theory and practice. In the theoretical analysis, they solely focus on the convergence of the proposed L2T-DLN and fail to illustrate how the proposed network with a memory unit for the teacher can benefit the training of L2T task. Additionally, the experiments show the better performance of the proposed method compared with other SOTA methods. However, based on section 5.1, this better performance is based on many combinations of loss functions and tricks. It remains unclear whether the proposed network with a memory unit for the teacher contributes to the enhanced performance observed in the L2T task.\n\n(-) The paper lacks sufficient ablation studies to support its claims. In the ablation study section, the authors include some experiments related to selecting the best learning parameters for their proposed method; these parameters may primarily influence performance rather than directly relating to the main contributions of the proposed method. Therefore, it is crucial for the authors to conduct a series of ablation studies to demonstrate how the proposed network with a memory unit for the teacher can benefit the training of L2T tasks.\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments.\n\nWeaknesses:\n\n1. The novelty of our approach is introduced in Lines 8-13 and Lines 50-55, main paper, which are:\n\n    (1) design a teaching strategy based on the gradient concerning DLN; \n\n    (2) use LSTM as the teacher model to update the DLN with the temporal information; \n\n    (3) provide a convergence analysis of the approach. \n\n    Using a network with a memory unit to enhance the temporal analyzing ability of the teacher is one of our contributions.\n\n    The introduction of background knowledge, i.e., existing studies of L2T with dynamic loss, is necessary. Lines 2, 23, and Figure 1 (main paper), aim to help readers understand the difference between L2T-DLN and existing works about L2T with dynamic loss (DL). By realizing the differences, the readers will not be misled that the usage of DL is our contribution.\n\n2. Please kindly refer to Section 3 and Section 5 for theoretical analysis and effectiveness demonstration of our approach. In practice, we evaluate three downstream tasks, i.e., image classification, object detection, and semantic segmentation. Experiments on three downstream tasks (refer to Section 5.2, main paper) and ablation study (refer to Lines 284-291, main paper) demonstrate that:\n\n    (1) L2T-DLN has shown state-of-the-art performance compared with other methods on different downstream tasks; \n\n    (2) The ablation study has drawn two conclusions: the first is that algorithms that can use historical information perform better than those that cannot, and the second is that the adaptability to capture and maintain short- and long-term dependencies can further enhance the loss function teaching, compared to handcrafted method.\n\n3. Please kindly refer to Section 5.3 in our main paper and Section 5.2 in the supplementary material for the ablation study. The ablation study of w/o memory unit is shown in Lines 284-291, main paper, and Lines 215-221, supplementary material."}, {"review_id": "VADdPVTjZl", "reviewer": "Reviewer_79Na", "summary": "This paper introduces an improvement towards the learning-to-teach framework. Compared with the previous works, the authors made an innovation that a dynamic loss network is added with LSTM acting as teacher model to enhance the temporal memorization. A three step optimization procedure is also proposed with an convergence analysis theoretically. Last extensive experiments are conducted on various computer vision tasks that demonstrated the effectiveness of the proposed approach. ", "questions": "While the memory units of LSTM make sense in this scenario, how about use Transformer as another sequential model which may even enhance the temporal memorization compared with LSTM? ", "limitations": "Stated in the paper as well as the above point, that the computational complexity is one concern while it seems feasible and justifiable to further solve this, as the future potential value of this direction is likely large.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. The problem this paper studied is an important one yet insufficiently studied in the current literature, that is how to use a better teaching strategy in addition to merely conduct \"learning\". It's a very meaningful step forward of this paper that pushes forward the frontier of such a literature; Furthermore the design of dynamic loss network is a new perspective for this area as well, going beyond the dynamic loss function design.\n\n2. The paper is technically very sound with enough insights and depth. I particularly appreciate the convergence analysis which is done first time for the machine teaching/learning-to-teach area, with a good depth leveraging the negative curvature.", "weaknesses": "As also stated in the paper, despite being theoretically sound and elegant, the proposed algorithm incurs much more computational overhead which will hinder its practical usefulness in real world applications. Particularly the compute of Hessian even three-order derivatives is a sign of its computational complexity. I agree with the author that this needs to be improved further in the future to enlarge the impact of this work. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1. Our target for the submission is to re-visit L2T and formulate the loss adjustment as a temporal task. We hope our discussions inspire other further work. Thank you for your valuable and attractive suggestions for using transformer architecture as a teacher. Using a transformer as the teacher model involves the simultaneous processing of sequential data across timestamps. It might meet challenges, e.g., the accumulation of historical gradients and pressure on storage resources. We will explore the suggestion in the future.\n\n2. Thank you for your valuable comments and efforts. We will attempt the following strategies in our future work: \n\n    (1) Design of appropriate prior knowledge for enhanced fusion to curtail the need for extensive higher-order gradient computations; \n\n    (2) Employing finite element method paradigm by partitioning the parameters associated with the student, DLN, and teacher models into discrete finite elements; \n\n    (3) Integration of Multivariate Taylor Series Expansion to more effectively navigate complex computations while optimizing resource consumption."}, {"review_id": "U2GMG6drrR", "reviewer": "Reviewer_HMfR", "summary": "The paper introduces the concept of teaching in machine learning and proposes a framework called L2T-DLN (Learning to Teach with Dynamic Loss Network). The framework aims to address the limitations of existing approaches by incorporating the temporal nature of loss function adjustment and utilizing the states of the loss function. The authors formulate the loss adjustment as a temporal task using a teacher model with memory units and propose a dynamic loss network to enhance the interactions between the teacher and student model. Extensive experiments demonstrate the effectiveness of the approach on various real-world tasks.", "questions": "1. Further clarification is needed in the paper on why it is necessary to consider the gradients concerning loss functions as a core idea. Besides experimental results, could there be theoretical analysis supporting this aspect?\n2. The authors introduce a Dynamic Loss Network instead of using a dynamic loss function. It would be helpful to explain if this choice leads to a larger number of parameters and potential unfairness in comparisons. Please clarify the impact of parameter quantity in this context.", "limitations": "The implementation of this approach necessitates substantial computational resources for calculating high-order derivatives.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper presents a novel framework, L2T-DLN, which integrates the concept of teaching with dynamic loss functions. This approach stands out by considering the temporal nature of loss function adjustment and utilizing the states of the loss function, setting it apart from existing works.\n2. The authors provide a thorough convergence analysis of the proposed framework, demonstrating its effectiveness and its ability to achieve convergence.", "weaknesses": "1. Unclear Motivation: The paper lacks a thorough discussion of the benefits of introducing the gradients concerning loss functions. It would be beneficial to include a clearer motivation.\n2. Marginal Improvement: The reported improvements of 0.4% lower than ALA on ImageNet for classification and only 0.3% improvement compared to the baseline on mIoU on the VOC dataset for segmentation are relatively small.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments.\n\nWeakness:\n\n1. Previous studies involve directly supplying certain states of the student model to a teacher for dynamic loss adjustment (kindly refer to Lines 28-30, main paper). This direct provision of states without integration hinders L2T convergence, as the teacher model necessitates further learning to integrate these state-based insights effectively. In contrast, the gradient concerning DLN achieves holistic information integration throughout the learning process (kindly refer to Lines 130-136, main paper), facilitated by prior knowledge (chain rule). Employing the gradient concerning the loss allows the teacher model to concentrate on capturing and preserving crucial information from gradients, negating the need for supplementary handling of dispersed states.\n\n2. The performance of ALA (reference [15], main paper) benefits from two aspects: \n\n    (1) The length of a student learning stage ALA set (200) is much larger than ours (25). Note our ablation showed that the size of the length is positively correlated with the test accuracy and computational consumption (Lines 276-283, main paper);\n\n    (2) ALA employed a multi-student training strategy to achieve stronger performance with an extra 20%-50% cost over other methods [A].\n\n    Therefore, considering its computational resources, our improvement is not marginal compared to ALA.\n\nQuestions:\n\n1. (1) For the benefits of the gradient concerning the loss, please refer to Weakness 1;\n\n    (2) Please kindly refer to Lines 130-136, main paper, for theoretical analysis. The gradient concerning the DLN involves the information of both training and validation data. As these pieces of information depend on each other, they are integrated into the temporal changes of the student. The gradient concerning DLN plays a crucial role in fusing these pieces of information. Therefore, compared to the state of the student, the gradient is the integration of information throughout the learning process, which provides more information to promote deep interaction between DLN, teacher, and student.\n\n2. (1) At the testing stages, the number of parameters of the student model is the same in ours and other baselines. The comparisons are fair. Note, in the L2T framework, the teacher and the dynamic loss network only affect the optimization of the student model during the training phase, while the student model operates independently during the testing phase; \n\n    (2) The number of parameters in DLN is 5000. Compared to commonly used models, e.g., the parameter of ResNet8 is 1235274, 5000 number of parameters is relatively small; \n\n    In response to the reviewer's concerns, we construct an experiment. We added 5000 parameters to the teacher model of baseline 'Stochastic Loss Function' (SLF, reference [22], main paper). The performance of SLF on CIFAR-10+ResNet8 with a testing accuracy of 89.97%, while our L2T-DLN still maintains a performance advantage (90.65%).\n\n[A] Huang, C., Zhai, S., Talbott, W., Bautista, M. A., Sun, S. Y., Guestrin, C., & Susskind, J. Addressing the Loss-Metric Mismatch with Adaptive Loss Alignment Supplementary Material.\n"}]

Author rebuttal:
We thank all reviewers' valuable comments and efforts. Our proposed L2T-DLN is technically very sound with enough insights and depth [79Na], provides a thorough convergence analysis [wSe9, HMfR], and outperforms state-of-the-art methods in various downstream tasks [9Vkw,wSe9]. We provide point-to-point responses in the Official Review system.

Meta-review: This paper proposes a novel L2T-DLN to enhance the temporal analyzing ability of the teacher in the learning to teach (L2T) tasks. Reviewer wSe9, 79Na and HMfR acknowledged the method novelty and experiments. After rebuttals, reviewer wSe9 think his/her concerns have been addressed. The area chair checks that authors also resolved well the problems of reviewer 79Na and HMfR. Reviewer 9Vkw gives negative comments. However the AC found the reviewer did not carefully read the paper and missed most of contents. The area chair thus decides to accept it.
Decision: Accept (poster)