SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Zhaoyang Hai  $^{1, *}$ , Liyuan Pan  $^{1,2,*, \dagger}$ , Xiabi Liu  $^{1,*, \dagger}$ , Zhengzheng Liu $^{1}$ , Mirna Yunita $^{1}$
$^{1}$  School of Computer Science and Technology,  $^{2}$  BIT Special Zone
Beijing, China, 100081
{haizhaoyang, liyuan.pan, liuxiabi, liuzhengzheng, mirnayunita}@bit.edu.cn
With the concept of teaching being introduced to the machine learning community, a teacher model start using dynamic loss functions to teach the training of a student model. The dynamic intends to set adaptive loss functions to different phases of student model learning. In existing works, the teacher model 1) merely determines the loss function based on the present states of the student model, i.e., disregards the experience of the teacher; 2) only utilizes the states of the student model, e.g., training iteration number and loss/accuracy from training/validation sets, while ignoring the states of the loss function. In this paper, we first formulate the loss adjustment as a temporal task by designing a teacher model with memory units, and, therefore, enables the student learning to be guided by the experience of the teacher model. Then, with a dynamic loss network, we can additionally use the states of the loss to assist the teacher learning in enhancing the interactions between the teacher and the student model. Extensive experiments demonstrate our approach can enhance student learning and improve the performance of various deep models on real-world tasks, including classification, objective detection, and semantic segmentation scenarios.
In pedagogy study, teachers refine their teaching ability based on student feedback, e.g., exam scores. Students benefited from the enhanced ability of teachers and then achieved high scores on the exam. Both teachers and students are developed in this interaction iteratively and constantly [27-29, 17]. The phenomenon is known as teaching-learning transaction [5], or learning to teach (L2T) in machine learning [7].
In L2T, a teacher model uses a dynamic loss function, which acts as an exam paper, to train and optimize the student model (as shown in Figure 1(a)). However, existing approaches adjust the loss functions by only employing a simple feedforward network as the teacher model, and neglecting the temporal nature of loss function adjustment. The disregarding of the experience accumulation for teachers (e.g., the ability to analyze all previous exam scores for a student), and, therefore, limits the potential of L2T. In addition, in previous works, the teacher model only focuses on the state of the student model, i.e., training iteration number [37], training/validation accuracy [37, 16], training/validation loss[2], and the output of the student model[16, 2, 23]. However, the states of loss functions (e.g., the gradients concerning loss functions) are neglected, which dilutes the benefit of improving the exam paper. In other words, the teacher needs to consider that the question changes of an exam paper also influence the performance of a student.
(b)
Figure 1: Illustration of the common L2T and our L2T-DLN framework. (a) The common framework in existing L2T works. (b) The framework of our L2T-DLN. Both (a) and (b) contain three models: a student model  $S_{\theta}$  with the parameter  $\theta$ , a dynamic loss model  $L^{m}$ , and a teacher model  $T_{\varphi}$  with the parameter  $\varphi$ . Here,  $m$  denotes the  $m^{th}$  iteration,  $x$  and  $y$  denote the input data of the student and corresponding label, respectively. Different from existing works that only use feedforward networks, we employ a network with a memory unit to enhance the temporal analyzing ability of the teacher. Then, we use the gradient  $\nabla \phi$  concerning  $L_{\phi}^{m}$  by designing a dynamic loss network to provide more information to the teacher model, compared to the state of the student  $\tau$ .
In this paper, we propose an L2T framework with a Dynamic Loss Network (L2T-DLN), to address the above-mentioned issue (in Figure 1(b)). First, we adopt a Long-Short Term Memory (LSTM) model as the teacher and design a differentiable three-stage asynchronous optimization strategy. Then, to ensure the teacher model can be optimized with the state of loss functions, we design a Dynamic Loss Network (DLN) instead of using the dynamic loss function (DLF). Specifically, we start by optimizing the student model through backpropagation in the first step with a fixed DLN as the loss function. Then, compute the gradient of the validation error of the student model with respect to the DLN. Next, we input this gradient into the teacher model, and the output of the teacher model is used to update the DLN. To achieve the updating of the teacher model, we perform another round of student learning with the updated DLN and obtain the gradient of the validation error of the updated student with respect to the teacher model. Moreover, we analyze how L2T-DLN exploits the negative curvature by using a special alternating gradient descent (AGD) sequence, achieving a differentiable asynchronous optimization.
In summary, the usage of the gradient concerning DLN and the LSTM teacher model both ensure the teacher model captures and maintains short- and long-term temporal information, which can further improve the performance of loss function teaching, compared to feedforward teachers [37, 16, 23].
Our main contributions are 1) design a dynamic loss network-based teaching strategy to let the teacher model learn optimized by the gradient of DLN; 2) use LSTM as the teacher model to update the DLN with the temporal information of the teacher model; and 3) a convergence analysis of the approach, which is treated as a special AGD sequence and has the potential to escape strict saddle points. We conduct extensive experiments on a wide range of loss functions and tasks to demonstrate the effectiveness of our approach.
Recent work by L2T [7] provides a comprehensive view of teaching for machine learning, encompassing aspects such as training data teaching, loss function teaching, and hypothesis teaching. In contrast to previous literature on machine teaching [41, 25, 10, 38], L2T breaks the strong assumption regarding the existence of an optimal off-the-shelf student model [37]. Instead, L2T employs automatic techniques to reduce the reliance on prior human knowledge, aligning with principles such as learning to learn and meta-learning [34, 36, 42, 1]. The recent focus of L2T has been mainly on loss function teaching [37, 16, 23, 2] and training data teaching [7, 25, 35, 32, 8].
During the training of a student model, there is a variation in the distribution of predictions where earlier in the training the distribution tends to differ from that at convergence. Consequently, an adaptive loss function is crucial. Existing works [37, 16, 23, 2] formulate the loss adjustment as some independent tasks by performing a multi-layer perceptron (MLP) as the teacher model. The differences lie in the representation of dynamic loss functions and the input information of teacher models. The representation of the dynamic loss function includes the variation of handcrafted loss functions [37, 23] and neural network [16, 2]. The input information contains the training iteration number [37, 16], training/validation accuracy [37, 16], training/validation loss[2], and the output of the student model[16, 2, 23]. In detail, Wu et al. [37] trains a neural network with an attention mechanism to generate a coefficient matrix between the prediction of the student and the ground truth. Huang et al. [16] constructs the teaching-learning framework with reinforcement learning. Their teacher also employs an MLP and generates the policy gradient for a loss network. Liu and Lai [23] utilizes a teacher model to guide the selection and combination of handcrafted loss functions. Baik et al. [2] performs a teacher to generate two weights for each layer of a loss network, and then updates the parameters of the loss network by affine transformation with the two weights.
Assigning weights to different data points has been widely investigated in the literature, where the weights can be either continuous [18] or binary [7]. In detail, Fan et al. [7] proposed a learning paradigm where a teacher model guides the training of the student model. Based on the collected information, the teacher model provides signals to the student model, which can be the weights of training data. Liu et al. [25] leveraged a teaching way to speed up the training, where the teacher model selects the training data balancing the trade-off between the difficulty and usefulness of the data. Fan et al. [8] inputs internal states of the student model, e.g., feature maps of the student model, to the teacher model and obtains the output of the teacher as the weight for corresponding training data.
In this section, we overview our L2T-DLN in Section 3.1, introduce the corresponding framework for student learning in Section 3.2, describe the DLN learning framework in Section 3.3, and discuss teacher learning in Section 3.4.
Our L2T-DLN is a differentiable teaching framework that enhances the performance of a student model. The L2T-DLN contains stages: (I) student learning, which optimizes a student model  $S_{\theta}$  with parameter  $\theta$ ; (II) DLN learning, which optimizes the DLN  $L_{\phi}$  with parameter  $\phi$ ; (III) teacher learning, which optimizes the teacher model  $T_{\varphi}$  with parameter  $\varphi$ .
Starting from  $S_{\theta}^{0}$ , in stage (I), we optimize the student model  $S_{\theta}^{0} \rightarrow S_{\theta}^{N}$  on training data  $x_{train}$  by leveraging the DLN  $L_{\phi}^{0}$  as the loss function, where  $N$  denotes the number of iterations in a student learning stage. In stage (II), we compute the error  $e_{val}$  of the student model  $S_{\theta}^{N}$  on validation data  $x_{val}$ , and then determine the gradient  $\nabla \phi^{0} = \partial e_{val} / \partial \phi^{0}$ . This gradient is then given to the teacher model, and the DLN is updated as  $\phi^{1} = \phi^{0} + g^{0}$ , where  $g^{0}$  indicates the output of the teacher model  $T_{\varphi}^{0}$ . In stage (III), we first train the student model  $S_{\theta}^{N} \rightarrow S_{\theta}^{2N}$  with the updated DLN  $L_{\phi}^{1}$ . Then, we obtain the validation error  $e_{val}$  of the student model  $S_{\theta}^{2N}$  and optimize the teacher model by backpropagation (BP) based on  $e_{val}$ . The objective function of our L2T-DLN is:
$$
\left(\theta^ {(M + K) N}, \phi^ {M}, \varphi^ {K}\right) \leftarrow \left(\theta^ {0}, \phi^ {0}, \varphi^ {0}\right) - \bigtriangledown_ {\left(\theta , \phi , \varphi\right)} \sum_ {k = 0} ^ {K} e _ {v a l} \left(\theta^ {2 (k + 1) N}\right). \tag {1}
$$
Our goal is to achieve  $\theta^{(M + K)N},\phi^M$  and  $\varphi^K$ , where  $M$  and  $K$  denote the number of iterations of DLN and teacher learning, respectively (details are shown in Figure 2).
3.2 Student learning
For a given task, we define the input and output space as  $X$  and  $Y$ , respectively. The student model is then denoted by  $S_{\theta}: X \to Y$ . Our student learning involves minimizing the output value of the
Figure 2: The pipeline of L2T-DLN. Grey, black, and red lines represent the optimization of the student parameter  $\theta$ , the DLN parameter  $\phi$ , and the teacher parameter  $\varphi$ , respectively. We aim at obtaining  $\theta^{(M + K)N}$ ,  $\phi^M$ , and  $\varphi^K$ , where  $M$  and  $K$  denote the number of iterations for DLN and the teacher and  $M = K$ . There are three stages: student, DLN, and teacher learning. During student learning, a student model is optimized by backpropagation with  $L_{\phi}^0$ . During a DLN learning, the teacher  $T_{\varphi}^0$  accepts the gradient  $\nabla \phi^0$  and output  $g^0$  to update  $L_{\phi}^0$ . During a teacher learning, we perform another student learning with  $L_{\phi}^1$  and optimize the teacher  $\varphi^0$  with the validation error of the updated student.
DLN, i.e.,  $\underset {\theta \in \Omega}{\arg \min}\sum_{(x,y)\in x_{train}}wL_{\phi}^{m}(S_{\theta}(x),y)$ , under a hypothesis space  $\Omega$  using training data  $x_{train}$ . Note,  $w$  is a weight parameter of  $x$ ,  $L_{\phi}^{m}$  is the loss function, and  $m$  is the  $m^{th}$  iteration for the DLN. During each stage of student learning, we iteratively train the student model  $N$  times with  $L_{\phi}^{m}$ . The optimization of the student model during the current stage is:
$$
\theta^ {i} = \theta^ {i - 1} - \eta \partial w L _ {\phi} ^ {m} \left(S _ {\theta} ^ {i - 1} (x), y\right) / \partial \theta^ {i - 1}, i = \{1, 2, \dots , N \}, \tag {2}
$$
where  $\eta$  denotes the learning rate of the student model. In our framework, the  $L_{\phi}^{m}$  that with learnable parameters are optimized in different student learning stages for providing seemingly guidance.
3.3 DLN learning
After the student learning stage (e.g.,  $S_{\theta}^{0} \rightarrow S_{\theta}^{N}$  with  $L_{\phi}^{0}$ ), we use the teacher model  $T_{\varphi}^{0}$  to adjust the DLN parameters  $\phi$  (e.g.,  $\phi^0 \rightarrow \phi^1$ ). To enable the temporal property of DLN, we use an LSTM to transform  $\phi$  dynamically:
$$
\phi^ {1} = \phi^ {0} + \gamma g ^ {0}, \left[ \begin{array}{l} g ^ {0} \\ h ^ {1} \end{array} \right] = T _ {\varphi} ^ {0} (\nabla \phi^ {0}, h ^ {0}), \tag {3}
$$
where  $\gamma$  denotes the learning rate of DLN and  $\nabla \phi^0$  represents the gradient of the validation error  $e_{val}$  of  $S_{\theta}$  with respect to  $\phi^0$ .
Considering the gap between training data and validation data, we employ the Reverse-Mode Differentiation (RMD) to calculate  $\nabla \phi^0$ . The RMD involves performing the SGD process in reverse order from  $N$  to 1, as depicted by the black lines in Figure 2. According to Eq. (2), the gradient of the validation error  $e_{val}(S_{\theta}^{N})$  with respect to  $\theta^N$  can be calculated as follows:
$$
\nabla \theta^ {N} = \partial e _ {\text {v a l}} \left(S _ {\theta} ^ {N}\right) / \partial \theta^ {N}. \tag {4}
$$
Then looping backward from  $N$ . At each step  $i = \{N - 1, \dots, 1\}$ , the calculation of gradient is shown as:
$$
\nabla \theta^ {i - 1} = \nabla \theta^ {i} - \eta w \frac {\partial^ {2} L _ {\phi} ^ {0} \left(S _ {\theta} ^ {i - 1} (x) , y\right)}{(\partial \theta^ {i - 1}) ^ {2}} \nabla \theta^ {i}. \tag {5}
$$
At the same time, the gradient of  $e_{val}(S_{\theta}^{N})$  with respect to  $\phi^0$  is accumulated as:
$$
\nabla \phi^ {0} = \nabla \phi^ {0} - \eta w \frac {\partial^ {2} L _ {\phi} ^ {0} \left(S _ {\theta} ^ {i - 1} (x) , y\right)}{\partial \theta^ {i - 1} \partial \phi^ {0}} \nabla \theta^ {i}. \tag {6}
$$
Reverting backward from  $N$  to 1, we get  $\nabla \phi^0$ .
Taking  $N = 1$ , we rewrite Eq. (6) by using the gradient, e.g.,  $\nabla \phi^0$ , as:
$$
\nabla \phi^ {0} = \eta w \frac {\partial e _ {v a l}}{\partial S _ {\theta} ^ {1} (x _ {v a l})} \frac {\partial S _ {\theta} ^ {1} (x _ {v a l})}{\partial \theta^ {1}} \left(\frac {\partial \theta^ {1}}{\partial \frac {\partial L _ {\phi} ^ {0} (S _ {\theta} ^ {0} (x) , y)}{\partial S _ {\theta} ^ {0} (x)}} \frac {\partial \frac {\partial L _ {\phi} ^ {0} (S _ {\theta} ^ {0} (x _ {t r a i n}) , y)}{\partial S _ {\theta} ^ {0} (x)}}{\partial \phi^ {0}}\right) \frac {\partial S _ {\theta} ^ {0} (x)}{\partial \theta^ {0}}, \tag {7}
$$
where  $\nabla \phi^0$  contains the information of both training and validation data,  $(\partial S_{\theta}^{1}(x_{val}) / \partial \theta^{1}$  and  $\partial S_{\theta}^{0}(x) / \partial \theta^{0})$ . As these pieces of information are dependent on each other, they are integrated into temporal changes of  $\theta$ . The gradient concerning  $\phi^0$  achieves holistic information integration throughout the learning process, facilitated by prior knowledge (chain rule). Employing the gradient concerning the loss allows the teacher model to concentrate on capturing and preserving crucial information from gradients, negating the need for supplementary handling of dispersed states. Therefore, compared to the state of the student, e.g., training/validation error, prediction, and numbers of iteration,  $\nabla \phi^0$  provides more information to promote deep interaction between DLN, the teacher, and the student.
3.4 Teacher learning
To retain the teaching experience, we utilize an LSTM to play the role of teacher. The LSTM utilizes a long-term memory unit, called the cell, to allow itself to maintain effective and long-term dependencies to make predictions, both in the current and future [9]. We employ the coordinate-wise manner [1] to process each value of the input information independently. Different behavior on each coordinate is achieved by using separate activations for each input value. This property leads the student to converge to a good solution.
To evaluate the teacher model, we perform another student learning (e.g.,  $S_{\theta}^{N} \rightarrow S_{\theta}^{2N}$ ) with the new DLN (e.g.,  $L_{\phi}^{1}$ ). The RMD is also utilized to calculate the gradient of validation error  $e_{val}(S_{\theta}^{2N})$  with respect to the teacher model parameters  $\varphi^0$ . We represent the computation using red lines in Figure 2. To obtain  $\nabla \varphi^0$ , we loop the SGD process backward from  $2N$  to  $N + 1$  with the updated DLN  $L_{\phi}^{1}$  according to Eq. (4), (5) and (6). At each step  $i = \{2N - 1,\dots ,N + 1\}$ , the gradient  $\nabla \varphi^0$  is updated as:
$$
\nabla \varphi^ {0} = \nabla \varphi^ {0} - \eta \gamma w \frac {\partial^ {3} L _ {\phi} ^ {1} \left(S _ {\theta} ^ {i - 1} \left(x _ {t r a i n}\right) , y\right)}{\partial \theta^ {i - 1} \partial \phi^ {1} \partial \varphi^ {0}} \nabla \theta^ {i}. \tag {8}
$$
The process of L2T-DLN is summarized in Algorithm 1.
Since our L2T-DLN is updated asynchronously, we can only access partial second-order information at each training stage. For example, given a quadratic objective function, while fixing one part of L2T-DLN, the problem is strongly convex with respect to the other part, but the entire problem is nonconvex. Even if the iterates converge for each part to the corresponding minimum points, the stationary point could still be a saddle point for the overall objective function [26]. Therefore, the analysis of how L2T-DLN exploits the negative curvature is necessary.
The Alternating Gradient Descent (AGD) [4] algorithm only updates partial variables of vector, e.g.,  $v = \{\theta, \varphi + \phi\}$ , which belongs to a subset of the feasible set. From the mean value theorem, we can
Algorithm 1 Obtaining the optimal student, DLN and teacher in L2T-DLN.
1: Hyperparameter: length of the student training  $N$ , total number of iterations for DLN learning  $M$ , total number of iterations for teacher learning  $K$ .  
2: Input: Random initialization parameter  $\theta^0, \phi^0$ , and  $\varphi^0$ .  
3:  $m = 0$   
4: for  $k = (1, \dots, K)$  do  
5: for  $i = ((m + k)N + 1, \dots, (m + k + 1)N)$  do ▷ A student learning stage  
6: Conduct student model training step via Eq. (2) ( $S_{\theta}^{i-1} \to S_{\theta}^{i}$ ).  
7: end for  
8:  $\nabla \phi^m = 0$ , compute  $\nabla \theta^{(m+k+1)N}$  via Eq. (4).  
9: for  $i = ((m + k + 1)N, \dots, (m + k)N + 1)$  do ▷ Reversely calculating the gradient  $\nabla \phi$   
10: Update  $\nabla \phi^m$  via Eq. (6).  
11: Compute  $\nabla \theta^i$  via Eq. (5).  
12: end for  
13: update  $\phi^m \to \phi^{m+1}$  via Eq. (3).  
14:  $m = m + 1$  ▷ Updating  $m$   
15: for  $i = ((m + k)N + 1, \dots, (m + k + 1)N)$  do ▷ A student learning stage  
16: Conduct student model training step via Eq. (2) ( $S_{\theta}^{i-1} \to S_{\theta}^{i}$ ).  
17: end for  
18:  $\nabla \varphi^k = 0$ ,  $\nabla \phi^m = 0$ , compute  $\nabla \theta^{(m+k+1)N}$  via Eq. (4)  
19: for  $i = ((m + k + 1)N, \dots, (m + k)N + 1)$  do ▷ Reversely calculating the gradient  $\nabla \varphi$   
20: update  $\nabla \phi^m$  via Eq. (6).  
21: Update  $\nabla \varphi^k$  via Eq. (8).  
22: Compute  $\nabla \theta^i$  via Eq. (5).  
23: end for  
24: Update  $\varphi^k \to \varphi^{k+1}$  using  $\nabla \varphi^k$  via gradient based optimization algorithm.  
25: end for  
26: output:  $\theta^{(M+K)N}, \phi^M, \varphi^K$
express the AGD rule of updating variables by assuming  $v^{(0)} = 0$  as follows, with  $e$  denoting the validation error.
$$
\begin{array}{l} v ^ {k + 1} = v ^ {k} - \eta \binom {\bigtriangledown_ {1} e \left(v _ {1} ^ {2 k N}, v _ {2} ^ {k}\right)} {\bigtriangledown_ {2} e \left(v _ {1} ^ {2 (k + 1) N}, v _ {2} ^ {k}\right)} \tag {9} \\ = v ^ {k} - \eta \int_ {0} ^ {1} \mathcal {H} _ {l} ^ {k} d v ^ {2 (k + 1) N} - \eta \int_ {0} ^ {1} \mathcal {H} _ {u} ^ {k} d v ^ {k}, \\ \end{array}
$$
where  $\mathcal{H}_l^k\triangleq \left[ \begin{array}{cc}0 & 0\\ \nabla_{21}^2 e(v_1^{2(k + 1)N},v_2^k) & 0 \end{array} \right]$  and  $\mathcal{H}_u^k\triangleq \left[ \begin{array}{cc}\bigtriangledown_{11}^2 e(v_1^{2kN},v_2^{k}) & \bigtriangledown_{12}^2 e(v_1^{2kN},v_2^k)\\ 0 & \bigtriangledown_{22}^2 e(\theta v_1^{2(k + 1)N},\theta v_2^k) \end{array} \right].$
The right-hand side of Eq. (9) not only contains the second order information of the previous point, i.e.,  $[v_1^{2kN}, v_2^k]$ , but also the one of the most recently updated point, i.e.,  $[v_1^{2(k+1)N}, v_2^k]$ .
Different from traditional AGD, the dynamic system in L2T-DLN takes the first-order information to update the student and the second-order information to update the teacher. Specifically,  $\nabla_1 e(v_1^{2kN}, v_2^k) = \nabla e(v_1^{2kN}, v_2^k)$  and  $\nabla_2 e(v_1^{2(k+1)N}, v_2^k) = \nabla^2 e(v_1^{2(k+1)N}, v_2^k)$ . These represent the main challenges in understanding the behavior of the sequence generated by the AGD algorithm.
Although the higher-order information is divided into two parts, we can still characterize the recursion of the iterates around strict saddle points  $v^{*}$ . We can also split  $\mathcal{H}$  as two parts, which are
$$
\mathcal {H} _ {u} = \left[ \begin{array}{c c} \nabla_ {1 1} ^ {2} e (v ^ {*}) & \nabla_ {1 2} ^ {2} e (v ^ {*}) \\ 0 & \nabla_ {2 2} ^ {2} e (v ^ {*}) \end{array} \right], \mathcal {H} _ {l} = \left[ \begin{array}{c c} 0 & 0 \\ \nabla_ {2 1} ^ {2} e (v ^ {*}) & 0 \end{array} \right], \tag {10}
$$
and obviously, we have  $\mathcal{H} = \mathcal{H}_u + \mathcal{H}_l$ .
Then recursion Eq. (9) can be written as
$$
v ^ {2 (k + 1) N} + \eta \mathcal {H} _ {l} v ^ {2 (k + 1) N} = x ^ {k} - \eta \mathcal {H} _ {u} v ^ {k} - \eta \triangle_ {u} ^ {k} v ^ {k} - \eta \triangle_ {l} ^ {k} v ^ {2 (k + 1) N}, \tag {11}
$$
where  $\triangle_u^k\triangleq \int_0^1 (\mathcal{H}_u^k (v) - \mathcal{H}_u)dv,\triangle_l^k\triangleq \int_0^1 (\mathcal{H}_l^k (v) - \mathcal{H}_l)dv.$  However, it is still unclear from Eq. (11) how the iteration evolves around the strict saddle point. To highlight ideas, let us define
$$
M \triangleq I + \eta \mathcal {H} _ {l}, G \triangleq I - \eta \mathcal {H} _ {u}. \tag {12}
$$
It can be observed that  $M$  is a lower triangular matrix where the diagonal entries are all 1s; therefore it is invertible. After taking the inverse of matrix  $M$  on both sides of Eq. (11), we can obtain
$$
v ^ {k + 1} = M ^ {- 1} G v ^ {k} - \eta M ^ {- 1} \triangle_ {u} ^ {k} v ^ {k} - \eta M ^ {- 1} \triangle_ {l} ^ {k} v ^ {2 (k + 1) N}. \tag {13}
$$
Our goal of analyzing the recursion of  $v^k$  becomes to find the maximum eigenvalue of  $M^{-1}G$ . With the help of the matrix perturbation theory, we can quantify the difference between the eigenvalues of matrix  $\mathcal{H}$  that contains the negative curvature and matrix  $M^{-1}G$  that we are interested in analyzing. With the gradient Lipschitz constants  $\{\tilde{C}_k\}$ , we set  $L_{max} \triangleq \max \{C_k, \tilde{C}_k, \forall k\} \leq C$  and give the following conclusion.
Conclusion 1. Let  $\mathcal{H} \triangleq \bigtriangledown^2 e(x)$  denote the Hessian matrix at an  $\epsilon$ -second-order stationary solution (SS2)  $v^*$  where  $\lambda_{min}(\mathcal{H}) \leq -\gamma$  and  $\gamma > 0$ . We have
$$
\lambda_ {m a x} \left(M ^ {- 1} G\right) > 1 + \frac {\eta \gamma}{1 + C / C _ {m a x}}. \tag {14}
$$
The proof of Conclusion 1 contains the following steps: