## 4 Convergence Analysis  
In this section we discuss the convergence proof of the proposed [PERSON] accelerated Symmetric Rank-1 (L-SR1-N) algorithm in its limited memory form. As mentioned earlier, the [PERSON]'s acceleration approximates the quadratic model at \(\mathbf{w}_{k}+\mu_{k}\mathbf{v}_{k}\) instead of the iterate at \(\mathbf{w}_{k}\). For ease of representation, we write \(\mathbf{w}_{k}+\mu_{k}\mathbf{v}_{k}=\widehat{\mathbf{w}}_{k}\ \forall\ k=1,2,...,k_{max}\in\mathbb{N}\). In the limited memory scheme, the Hessian matrix can be implicitly constructed using the recent \(m_{L}\) number of curvature information pairs \(\{\mathbf{s}_{i},\mathbf{y}_{i}\}_{i=k-1}^{k-m_{L}-1}\). At a given iteration \(k\), we define matrices \(\mathbf{S}_{k}\) and \(\mathbf{Y}_{k}\) of dimensions \(d\times m_{L}\) as  
\[\mathbf{S}_{k}=[\mathbf{s}_{k-1},\mathbf{s}_{k-2},...,\mathbf{s}_{k-m_{L}-1}] \quad\text{and}\quad\mathbf{Y}_{k}=[\mathbf{y}_{k-1},\mathbf{y}_{k-2},..., \mathbf{y}_{k-m_{L}-1}], \tag{35}\]where the curvature pairs \(\{\mathbf{s}_{i},\mathbf{y}_{i}\}_{i=k-1}^{k-m_{L}-1}\) are each vectors of dimensions \(d\times 1\). The Hessian approximation in (30) can be expressed in its compact representation form [30] as  
\[\mathbf{B}_{k}=\mathbf{B}_{0}+(\mathbf{Y}_{k}-\mathbf{B}_{0}\mathbf{S}_{k})( \mathbf{L}_{k}+\mathbf{D}_{k}+\mathbf{L}_{k}^{T}-\mathbf{S}_{k}^{T}\mathbf{B}_ {0}\mathbf{S}_{k})^{-1}(\mathbf{Y}_{k}-\mathbf{B}_{0}\mathbf{S}_{k}), \tag{36}\]  
where \(\mathbf{B}_{0}\) is the initial \(d\times d\) Hessian matrix, \(\mathbf{L}_{k}\) is a \(m_{L}\times m_{L}\) lower triangular matrix and \(\mathbf{D}_{k}\) is a \(m_{L}\times m_{L}\) diagonal matrix as given below,  
\[\mathbf{B}_{0}=\gamma_{k}\mathbf{I},\]  
\[(\mathbf{L}_{k})_{i,j}= \begin{cases}\mathbf{s}_{i}^{T}\mathbf{y}_{j}&\text{ if }i>j,\\ 0&\text{otherwise},\end{cases} \]  
\[\mathbf{D}_{k}=\text{diag }[\mathbf{S}_{k}^{T}\mathbf{Y}_{k}]. \tag{37}\]  
Let \(\mathbf{\Omega}\) be the level set such that \(\mathbf{\Omega}=\{\mathbf{w}\in\mathbb{R}^{d}:E(\mathbf{w})\leq E(\mathbf{w}_ {\mathbf{0}})\}\) and \(\{\mathbf{s}_{k}\}\ \forall\ k=1,2,...,k_{max}\in\mathbb{N}\), denote the sequence generated by the explicit trust-region algorithm where \(\Delta_{k}\) be the trust-region radius of the successful update step. We choose \(\gamma_{k}=0\). Since the curvature information pairs \((\mathbf{s}_{k},\mathbf{y}_{k})\) given by (33) are stored in \(\mathbf{S}_{k}\) and \(\mathbf{Y}_{k}\) only if they satisfy the condition in (31), the matrix \(\mathbf{M}_{k}=(\mathbf{L}_{k}+\mathbf{D}_{k}+\mathbf{L}_{k}^{T}-\mathbf{S}_{ k}^{T}\mathbf{B}_{0}\mathbf{S}_{k})\) is invertible and positive semi-definite. **Assumption 1**. : _The sequence of iterates \(\mathbf{w}_{k}\) and \(\hat{\mathbf{w}}_{k}\ \forall\ k=1,2,...,k_{max}\in\mathbb{N}\) remains in the closed and bounded set \(\mathbf{\Omega}\) on which the objective function is twice continuously differentiable and has Lipschitz continuous gradient, i.e., there exists a constant \(L>0\) such that_  
\[||\
abla E(\mathbf{w}_{k+1})-\
abla E(\hat{\mathbf{w}}_{k})||\leq L||\mathbf{ w}_{k+1}-\hat{\mathbf{w}}_{k}||\ \ \ \forall\ \mathbf{w}_{k+1},\ \hat{\mathbf{w}}_{k}\in\mathbb{R}^{d}. \tag{38}\]  
**Assumption 2**. : _The Hessian matrix is bounded and well-defined, i.e., there exists constants \(\rho\) and \(M\), such that_  
\[\rho\leq||\mathbf{B}_{k}||\leq M\ \ \ \forall\ \ k=1,2,...,k_{max}\in\mathbb{N}. \tag{39}\]  
_and for each iteration \(k\)_  
\[|\mathbf{s}_{k}^{T}(\mathbf{y}_{k}-\mathbf{B}_{k}\mathbf{s}_{k})|\geq\rho\ ||\mathbf{s}_{k}||\ || \mathbf{y}_{k}-\mathbf{B}_{k}\mathbf{s}_{k}||. \tag{40}\]  
**Assumption 3**. : _Let \(\mathbf{B}_{k}\) be any \(n\times n\) symmetric matrix and \(\mathbf{s}_{k}\) be an optimal solution to the trust region subproblem,_  
\[\min_{\mathbf{d}}\ \ m_{k}(\mathbf{d})=E(\hat{\mathbf{w}}_{k})+\mathbf{d}^{T} \
abla E(\hat{\mathbf{w}}_{k})+\frac{1}{2}\mathbf{d}^{T}\mathbf{B}_{k}\mathbf{ d}, \tag{41}\]  
_where \(\hat{\mathbf{w}}_{k}+\mathbf{d}\) lies in the trust region.