## 3 Proposed Method  
Second order quasi-Newton (QN) methods build an approximation of a quadratic model recursively using the curvature information along a generated trajectory. In this section, we first show that the Nesterov's acceleration when applied to QN satisfies the secant condition and then show the derivation of the proposed Nesterov Accelerated Symmetric Rank-1 Quasi-Newton Method. _Nesterov Accelerated Symmetric Rank-1 Quasi-Newton Method_  
Suppose that \(E:R^{d}\to R\) is continuosly differentiable and that \(\mathbf{d}\in\mathbb{R}^{d}\), then from Taylor series, the quadratic model of the objective function at an iterate \(\mathbf{w}_{k}\) is given as  
\[E(\mathbf{w}_{k}+\mathbf{d})\approx m_{k}(\mathbf{d})\approx E(\mathbf{w}_{k} )+\
abla E(\mathbf{w}_{k})^{T}\mathbf{d}+\frac{1}{2}\;\mathbf{d}^{T}\
abla^{2 }E(\mathbf{w}_{k})\mathbf{d}. \tag{13}\]  
In order to find the minimizer \(\mathbf{d}_{k}\), we equate \(\
abla\mathbf{m}_{k}(\mathbf{d})=0\) and thus have  
\[\mathbf{d}_{k}=-\
abla^{2}E(\mathbf{w}_{k})^{-1}\
abla E(\mathbf{w}_{k})=- \mathbf{B}_{k}^{-1}\
abla E(\mathbf{w}_{k})\;. \tag{14}\]  
The new iterate \(\mathbf{w}_{k+1}\) is given as,  
\[\mathbf{w}_{k+1}=\mathbf{w}_{k}-\alpha_{k}\mathbf{B}_{k}^{-1}\
abla E( \mathbf{w}_{k}), \tag{15}\]  
and the quadratic model at the new iterate is given as  
\[E(\mathbf{w}_{k+1}+\mathbf{d})\approx m_{k+1}(\mathbf{d})\approx E(\mathbf{w} _{k+1})+\
abla E(\mathbf{w}_{k+1})^{T}\mathbf{d}+\frac{1}{2}\;\mathbf{d}^{T} \mathbf{B}_{k+1}\mathbf{d}\;, \tag{16}\]  
where \(\alpha_{k}\) is the step length and \(\mathbf{B}_{k}^{-1}=\mathbf{H}_{k}\) and its consecutive updates \(\mathbf{B}_{k+1}^{-1}=\mathbf{H}_{k+1}\) are symmetric positive definite matrices satisfying the secant condition. The Nesterov's acceleration approximates the quadratic model at \(\mathbf{w}_{k}+\mu_{k}\mathbf{v}_{k}\) instead of the iterate at \(\mathbf{w}_{k}\). Here \(\mathbf{v}_{k}=\mathbf{w}_{k}-\mathbf{w}_{k-1}\) and \(\mu_{k}\) is the momentum coefficient in the range \((0,1)\). Thus we have the new iterate \(\mathbf{w}_{k+1}\) given as,  
\[\mathbf{w}_{k+1}=\mathbf{w}_{k}+\mu_{k}\mathbf{v}_{k}-\alpha_{k}\mathbf{B}_{k }^{-1}\
abla E(\mathbf{w}_{k}+\mu_{k}\mathbf{v}_{k}), \tag{17}\]  
\[=\mathbf{w}_{k}+\mu_{k}\mathbf{v}_{k}+\alpha_{k}\mathbf{d}_{k}. \tag{18}\]  
In order to show that the Nesterov accelerated updates also satisfy the secant condition, we require that the gradient of \(m_{k+1}\) should match the gradient of the objective function at the last two iterates \((\mathbf{w}_{k}+\mu_{k}\mathbf{v}_{k})\) and \(\mathbf{w}_{k+1}\). In other words, we impose the following two requirements on \(\mathbf{B}_{k+1}\),  
\[\
abla m_{k+1}|_{\mathbf{d}=0}=\
abla E(\mathbf{w}_{k+1}+\mathbf{d})|_{ \mathbf{d}=0}=\
abla E(\mathbf{w}_{k+1}), \tag{19}\]  
\[\
abla m_{k+1}|_{\mathbf{d}=-\alpha_{k}\mathbf{d}_{k}}=\
abla E(\mathbf{w}_{k +1}+\mathbf{d})|_{\mathbf{d}=-\alpha_{k}\mathbf{d}_{k}}=\
abla E(\mathbf{w}_{ k+1}-\alpha_{k}\mathbf{d}_{k})=\
abla E(\mathbf{w}_{k}+\mu_{k}\mathbf{v}_{k}). \tag{20}\]  
From (16),  
\[\
abla m_{k+1}(\mathbf{d})=\
abla E(\mathbf{w}_{k+1})+\mathbf{B}_{k+1} \mathbf{d}. \tag{21}\]  
Substituting \(\mathbf{d}=0\) in (21), the condition in (19) is satisfied.