# Improved Gradient Descent Iterations for Solving Systems of Nonlinear Equations
## 3 Convergence Analysis  
The level set is defined as  
\[\Omega=\{\mathbf{x}\in\mathbb{R}^{n}|\ \ \|F(\mathbf{x})\|\leq\|F(\mathbf{x}_{0} )\|\}, \tag{49}\]  
where \(\mathbf{x}_{0}\in\mathbb{R}^{n}\) is an initial approximation. Therewith, the next assumptions are needed:  
\((A_{1})\) The level set \(\Omega\) defined in (49) is bounded below. \((A_{2})\) Lipschitz continuity holds for the vector function \(F\), i.e., \(\|F(\mathbf{x})-F(\mathbf{y})\|\leq r\|\mathbf{x}-\mathbf{y}\|\) for all \(\mathbf{x},\mathbf{y}\in\mathbb{R}^{n}\) and \(r>0\). \((A_{3})\) The Jacobian \(F^{\prime}(x)\) is bounded. Lemma 1: _Suppose the assumption \((A_{2})\) holds. If the sequence \(\{\mathbf{x}_{k}\}\) is obtained by the \(IGDN\) (29) iterations, then_  
\[\mathbf{y}_{k}^{\mathrm{T}}\mathbf{s}_{k}\leq r\|\mathbf{s}_{k}\|^{2},\quad r >0. \tag{50}\]  
Proof: Obviously,  
\[\mathbf{y}_{k}^{\mathrm{T}}\mathbf{s}_{k}=\mathbf{s}_{k}^{\mathrm{T}}\mathbf{y }_{k}=\mathbf{s}_{k}^{\mathrm{T}}(F_{k+1}-F_{k}). \tag{51}\]  
Therefore, assuming \((A_{2})\), it is possible to derive  
\[\mathbf{y}_{k}^{\mathrm{T}}\mathbf{s}_{k}\leq\|\mathbf{s}_{k}\|\|F_{k+1}-F_{k} \|\leq r\|\mathbf{s}_{k}\|^{2}. \tag{52}\]Previous estimation confirms that (50) is satisfied with \(r\) defined by the Lipschitz condition in \((A_{2})\). For the convergence results of the remaining algorithms, we need to prove the finiteness of \(\gamma_{k}\), \(\mathbf{d}_{k}\), and the remaining results follow trivially. **Lemma 2**. : _The \(\gamma_{k}\) generated by \(IGDN\) (29) is bounded by the Lipschitz constant \(r\)._  
Proof. : Clearly, the complemental step size \(\gamma_{k}\) defined by (29) satisfies  
\[\gamma_{k+1}=\frac{\mathbf{y}_{k}^{\mathrm{T}}\mathbf{s}_{k}}{\|\mathbf{s}_{ k}\|^{2}}\leq\frac{r\|\mathbf{s}_{k}\|^{2}}{\|\mathbf{s}_{k}\|^{2}}=r, \tag{53}\]  
which leads to the conclusion \(\gamma_{k}\leq r\). **Lemma 3**. : _The additional step size \(\gamma_{k}\) generated by \(IGDN\) (34) is bounded as follows:_  
\[\gamma_{k}\leq\frac{1}{\prod\limits_{i=0}^{k-1}t_{i}} \tag{54}\]  
Proof. : The updating rule (34) satisfies \(\gamma_{k+1}\leq\frac{\gamma_{k}}{t_{k}}\). Continuing in the same way, one concludes  
\[\gamma_{k+1}\leq\frac{\gamma_{0}}{\prod\limits_{i=0}^{k}t_{i}}.\]  
The proof can be finished using \(\gamma_{0}=1\). **Lemma 4**. : _The additional scaling parameter \(\gamma_{k}\) generated by (42) is bounded as follows:_  
\[\gamma_{k}\leq\frac{1}{\prod\limits_{i=0}^{k-1}(t_{i}+\gamma_{i}l_{i})}. \tag{55}\]  
**Lemma 5**. : _The directions \(\mathbf{d}_{k}\) used in \(IGDN\) (29) and \(IGDN\) (34) algorithms are descent directions._  
Proof. : Since  
\[\mathbf{d}_{k}=-\gamma_{k}^{-1}F_{k}, \tag{56}\]  
an application of the scalar product of both sides in (56) with \(F_{k}^{\mathrm{T}}\) in conjunction with Lemma 2 leads to the following conclusion for \(IGDN\) (29) iterations:  
\[F_{k}^{\mathrm{T}}\mathbf{d}_{k}=-\gamma_{k}^{-1}F_{k}^{\mathrm{T}}F_{k}\leq- \frac{1}{r}\|F_{k}\|^{2}<0. \tag{57}\]  
With Lemma 3, it can be concluded that \(IGDN\) (34) iterations imply the following:  
\[\mathbf{y}_{k}^{\mathrm{T}}\mathbf{s}_{k}\leq r\|\mathbf{s}_{k}\|^{2},\quad r >0. \tag{50}\]0  
The proof is complete. **Lemma 6**. : _The direction \(\mathbf{d}_{k}\) used in \(ADSSN\) algorithms is a descent direction._  
Proof. : Since  
\[\mathbf{y}_{k}^{\mathrm{T}}\mathbf{s}_{k}\leq r\|\mathbf{s}_{k}\|^{2},\quad r >0. \tag{50}\]1after using the scalar product of both sides in (59) with \(F_{k}^{\rm T}\) and taking into account Lemma 4, we obtain  
\[\mathbf{y}_{k}^{\mathrm{T}}\mathbf{s}_{k}\leq r\|\mathbf{s}_{k}\|^{2},\quad r >0. \tag{50}\]2 \[\mathbf{y}_{k}^{\mathrm{T}}\mathbf{s}_{k}\leq r\|\mathbf{s}_{k}\|^{2},\quad r >0. \tag{50}\]3 \[\mathbf{y}_{k}^{\mathrm{T}}\mathbf{s}_{k}\leq r\|\mathbf{s}_{k}\|^{2},\quad r >0. \tag{50}\]4 \[\mathbf{y}_{k}^{\mathrm{T}}\mathbf{s}_{k}\leq r\|\mathbf{s}_{k}\|^{2},\quad r >0. \tag{50}\]5  
The proof is complete. Theorem 3.: _The vector \(F_{k+1}\) generated by \(1 GDN\) (34) is a descent direction._  
Proof.