# Improved Gradient Descent Iterations for Solving Systems of Nonlinear Equations
## 2 Multiple Step-Size Methods for Solving SNE
### Simplified ADSSN  
Applying the relation  
\[t_{k}+l_{k}=1 \tag{45}\]  
between the step size parameters \(t_{k}\) and \(l_{k}\) in the \(ADSSN\) iterative rule (41), the \(ADSSN\) iteration is transformed to  
\[x_{k+1}=x_{k}-\Big{[}t_{k}(\gamma_{k}^{-1}-1)+1\Big{]}F(\textbf{x}_{k}). \tag{46}\]  
The convex combination (45) of step size parameters \(t_{k}\) and \(l_{k}\) that appear in the \(ADSSN\) scheme (41) was originally proposed in [62] and applied in an iterative method for solving the unconstrained optimization problem (7). The assumption (45) represents a trade-off between the steplength parameters \(t_{k}\) and \(l_{k}\). In [62], it was shown that the induced single step size method shows better performance characteristics in general. The constraint (45) initiates the reduction of the two-parameter \(ADSSN\) rule into a single step size transformed \(ADSSN\) (shortly \(TADSSN\)) iterative method (46).  
We can spot that the \(TADSSN\) method is a modified version of \(IGDN\) iterations, based on the replacement of the product \(t_{k}\gamma_{k}^{-1}\), from the classical \(IGDN\) iteration, by the multiplying factor \(t_{k}(\gamma_{k}^{-1}-1)+1\).  
The substitution \(\phi_{k}:=t_{k}(\gamma_{k}^{-1}-1)+1\) will be used to simplify the presentation. Here, the accelerated parameter value \(\gamma_{k+1}\) is calculated by (29).  
Corollary 3: _Iterations (46) satisfy_  
\[F_{k+1}=F_{k}-\gamma_{k+1}\phi_{k}F_{k}. \tag{47}\]  
Proof: It follows from (27) and (46).  
In view of (47), it is possible to conclude  
\[\gamma_{k+1}=-\frac{F_{k}^{\mathrm{T}}\mathbf{y_{k}}}{\phi_{k}F_{k}^{\mathrm{T}}F _{k}}.\]  
Corollary 4 gives some useful restrictions on this rule.  
Corollary 4: _If the condition \(\gamma_{k+1}\leq\frac{\gamma_{k}}{t_{k}+\gamma_{k}(1-t_{k})}\) holds, then the iterations (41) satisfy \(F_{k+1}\leq F_{k}\)._  
Proof: It follows from Theorem 1 and \(l_{k}=1-t_{k}\).  
In view of Corollary 4, it is reasonable to define the following update for \(\gamma_{k+1}\) in the \(TADSSN\) method:  
\[\gamma_{k+1}=\left\{\begin{array}{ll}-\frac{F_{k}^{\mathrm{T}}\mathbf{y_{k} }}{\phi_{k}F_{k}^{\mathrm{T}}F_{k}},&\frac{F_{k}^{\mathrm{T}}\mathbf{y_{k}}}{ \phi_{k}F_{k}^{\mathrm{T}}F_{k}}\leq 0\\ \frac{\gamma_{k}}{t_{k}+\gamma_{k}(1-t_{k})},&\frac{F_{k}^{\mathrm{T}}\mathbf{y _{k}}}{\phi_{k}F_{k}^{\mathrm{T}}F_{k}}>0.\end{array}\right. \tag{48}\]  
Then, \(x_{k+2}\) is equal to  
\[\mathbf{x}_{k+2}=\mathbf{x}_{k+1}-\phi_{k+1}F_{k+1}.\]  
```
0: Chosen \(F(\mathbf{x})\), \(\varepsilon>0\) and \(\mathbf{x}_{0}\in\mathbb{R}^{n}\).
1: For \(k=0\) chose \(\gamma_{0}=1\) and \(F(\mathbf{x}_{0})\).
2: Check the termination criterion; if \(\|F(\mathbf{x}_{k})\|\leq\varepsilon\) holds then stop; else, go to Step 3:.
3: (Line search) Apply (31) and generate the step size value \(t_{k}\).
4: Compute \(l_{k}=1-t_{k}\).
5: Compute \(\mathbf{x}_{k+1}\) using (46).
6: Determine the scaling factor \(\gamma_{k+1}\) using (48).
7:\(k:=k+1\).
8: Return to Step 2.
9: Output: \(\mathbf{x}_{k+1}\), \(F(\mathbf{x}_{k+1})\).
```  
**Algorithm 4** The ADSSN iteration based on (46) and (48).