# Improved Gradient Descent Iterations for Solving Systems of Nonlinear Equations
## 1 Introduction, Preliminaries, and Motivation
### Overview of Methods for Solving SNE  
Most popular iterations for solving (1) use appropriate approximations \(B_{k}\) of the Jacobian matrix \(F^{\prime}(\mathbf{x}_{k})\). These iterations are of the form \(\mathbf{x}_{k+1}=\mathbf{x}_{k}+t_{k}\mathbf{d}_{k}\), where \(t_{k}\) is the steplength, and \(\mathbf{d}_{k}\) is the search direction obtained as a solution to the SNE  
\[B_{k}\mathbf{d}_{k}+F(\mathbf{x}_{k})=0. \tag{5}\]  
For simplicity, we will use notations  
\[F_{k}:=F(\mathbf{x}_{k}),\ \ \ \mathbf{y}_{k}:=F_{k+1}-F_{k},\ \ \ \mathbf{s}_{k}:= \mathbf{x}_{k+1}-\mathbf{x}_{k}. \tag{6}\]  
The BFGS approximations are defined on the basis of the secant equation \(B_{k+1}\mathbf{s}_{k}=\mathbf{y}_{k}\). The BFGS updates  
\[B_{k+1}=B_{k}-\frac{B_{k}\mathbf{s}_{k}\mathbf{s}_{k}^{\mathrm{T}}B_{k}}{ \mathbf{s}_{k}^{\mathrm{T}}B_{k}\mathbf{s}_{k}}+\frac{\mathbf{y}_{k}\mathbf{ y}_{k}^{\mathrm{T}}}{\mathbf{y}_{k}^{\mathrm{T}}\mathbf{s}_{k}}\]  
with an initial approximation \(B_{0}\in\mathbb{R}^{n\times n}\) were considered in [1]. Further on, we list and briefly describe relevant minimization methods that exploit the equivalence between (1) and (2). The efficiency and applicability of these algorithms highly motivated the research presented in this paper. The number of methods that we mention below confirms the applicability of this direction in solving SNE. In addition, there is an evident need to develop and constantly upgrade the performances of optimization methods for solving (1). There are numerous methods which can be used to solve the problem (1). Many of them are developed in [2; 3; 4; 5; 6; 7]. Derivative-free methods for solving SNE were considered in [8; 9; 10]. These methods are proposed as appropriate adaptations of double direction and steplength methods in nonlinear optimization and the approximation of the Jacobian with a diagonal matrix whose entries are defined utilizing of an appropriate parameter. One approach based on various modifications of the Broyden method was proposed in [11; 12]. A derivative-free conjugate gradient (CG) iterations for solving SNE were proposed in [13]. A descent Dai-Liao CG method for solving large-scale SNE was proposed in [14]. Novel hybrid and modified CG methods for finding a solution to SNE were originated in [15; 16], respectively. An extension of a modified three-term CG method that can be applied for solving equations with convex constraints was presented in [17]. A diagonal quasi-Newton approach for solving large-scale nonlinear systems was considered in [18; 19]. A quasi-Newton method, defined based on an improved diagonal Jacobian approximation, for solving nonlinear systems was proposed in [20]. [PERSON] et al. in [21] proposed a double direction method for solving nonlinear equations. The first direction is the steepest descent direction, while the second direction is the proposed CG direction. Two derivative-free modifications of the CG-based method for solving large-scale systems \(F(\mathbf{x})=0\) were presented in [22]. These methods are applicable in the case when the Jacobian of \(F(\mathbf{x})\) is not accessible. An efficient approximation to the Jacobian matrix with a computational effort similar to that of matrix-free settings was proposed in [23]. Such efficiency was achieved when a diagonal matrix generates a Jacobian approximation. This method possesses low memory space requirements because the method is defined without computing exact gradient and Jacobian. [PERSON] et al. in [24] followed the approach based on the approximation of the Jacobian inverse by a nonsingular diagonal matrix. A fast and computationally efficient method concerning memory requirements was proposed in [25], and it uses an approximation of the Jacobian by an adequate diagonal matrix. A two-step generalized scheme of the Jacobian approximation was given in [26]. Further on, an iterative scheme which is based on a modification of the Dai-Liao CG method, classical Newton iterates, and the standard secant equation was suggested in [27].