# A Novel Value for the Parameter in the Dai-Liao-Type Conjugate Gradient Method

Authors: Branislav Ivanov; Predrag S. Stanimirović; Bilall I. Shaini; Hijaz Ahmad; Miao-Kun Wang
Journal: Journal of Function Spaces
 (2021)

---

## Abstract

A new rule for calculating the parametert involved in each iteration of the MHSDL (Dai-Liao) conjugate gradient (CG) method is presented. The new value of the parameter initiates a more efficient and robust variant of the Dai-Liao algorithm. Under proper conditions, theoretical analysis reveals that the proposed method in conjunction with backtracking line search is of global convergence. Numerical experiments are also presented, which confirm the influence of the new value of the parameter t on the behavior of the underlying CG optimization method. Numerical comparisons and the analysis of obtained results considering Dolan and Moré’s performance profile show better performances of the novel method with respect to all three analyzed characteristics: number of iterative steps, number of function evaluations, and CPU time.


---

## Body

## 1. Introduction and Background Results

The topic of our research is solving the unconstrained nonlinear optimization problem(1)minfx,x∈ℝn,where the function f:ℝn⟶ℝ is continuously differentiable and bounded below. Following the standard notation, gk=∇fxk denotes the gradient, sk−1=xk−xk−1 and yk−1=gk−gk−1. Using an extended conjugacy condition(2)dkTyk−1=−tgkTsk−1,t>0,Dai and Liao in [1] proposed the conjugate gradient (CG) method(3)xk+1=xk+αkdk,where the step size αk is a positive parameter, xk is an already generated point, xk+1 is a new iterative point, and dk is a suitable search direction. The search directions dk are generated by the conceptual formula(4)dk=−g0,k=0,−gk+βkDLdk−1,k≥1,where the conjugate gradient coefficient βkDL is defined by(5)βkDL=Yt≔gkTyk−1dk−1Tyk−1−tgkTsk−1dk−1Tyk−1,t>0,wherein t>0 is a scalar.Some well-known formulas for definingβk have been created by modifying the conjugate gradient parameter βkDL [2–9]. One of them is denoted as βkMHSDL and defined in [7] by(6)βkMHSDL=Y1t≔gkTyk−1^dk−1Tyk−1−tgkTsk−1dk−1Tyk−1,where t>0 is a scalar as in (5) and yk−1^=gk−∥gk∥/∥gk−1∥gk−1.The family of CG methods for nonlinear optimization has reached great popularity lately, thanks to the various benefits and advantages it possesses. The most important property is based on computationally efficient iterations arising from a simple CG rule. This property initiates the high efficiency of CG methods with respect to analogous methods for nonlinear optimization. Moreover, global convergence is ensured under suitable conditions. Finally, the application of various CG methods in solving image restoration problems has become an important research topic [10, 11].Since the parametert is important for the numerical behavior of Dai-Liao (DL) CG methods [12], one of the most important problems in the implementation of the DL class CG method is to determine a proper value t>0 which will give desirable results. Many scientists have invested a lot of time and effort in the previous period to determine the best definition of the nonnegative parameter t in the DL class CG methods. So far, the research in finding the appropriate value of t has evolved in two directions. One group of methods is aimed at finding an appropriate fixed value for t [1, 2, 6–8], while methods from another group promote appropriate rules for computing values of t in each iteration, which ensure a satisfactory decrease of the objective. In our research, we will pay attention to the second research stream: find the parameter t whose values change through iterations so that the faster convergence is achieved. The value of the parameter t defined in the kth iteration will be denoted by tk≔tk.In order to complete the presentation, we will restate the main principles proposed so far for computingtk. Hager and Zhang in [13, 14] proposed the DL CG method (5), known as CG-DESCENT, where tk≡tk1 is defined by(7)tk≡tk1≔2∥yk−1∥2yk−1Tsk−1.Dai and Kou [15] suggested the conjugate gradient coefficient βkDK of the form(8)βkDK=Yτk+yk−12yk−1Tsk−1−yk−1Tsk−1sk−12=gkTyk−1yk−1Tdk−1−τk+∥yk−1∥2yk−1Tsk−1−yk−1Tsk−1∥sk−1∥2gkTsk−1dk−1Tyk−1,where τk is the scaling parameter arising from the self-scaling memoryless BFGS method. Clearly, the Dai and Kou (DK) method is a member of the DL class CG methods, which is determined by(9)tk≡tk2≔τk+∥yk−1∥2yk−1Tsk−1−yk−1Tsk−1∥sk−1∥2.The results given in [15] confirm that the DK iterations outperform many existing CG methods. Following the development of DL methods, Babaie-Kafaki and Ghanbari [16] defined two new ways to calculate the value of the parameter t in (5), as in the following two formulas:(10)tk≡tk3≔sk−1Tyk−1sk−12+yk−1sk−1,tk≡tk4≔yk−1sk−1.Andrei in [17] proposed the new rule for calculating t in order to define Yt in (5) and defined a new variant of the DL class CG methods, denoted by DLE, with(11)tk≡tk5≔sk−1Tyk−1sk−12.Lotfi and Hosseini in [18] proposed the following rule for determining the parameter tk, using the expression(12)tk≡tk6≔maxtk6∗,υyk−12sk−1Tyk−1,where(13)tk6∗≔1−hkgk−1rsk−1Tgk+gkTyk−1/yk−1Tsk−1hkgk−1rsk−12gkTsk−1+gkTsk−1/sk−1Tyk−1hkgk−1rsk−12,hk=C+max−sk−1Tyk−1sk−12,0gk−1−r,and υ>1/4, C, and r are three positive constants.On the basis of the above overview of the main CG methods and motivated by the strong theoretical properties and computational efficiency of modified Dai-Liao CG methods proposed by many researchers, we suggest a new way of calculating the value of the parametertk. As a consequence, the corresponding CG method of DL type, termed as the Effective Dai-Liao (EDL) method, is proposed and its convergence is proven. Numerical testing and comparison with other known DL variants are presented in order to show the effectiveness of the introduced method. Analysis of generated numerical results exhibits that the proposed EDL method is efficient compared with other DL-type methods.The global organization of sections is described as follows. Introduction, motivation, and a brief overview of the preliminary results are given in Section1. A new rule for calculating the variable parameter tk is proposed in Section 2. An effective algorithm and global convergence of the EDL method initiated by tk are given in the same section. The new EDL method is tested in Section 3 on some unlimited optimization test problems and compared against some known variants of the DL class methods. Finally, concluding remarks are presented in the last concluding section.


## 2. A Modified Dai-Liao Method and Its Convergence

Popularity in defining new rules for calculatingtk is a guarantee that such an approach is effective and still insufficiently explored. The idea for defining a new parameter tk∗ comes from previously described rules for computing tk, particularly from the paper Li and Ruan [19] and from the idea which can be found in the paper Yuan et al. [11]. Further, analyzing the results from [1, 2, 6–8], we conclude that the scalar t was defined by a fixed value of 0.1 in related numerical experiments. Also, numerical experience related to the fixed valued t=1 was reported in [1]. According to this experience, our intention is to define variable values tk inside the interval 0,1.To successfully definetk with values belonging to the interval 0,1, let us start from the definition of the quantity Lk which was used in defining the direction dk in [19]. The parameter Lk was defined by Lk=sk−1Tsk−1/sk−1Tyk−1∗∈0,1,k≥0, where(14)yk−1∗=yk−1+max0,−sk−1Tyk−1∥sk−1∥2+1sk−1.By puttingyk−1∗ into Lk, the following can be obtained:(15)Lk=sk−1Tsk−1sk−1Tyk−1+max0,−sk−1Tyk−1/sk−12+1sk−1=sk−12sk−1Tyk−1+max0,−sk−1Tyk−1/∥sk−1∥2+1sk−12.Further, with certain modifications and substitutions in the equation definingLk, as well as using the function max, which chooses the maximum between the value of the expression dk−1Tgk and 1, we come to a new definition of the parameter tk. As described in advance imposed desired restrictions, the novel parameter tk∗ is defined by(16)tk∗=gk2max1,dk−1Tgk+max0,dk−1Tgk/∥gk∥2+1gk2.It is easy to verify thattk∗ defined by (16) satisfies(17)0<tk∗≤gk21+0+1gk2=gk21+gk2<1.Accordingly,tk∗∈0,1, which was our initial intention. Clearly, greater values of ∥gk∥ lead to values tk∗↗1. Further, since the trend ∥gk∥⟶0 is expectable, we can expect smaller values tk∗↘0 in late iterations. Therefore, tk∗ is suitable for defining corresponding conjugate gradient coefficient Yt or Y1t and further DL CG iterations (4).Consideringt=tk∗ in (6), it is reasonable to propose a novel variant of the Dai-Liao CG parameter βkEDL which is subject to the following rule during the iterative process:(18)βkEDL=Y1tk∗≔gk2−gk/gk−1gkTgk−1dk−1Tyk−1−tk∗gkTsk−1dk−1Tyk−1.Before the main algorithm, it is necessary to define the backtracking line search as one of the most popular and practical methods for computing the step lengthαk in (3). The procedure for the backtracking line search proposed in [20] starts from the initial value α=1 and generates output values which ensure that the goal function decreases in each iteration. Consequently, it is appropriate to use Algorithm 1, restated from [21], in order to determine the primary step size αk.Algorithm 1:
The backtracking line search.Require: Nonlinear objective function fx, search direction dk, previous point xk, and real quantities 0<ω<0.5 and φ∈0,1.1:α=1.2: Whilefxk+αdk>fxk+ωαgkTdk, do α≔αφ.3: Returnαk=α.Algorithm2 describes a computational framework for the EDL method.It is necessary to examine the properties of the EDL method and prove its convergence.Assumption 1.
(1)
The level setM=x∈ℝn∣fx≤fx0, defined upon the initial point x0 of the iterative method (3), is bounded.(2)
The goal functionf is continuous and differentiable in a neighborhood P of M with the Lipschitz continuous gradient g. This assumption implies the existence of a positive constant L>0 satisfying(19)gu−gv≤Lu−v,∀u,v∈P.Assumption1 initiates the existence of positive constants D and γ satisfying(20)u−v≤D,∀u,v∈P,gu≤γ,∀u∈P.The conditions from Assumption1 are assumed. In view of the uniform convexity of f, there is a constant θ>0 that satisfies(21)gu−gvTu−v≥θu−v2,forallu,v∈M,or equivalently,(22)fu≥fv+gvTu−v+θ2u−v2,forallu,v∈M.Algorithm 2:
Effective Dai-Liao (EDL) CG method.Require: An initial point x0 and quantities 0<ε<1, 0<δ<1.1: Assignk=0 and d0=−g0.2: Ifgk≤ε and ∣fxk+1−fxk∣/1+fxk≤δ,STOP;else go to Step 3.3: Calculateαk∈0,1 using Algorithm 1 (backtracking line search).4: Computexk+1=xk+αkdk.5: Calculategk+1, yk=gk+1−gk, sk=xk+1−xk.6: Computetk∗ by (16).7: Calculateβk+1EDL by (18).8: Computedk+1=−gk+1+βk+1EDLdk.9: Letk≔k+1, and go to Step 2.It follows from (21) and (22) that(23)sk−1Tyk−1≥θsk−12,(24)fxk−1−fxk≥−gxkTsk−1+θ2sk−12.By (19) and (23), one concludes(25)θsk−12≤sk−1Tyk−1≤Lsk−12,where the inequality implies θ≤L.The inequality (25) initiates(26)sk−1Tyk−1=αk−1dk−1Tyk−1>0.Taking into accountαk−1>0 and the last inequality, we conclude(27)dk−1Tyk−1>0.Lemma 2.
[22, 23]. Let Assumption 1 be accomplished and the points xk be generated by the method (3)–(4). Then, it holds(28)∑k=0∞∥gk∥4∥dk∥2<+∞.Lemma 3.
Consider the proposed Dai-Liao CG method, including (3), (4), and (18). If the search procedure guarantees (27), for all k≥0, then the next inequality holds(29)gkTdk≤−cgk2,for some 0≤c≤1.Proof.
The inequality (29) will be verified by induction. In the initial situation k=0, one obtains g0Td0=−g02. Since c≤1, obviously (29) is satisfied in the basic case. Suppose that (29) is valid for some k≥1. Taking the inner product of both the left- and right-hand sides in (4) with the vector gkT, the following can be obtained:(30)gkTdk=−gk2+βkEDLgkTdk−1=−gk2+gk2−gk/gk−1gkTgk−1dk−1Tyk−1−tk∗gkTsk−1dk−1Tyk−1gkTdk−1=−gk2+gk2−gk/gk−1gkTgk−1dk−1Tyk−1gkTdk−1−tk∗gkTsk−1dk−1Tyk−1gkTdk−1=−gk2+gk2−gk/gk−1gkTgk−1dk−1Tyk−1gkTdk−1−tk∗αk−1gkTdk−12dk−1Tyk−1.
Using (17) in common with (27) and αk−1>0, we conclude(31)tk∗αk−1gkTdk−12dk−1Tyk−1>0.
Now from (30), (31), and(32)0≤βkMHS=gk2−gk/gk−1gkTgk−1dk−1Tyk−1≤∥gk∥2λ∣gkTdk−1∣,λ≥1,it follows that(33)gkTdk≤−gk2+gk2−gk/gk−1gkTgk−1dk−1Tyk−1gkTdk−1≤−gk2+gk2λgkTdk−1gkTdk−1=−1−1λgk2.
In view ofλ≥1, the inequality (29) is satisfied for c=1−1/λ in (33) and arbitrary k≥0.The global convergence of the proposed EDL method is confirmed by Theorem4.Theorem 4.
Let Assumption1 be true and f be uniformly convex. Then, the sequence xk generated by (3), (4), and (18) fulfills(34)liminfk→∞gk=0.Proof.
Suppose the opposite, i.e., (34) is not true. This implies the existence of a constant c1>0 such that(35)gk≥c1,forallk.
Squaring both sides of (4) implies(36)dk2=gk2−2βkEDLgkTdk−1+βkEDL2dk−12.
Taking into account (18), we can get(37)−2βkEDLgkTdk−1=−2gk2−gk/gk−1gkTgk−1dk−1Tyk−1−tk∗gkTsk−1dk−1Tyk−1gkTdk−1=−2gk2−gk/gk−1gkTgk−1dk−1Tyk−1gkTdk−1−tk∗αk−1gkTdk−12dk−1Tyk−1.
Now from (31) and (32), it follows that(38)−2βkEDLgkTdk−1≤2gk2−gk/gk−1gkTgk−1dk−1Tyk−1gkTdk−1≤2∥gk∥2λgkTdk−1gkTdk−1=2∥gk∥2λ.
Now, an application of (18) initiates(39)βkEDL=gk2−gk/gk−1gkTgk−1−tk∗gkTsk−1dk−1Tyk−1≤gkTgk−gk/gk−1gkTgk−1−tk∗gkTsk−1dk−1Tyk−1≤gkTgk−gk/gk−1gk−1−tk∗sk−1θαk−1dk−12=gkTgk−gk−1+gk−1−gk/gk−1gk−1−tk∗sk−1θαk−1dk−12≤gkgk−gk−1+gk−11−gk/gk−1+tk∗sk−1θαk−1dk−12=gkgk−gk−1+1−gk/gk−1gk−1+tk∗sk−1θαk−1dk−12=gkgk−gk−1+gk−1−gk+tk∗sk−1θαk−1dk−12≤gkgk−gk−1+gk−1−gk+tk∗sk−1θαk−1dk−12=gk2gk−gk−1+tk∗sk−1θαk−1dk−12≤gk2Lsk−1+tk∗sk−1θαk−1dk−12=2L+tk∗gksk−1θαk−1dk−12=2L+tk∗gkαk−1dk−1θαk−1dk−12=2L+tk∗gkθdk−1.
Usingtk∗∈0,1 and (38) and (39) in (36), we obtain(40)∥dk∥2≤∥gk∥2+2∥gk∥2λ+2L+tk∗2gk2θ2dk−12∥dk−1∥2=∥gk∥2+2∥gk∥2λ+2L+tk∗2θ2gk2=1+2λ+2L+tk∗2θ2gk2=λ+2λ+2L+tk∗2θ2gk2=λ+2θ2+λ2L+tk∗2λθ2gk2.
Next, dividing both sides of (40) by gk4 and using (35), it can be concluded that(41)∥dk∥2∥gk∥4≤λ+2θ2+λ2L+tk∗2λθ2·1c12,∥gk∥4∥dk∥2≥λθ2·c12λ+2θ2+λ2L+tk∗2.
The inequalities in (41) imply(42)∑k=0∞gk4dk2≥∑k=0∞λθ2·c12λ+2θ2+λ2L+tk∗2=∞.
Therefore,gk≥c1 causes a contradiction with Lemma 2.


## 3. Numerical Experiments

The implementation of the EDL method is based on Algorithm2. This section is intended to analyze and compare the numerical results obtained by the EDL method and four variants of the MHSDL class methods (6). These variants are defined by t≡tk3, t≡tk4, t≡tk5, and t≡tk6 and denoted, respectively, as MHSDL3, MHSDL4, MHSDL5, and MHSDL6. The obtained results are not compared with the values tk1 and tk2, because in [16], the authors have already shown that tk3 and tk4 initiate better numerical performances compared to tk1 and tk2.The codes used in the testing experiments for the above methods are written in MATLAB R2017a and executed on the Intel Core i3 2.0 GHz workstation with the Windows 10 operating system. Three important criteria are analyzed in each individual test case: number of iterations (NI), number of function evaluations (NFE), and processor time (CPU).The numerical experiment is performed using 28 test functions presented in [24], where much of the problems are taken over from the CUTEr collection [25]. All methods used in the testing of an arbitrary objective function start from the same initialization x0. Each function is tested 10 times with gradually increasing dimensions n=100, 500, 1000, 3000, 5000, 7000, 8000, 10000, 15000, and 20000.The uniform terminating criteria for each of the five considered algorithms (EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6) are(43)gk≤ε,fxk+1−fxk1+fxk≤δ,where ε=10−6 and δ=10−16. The backtracking line search is based on the parameters ω=0.0001 and φ=0.8 for all five algorithms. Specific parameters used only in the MHSDL6 method are defined as C=1, υ=0.26, and r=rk=υ∥gk−1∥.Summary numerical results for EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6 methods, executed on 28 test functions, are arranged in Tables1–3. Tables 1–3 show the numerical outcomes corresponding to all three criteria (NI, NFE, and CPU) for the EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6 methods.Table 1
Summary results of EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6 methods with respect to NI.
Test functionMHSDL3MHSDL4MHSDL5EDLMHSDL6Extended penalty14662243223112591371Perturbed quadratic1203710754291746557305622423037Raydan 11590551105871065865547775154Raydan 2163644144170209Diagonal 111678878844735123097820332Diagonal 2176983270434271595515000271295Diagonal 3150328986471044174715537711Hager86665219515732343625Generalized tridiagonal 1186214711485639877Extended TET13575954591540302664Diagonal 4306931958919332804012012Diagonal 51721251202512060216Extended Himmelblau17778023794613763682Perturbed quadratic diagonal29409702115659202712811364141352704Quadratic QF11270802799192786032309509325415Extended quadratic penalty QP1770594575560543Extended quadratic penalty QP239967124053024525496620137799Extended quadratic exponential EP1462606606513526Extended tridiagonal 231192176217711321455ARWHEAD (CUTE)8882469868674134071348669ENGVAL1 (CUTE)232314071415552820INDEF (CUTE)203110802336240QUARTC (CUTE)173913262291262291524299262181Diagonal 6182450850870227Generalized quartic12081403284612651154Diagonal 73217655655653580Diagonal 8511698698686596Full Hessian FH314565353535025233176Table 2
Summary results of EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6 methods with respect to NFE.
Test functionMHSDL3MHSDL4MHSDL5EDLMHSDL6Extended penalty5487673764734294682049791Perturbed quadratic5669173734287604338857011316868818486375Raydan 150667393364983323633515518462170553Raydan 2655411621162159428Diagonal 15004640325627430220151200086744278Diagonal 23539765408785432001030010542600Diagonal 363391463998904422956517980321400076Hager1924741074131065345918769735Generalized tridiagonal 13742927860281381076015177Extended TET1954677422769254034029334Diagonal 4713120425023418666155027242443Diagonal 568745046050460140442Extended Himmelblau459721923621905242610480854Perturbed quadratic diagonal13590122294177165902384414814751257702654Quadratic QF15597269733836473332437111231672112853424Extended quadratic penalty QP11701612882125651111610544Extended quadratic penalty QP2130158887454686758496027433584030601Extended quadratic exponential EP11491418463184631413215133Extended tridiagonal 2364502256422379968712920ARWHEAD (CUTE)42960283305257318213818466062230650ENGVAL1 (CUTE)404622243222898820912858INDEF (CUTE)1808218259952060104962QUARTC (CUTE)3479265246625246621048648524422Diagonal 6739414161408159468Generalized quartic1436421842487701669514103Diagonal 764546838683838914521Diagonal 860986938693841615494Full Hessian FH36079221279921270189890114962Table 3
Summary results of EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6 methods with respect to CPU time (sec).
Test functionMHSDL3MHSDL4MHSDL5EDLMHSDL6Extended penalty29.7534.1131.4218.3024.27Perturbed quadratic40532.6624358.2024947.848335.8013225.80Raydan 13054.671904.481692.06690.911184.86Raydan 26.771.581.660.310.77Diagonal 17834.035106.414592.281476.89486.09Diagonal 2885.131428.051447.022352.111513.50Diagonal 313614.278416.779064.303132.021916.30Hager586.63325.75333.41142.06198.13Generalized tridiagonal 166.1435.5934.4215.1921.63Extended TET20.5078.3482.9441.2331.45Diagonal 4134.5377.8687.8830.4155.34Diagonal 518.06134.73121.090.561.84Extended Himmelblau11.1344.4744.366.1918.30Perturbed quadratic diagonal91655.5558226.1660920.0632179.3836383.83Quadratic QF162610.5031552.4828679.918832.118465.34Extended quadratic penalty QP17.567.256.984.984.94Extended quadratic penalty QP23814.162128.862288.55671.521204.72Extended quadratic exponential EP19.1110.238.558.008.02Extended tridiagonal 211.138.836.954.085.25ARWHEAD (CUTE)2709.422336.922369.281266.801689.80ENGVAL1 (CUTE)19.4711.3311.814.036.70INDEF (CUTE)2.442.8910.701.92774.34QUARTC (CUTE)3106.564818.584808.707138.724735.39Diagonal 66.751.922.030.381.34Generalized quartic7.1611.5321.057.539.78Diagonal 75.988.208.284.566.25Diagonal 86.178.208.084.727.69Full Hessian FH330.0866.4579.4835.7743.42We utilized the performance profile given in [26] to compare numerical results for three criteria (NI, NFE, and CPU) generated by five methods (EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6). The upper curve of the selected performance profile corresponds to the method that shows the best performance.Figures1–3 plot the performance profiles for the numerical values included in Tables 1–3, respectively. Figure 1 presents the performance profiles of the NI criterion generated by the EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6 methods. In this figure, it is noticeable that EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6 methods solved all tested functions, wherein the EDL method shows the best performances in 57.14% of test functions compared with MHSDL3 (25.00%), MHSDL4 (0.00%), MHSDL5 (0.00%), and MHSDL6 (17.86%). From Figure 1, it is observable that the graph of the EDL method comes first to the top, which means that the EDL outperforms other considered methods with respect to the NI.Figure 1
NI performance profile for EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6 methods.Figure 2
NFE performance profile for EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6 methods.Figure 3
CPU performance profile for EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6 methods.Figure2 presents the performance profiles of the NFE of the EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6 methods. It is observable that EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6 generated solutions to all tested cases, and the EDL method is the best in 67.86% of the functions compared with MHSDL3 (17.86%), MHSDL4 (0.00%), MHSDL5 (0.00%), and MHSDL6 (14.28%). From Figure 2, it is observed that the EDL graph first comes to the top, which confirms that the EDL is the winner with respect to the NFE.Figure3 contains graphs of the performance profiles corresponding to the CPU time of the EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6 methods. It is obvious that EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6 solved all tested functions. Further analysis gives that the EDL method is the winner in 67.86% of the test cases compared with MHSDL3 (17.86%), MHSDL4 (0.00%), MHSDL5 (0.00%), and MHSDL6 (14.28%). Figure 3 demonstrates that the graph of the EDL method first comes to level 1, which indicates its superiority with respect to the CPU time.From the previous analysis of the results shown in Tables1–3 and Figures 1–3, it can be concluded that the EDL method produces superlative results in terms of all three basic metrics: NI, NFE, and CPU.


## 4. Conclusion

A novel rule which determines the valuetk of the parameter t in each iteration of the Dai-Liao-type CG method is presented. The proposed expression for defining tk is denoted by tk∗. Considering t=tk∗ in (6), a novel variant of the Dai-Liao CG parameter βkEDL is defined and a novel Effective Dai-Liao (EDL) conjugate gradient method is proposed. The convergence of the EDL method is investigated, and the global convergence on a class of uniformly convex functions is established. By numerical testing, we have shown that there is a significant influence of the scalar size of tk∗ on the convergence speed of the EDL method. Numerical comparisons on large-scale unconstrained optimization test functions of different structures and complexities confirm the computational efficiency of the algorithm EDL and its superiority over the previously known DL CG variants, such as MHSDL3, MHSDL4, MHSDL5, and MHSDL6. During the testing, we tracked the number of iterations (NI), number of function evaluations (NFE), and spanned processor time (CPU) performances for each function and each method. Analysis of the obtained performance profiles introduced by Dolan and Moré revealed that the EDL method is the most efficient.We are convinced that the obtained results will be a motivation for further research in defining new values of the parametertk in the Dai-Liao CG methods. Future research would include research in finding some more efficient rules to calculate the parameter tk during the iterative process. We hope that our proposal of the new expression for defining the parameter t will initiate further research in that direction. It is evident that finding novel approaches in defining different values of t and the conjugate gradient parameter βk is an inexhaustible topic for scientific research, and our approach is only one possible direction in this research.



---

*Source: 6693401-2021-01-30.xml*