# A New Modified Three-Term Conjugate Gradient Method with Sufficient Descent Property and Its Global Convergence

Authors: Bakhtawar Baluch; Zabidin Salleh; Ahmad Alhawarat; U. A. M. Roslan
Journal: Journal of Mathematics
 (2017)

---

## Abstract

A new modified three-term conjugate gradient (CG) method is shown for solving the large scale optimization problems. The idea relates to the famous Polak-Ribière-Polyak (PRP) formula. As the numerator of PRP plays a vital role in numerical result and not having the jamming issue, PRP method is not globally convergent. So, for the new three-term CG method, the idea is to use the PRP numerator and combine it with any good CG formula’s denominator that performs well. The new modification of three-term CG method possesses the sufficient descent condition independent of any line search. The novelty is that by using the Wolfe Powell line search the new modification possesses global convergence properties with convex and nonconvex functions. Numerical computation with the Wolfe Powell line search by using the standard test function of optimization shows the efficiency and robustness of the new modification.


---

## Body

## 1. Introduction

The conjugate gradient method is an efficient and organized tool for solving the large-scale nonlinear optimization problem, due to its simplicity, easiness, and low memory requirements. This method is very popular for mathematician and engineers and those who are interested in solving the large-scale optimization problems [1–3].Consider the unconstrained optimization problem(1)minfx,x∈Rn,where f:Rn→R is continuously differentiable and its gradient is available g(x). Generally CG method generates an iterative sequence (xk) defined by(2)xk+1=xk+αkdk,k=0,1,…,where αk>0 is a line search and dk is a search direction defined by (3)dk=-gk,ifk=0,-gk+βkdk-1,ifk≥1,where the term βk is a scalar. There are six essential formulas for βk, which are stated as(4)βkHS=gkTgk-gk-1gk-gk-1Tdk-1(Hestenes and Stiefel [4], 1952),(5)βkFR=gkTgkgk-1Tgk-1(Fletcher and Reeves [5], 1964),(6)βkPRP=gkTgk-gk-1gk-1Tgk-1(Polak et al. [6, 7], 1969),(7)βkCD=-gkTgkdk-1Tgk-1(Conjugate-Descent [8], 1997),(8)βkLS=-gkTgk-gk-1dk-1Tgk-1(Liu and Storey [9], 1991),(9)βkDY=gkTgkgk-gk-1Tdk-1 (Dai and Yuan [10], 2000).Generally inexact line search is used in order to get the global convergence of conjugate gradient method, such as Wolfe line search or strong Wolfe line search; the Wolfe line search is given as(10)fxk+αkdk≤fxk+ραkgkTdk,gxk+αkdkTdk≥σgkTdk,where 0<ρ<σ<1. The strong Wolfe line search is computing αk such that (11)fxk+αkdk≤fxk+ραkgkTdk,gxk+αkdkTdk≤σgkTdk.Recently Alhawarat et al. [11, 12] and Alhawarat and Salleh [13, 14] have proposed an efficient and hybrid conjugate gradient method that satisfies the global convergence properties. To enhance the effectiveness of two-term conjugate gradient method, the three-term conjugate gradient has been widely studied and given much importance. The three-term conjugate gradient method attains different numerical outcomes, depending on how the scalar parameter is being selected. The papers by Beale [15], McGuire and Wolfe [16], Nazareth [17], Deng and Li [18], Dai and Yuan [19], Zhang et al. [20, 21], Cheng [22], Zhang et al. [23], Al-Bayati and Sharif [24], Narushima et al. [25], Andrei [26–28], Sugiki et al. [29], Al-Baali et al. [30], Babaie-Kafaki and Ghanbari [31], and Sun and Liu [32] presented different types of three-term conjugate gradient method along with their numerical performance and efficiency and proved their global convergence properties. As a comparison with classical conjugate gradient algorithms, the proposed three-term conjugate gradient algorithms are numerically strong, efficient, reliable, and robust compared to the classical conjugate gradient algorithms and Beale [15] was the first to propose the three-term conjugate gradient method.In the new three-term modification, we put our attention on the numerator of PRP method, in which the parameterβk is given as(12)βkPRP=gkTgk-gk-1gk-12.The PRP method is among one of the most efficient and reliable conjugate gradient method due to good numerical performance. The global convergence of PRP is established when the objective function f is strongly convex and the line search is exact [6]. On the other hand Powell [33] through his analysis expressed that there exist nonconvex functions for which PRP method does not converge globally. Gilbert and Nocedal [34] established the so-called PRP+ method; in this method βkPRP+ is restricted to be nonnegative denoted as βkPRP+=max⁡0,PRP. If the standard Wolfe line search (10) is used, thenPRP+method attains the global convergence and also sufficient descent conditions are being satisfied.Recently Sun and Liu [32] proposed a new conjugate gradient method called TMPRP 1 method by using the VFR formula from Wei et al. [35], in which the search direction is stated as(13)dkTMPRP1=-gk,ifk=0,-1+βkMPRPgkTdk-1gk2gk+βkMPRPdk-1,ifk≥1,where βkMPRP=gkT(gk-gk-1)/μgkTdk-1+gk-12 or βkMPRP+=max⁡gkTgk-gk-1/μgkTdk-1+gk-12,0.This method has attractive property of satisfying the sufficient descent conditiongkTdk=-gk2 independent of any line search and attains global convergence if standard Wolfe line is used. As compared with the strong Wolfe line search, the standard Wolfe line search takes less computation in order to get an acceptable step size at each iteration. Hence the standard Wolfe line search increases the effectiveness of the conjugate gradient method [32].The rest of the paper is organized as follows. In Section2, the motivation and formula for construction of three-term conjugate gradient method are given. In Section 3 we have presented Algorithm  1.1 in which the general form of three-term conjugate gradient method is shown. In Section 4 the sufficient descent condition and the global convergence properties for convex and nonconvex function are proven. In Section 5, the detailed numerical results to test the proposed method are reported.


## 2. Motivation and Formula

Wei et al. [35] proposed three new formulas which are given in the following:(14)βk2∗μ,η=ηgk2μgkTdk-1-ηgk-1Tdk-1,forη∈0,+∞,μ∈η,+∞,βk∗∗μ1,μ2,μ3=μ1gk2μ2gkTdk-1-μ3gk-1Tdk-1,forμ1∈0,+∞,μ2∈μ1+ε1,+∞,μ3∈0,+∞,βkVFRμ=μ1gk2μ2gkTdk-1+μ3gk-12,forμ1∈0,+∞,μ2∈μ1+ε1,+∞,μ3∈0,+∞.There is an efficient conjugate gradient method named βk2∗μ,η. In this formula, the denominator plays an important role in satisfying the sufficient descent condition and performs well in terms of global convergence and numerical result. This motivated us to take the denominator from this formula. Secondly, the PRP [6, 7] method is considered to be one of the most proficient CG parameters due to the properties of its numerator gk-gk-1. If the step taken becomes very small, then gk-gk-1 reaches zero such as gk-gk-1≈0. Afterwards βkPRP≈0; then the search direction continued as the steepest descent method. So the numerator of PRP method worked efficiently and does not jam.This motivated us to construct a new modified three-term conjugate gradient method, such as(15)βkBZAU=gkTgk-gk-1-ηgk-1Tdk-1+μgkTdk-1,forη∈1,+∞,μ∈η,+∞.Further, Powell [36] showed that the PRP method can cycle infinitely without approaching a minimum point, even if the step size αk is chosen to the least positive minimizer. To overcome this, Gilbert and Nocedal [34] showed their analysis(16)βkPRP+=max⁡βkPRP,0.So(17)βkBZAU+=max⁡gkTgk-gk-1-ηgk-1Tdk-1+μgkTdk-1,0forη∈1,+∞,μ∈η,+∞,(18)θkBZAU=gkTdk-1-ηgk-1Tdk-1+μgkTdk-1forη∈1,+∞,μ∈η,+∞.In βkBZAU, βkBZAU+, and θkBZAU, the parameters η and μ have an important role in the sense that when η is getting smaller, the numbers of iteration, function evaluation, and gradient evaluation are decreased and when μ is getting larger, the numbers of iteration, function evaluation, and gradient evaluation are also decreased. So we observe that the best value for the parameters is η,μ=1,2.


## 3. Algorithm  1.1

Step  0. Given an initial point x0∈Rn, μ=2, η=1, 0<ρ<σ<1, and set d0=-g0, k≔0.Step  1. If gk≤ε, where ε=10-6, then the algorithm stops; otherwise, go to Step  2.(Note: all the norm· we use in this paper means ·2).Step  2. Compute the search direction (19) by using βkBZAU and θkBZAU where(19)βkBZAU=gkTgk-gk-1-ηgk-1Tdk-1+μgkTdk-1,θkBZAU=gkTdk-1-ηgk-1Tdk-1+μgkTdk-1dk=-gkifk=0,-gk+βkBZAUdk-1-θkBZAUyk-1ifk≥1, where yk-1=gk-gk-1.Step  3. Determine the step size αk>0 by the Wolfe line search (10).Step  4.Compute xk+1=xk+αkdk where αk is given in Step  3 and dk is given in Step  2.Step  5. Set k=k+1 and go to Step  1.


## 4. Global Convergence of Modified Three Term

Assumptions 1.
(A1) The level setR0=x∣fx≤fx0 is bounded.
(A2) In some neighborhoodN of R0, the gradient gx is Lipschitz continuous on an open convex set E that contains R0; that is, there exists a positive constant L>0 such that(20)gxk-gxk-1≤Lxk-xk-1for anyxk,xk-1∈E;then Assumptions (A1) and (A2) and [32, 34] imply that there exist positive constants γ and b such that(21)gxk≤γ∀xk∈R0,(22)xk-xk-1≤b∀xk,xk-1∈R0.Sincefxk is decreasing as k→+∞, from Assumption (A1) it is shown that the sequence xk created by Algorithm  1.1 will be contained in a bounded region. Then the sequence xk is convergent.Now we will prove the sufficient descent condition independent of line searchgkTdk=-gk2 and also gk≤dk. From (19),(23)dk=-gk+βkBZAUdk-1-θkBZAUyk-1.Multiplying by gkT, we obtain (24)gkTdk=-gk2+gkTgk-gk-1gkTdk-1-ηgk-1Tdk-1+μgkTdk-1-gkTdk-1gkTgk-gk-1-ηgk-1Tdk-1+μgkTdk-1;that is,(25)gkTdk=-gk2.Hence the sufficient descent condition independent of line search holds.Now we prove thatgk≤dk. As we have gkTdk=-gk2, by taking modulus on both sides(26)gkTdk=-gk2=gk2.By Schwarz inequality we have (27)gkTdk≤gkdk.So,(28)gk2≤gkdkorgk≤dk.Thus we have (29)gkTdk=-gk2,gk≤dk.Lemma 1.
Assumptions (A1) and (A2) hold ifx0 is supposed to be an initial point. Now consider any method in the form of (2), in which dk is a descent direction and αk satisfies the Wolfe condition (10) or the strong Wolfe line search condition (11). Then we have the Zoutendijk condition: (30)∑k=0∞gkTdk2dk2<+∞,which is normally used to prove the global convergence of CG method. From (29) the Zoutendijk condition is equivalent to the following inequality:(31)∑k=0∞gk4dk2<+∞.Definition 2 (see [32]).
The functionf is called uniformly convex on Rn, if there exists a positive constant m such that(32)mdk2≤dT∇2fxkd∀x,d∈Rn,where the function f has the Hessian matrix ∇2fxk.We now show the global convergence of Algorithm  1.1 for uniformly convex functions.Lemma 3.
Let both sequencesxk and dk be generated by Algorithm  1.1 and suppose that (32) holds; then(33)z1αkdk2≤-gkTdk,where z1=1-ρ-1m/2, m is a positive constant, and ρ is a positive number whose range is 0<ρ<σ<1, from the Wolfe line search (10).Proof.
Detail of proof can be seen in Lemma2.1 of [37].Theorem 4.
Suppose that Assumptions (A1) and (A2) hold and the functionf is uniformly convex; then (34)limk→∞gk=0.Proof.
From (15), (33), and (A2), we have(35)βkBZAU≤gkTgk-gk-1η-gk-1Tdk-1≤gkLxk-xk-1ηz1αk-1dk-12=gkLαk-1dk-1ηz1αk-1dk-12,βkBZAUdk-1≤Lgkηz1dk-1dk-1=Lgkηz1.Now (36)θkBZAUyk-1≤gkTdk-1η-gk-1Tdk-1yk-1.From (18), (33), and (A2), we have (37)θkBZAUyk-1≤gkLxk-xk-1ηz1αk-1dk-12dk-1=gkLαk-1dk-1ηz1αk-1dk-12dk-1=Lgkηz1.Combining (35) and (37) with (19),(38)dk≤gk+βkBZAUdk-1+θkBZAUyk-1≤gk+Lgkηz1+Lgkηz1=1+2Lηz1gk.Now letting B=1+2L/ηz1, we get dk2≤Bgk2,(39)1dk2≥1Bgk2,Bgk4dk2≥gk4gk2=gk2.So by (31), we get(40)limk→∞⁡gk2≤Blimk→∞gk4dk2=0.We now show the global convergence for nonconvex functions.Lemma 5.
Let Assumptions (A1) and (A2) hold. Consider the sequence (xk) to be generated by Algorithm  1.1. If there exists a positive constant ϵ>0 in such a way that gk≥ϵ for every k≥0,(41)∑k=0∞uk+1-uk2<+∞,where uk=dk/dk.Proof.
SincegkTdk=-gk2, gk≤dk, and gk≥ϵ for every k, then we get dk>0 for every k, so that uk is well defined. If(42)rk=-1+θkBZAUyk-1gkT/gk2gkdk,δk=βkBZAU+dk-1dk,then we get uk=rk+δkuk-1. Also uk and uk-1 are unit vectors, so(43)rk=uk-δkuk-1=δkuk-uk-1as we know δk≥0,(44)uk-uk-1≤1+δkuk-uk-1≤uk-δkuk-1+δkuk-uk-1=2rk.Now from (18), (22), and (A2) (45)θkBZAUyk-1gkTgk2≤gkTdk-1μgkTdk-1yk-1gkTgk2≤Lxk-xk-1μgk≤Lbϵμ.Now from (21), (22), and (45), there is a constant N1≥0 as follows:(46)-1+θkBZAUyk-1gkTgk2gk≤gk+θkBZAUyk-1gkTgk2gk≤γ+Lbϵμγ=N1.Therefore, from (31) and (46), we have(47)∑k=0∞rk2≤∑k=0∞N12dk2≤∑k=0∞N12gk4gk4dk2=N12ϵ4∑k=0∞gk4dk2<+∞.This along with (44) completes the proof.Theorem 6.
Suppose that Assumptions (A1) and (A2) hold. Then the sequence (xk) generated by Algorithm  1.1 satisfies(48)limk→∞⁡inf⁡gk=0.Proof.
Suppose that the conclusion (48) is not true. Then there exists a positive constant ϵ>0 in such a way that gk≥ϵ∀k≥0.
The proof has the following two parts.
Part 1. We noticed that for any l and k we have l≥k, such that(49)xl-xk=∑i=kl-1xi+1-xi=∑i=kl-1siui.Proceeding the same proof of Theorem 2.2, step 1 from [32], we have(50)∑i=kl-1si≤2b.
Part 2. Taking a bound on the direction dk. Now from (19) and (46) we have(51)dk2≤-1+θkBZAUgkTyk-1gk2gk+βkBZAU+dk-12≤N1+βkBZAU+dk-12≤2N12+2βkBZAU+dk-12≤2N12+2γ2L2sk-12ε2dk-12.At the beginning of proof we assume that limk→∞⁡infgk≠0; then there exists a constant ϵ>0 and also there exists γ>0 such that gk>γ>0. Then (52)∑k=0∞gkTdk2dk2>+∞⟹1dk2>+∞,which contradicts with (A2), (31), and (51). Hence it is proved that limk→∞⁡inf⁡gk=0.


## 5. Numerical Results

In this part we compare the numerical results of proposed three-term BZAU (Bakhtawar, Zabidin, Ahmad and Ummu) method with recently developed TMPRP1 method and also compare their performance. The Wolfe line search (10) is used and the values of the parameters for BZAU and TMPRP1 method areμ= 2, 10(−4);η= 1, 0; ρ = 0.1, 0.1; and σ = 0.5, 0.5, respectively. The code was written in Matlab 7.1 and run on an i5 computer with 2.40 GHz CPU processor, 2.0 GB RAM memory. We test the functions taken from [38] with dimension ranges 2,5000. The main purpose in optimization for the selection of large number of test functions is to test the unconstrained optimization algorithms properly. Dantzig (1914–2005) said the final test of a theory is its capacity to solve the problems which originated it. This is one of the main reasons we select the large-scale unconstrained optimization problems to test the theoretical progress in numerical form through mathematical programming [38].Moré et al. [39] claimed the efficiency of a method and that algorithm for a small number of test functions is not suitable because this will lead to the choice of an algorithm that is not favorable. Testing a method or algorithm for a large number of test functions would lead to large amount of data and from that data we can interpret which method or algorithm is more efficient and robust. But the number of test functions should not be very large nor very small, so there is a benchmark of 75 numbers of test functions which are chosen to test the efficiency of any method.Practically, optimizers need to evaluate nonlinear optimization method. To prove the global convergence properties of any method, the theory is not enough to determine the reliability and efficiency of a method. As a result, the robustness of any method is established by testing the large number of test problems [38].In global convergence propertyβkBZAU is used in case of proving convex function and property βkBZAU+ is used for proving nonconvex function. But in the numerical part βkBZAU+ is used for a comparison with TMPRP1 method. The TMPRP1 possesses the sufficient descent property without any line searches. Theoretically, TMPRP1 method established well and converges globally. When it comes to numerical computation, the TMPRP1 method is tested by a benchmark of 75 numbers of test functions and shows a promising result. Hence the TMPRP1 method is then compared with our BZAU method.In Table1 number of iterations, number of function evaluations, number of gradient evaluations, and CPU time are represented by NI/NF/GE/CT. If the CT exceeds 500 seconds and the NI is more than 10000 iterations, then the function is given the name of Fail F. This standard is followed by every paper. For most of the function we can get the result within this limit and the function that does not come in this limit is named Fail F.Table 1
A list of test problem functions.
Problem
n
BZAUNI/GE/FE/CT
TMPRP1NI/GE/FE/CT
Extended white and holst function
500
31/223/191/1.4647
32/193/160/1.1246
1000
31/223/191/1.845
32/193/160/1.4398
Extended Rosenbrock function
500
28/170/141/1.1573
21/110/88/0.8869
1000
28/170/141/1.152
21/110/88/0.9232
Extended Himmelblau function
500
7/34/26/0.6446
10/47/36/0.6638
1000
7/34/26/0.7544
11/51/39/0.697
5000
7/34/26/0.9163
12/55/42/1.0252
Extended DENSCHNB function
2
5/22/16/0.6135
6/26/19/0.5955
1000
6/26/16/0.68
6/26/19/0.7011
Shallow function
500
9/41/31/0.6188
8/39/30/0.6341
1000
9/41/31/0.7291
8/39/30/0.764
DIXMAANA function
6000
8/35/26/1.65
10/43/32/1.7494
60
8/35/26/2.1339
10/67/56/2.7067
NONDIA
500
8/59/50/0.6976
11/66/54/0.7215
5000
14/84/69/1.4958
16/90/73/1.5903
DQDRTIC
100
5/21/15/0.6202
5/21/15/0.658
1000
5/21/15/0.5997
5/21/15/0.6591
Extended Block Diagonal BD 1
500
32/37000/28750/1.9822
35/73750/64750/2.6201
1000
32/75500/59000/2.4557
36/147000/128500/4.261
Extended Tridiagonal 1
2
4/18/13/0.5789
1694/5096/3401/18.6813
5000
2072/7767/5694/130.172
F
Diagonal 4
100
2/9/6/0.5781
2/9/6/0.589
5000
2/9/6/0.6138
2/9/6/0.623
Extended Powell
100
400/41100/31075/7.6346
92/9925/7600/2.4096
500
400/205500/155375/7.58
92/49625/38000/2.3136
Diagonal 2
5
44/135/90/1.008
48/147/98/1.2072
100
964/2895/1930/12.4522
968/2907/1938/12.5155
Sum Squares
100
57/229/171/1.4846
57/229/171/1.5077
1000
165/661/495/3.1385
165/661/495/3.2041
SINQUAD
70
75/376/300/2.0587
138/653/514/3.5782
100
131/630/498/3.9387
594/2252/1657/11.1859
DIXMAANC
66
6/29/22/0.7458
7/33/25/0.7998
120
6/29/22/0.7667
7/33/25/0.7857
DIXMAANE
150
60/300/239/2.1251
434/1374/939/7.1568
3000
244/1220/975/25.8693
F
DIXMAANG
150
66/330/263/2.3892
681/2073/1391/9.6909
3000
2961/13363/10401/281.4836
F
DIXMAANI
3
20/90/69/0.8965
43/136/92/1.0161
12
287/1269/961/6.297
342/1375/1032/6.5883
DIXMAANK
3
21/94/72/0.87
52/163/110/0.99
12
684/2069/1384/9.0948
695/2904/1398/9.2842
DENSCHNA
100
8/40/31/0.6462
8/37/28/0.7929
500
8/40/31/0.8236
8/37/28/0.8122
DENSCHNB
2
5/22/16/0.674
6/26/19/0.997
20
5/22/16/0.6559
6/26/19/0.6583
DENSCHNC
100
7/38/30/0.7599
8/42/33/0.6927
500
7/38/30/0.8327
8/42/33/0.8875
ARGLINB
500
2/9/6/0.5714
2/9/6/0.5631
1000
2/9/6/0.5777
2/9/6/0.6239
Power
2
2/9/6/0.5587
2/9/6/0.5702
50
66/265/198/1.7672
66/265/198/1.7369
Fletcher
5
22/120/97/0.964
26/143/116/1.0727
50
242/1333/1090/8.6331
278/1495/1216/9.7661
HIMMELBH
100
9/77/67/0.8398
8/74/65/1.1388
5000
9/77/67/1.1324
8/74/65/1.7183
NONDQUAR
5
660/2649/1988/14.3065
739/2964/2224/16.1369
30
7170/28703/21532/110.70
F
Generalized quartic GQ1
1000
11/47/35/0.706
13/55/41/0.6912
5000
11/47/35/0.987
12/52/39/1.219
Generalized quartic GQ2
500
24/106/81/0.9497
26/115/88/1.0062
1000
23/102/78/0.9412
24/107/82/0.9761
Extended Quadratic Penalty
100
12/61/48/0.7067
10/43/32/0.642
5000
15/92/76/1.235
18/119/100/1.2217
ARWHEAD
2
8/36/27/0.6249
7/32/24/0.5877
100
7/39/31/0.6106
6/35/28/0.6174
Quartic
2
3941/11826/7884/56.888
3945/11838/7892/59.0699
5
5358/16077/10718/80.029
5362/16089/10726/78.05
Extrosnb
100
0/1/0/0.5027
0/1/0/0.5208
5000
0/496/334/0.5188
0/1/0/0.5168
LIARWHD
2
11/56/44/0.7354
12/66/53/0.794
100
16/94/77/0.8861
20/134/113/1.3092
ENGVAL8
2
7/36/26/0.6582
8/37/28/0.6912
100
F
F
Hager
2
4/17/12/0.6103
4/17/12/0.5686
100
25/151/125/1.2086
25/102/76/0.9215
Extended Wood
100
91/509/417/3.0717
116/627/510/3.6229
500
68/423/354/2.7678
87/535/447/3.3762
BIGGSB1
500
1579/6316/4736/31.3271
1627/6508/4880/32.1275
1000
2463/9852/7388/48.4366
3509/14036/10526/68.93
Extended Beale
100
15/71/55/0.7541
12/59/46/0.7065
500
15/71/55/0.8143
12/59/46/0.8638
Extended Penalty
2
9/43/33/0.6341
10/46/35/0.6735
1000
37/252/214/1.7596
F
Extended Maratos Function
500
33/223/189/2.1234
31/213/181/1.5054
1000
F
31/213/181/1.5017
Quadratic QF2 Function
2
6/26/19/0.6122
6/26/19/0.6209
500
293/1269/975/6.6934
277/996/768/5.1248
SINCOS
1000
8/39/30/0.8623
12/51/38/0.9807
5000
8/39/30/1.1289
12/51/38/1.2905
STAIRCASE S1
10
31/125/93/1.0555
32/129/96/1.0384
50
135/540/404/2.7473
197/790/592/4.0499
GENROSEN-2
2
25/159/133/1.3379
21/110/88/0.9879
500
6889/36000/29110/185.5
74/488/413/3.4692
TRIDIAGONAL DOUBLE BORDED
50
395/1585/1189/7.8008
540/2165/1624/11.11
100
773/3099/2325/15.481
1016/4073/3056/20.09
EDENSCH FUNCTION
500
3090/9276/6185/40.9297
3094/9288/6193/34.4141
1000
3090/9276/6185/135.232
3094/9288/6193/131.676
DENSCHNF
500
10/59/48/0.9205
12/73/60/0.9032
1000
10/59/48/0.7546
12/73/60/0.8977
Generalized Triagonal-1
100
23/95/71/0.9284
F
500
25/124/98/1.5822
24/100/75/0.9883
Generalized Triagonal-2
50
43/205/161/1.225
35/165/129/1.5369
100
39/188/148/1.2548
39/191/151/1.2668
Extended Trigonomenctric
5
313/965/651/5.1384
324/990/665/5.2108
100
97/437/339/2.6942
318/997/678/5.3339
DIAGONAL-1
2
5/22/16/0.6467
6/27/20/0.6174
10
23/95/71/0.889
25/103/77/0.9587
EG3
50
32/138/105/1.1712
33/141/107/1.1336
200
35/180/144/1.4381
F
Extended Three Exponential
100
8/1700/1250/0.6441
6/1300/950/0.6111
1000
8/17000/12500/0.999
6/13000/9500/0.8019
Extended Freudenstein and Roth
30
16/91/74/0.7806
F
50
16/141/124/1.0795
F
Dixon3DQ
1000
3049/12196/9146/49.19
3409/13636/10226/55.20
2000
4959/19836/14876/80.41
6473/25892/19418/117.9
Quadratic Qf1
1000
187/749/561/4.2752
187/749/561/4.3368
5000
426/1705/1278/10.7051
426/1705/1278/11.6042
Extended Quadratic penalty Q2
500
46/466/419/2.9755
51/576/524/4.0679
5000
61/740/678/6.627
86/1052/970/9.3083
Sphere
2
1/5/3/0.5581
1/5/3/0.549
500
1/5/3/0.5498
1/5/3/0.5447
Raydan1
2
101/304/202/1.6957
105/316/210/2.0696
100
66/266/199/1.5276
69/278/208/2.0739
BDQRTIC
10
37/179/141/1.1653
52/240/187/1.4611
500
F
F
DIXMAANJ
3
23/88/64/0.809
52/162/109/1.0504
45
3015/13079/10063/63.498
7074/21256/14181/93.00
DIXMAANB
3
5/23/17/0.5848
6/27/20/0.652
900
8/34/25/0.945
8/34/25/0.954
DIXMAAND
300
8/38/29/0.932
9/42/32/0.9416
900
7/34/26/1.0641
10/46/35/1.215
DIXMAANF
300
90/451/360/3.386
1327/3998/2670/19.805
900
1071/4832/3760/29.3661
3488/10489/7000/103.27
DIXMAANH
300
81/408/326/2.6333
1319/3974/2654/17.820
900
1476/6413/4936/76.60
3462/10409/6946/121.09
DIXMAANL
3
16/76/59/0.7726
52/164/111/1.1245
45
3023/13115/10091/57.84
7031/21130/14098/85.66
Raydan 2
500
4/17/12/0.5985
2/9/6/0.5584
1000
5/21/15/0.6352
2/9/6/0.5593
Perturbed Quadratic
1000
187/749/561/3.9731
187/749/561/3.7987
5000
425/1701/1275/11.039
425/1701/1275/10.9626
A Quadratic Function
2
6/26/19/0.6334
6/26/19/0.6101
100
91/396/304/2.2252
96/417/320/2.421
TRIDIA
500
240/961/720/5.146
240/961/720/5.5388
1000
349/1397/1047/7.1665
349/1397/1047/7.090
Zett1
2
12/52/39/0.6578
14/61/46/0.6748
Six hump
2
7/31/23/0.6482
8/36/27/0.6714
Booth
2
2/9/6/0.5722
2/9/6/0.5513
Three hump
2
6/28/21/0.5959
7/31/23/0.6031
EG2
2
4/20/15/0.6011
5/26/20/1.3894The performance profiles are adopted by Dolan and Moré [40]. In Figures 1–4 we compare the performance based on the NI/NF/GE/CT. For every method, we plot fraction P of problems for which the method is within a factor t of best time. The left hand side of the figure represents the percentage of the test problem of which method is robust and the fastest; the right hand side of the figure shows the percentage of test problems that are solved successfully by either the BZAU or TMPRP1 method. In the graph there are two axes x and y, as there are much values of x which creates difficulty in understanding the graph. The value of x-axis is then converted in natural log of t so it shows x-axis values in exponent like e0,e1,e2, e3 and the values of y-axis are taken in linear form of P. Comparing Figures 1–4 shows that the BZAU method outperforms the TMPRP1 method in every case. The top curve is the most efficient method, so the new modified three-term CG method is also efficient in terms of numerical result.Figure 1
Performance profiles based on the number of iterations.Figure 2
Performance profiles based on the CPU time.Figure 3
Performance profiles based on number of gradient evaluations.Figure 4
Performance profiles based on number of function evaluations.


## 6. Conclusion

In this paper, we have proposed a new modified three-term conjugate gradient method for unconstrained optimization. The new modified three-term BZAU method possesses the sufficient descent property independent of any line search. Global convergence is shown for both convex and nonconvex functions using the Wolfe line search. In numerical result we compare the three-term BZAU method with TMPRP1 method [32]. As in [32] the TMPRP1 method is shown to be numerically efficient when it comes to comparison with other two robust methods such as CG_Descent method [41] and DTPRP method [42]. That is the reason for comparing our BZAU method with TMPRP1 method.



---

*Source: 2715854-2017-09-13.xml*