# New Investigation for the Liu-Story Scaled Conjugate Gradient Method for Nonlinear Optimization

Authors: Eman T. Hamed; Rana Z. Al-Kawaz; Abbas Y. Al-Bayati
Journal: Journal of Mathematics
 (2020)

---

## Abstract

This article considers modified formulas for the standard conjugate gradient (CG) technique that is planned by Li and Fukushima. A new scalar parameterθkNew for this CG technique of unconstrained optimization is planned. The descent condition and global convergent property are established below using strong Wolfe conditions. Our numerical experiments show that the new proposed algorithms are more stable and economic as compared to some well-known standard CG methods.


---

## Body

## 1. Introduction

Conjugate gradient (CG) strategies consists of a category of nonlinear optimization algorithms, which needs low memory and powerful local and global convergence properties [1,2]. Typically, a CG method is meant to resolve massive scaled nonlinear optimization problem:(1)minimizex∈Rnfx.On the understanding that the function is defined in the formf:Rn⟶R is smooth nonlinear function. The repetitive formula is in the form(2)xk+1=xk+αkdk.The most important component of this formula isαk step-size, and the search direction dk consists of(3)dk+1=−gk+1,fork=0,−gk+1+βkdk,fork≥1,whereas gk=gxk denotes ∇fxk and βk denotes a positive scalar. The step-size αk is sometimes chosen to satisfy bound line search condition [3]. Among these search direction conditions, the strong Wolfe line search condition is sometimes outlined as follows:(4)fxk+αkdk−fxk≤δαkgkTdk,(5)gxk+αkdkTdk≤−σgkTdk,and 0<δ<σ<1. There are many different formulas for conjugate coefficients as in the following sources, e.g., Hestenes and Stiefel, HS [4]; Fletcher and Reeves, FR [5]; Polak and Ribière, PR [6]; Conjugate Descent, CD [7]; Li and Fukushima, LF [1]; and Liu and Story, LS [8], correspond to different choice for the scalar parameter βk.


## 2. A New Scalar Formula for the ParameterθkNew

Here in this part of this article, we proposed a new version for the parameterθk by relying on the modified BFGS method proposed by Li and Fukushima [1]. In the BFGS method, the matrix Bk+1 is updated to the following formula [9]:(6)Bk+1=Bk−BkskskTBkskTBksk+ykykTskTyk,where yk=gk+1−gkandsk=xk+1−xk. In addition, the normal secant relation is outlined consistent with the subsequent formula:(7)Bk+1sk=yk.The researchers Li and Fukushima presented an appropriate modified BFGS technique which is globally and super-linearly convergent, even though while not requiring convex objective functions. The subsequent modified secant equation is outlined consistent with the subsequent formula as follows:(8)Bk+1sk=y˜k,where(9)y˜k=yk+hkgkrsk,and r > 0; hk > 0, hk is a parameter defined as(10)hk=c+max−skTyksk2,0gkr.Specifically, take valuec is constant, and it is greater than zero.There are three different cases for the termskTyk:Case 1: ifskTyk≤0,in this case we have the problem of the nonpositive definite matrix, so Li and Fukushima proposed y˜k formula as in (9) and developed the corresponding BFGS formula as follows:(11)Bk+1=Bk−BkskskTBkskTBksk+y˜ky˜kTskTy˜k.Moreover, the form ofhk in (10), when max is used so that the value (0; zero), is not selected in this case. Through this formula, the researchers proved that the modified symmetric matrix is positive definite [10].Case 2: ifskTyk>0, in this case, we can say surely that the BFGS update matrix is symmetric and positive definite when applied within this formula (in other words, when applying the inequality skTyk>0 in the formula hk, max = 0) [11].If we useβkLSof Liu and Story (LS), we use any scalar θk; then, (3) becomes(12)dk+1=−θkgk+1+βkLSsk,where(13)βkLS=−gk+1TykskTgk.When any positive value tok is greater than one, the new parallel search direction dk+1 provided in equation (12) is the Newton direction. Hence, Newton’s direction is(14)−Gk+1−1gk+1=−θkgk+1+βkLSsk,−Gk+1−1gk+1=−θkgk+1−gk+1TykskTgksk.Hence,(15)θk=skTgk+1skTgk−gk+1TykskTGk+1skskTgkskTGk+1gk+1.Using equation (8), the new scalar θkNew becomes(16)θkNew=skTgk+1skTgk−gk+1Tyky˜kTskskTgky˜kTgk+1.By substituting equations (9) and (10) and by taking max=0 (because we use the strong Wolfe line search in equation (10), yields hk=c. Therefore, the new scalar within the new search direction is(17)θkNew=skTgk+1+βkLSykTsk−cgkrsk2ykTgk+1+cgkrskTgk+1.Hence, we conclude from equation (17) that the new parameter θkNew is best because it is up to date to find the value of y, and also we find different forms when changing the value of c as we will notice in the section of numerical results.


## 3. New Theorem (Sufficient Descent Direction)

If we presume that the line search satisfies conditions (4) and (5), then the new search direction which is generated from equations (12) and (17) could be a sufficient descent direction.Proof.
From equations (12) and (17) we obtained(18)dk+1=−θkNewgk+1−gk+1TykskTgksk=−skTgk+1+βkLSykTsk−cgkrsk2ykTgk+1+cgkrskTgk+1gk+1−gk+1TykskTgksk,dk+1Tgk+1=−skTgk+1skTgk−gk+1TykykTsk−cgkrsk2skTgkykTgk+1+cgkrskTgk+1.gk+12−ykTgk+1skTgkskTgk+1≤−ykTskskTgk−gk+12ykTsk−cgkrsk2skTgk0.2gk+12+cgkrykTsk.gk+12−ykTskskTgkskTgk+1.
By using Powell restart equation (i.e.,gkTgk+1≥0.2gk+12),(19)dk+1Tgk+1≤−ykTskskTgk−gk+12ykTsk−cgkrsk2skTgk0.2gk+12+cgkrykTsk.gk+12−0.2ykTskskTgkgk+12,dk+1Tgk+1≤−ykTskskTgk−gk+12ykTsk−cgkrsk2+0.2ykTsk0.2gk+12+ykTskcgkr1skTgk0.2gk+12+cgkrykTskgk+12.
IfskTgk0.2gk+12+cgkrskTyk > 0, the next inequality is true:(20)dk+1Tgk+1≤−ykTskskTgk−gk+12ykTsk+cgkrsk2gk+12+0.04ykTskgk+12+0.2yk2sk2cgkr1skTgk0.2gk+12+cgkrykTsk.gk+12≤−ykTskskTgk−0.96gk+12ykTsk+cgkrsk2.0.2yk2+gk+12skTgk0.2gk+12+cgkrykTsk.gk+12.
Using strong Wolfe line search condition (5a) yields(21)skTyk=skTgk+1−skTgk≥σskTgk−skTgk≥−1−σskTgk,dk+1Tgk+1≤−skTgkskTgk0.2gk+12+cgkrykTskgk+12−1−σskTgk+0.961−σgk+12+cgkr−1sk0.2yk2+gk+12.
This latter equation implies that(22)dk+1Tgk+1≤−υgk+12.
Thus, our requirement is complete.


### 3.1. Outlines of the New CG-Algorithms

Step 1: select the initial pointx0∈Rn,∈>0, and select some positive values for δ and σ. Then, set d0=−∇fx0 and set k=0.
Step 2: test for stopping criterion. If satisfied, then stop; otherwise, continue.
Step 3: determineαk by Wolfe conditions, which are defined in equations (4) and (5).
Step 4: compute the second iterative pointxk+1 from equation (2).
Step 5: calculate the scalar parameterθkNew from equation (17).
Step 6: calculate the new search directions, namely,dk+1=−θkNewgk+1+βkLSsk
Step 7: test Powell restarting criterion, namely, ifgk+1Tgk≥0.2gk+12, then restart the new search direction with −gk+1.
Step 8: set the next iterationk = k + 1, and go to Step 2.


## 3.1. Outlines of the New CG-Algorithms

Step 1: select the initial pointx0∈Rn,∈>0, and select some positive values for δ and σ. Then, set d0=−∇fx0 and set k=0.
Step 2: test for stopping criterion. If satisfied, then stop; otherwise, continue.
Step 3: determineαk by Wolfe conditions, which are defined in equations (4) and (5).
Step 4: compute the second iterative pointxk+1 from equation (2).
Step 5: calculate the scalar parameterθkNew from equation (17).
Step 6: calculate the new search directions, namely,dk+1=−θkNewgk+1+βkLSsk
Step 7: test Powell restarting criterion, namely, ifgk+1Tgk≥0.2gk+12, then restart the new search direction with −gk+1.
Step 8: set the next iterationk = k + 1, and go to Step 2.


## 4. Convergence Analysis for the New Proposed Algorithm

In the following parts, we have a tendency to discuss the convergence analysis property for the new algorithm thoroughly. First, we offer an assumption for the convergence analysis property for the new algorithms. Then, we offer another well-known lemma needed within the study of convergence analysis property. Finally, we have a tendency to set new theorems aboard their proofs that area unit associated with the convergence analysis for the new algorithm.Assumption.
(i) The level set(23)S=x:fx≤fx0,is bounded, that is, there exists a constantz > 0, such as [12](24)x≤z,∀x∈S.(ii)
In neighbourhood N of S,f is continuously differentiable, and its gradient is Lipschitz continuous, that is, there exists a constant L > 0, such as(25)gx−gy≤Lx−y,∀x,y∈N.
From the assumptions (i) and (ii) onf, we are able to deduce that there exists γ > 0 such as(26)γ≤∇fx≤γ¯.Lemma.
If we suppose that [3,13](1)
Assumption holds.(2)
Search directiondk+1 in the standard CG method is a descent direction.(3)
Optimal stepαk is calculated by equations (4) and (5).(4)
The convergence condition is satisfied, i.e., if(27)∑k≥11dk+12=∞.
Then,(28)limk⟶∞infgk=0″.


## 5. New Theorem (Uniformly Convex Function)

If we suppose that(1)
Assumption holds.(2)
The new search directiondk+1 defined by equations (12) and (17) is a descent direction.(3)
The optimal stepαk is calculated by equations (4) and (5).(4)
The objective functionf is uniformly convex; then,(29)limk⟶∞gk=0.Proof.
Consider the new direction in equation (12) and the parameter of equation (17) satisfy the next absolute value condition:(30)θkNew=skTgk+1skTgk−gk+1Tyky˜kTskskTgky˜kTgk+1≤skTgk+1y˜kTgk+1+gk+1Tyk.y˜kTskskTgk.y˜kTgk+1,Sincey˜kTsk=ykTsk+cgkrsk2s.t.hk=c≤Lsk2+cγ¯rsk2≤cγ¯r+Lsk2,alsoy˜kTgk+1=ykTgk+1+cgkrskTgk+1≥cgkrykTsk≥cγrμsk2,θkNew≤Lsk2cγrμsk2+γ¯Lskcγ¯r+Lsk2skγcγrμsk2≤Lcγrμ+γ¯Lcγ¯r+Lcγr+1μ.
Well parameterβkLS(31)βkLS≤γ¯Lskμsk2.
Moreover, by combining the results, we obtained(32)dk+1≤θkgk+1+βkLS.sk≤Lcγr+γ¯Lcγ¯r+Lcγr+1μγ¯+γ¯Lμ=D1,∑k≥11dk+12≥1D1∑k≥11=∞.
We got the required proof. We put similar points to the previous hypotheses, but there are some variations in the formulas.


## 6. New Theorem (General Function)

If we suppose that(1)
Assumption holds.(2)
The new search directiondk+1 defined by equations (12) and (17)is a descent direction.(3)
The optimal stepαk is calculated by equations (4) and (5).(4)
The objective functionf is general function; then,(33)limk⟶∞infgk=0.Proof.
Using the same proof style of the previous theorem with the difference in the fact that the functions of the algorithm are general functions,(34)Sincey˜kTsk=ykTsk+cgkrsk2≥γsk+cγrsk2,dk+1≤θkNewgk+1+βkLS.sk≤skγ¯cγrLsk+γ¯L2sk2sk2cγr+1Lγ¯+γ¯Lsk2Lsk2≤γ¯2cγrL+γ¯2L2cγr+1L+γ¯=D2.
Then, we obtain(35)∑k≥11dk+12≥1D2∑k≥11=∞.
Therefore, the proof of the new theorems in regards to the convergence analyses of the proposed algorithms is complete.


## 7. Numerical Experiments

In this section, we have reported some numerical experiments that are performed on a set of (60) unconstrained optimization test problems to analyse the efficiency ofθkNew. Detail of these test problems, with their given initial points, can be found in [14,15]. We handled each of these (60) test functions by adding 1000 for each n to arrive at maximum number of n which is equal to 10000. The termination criterion used in our experiments is gk≤10−6,whereδ=0.01andσ=0.1.In our comparisons below, we employ the following algorithms:(i)
LS: with the Wolfe line search(ii)
CD: with the Wolfe line search(iii)
HS: with the Wolfe line search(iv)
PR: with the Wolfe line search(v)
New Algorithm, using equation (17) and the scalar c = 0.1(vi)
New Algorithm, using equation (17) and the scalar c = 0.001In Tables1 and 2, we numerically compare the proposed new CG algorithms against other well-known CG algorithms to verify their performance using the known comparison tools for such algorithms which are as follows:NOI = the total number of calculated iterative iterationsNOFG = the total number of function and gradient calculationsTIME = the total CPU time required for the processor to execute the CG algorithm and reach the minimum value of the required function minimizationTable 1
Comparisons of new algorithm against (LS) & (CD) algorithms for the total of (60) test problems with (1000≤ n ≤ 10000, and the increasing size in n is equal to 1000) with (c = 0.1 and 0.001).
Prob.
New algorithm (C = 0.1) NOI/NOFG/TIME
New algorithm (C = 0.001) NOI/NOFG/TIME
LSNOI/NOFG/TIME
CDNOI/NOFG/TIME
1
99/252/0.14
101/263/0.14
323/7045/2.31
414/9608/2.64
2
408/884/2.02
405/880/1.97
412/888/1.98
395/878/2.08
3
853/2183/1.06
830/2171/0.98
823/2159/1.02
824/2154/0.97
4
123/308/0.14
123/308/0.15
114/300/0.17
121/314/0.15
5
100/388/0.15
100/388/0.20
274/5586/1.14
167/1918/0.77
6
585/995/1.56
585/995/1.54
6576/19689/4.54
6848/20496/5.70
7
30/80/0.15
30/80/0.13
40/100/0.21
40/100/0.16
8
1032/2705/1.58
1020/2668/1.42
1033/2768/1.49
1026/2972/1.52
9
2388/4881/2.40
2503/5048/3.06
3520/7182/7.49
3676/7485/8.83
10
477/871/2.66
478/858/2.62
18795/503441/7.60
19346/518146/9.14
11
182/423/1.03
174/418/1.09
8669/270951/5.83
8252/257171/6.83
12
113/302/0.16
113/302/0.20
318/3881/1.69
279/2792/5.80
13
80/226/0.11
78/222/0.08
73/241/0.10
73/241/0.09
14
61/131/0.50
61/131/0.56
124/2140/7.12
160/2833/3.54
15
460/991/0.62
452/969/0.59
456/953/0.61
467/997/0.64
16
66/132/0.03
66/132/0.04
67/134/0.14
68/136/0.06
17
70/160/0.11
70/160/0.06
69/158/0.10
71/162/0.08
18
753/1577/0.79
791/1668/0.85
691/1477/0.74
782/1687/0.82
19
74/158/0.33
74/158/0.42
123/1905/2.31
70/150/0.30
20
110/349/0.41
109/349/0.37
114/334/0.43
111/329/0.41
21
806/3224/1.60
600/1665/1.26
429/1895/4.93
561/5284/5.69
22
72/275/0.51
72/285/0.49
84/366/0.70
2084/2322/9.54
23
4470/9572/1.33
5033/10748/2.42
20010/168275/4.95
19334/128670/5.02
24
62/201/0.42
62/201/0.40
546/15413/3.73
495/13595/3.48
25
459/1091/0.60
521/1192/0.78
494/1101/0.65
531/1154/0.76
26
56/153/0.08
65/373/0.13
56/155/0.06
66/328/0.12
27
85/203/0.11
85/203/0.12
80/190/0.07
80/190/0.09
28
534/1139/0.65
514/1073/0.68
493/1055/0.63
492/1075/0.61
29
540/1274/0.62
537/1267/0.58
941/2237/1.03
943/2239/1.11
30
576/1440/0.96
591/1504/1.05
19026/581537/2.57
18144/543276/2.03
31
113/236/0.16
113/236/0.11
118/245/0.11
123/253/0.11
32
813/2181/3.28
735/1997/3.03
18140/560188/8.67
18320/564064/8.04
33
98/268/0.18
98/268/0.17
359/5355/2.21
387/6200/3.06
34
346/766/0.48
348/766/0.50
356/786/0.44
332/732/0.44
35
7635/12820/8.59
7554/12715/8.73
7463/12576/8.29
7183/12051/8.37
36
280/978/0.43
280/978/0.35
264/935/0.45
273/978/0.39
37
217/534/0.31
217/534/0.27
221/551/0.31
219/546/0.31
38
121/287/0.14
120/285/0.13
814/20969/8.83
786/20117/7.46
39
150/329/0.64
153/328/0.65
141/305/0.59
137/297/0.58
40
107/217/0.65
107/217/0.68
144/1165/3.67
124/476/1.35
41
120/330/0.23
120/330/0.25
100/290/0.17
100/290/0.18
42
3832/9998/2.51
3373/8928/7.18
3404/9046/8.47
3182/8618/7.82
43
40/80/0.11
40/80/0.08
40/80/0.12
40/80/0.12
44
50/110/0.10
50/110/0.05
50/110/0.09
50/110/0.05
45
43/184/0.08
43/184/0.11
3903/129109/7.10
7265/242193/4.78
46
427/1323/2.56
427/1320/2.60
413/1307/2.53
428/1252/2.44
47
64/249/0.09
64/249/0.06
118/447/0.22
114/425/0.19
48
308/798/1.39
293/772/1.28
773/7349/0.72
1731/23050/5.88
49
22/89/0.11
22/89/0.13
316/8561/9.47
298/8022/8.33
50
20/50/0.03
20/50/0.02
20/50/0.00
20/50/0.04
51
92/1755/2.25
42/136/0.17
361/9646/9.02
315/8202/8.61
52
107/418/0.16
107/418/0.15
107/418/0.16
107/418/0.16
53
6199/52426/6.75
6199/52426/6.78
8071/53291/6.20
8071/53291/9.30
54
51/151/0.20
51/151/0.19
51/151/0.21
51/151/0.18
55
60/140/0.19
60/140/0.19
60/140/0.18
60/140/0.20
56
70/140/0.20
70/140/0.17
70/140/0.19
70/140/0.16
57
79/158/0.22
79/158/0.22
79/158/0.24
74/148/0.22
58
143/570/0.25
143/570/0.25
143/570/0.23
143/570/0.21
59
188/498/0.29
172/453/0.27
176/449/0.27
177/461/0.22
60
83/236/0.13
83/236/0.11
985/27409/9.27
917/25010/8.38
Total
37602/124886/55.54
37426/121643/59.2
131933/2455353/154.98
136949/1242169/166.56Table 2
Comparisons of new algorithm against (HS) & (PR) algorithms for the total of (60) test problems with (1000≤ n ≤ 10000, and the increasing size in n is equal to 1000) with (c = 0.1 and 0.001).
Prob.
New algorithm (C = 0.1) NOI/NOFG/TIME
New algorithm (C = 0.001) NOI/NOFG/TIME
HSNOI/NOFG/TIME
PRNOI/NOFG/TIME
1
99/252/0.14
101/263/0.14
5902/173484/0.72
11798/270815/6.07
2
408/884/2.02
405/880/1.97
362/637/1.83
416/720/2.07
3
853/2183/1.06
830/2171/0.98
789/1817/0.86
989/1979/1.11
4
123/308/0.14
123/308/0.15
141/281/0.19
254/430/0.26
5
100/388/0.15
100/388/0.20
653/17073/3.79
410/7948/2.73
6
585/995/1.56
585/995/1.54
9881/16243/3.07
20010/22091/4.04
7
30/80/0.15
30/80/0.13
40/90/0.18
40/90/0.18
8
1032/2705/1.58
1020/2668/1.42
997/2279/1.28
8780/10109/1.57
9
2388/4881/2.40
2503/5048/3.06
4658/7644/2.48
14945/16013/9.92
10
477/871/2.66
478/858/2.62
20010/98744/4.70
20010/292528/9.39
11
182/423/1.03
174/418/1.09
14053/39805/1.28
15593/489487/8.58
12
113/302/0.16
113/302/0.20
428/6427/2.19
609/11752/4.81
13
80/226/0.11
78/222/0.08
113/234/0.08
291/509/0.26
14
61/131/0.50
61/131/0.56
906/23792/1.66
318/6032/3.76
15
460/991/0.62
452/969/0.59
636/1006/0.73
964/1479/1.25
16
66/132/0.03
66/132/0.04
60/120/0.03
1043/1116/0.38
17
70/160/0.11
70/160/0.06
207/339/0.19
110/230/0.10
18
753/1577/0.79
791/1668/0.85
821/1545/0.78
3732/4630/3.99
19
74/158/0.33
74/158/0.42
108/1352/2.44
303/7194/2.66
20
110/349/0.41
109/349/0.37
135/321/0.42
154/339/0.45
21
806/3224/1.60
600/1665/1.26
875/14122/5.89
929/10442/9.02
22
72/275/0.51
72/285/0.49
2104/2442/3.61
161/440/0.92
23
4470/9572/1.33
5033/10748/2.42
18912/38658/8.87
20010/25808/6.95
24
62/201/0.42
62/201/0.40
1853/6983/3.39
2527/76854/9.01
25
459/1091/0.60
521/1192/0.78
304/606/0.40
1199/1793/1.55
26
56/153/0.08
65/373/0.13
128/1103/0.37
2697/14700/7.09
27
85/203/0.11
85/203/0.12
91/193/0.13
132/264/0.16
28
534/1139/0.65
514/1073/0.68
288/558/0.35
556/925/0.60
29
540/1274/0.62
537/1267/0.58
852/1783/0.98
1014/2180/1.14
30
576/1440/0.96
591/1504/1.05
20010/98171/6.44
20010/317766/8.02
31
113/236/0.16
113/236/0.11
79/168/0.10
147/287/0.13
32
813/2181/3.28
735/1997/3.03
20010/91480/9.47
20010/179051/5.16
33
98/268/0.18
98/268/0.17
631/11069/5.15
837/17999/8.99
34
346/766/0.48
348/766/0.50
716/1148/0.90
744/1213/0.93
35
7635/12820/8.59
7554/12715/8.73
8375/13146/9.98
8539/12513/15.46
36
280/978/0.43
280/978/0.35
330/695/0.37
401/868/0.47
37
217/534/0.31
217/534/0.27
610/6778/2.40
820/11112/5.50
38
121/287/0.14
120/285/0.13
1565/42467/4.89
1624/45918/6.43
39
150/329/0.64
153/328/0.65
174/290/0.54
193/323/0.69
40
107/217/0.65
107/217/0.68
253/426/1.03
4281/4461/11.17
41
120/330/0.23
120/330/0.25
118/286/0.19
124/298/0.20
42
3832/9998/2.51
3373/8928/7.18
3685/8619/8.30
17419/22467/5.83
43
40/80/0.11
40/80/0.08
99/119/0.19
99/119/0.19
44
50/110/0.10
50/110/0.05
50/110/0.02
70/282/0.13
45
43/184/0.08
43/184/0.11
12047/91618/9.93
15601/521624/4.14
46
427/1323/2.56
427/1320/2.60
409/1040/2.07
597/1232/2.51
47
64/249/0.09
64/249/0.06
133/380/0.15
143/393/0.40
48
308/798/1.39
293/772/1.28
2549/27198/2.61
15662/156182/6.10
49
22/89/0.11
22/89/0.13
1377/37674/8.58
1571/43940/8.14
50
20/50/0.03
20/50/0.02
20/50/0.02
20/50/0.03
51
92/1755/2.25
42/136/0.17
1620/5439/3.43
1680/46332/4.04
52
107/418/0.16
107/418/0.15
125/360/0.14
125/360/0.16
53
6199/52426/6.75
6199/52426/6.78
4160/9534/3.23
2233/8070/9.66
54
51/151/0.20
51/151/0.19
75/173/0.25
77/144/0.20
55
60/140/0.19
60/140/0.19
60/120/0.14
60/120/0.17
56
70/140/0.20
70/140/0.17
70/140/0.24
80/160/0.22
57
79/158/0.22
79/158/0.22
79/158/0.24
86/172/0.23
58
143/570/0.25
143/570/0.25
177/506/0.25
177/506/0.25
59
188/498/0.29
172/453/0.27
209/460/0.30
641/917/1.01
60
83/236/0.13
83/236/0.11
1615/43549/5.17
2005/57225/7.75
Total
37602/124896/60.1
37426/121943/70.14
167737/10741186/139.61
271227/7136001/214.33Therefore, among these CG algorithms, the new algorithm appears to generate the best search direction. In Table3, there is a clear evidence that the new algorithm outperforms the standard LS and CD algorithms detailed as follows (when c = 0.1):(a)
For 100% LS algorithm: the new algorithm is improved by (71.5%) NOI, improved by (94.92%) NOFG, and improved by (64.2%) time(b)
For 100% CD algorithm: the new algorithm is improved by (72.6%) NOI, improved by (89.95%) NOFG, and improved by (66.7%) timeTable 3
Standardizing perceptual of the new algorithm vs. LS and CD algorithms.
Algorithm
Tools
LS (1991) (%)
CD (1987) (%)
Whenc = 0.1
NOI
28.5
27.4
NOFG
5.08
10.05
TIME
35.8
33.3
Whenc = 0.001
NOI
28.3
27.3
NOFG
4.9
9.7
TIME
38.1
35.5And (whenc = 0.001):(c)
For 100% LS algorithm: the new algorithm is improved by (64.34%) NOI, improved by (16.99%) NOFG, and improved by (16.08%) time(d)
For 100% CD algorithm: the new algorithm is improved by (52.60%) NOI, improved by (14.63%) NOFG, and improved by (12.75%) timeIn Table4, there is a clear evident that the new algorithm outperforms the standard HS and PR algorithms as detailed below (when c = 0.1):(e)
For 100% HS algorithm: the new algorithm is improved by (77.6%) NOI, improved by (98.9%) NOFG, and improved by (57%) time(f)
For 100% PR algorithm: the new algorithm is improved by (86.2%) NOI, improved by (98.3%) NOFG, and improved by (72%) timeand (whenc = 0.001)(g)
For 100% HS algorithm: the new algorithm is improved by (77.7%) NOI, improved by (98.9%) NOFG, and improved by (49.8%) time(h)
For 100% PR algorithm: the new algorithm is improved by (86.3%) NOI, improved by (98.3%) NOFG, and improved by (67.3%) timeTable 4
Standardizing perceptual of the new algorithm vs. HS and PR algorithms.
Algorithm
Tools
HS (1952) (%)
PR (1969) (%)
Whenc = 0.1
NOI
22.4
13.8
NOFG
1.1
1.7
TIME
43
28
Whenc = 0.001
NOI
22.3
13.7
NOFG
1.1
1.7
TIME
50.2
32.7What can be deduced from the above tables and experiments are summarized in the following:(i)
Points (a to d) above are that our new proposed algorithms in the field of CG-type methods are economic and robust as compared to the standard LS and CD algorithms(ii)
The abovementioned points (e to h) are that our new proposed algorithms in the field of CG-type methods are economic and robust as compared to the standard HS and PR algorithmsAll these comparisons were made using the performance profile of Dolan and Moré [16], and we can conclude that(1)
Figure1 illustrates the new algorithm versus (LS, CD, HS, and PR) the activity of the new algorithms in calculating the number of iterations(2)
Figure2 explains the new algorithm versus (LS, CD, HS, and PR) the activity of the new algorithms in calculating the number of function and gradient evaluations(3)
Figure3 displays how long the algorithms take to reach the solution (i.e., the required CPU time)(4)
Figure4 shows the functions that perform well in the new algorithm with two different constants compared to the basic algorithms (LS, CD, HS, and PR) based on the number of iterations(5)
Figure5 demonstrates the outstanding performance of a number of functions in the new algorithm with two different constants compared to basic algorithms (LS, CD, HS, and PR) based on the number of function and gradient evaluationsFigure 1
Performance profiles based on number of iterations (NOI). (a) Comparison of the new algorithm vs. LS and CD. (b) Comparison of the new algorithm vs. HS and PR.
(a)
(b)Figure 2
Performance profiles based on function and gradient evaluations (NOFG). (a) Comparison of the new algorithm vs. LS and CD. (b) Comparison of the new algorithm vs. HS and PR.
(a)
(b)Figure 3
Performance profiles based on CPU Time. (a) Comparison of the new algorithm vs. LS and CD. (b) Comparison of the new algorithm vs. HS and PR.
(a)
(b)Figure 4
Performance profiles based on the best NOI. (a) Comparison of the new algorithm vs. LS and CD. (b) Comparison of the new algorithm vs. HS and PR.
(a)
(b)Figure 5
Performance profiles based on the best NOFG. (a) Comparison of the new algorithm vs. LS and CD. (b) Comparison of the new algorithm vs. HS and PR.
(a)
(b)


## 8. Conclusions

In this study, we have submitted two proposed new CG methods (by changing the value ofc). A crucial property of proposed CG methods is that it secures sufficient descent directions. Under mild conditions, we have demonstrated that the new algorithms are globally convergent for each uniformly convex and general functions using the strong Wolfe line search conditions. The preliminary numerical results show that if we decide a good value of parameter c, the new algorithms perform very well. However, an optimal value of the parameter c can be handled theoretically (in future research studies) to achieve more best numerical results.



---

*Source: 3615208-2020-01-25.xml*