# Variant Gradient Projection Methods for the Minimization Problems

Authors: Yonghong Yao; Yeong-Cheng Liou; Ching-Feng Wen
Journal: Abstract and Applied Analysis
 (2012)

---

## Abstract

The gradient projection algorithm plays an important role in solving constrained convex minimization problems. In general, the gradient projection algorithm has only weak convergence in infinite-dimensional Hilbert spaces. Recently, H. K. Xu (2011) provided two modified gradient projection algorithms which have strong convergence. Motivated by Xu’s work, in the present paper, we suggest three more simpler variant gradient projection methods so that strong convergence is guaranteed.


---

## Body

## 1. Introduction

LetH be a real Hilbert space and C a nonempty closed and convex subset of H. Let f:H→R  be a real-valued convex function. Now we consider the following constrained convex minimization problem:
(1.1)min⁡x∈Cf(x).
Assume that (1.1) is consistent; that is, it has a solution and we use S to denote its solution set. If f is Fréchet differentiable, then x*∈C solves (1.1) if and only if x*∈C satisfies the following optimality condition:
(1.2)〈∇f(x*),x-x*〉≥0,∀x∈C,
where ∇f denotes the gradient of f. Note that (1.2) can be rewritten as
(1.3)〈x*-(x*-∇f(x*)),x-x*〉≥0,∀x∈C.
This shows that the minimization (1.1) is equivalent to the fixed-point problem:
(1.4)ProjC(x*-γ∇f(x*))=x*,
where γ>0 is an any constant and ProjC is the nearest point projection from H onto C. By using this relationship, the gradient-projection algorithm is usually applied to solve the minimization problem (1.1). This algorithm generates a sequence {xn} through the recursion:
(1.5)xn+1=ProjC(xn-γn∇f(xn)),n≥0,
where the initial guess x0∈C is chosen arbitrarily and {γn} is a sequence of step sizes which may be chosen in different ways. The gradient-projection algorithm (1.5) is a powerful tool for solving constrained convex optimization problems and has well been studied in the case of constant stepsizes γn=γ for all n. The reader can refer to [1–9]. It has recently been applied to solve split feasibility problems which find applications in image reconstructions and the intensity modulated radiation therapy (see [10–17]).It is known [3] that if f has a Lipschitz continuous and strongly monotone gradient, then the sequence {xn} can be strongly convergent to a minimizer of f in C. If the gradient of f is only assumed to be Lipschitz continuous, then {xn} can only be weakly convergent if H is infinite dimensional. This gives naturally rise to a question.Question 1.
How to appropriately modify the gradient projection algorithm so as to have strong convergence?For this purpose, recently, Xu [18] first introduced the following modification:
(1.6)xn+1=θnh(xn)+(1-θn)ProjC(xn-γn∇f(xn)),n≥0,
where the sequences {θn}⊂(0,1) and {γn}⊂(0,∞) satisfy the following conditions: (i)
limn→∞θn=0, ∑n=0∞‍θn=∞ and ∑n=0∞‍|θn+1-θn|<∞;(ii)
0<liminfn→∞γn≤limsupn→∞γn<2/L and ∑n=0∞‍|γn+1-γn|<∞. Xu [18] proved that the sequence {xn} converges strongly to a minimizer of (1.1).Remark 1.1.
Xu's modification (1.6) is a convex combination of the gradient-projection algorithm (1.5) and a self-mapping h(x) which is usually referred as a so-called viscosity item.In [18], Xu presented another modification as follows:
(1.7)yn=ProjC(xn-γn∇f(xn)),n≥0,Cn={z∈C:‖yn-z‖≤‖xn-z‖},Qn={z∈C:〈xn-z,x0-xn〉≥0},xn+1=ProjCn∩Qnx0.
Consequently, Xu [18] proved that Algorithm (1.7) also converges strongly to x* which solves the minimization problem (1.1).Remark 1.2.
Equation (1.7) involved in additional projections which couple the gradient projection method (1.5) with the so-called CQ method.It should be pointed out that Xu's modifications (1.6) and (1.7) are interesting and provide us with a direction for solving (1.1) in infinite-dimensional Hilbert spaces.Motivated by Xu's work, in the present paper, we suggest three variant gradient projection methods so that strong convergence is guaranteed for solving (1.1) in infinite-dimensional Hilbert spaces. Our motivations are mainly in the two respects.Reason 1. The solution of the minimization problem (1.1) is not always unique, so that there may be many solutions to the problem. In that case, a special solution (e.g., the minimum norm solution) must be found from among candidate solutions. The minimum norm problem is motivated by the following least squares solution to the constrained linear inverse problem:
(1.8)Bx=b,x∈Ω,
where Ω is a nonempty closed convex subset of a real Hilbert space H, B is a bounded linear operator from H to another real Hilbert space H1, B* is the adjoint of B, and b is a given point in H1. The least-squares solution to (1.8) is the least-norm minimizer of the minimization problem:
(1.9)min⁡x∈Ω‖Bx-b‖.
For some related works, please see  Solodov and Svaiter  [19], Goebel and Kirk [20], and  Martinez-Yanes and Xu [21].Reason 2. Projection methods are used extensively in a variety of methods in optimization theory. Apart from theoretical interest, the main advantage of projection methods, which makes them successful in real-world applications, is computational (see [22–31]). In this respect, (1.7) is particularly useful. But we observe that (1.7) involves two half-spaces Cn and Qn. If the sets Cn and Qn are simple enough, then PCn and PQn are easily executed. But Cn∩Qn may be complicate, so that the projection PCn∩Qn is not easily executed. This might seriously affect the efficiency of the method. Hence, it is interesting that one can relax Cn or Qn from (1.7).In the present paper, we suggest the following three methods:(1.10)xn+1=ProjC(θnh(x)+(1-θn)xn-γn∇f(xn)),n≥0,xn+1=ProjC(I-γ∇f)ProjC((1-θn)xn),n≥0,(1.11)yn=ProjC(xn-γn∇f(xn)),n≥0,Cn={z∈Cn-1:‖yn-z‖≤‖xn-z‖},xn+1=ProjCnx0.
We will show that (1.10) can be used to find the minimum norm solution of the minimization problem (1.1), and (1.11) which is only involved in Cn also has strong convergence.


## 2. Preliminaries

LetC be a nonempty closed convex subset of a real Hilbert space H. A mapping T:C→C is called nonexpansive if
(2.1)‖Tx-Ty‖≤‖x-y‖,∀x,y∈C.
Recall that the (nearest point or metric) projection from H onto C, denoted ProjC, assigns, to each x∈H, the unique point ProjC(x)∈C with the property
(2.2)‖x-ProjC(x)‖=inf⁡{‖x-y‖:y∈C}.
It is well known that the metric projection ProjC of H onto C has the following basic properties:  (i)
∥ProjC(x)-ProjC(y)∥≤∥x-y∥, for all x,y∈H;(ii)
〈x-y,ProjC(x)-ProjC(y)〉≥∥ProjC(x)-ProjC(y)∥2, for every x,y∈H;(iii)
〈x-ProjC(x),y-ProjC(x)〉≤0, for all x∈H, y∈C. Next we adopt the following notation: (i)
xn→x means that xn converges strongly to x;(ii)
xn⇀x means that xn converges weakly to x;(iii)
ωw(xn):={x:∃xnj⇀x} is the weak ω-limit set of the sequence {xn}.Lemma 2.1 (see [32] (Demiclosedness Principle)).
LetC be a closed and convex subset of a Hilbert space H, and let T:C→C be a nonexpansive mapping with Fix(T)≠∅. If {xn} is a sequence in C weakly converging to x and if {(I-T)xn} converges strongly to y, then
(2.3)(I-T)x=y.
In particular, ify=0, then x∈Fix(T).Lemma 2.2 (see [33]).
LetC be a closed convex subset of H. Let {xn} be a sequence in H and x0∈H. If {xn} is such that ωw(xn)⊂C and satisfies the condition
(2.4)‖xn-x0‖≤‖x0-ProjC(x0)‖,∀n≥0,
then xn→ProjC(x0).Lemma 2.3 (see [34]).
Assume{an} is a sequence of nonnegative real numbers such that
(2.5)an+1≤(1-γn)an+δn,
where {γn} is a sequence in (0,1) and {δn} is a sequence such that(1)
∑n=1∞‍γn=∞;(2)
limsupn→∞δn/γn≤0 or ∑n=1∞‍|δn|<∞. Then limn→∞an=0.Lemma 2.4 (see [35]).
Let{xn} and {yn} be bounded sequences in a Banach space X, and let {βn} be a sequence in [0,1] with
(2.6)0<liminf⁡n→∞βn≤limsup⁡n→∞βn<1.
Suppose
(2.7)xn+1=(1-βn)yn+βnxn,
for all n≥0, and
(2.8)limsup⁡n→∞(‖yn+1-yn‖-‖xn+1-xn‖)≤0.
Then, limn→∞∥yn-xn∥=0.


## 3. Main Results

In this section, we will state and prove our main results.Theorem 3.1.
LetC be a closed convex subset of a real Hilbert space H. Let f:C→R be a real-valued Fréchet differentiable convex function. Assume that the solution set S of (1.1) is nonempty. Assume that the gradient ∇f is L-Lipschitzian. Let h:C→H be a ρ-contraction with ρ∈[0,1). Let {xn} be a sequence generated by the following hybrid gradient projection algorithm:
(3.1)xn+1=ProjC(θnh(xn)+(1-θn)xn-γn∇f(xn)),n≥0,
where the sequences {θn}⊂(0,1) and {γn}⊂(0,∞) satisfy the following conditions: (i)
limn→∞θn=0, ∑n=0∞‍θn=∞ and ∑n=0∞‍|θn+1-θn|<∞;(ii)
0<liminfn→∞γn≤limsupn→∞γn<2/L and ∑n=0∞‍|γn+1-γn|<∞. Then the sequence {xn} generated by (3.1) converges to a minimizer x^ of (1.1) which is the unique solution of the following variational inequality:
(3.2)x^∈S,〈(I-h)x^,x-x^〉≥0,x∈S.Proof.
Take anyx*∈S. Since x*∈C solves the minimization problem (1.1) if and only if x* solves the fixed-point equation, x*=ProjC(I-γ∇f)x* for any fixed positive number γ. So, we have x*=ProjC(I-γn∇f)x* for all n≥0. It can be rewritten as
(3.3)x*=ProjC(θnx*+(1-θn)(x*-γn1-θn∇f(x*))),n≥0.
From condition (ii) 0<liminfn→∞γn≤limsupn→∞γn<2/L, there exist two constants a and b such that 0<a≤γn≤b<2/L for sufficiently large n; without loss of generality, we can assume 0<a≤γn≤b<2/L for all n. Since limn→∞θn=0, without loss of generality, we can assume that 0<θn<1-bL/2 for all n≥0. So, 0<liminfn→∞(γn/(1-θn))≤limsupn→∞(γn/(1-θn))<2/L. Hence, I-(γn/(1-θn))∇f is nonexpansive.
From (3.1), we get
(3.4)‖xn+1-x*‖=‖ProjC(θnh(xn)+(1-θn)xn-γn∇f(xn))-ProjC(I-γn∇f)x*‖=‖ProjC(θnh(xn)+(1-θn)(xn-γn1-θn∇f(xn)))-ProjC(θnx*+(1-θn)(x*-γn1-θn∇f(x*)))‖≤θn‖h(xn)-x*‖+(1-θn)×‖(I-γn1-θn∇f)xn-(I-γn1-θn∇f)x*‖≤θn‖h(xn)-h(x*)‖+θn‖h(x*)-x*‖+(1-θn)‖xn-x*‖≤θnρ‖xn-x*‖+θn‖h(x*)-x*‖+(1-θn)‖xn-x*‖=(1-(1-ρ)θn)‖xn-x*‖+θn‖h(x*)-x*‖≤max⁡{‖xn-x*‖,11-ρ‖h(x*)-x*‖}.
Thus, we deduce by induction that
(3.5)‖xn-x*‖≤max⁡{‖x0-x*‖,11-ρ‖h(x*)-x*‖}.
This indicates that the sequence {xn} is bounded and so are the sequences {h(xn)} and {∇f(xn)}. Then, we can chose a constant M>0 such that
(3.6)sup⁡n≥0{‖h(xn)‖+‖xn‖+‖∇f(xn)‖}≤M.
Next, we estimate ∥xn+1-xn∥. By (3.1), we have
(3.7)‖xn+1-xn‖=‖ProjC(θnh(xn)+(1-θn)xn-γn∇f(xn))-ProjC(θn-1h(xn-1)+(1-θn-1)xn-1-γn-1∇f(xn-1))‖≤‖((1-θn)xn-γn∇f(xn))-((1-θn-1)xn-1-γn-1∇f(xn-1))‖+‖θnh(xn)-θn-1h(xn-1)‖=‖(1-θn)(I-γn1-θn∇f)xn-(1-θn)(I-γn1-θn∇f)xn-1+(θn-1-θn)xn-1+(γn-1-γn)∇f(xn-1)γn1-θn‖+‖θn(h(xn)-h(xn-1))+(θn-θn-1)h(xn-1)‖≤(1-θn)‖(I-γn1-θn∇f)xn-(I-γn1-θn∇f)xn-1‖+|θn-θn-1|‖xn-1‖+|γn-γn-1|‖∇f(xn-1)‖+θn‖h(xn)-h(xn-1)‖+|θn-θn-1|‖h(xn-1)‖≤(1-(1-ρ)θn)‖xn-xn-1‖+|θn-θn-1|(‖h(xn-1)‖+‖xn-1‖)+|γn-γn-1|‖∇f(xn-1)‖≤(1-(1-ρ)θn)‖xn-xn-1‖+(|γn-γn-1|+|θn-θn-1|)M.
Then, we can combine the last inequality and Lemma 2.3 to conclude that
(3.8)lim⁡n→∞‖xn+1-xn‖=0.
Now we show that the weak limit set ωw(xn)⊂S. Choose any x~∈ωw(xn). Then there must exist a subsequence {xnj} of {xn} such that xnj⇀x~. At the same time, the real number sequence {γnj} is bounded. Thus, there exists a subsequence {γnji} of {γnj} which converges to γ. Without loss of generality, we may assume that γnj→γ. Note that 0<liminfn→∞γn≤limsupn→∞γn<2/L. So, γ∈(0,2/L). That is, γnj→γ∈(0,2/L) as j→∞. Next, we only need to show that x~∈S. First, from (3.8) we have that ∥xnj+1-xnj∥→0. Then, we have
(3.9)‖xnj-ProjC(I-γ∇f)xnj‖≤‖xnj-xnj+1‖+‖xnj+1-ProjC(I-γnj∇f)xnj‖+‖ProjC(I-γnj∇f)xnj-ProjC(I-γ∇f)xnj‖=‖ProjC(θnjh(xnj)+(1-θnj)xnj-γnj∇f(xnj))-ProjC(I-γnj∇f)xnj‖+‖ProjC(I-γnj∇f)xnj-ProjC(I-γ∇f)xnj‖+‖xnj-xnj+1‖≤θnj(‖h(xnj)‖+‖xnj‖)+|γnj-γ|‖∇f(xnj)‖+‖xnj-xnj+1‖⟶0.
Since γ∈(0,2/L), ProjC(I-γ∇f) is nonexpansive. It then follows from Lemma 2.1 (demi-closedness principle) that x~∈Fix(ProjC(I-γ∇f)). Hence, x~∈S because of S=Fix(ProjC(I-γ∇f)). So, ωw(xn)⊂S.
Finally, we prove thatxn→x^, where x^ is the unique solution of the VI (3.2). First, we show that limsupn→∞〈(I-h)x^,xn-x^〉≥0. Observe that there exists a subsequence {xnj} of {xn} satisfying
(3.10)limsup⁡n→∞〈(I-h)x^,xn-x^〉=lim⁡j→∞〈(I-h)x^,xnj-x^〉.
Since {xnj} is bounded, there exists a subsequence {xnji} of {xnj} such that xnji⇀x~. Without loss of generality, we assume that xnj⇀x~. Then, we obtain
(3.11)limsup⁡n→∞〈(I-h)x^,xn-x^〉=lim⁡j→∞〈(I-h)x^,xnj-x^〉=〈(I-h)x^,x~-x^〉≥0.
By using the property (ii) of ProjC, we have
(3.12)‖xn+1-x^‖2=‖ProjC(θnh(xn)+(1-θn)xn-γn∇f(xn))-ProjC(θnx^+(1-θn)x^-γn∇f(x^))‖2≤〈θn(h(xn)-x^)+(1-θn)((I-γn1-θn∇f)xn-(I-γn1-θn∇f)x^),xn+1-x^〉≤θn〈h(xn)-h(x^),xn+1-x^〉+θn〈h(x^)-x^,xn+1-x^〉+(1-θn)‖(I-γn1-θn∇f)xn-(I-γn1-θn∇f)x^‖‖xn+1-x^‖≤θnρ‖xn-x^‖‖xn+1-x^‖+θn〈h(x^)-x^,xn+1-x^〉+(1-θn)‖xn-x^‖‖xn+1-x^‖=(1-(1-ρ)θn)‖xn-x^‖‖xn+1-x^‖+θn〈h(x*)-x^,xn+1-x^〉≤1-(1-ρ)θn2‖xn-x^‖2+12‖xn+1-x^‖2+θn〈h(x^)-x^,xn+1-x^〉.
It follows that
(3.13)‖xn+1-x^‖2≤(1-(1-ρ)θn)‖xn-x^‖2+(1-ρ)θn{21-ρ〈h(x^)-x^,xn+1-x^〉}.
From Lemma 2.3, (3.11), and (3.13), we deduce that xn→x^. This completes the proof.From Theorem3.1, we obtain immediately the following theorem.Theorem 3.2.
LetC be a closed convex subset of a real Hilbert space H. Let f:C→R be a real-valued Fréchet differentiable convex function. Assume S≠∅. Assume that the gradient ∇f is L-Lipschitzian. Let {xn} be a sequence generated by the following hybrid gradient projection algorithm:
(3.14)xn+1=ProjC((1-θn)xn-γn∇f(xn)),n≥0,
where the sequences {θn}⊂(0,1) and {γn}⊂(0,∞) satisfy the following conditions: (i)
limn→∞θn=0, ∑n=0∞‍θn=∞ and ∑n=0∞‍|θn+1-θn|<∞;(ii)
0<liminfn→∞γn≤limsupn→∞γn<2/L and ∑n=0∞‍|γn+1-γn|<∞. Then the sequence {xn} generated by (3.14) converges to a minimizer x^ of (1.1) which is the minimum norm element in S.Proof.
In Theorem3.1, we note that h is a non-self mapping from C to the whole space H. Hence, if we chose h(x)≡0 for all x∈C, then Algorithm (3.1) reduces to (3.14). And sequence xn converges strongly to x^=ProjS(0) which is obviously the minimum norm element in S. The proof is completed.Next, we suggest another simple algorithm for dropping the assumption∑n=0∞‍|θn+1-θn|<∞.Theorem 3.3.
LetC be a closed convex subset of a real Hilbert space H. Let f:C→R be a real-valued Fréchet differentiable convex function. Assume S≠∅. Assume that the gradient ∇f is L-Lipschitzian. Let {xn} be a sequence generated by the following hybrid gradient projection algorithm:
(3.15)xn+1=ProjC(I-γ∇f)ProjC((1-θn)xn),n≥0,
where γ∈(0,2/L) is a constant and the sequences {θn}⊂(0,1) satisfy the following conditions: (1)
limn→∞θn=0;(2)
∑n=0∞‍θn=∞. Then the sequence {xn} generated by (3.15) converges to a minimizer x^ of (1.1) which is the minimum norm element in S.Proof.
Claim 1. The sequence {xn} is bounded.
Takex*∈S. Then we have
(3.16)‖xn+1-x*‖=‖ProjC(I-γ∇f)ProjC((1-θn)xn)-ProjC(I-γ∇f)x*‖≤‖(1-θn)xn-x*‖≤(1-θn)‖xn-x*‖+θn‖x*‖≤max⁡{‖xn-x*‖,‖x*‖}.
By induction,
(3.17)‖xn-x*‖≤max⁡{‖x0-x*‖,‖x*‖}.
Claim 2. ∥xn-ProjC(I-γ∇f)xn∥→0 and ωw(xn)⊂S.
By the similar argument as that in [18, page 366], we can write
(3.18)ProjC(I-γ∇f)=(1-β)I+βT,
where T is nonexpansive and β=(2+γL)/4⊂(0,1). Then we can rewrite (3.15) as
(3.19)xn+1=[(1-β)I+βT]ProjC((1-θn)xn)=(1-β)xn+βTProjC((1-θn)xn)+(1-β)(ProjC((1-θn)xn)-xn)=(1-β)xn+βyn,
where
(3.20)yn=TProjC((1-θn)xn)+1-ββ(ProjC((1-θn)xn)-xn).
It follows that
(3.21)‖yn+1-yn‖=‖TProjC((1-θn+1)xn+1)+1-ββ(ProjC((1-θn+1)xn+1)-xn+1)-TProjC((1-θn)xn)-1-ββ(ProjC((1-θn)xn)-xn)‖≤‖TProjC((1-θn+1)xn+1)-TProjC((1-θn)xn)‖+1-ββ‖ProjC((1-θn+1)xn+1)-xn+1‖+1-ββ‖ProjC((1-θn)xn)-xn‖≤‖xn+1-xn‖+θn+1β‖xn+1‖+θnβ‖xn‖.
So,
(3.22)limsup⁡n→∞(‖yn+1-yn‖-‖xn+1-xn‖)≤0.
This together with Lemma 2.4 implies that
(3.23)lim⁡n→∞‖yn-xn‖=0.
Thus,
(3.24)lim⁡n→∞‖xn+1-xn‖=lim⁡n→∞(1-β)‖yn-xn‖=0.
Note that
(3.25)‖xn-ProjC(I-γ∇f)xn‖≤‖xn-xn+1‖+‖xn+1-ProjC(I-γ∇f)xn‖=‖xn-xn+1‖+‖ProjC(I-γ∇f)ProjC((1-θn)xn)-ProjC(I-γ∇f)xn‖≤‖xn-xn+1‖+θn‖xn‖.
Therefore,
(3.26)lim⁡n→∞‖xn-ProjC(I-γ∇f)xn‖=0.
Now repeating the proof of Theorem 3.1, we conclude that ωw(xn)⊂S.
Claim 3. limsupn→∞〈x^,xn-x^〉≥0 where x^ is the minimum norm element in S.
Observe that there exists a subsequence{xnj} of {xn} satisfying
(3.27)limsup⁡n→∞〈x^,xn-x^〉=lim⁡j→∞〈x^,xnj-x^〉.
Since {xnj} is bounded, there exists a subsequence {xnji} of {xnj} such that xnji⇀x~∈S. Without loss of generality, we assume that xnj⇀x~∈S. Then, we obtain
(3.28)limsup⁡n→∞〈x^,xn-x^〉=lim⁡j→∞〈x^,xnj-x^〉=〈x^,x~-x^〉≥0.
Claim 4. xn→x^. From (3.15), we have
(3.29)‖xn+1-x^‖2=‖ProjC(I-γ∇f)ProjC((1-θn)xn)-ProjC(I-γ∇f)x^‖2≤‖(1-θn)xn-x^‖2=‖(1-θn)(xn-x^)-θnx^‖2=(1-θn)2‖xn-x^‖2-2θn(1-θn)〈x^,xn-x^〉+θn2‖x^‖2≤(1-θn)‖xn-x^‖2+θn{-2(1-θn)〈x^,xn-x^〉+θn‖x^‖2}.
It is obvious that limsupn→∞(-2(1-θn)〈x^,xn-x^〉+θn∥x^∥2)≤0. Then we can apply Lemma 2.3 to the last inequality to conclude that xn→x^. The proof is completed.Next, we suggest another algorithm with the additional projections applied to the gradient projection algorithm. We show that this algorithm has strong convergence.Theorem 3.4.
LetC be a closed convex subset of a real Hilbert space H. Let f:C→R be a real-valued Fréchet differentiable convex function. Assume S≠∅. Assume that the gradient ∇f is L-Lipschitzian. Let x0∈H. For C1=C and x1=ProjC1x0, define a sequence {xn} of C as follows:
(3.30)yn=ProjC(xn-γn∇f(xn)),n≥0,Cn={z∈Cn-1:‖yn-z‖≤‖xn-z‖},xn+1=ProjCnx0.
where the sequence {γn}⊂(0,∞) satisfies the condition 0<liminfn→∞γn≤limsupn→∞γn<2/L. Then the sequence {xn} generated by (3.30) converges to x^=ProjS(x0).Proof.
It is obvious thatCn is convex. For any x*∈S, we have
(3.31)‖yn-x*‖=‖ProjC(xn-γn∇f(xn))-ProjC(x*-γn∇f(x*))‖≤‖xn-x*‖.
This implies that x*∈Cn. Hence, S⊂Cn. From xn+1=ProjCnx0, we have
(3.32)〈x0-xn+1,xn+1-y〉≥0,∀y∈Cn.
Since S⊂Cn, we have
(3.33)〈x0-xn+1,xn+1-u〉≥0,∀u∈S.
So, for u∈S, we have
(3.34)0≤〈x0-xn+1,xn+1-u〉=〈x0-xn+1,xn+1-x0+x0-u〉=-‖x0-xn+1‖2+〈x0-xn+1,x0-u〉≤-‖x0-xn+1‖2+‖x0-xn+1‖‖x0-u‖.
Hence,
(3.35)‖x0-xn+1‖≤‖x0-u‖,∀u∈S.
This implies that {xn} is bounded.
Fromxn=ProjCn-1x0 and xn+1=ProjCnx0∈Cn⊂Cn-1, we have
(3.36)〈x0-xn,xn-xn+1〉≥0.
Hence,
(3.37)0≤〈x0-xn,xn-xn+1〉=〈x0-xn,xn-x0+x0-xn+1〉=-‖x0-xn‖2+〈x0-xn,x0-xn+1〉≤-‖x0-xn‖2+‖x0-xn‖‖x0-xn+1‖,
and therefore
(3.38)‖x0-xn‖≤‖x0-xn+1‖.
This implies that
(3.39)lim⁡n→∞‖xn-x0‖exists.
From (3.36) and (3.39), we obtain
(3.40)‖xn+1-xn‖2=‖(xn+1-x0)-(xn-x0)‖2=‖xn+1-x0‖2-‖xn-x0‖2-2〈xn+1-xn,xn-x0〉≤‖xn+1-x0‖2-‖xn-x0‖2⟶0.
By the fact xn+1∈Cn, we get
(3.41)‖yn-xn+1‖≤‖xn-xn+1‖⟶0.
Therefore, from (3.40) and (3.41), we deduce
(3.42)lim⁡n→∞‖xn-yn‖=lim⁡n→∞‖xn-ProjC(I-γn∇f)xn‖=0.
Now (3.42) and Lemma 2.1 guarantee that every weak limit point of {xn} is a fixed point of ProjC(I-γn∇f). That is, ωw(xn)⊂Fix(ProjC(I-γn∇f))=S. At the same time, if we choose u=ProjS(x0) in (3.35), we have
(3.43)‖x0-xn+1‖≤‖x0-ProjS(x0)‖.
This fact and Lemma 2.2 ensure the strong convergence of {xn} to ProjS(x0). This completes the proof.Now we give some remarks on our variant gradient projection methods.Remark 3.5.
Under the same control parameters, the gradient projection methods (3.1) and (1.6) are all strong convergent. However, (3.1) seems to  have  more advantage than (1.6) as  h is a non-self-mapping.Remark 3.6.
The gradient projection method (3.14) is similar to (1.5) by using (1-θn)xn instead of xn. But (3.1) has strong convergence, and especially (3.1) converges strongly to the minimum norm element of S.Remark 3.7.
The advantage of the gradient projection method (3.15) is that it has strong convergence under some weaker assumptions on parameter θn.Remark 3.8.
The gradient projection method (3.30) is simpler than (1.7).



---

*Source: 792078-2012-07-08.xml*