# Generalized Quadratic Augmented Lagrangian Methods with Nonmonotone Penalty Parameters

Authors: Xunzhi Zhu; Jinchuan Zhou; Lili Pan; Wenling Zhao
Journal: Journal of Applied Mathematics
 (2012)

---

## Abstract

For nonconvex optimization problem with both equality and inequality constraints, we introduce a new augmented Lagrangian function and propose the corresponding multiplier algorithm. New iterative strategy on penalty parameter is presented. Different global convergence properties are established depending on whether the penalty parameter is bounded. Even if the iterative sequence{xk} is divergent, we present a necessary and sufficient condition for the convergence of {f(xk)} to the optimal value. Finally, preliminary numerical experience is reported.


---

## Body

## 1. Introduction

This paper is concerned with the following nonlinear programming problem:(P)min⁡f(x)s.t.gi(x)≤0,i=1,…,m,hj(x)=0,j=1,…,l,x∈Ω,
where f,gi:ℝn→ℝ for i=1,…,m and hj:ℝn→ℝ for j=1,…,l are all continuously differentiable functions, and Ω is a nonempty and closed subset in ℝn. Denoted by X the feasible region and by X* the solution set.The classical Lagrangian function for this problem is(1.1)L(x,λ,μ)=f(x)+∑i=1mλigi(x)+∑j=1lμjhj(x),
where (λ,μ)∈ℝ+m×ℝl. It is well known that the above classic Lagrangian function would fail to find out the optimal solution of primal problem because a nonzero duality gap may be arisen for nonconvex programming. Augmented Lagrangians were proposed to overcome this difficulty. It combines the advantage from Lagrangian duality methods and penalty function methods; for example, it does not need to make the penalty parameter infinitely large, and hence successfully avoid ill-conditioning effects. In fact, if the penalty parameter is very large, the Lagrangian relaxation subproblem would be very difficult in the sense that the stabilized Newton step is not regarded as a candidate to be a decreasing direction.Global convergence properties of augmented Lagrangian methods have been studied by many researchers [1–8]. It should be noted that the multiplier sequence generated by the algorithm may be unbounded. Several strategies have proposed to deal with these questions, such as adding constraint qualifications [5, 6] or via safeguarding projection [1, 2, 9, 10].In this paper, for the optimization problem (P) with both equality and inequality constraints, we introduce a new class of augmented Lagrangian functions, which include the well-known quadratic augmented Lagrangian function as special case [11–13]. It should be noted that this function class is more general since we do not restrict ϕ to be convex (see (2.1) below for the definition of ϕ). Convergence properties of the corresponding multiplier algorithm are studied. Specially, a new updated strategy on penalty parameter is proposed, that is, the penalty parameter ck is improved only when the multipliers are too large, and the iterative point is far away from the feasible region. This strategy guarantees that the penalty parameter remains unchanged, provided that measure of infeasibility of iterative point and bounding of multipliers have enough progress (see (2.4)). Furthermore, we study the performance of the multiplier sequences generated by our algorithm. Finally, compared with [4, 9, 10], we further consider the case when {xk} is divergent, in which a necessary and sufficient condition for {f(xk)} converging to the optimal value is given as well.The organization of this paper is as follows. In the next section, we propose the multiplier algorithm and study the global convergence properties. Preliminary numerical results are reported in Section3. Conclusion is drawn in Section 4.


## 2. Multiplier Algorithms

A new generalized quadratic augmented Lagrangian function for (P) is defined as follows:(2.1)L(x,λ,μ,c):=f(x)+∑j=1lμjhj(x)+c2∑j=1lhj2(x)+12c∑i=1m{max2{0,ϕ(cgi(x))+λi}-λi2},
where (x,λ,μ,c)∈ℝn×ℝm×ℝl×ℝ++, and ℝ++ denotes the all positive real scalars, that is, ℝ++={a∈ℝ∣a>0}. The function ϕ:ℝ→ℝ involved in (2.1) satisfies the following properties:(A1)
continuously differentiable and strictly increasing onℝ with ϕ(0)=0 and ϕ(α)≥α for α≥0.Particularly, ifϕ(α)=α for α∈ℝ, then L reduces to the quadratic augmented Lagrangian function. Compared with [9, 14, 15], an important point made above is that ϕ is not required to be convex. Hence, it makes the augmented Lagrangian we introduce here more general.Given(x,λ,μ,c), the Lagrangian relaxation subproblem associated with the augmented Lagrangian L is defined as:(Lλ,μ,c)min⁡L(x,λ,μ,c),s.t.x∈Ω.
Its solution set is denoted by S*(λ,μ,c). We always assume throughout this paper that the set S*(λ,μ,c) is nonempty, which is ensured if Ω is compact, since f, gi, and hj are all continuous. In addition, we assume that:(A2)
f is bounded from below, that is, f*=infx∈ℝnf(x)>-∞.It is a rather mild assumption in optimization problem because otherwise the objective functionf can be replaced by ef(x). Recall that a vector x* is said to be a stationary point of (P) if it is a feasible point and there exist λi∈ℝ+ for all i=1,…,m and μj for j=1,…,l such that the following system holds:(2.2)0∈∇f(x*)+∑i=1mλi∇gi(x*)+∑j=1lμj∇hj(x*)+NΩ(x*),
where 𝒩Ω(x*) denotes the normal cone of Ω at x* [16, Chapter 6]. Let Λ(x*) denote the collection of multipliers satisfying (2.2). The set of Λ(x*) is larger than that of R-multiplier defined in [17], which reduces to the well-known Karush-Kuhn-Tucker (KKT) point of (P) when Ω=ℝn, since the complementarity condition is required for KKT point. The following is the multiplier algorithm based on the generalized quadratic augmented Lagrangian L. One of its main feature is that the Lagrangian multipliers associated with equality and inequality constraints are not restricted to be bounded.Algorithm 2.1 (multiplier algorithm based onL).
Step 1.
(initialization)
LetM>0 and {εk}k=1∞ be an arbitrary positive sequence with zero as limit (i.e., εk→0+). Select an initial point x0∈ℝn, λ0∈ℝ+m, μ0∈ℝl, and c0>0. Set k:=0.
Step 2.
(estimating multipliers)
Compute(2.3)λik+1=max⁡{0,ϕ(ckgi(xk))+λik}ϕ′(ckgi(xk)),∀i=1,…,m,μjk+1=μjk+ckhj(xk),∀j=1,…,l.
Step 3.
(updating penalty parameter)
Let(2.4)ck+1=ck,if|λik+1|≤M,|μjk+1|≤M,|hj(xk)|≤εk,gi(xk)≤εk,∀i,j(2.5)ck+1=(k+1)max⁡{1,∑i+1m(λik+1)2,∑j+1l(μjk+1)2},otherwise.
Step 4.
(solving lagrangian relaxation subproblem)
Findxk+1∈S*(λk+1,μk+1,ck+1).
Step 5.
(cycling)
Ifxk+1∈Ω and (λk+1,μk+1)∈Λ(xk+1), then stop; otherwise, let k:=k+1 and go back to Step 2.Consider the perturbation of (P); precisely, for any fixed α≥0, define the perturbation value function as follows:(2.6)v(α)=inf⁡{f(x)∣x∈Ω(α)},
where(2.7)Ω(α)={x∈Ω∣gi(x)≤α,|hj(x)|≤α,i=1,⋯,m,j=1,⋯,l}.
It is clear that Ω(0)=X. In practical implementation of the previous algorithm, we always choose the constant M large enough to strictly include ∂v(0), the set of subdifferential of v at origin, since according to the strong duality theory, the optimal Lagrangian multipliers belong to ∂v(0); see [18] for the detailed discussion. In [5, 6], the authors require the convergence of {λk/ck} to zero. It follows from (2.5) that this nice property holds automatically in our new strategy. In addition, (2.4) and (2.5) indicate that Lagrangian relaxation subproblem needs to place more emphasis on decreasing the constraint violations via increasing the penalty parameter, provided that the current point xk is far away from feasible region.Lemma 2.2.
Let{xk} be the iterative sequence generated by Algorithm 2.1, and let x* be one of its accumulate points. If {ck} is bounded, then x* is a stationary point of (P).Proof.
If the algorithm is terminated finitely, then the result follows immediately according to the stop criterion in Step5. On the other hand, if the condition (2.5) occurs infinitely, then ck≥k, and hence ck must be unbounded. Therefore, by hypothesis we knew that only condition (2.4) occurs when k is large enough. Because gi(xk)≤εk and |hj(xk)|≤εk by (2.4), taking the limit yields gi(x*)≤0 and hj(x*)=0, since εk converges to zero. This shows the feasibility of x*. Meanwhile, we know from (2.4) that {λik} and {μjk} are bounded. Hence, we can assume without loss of generality that λik→λi* and μjk→μj*. Since xk is an optimal solution of minimizing L(x,λk,μk,ck) over Ω by Step 4, then according to the well-known optimality conditions we have
(2.8)-∇xL(xk,λk,μk,ck)∈NΩ(xk),
which together with the formula of (2.3) and (2.4) yields
(2.9)-∇f(xk)-∑j=1lμjk+1∇hj(xk)-∑i=1mλik+1∇gi(xk)∈NΩ(xk).
Taking limits gives us
(2.10)-∇f(x*)-∑j=1lμj*∇hj(x*)-∑i=1mλi*∇gi(x*)∈NΩ(x*),
where we have used the upper semicontinuity of normal cone [16, Chapter 6]. This completes the proof.Now, we turn our main concern to the case when{ck} is unbounded.Theorem 2.3.
Let{xk} be the iterative sequence generated by Algorithm 2.1, and let x* be one of its accumulate points. If {ck} is unbounded, then x* is an optimal solution of (P).Proof.
It is clear that if{ck} is unbounded, then the iteration formula (2.5) occurs infinitely. Hence, for simplification, we assume that (2.5) always happens as k sufficiently large, and that xk converges to x* (not resorting to a subsequence). Taking into account of (2.5), the terms λk/ck,μk/ck,(λk)2/ck,(μk)2/ck are all converges to zero. This fact will be used in the following analysis.
Let us first show thatx* is a feasible point of (P). We argue it by contradiction. Note first that
(2.11)L(xk,λk,μk,ck)=inf⁡{L(x,λk,μk,ck)∣x∈Ω}≤inf⁡{L(x,λk,μk,ck)∣x∈Ω(0)}=inf⁡⁡{f(x)+12ck∑i=1m{max2{0,ϕ(ckgi(x))+λik}-(λik)2}∣x∈Ω(0)}≤inf⁡{f(x)∣x∈Ω(0)}=v(0),
where the third and fourth steps are due to the feasibility of x∈X. This implies that
(2.12)limsup⁡k→∞L(xk,λk,μk,ck)≤v(0)<+∞.
Now, let us consider the following two cases.Case 1.
There exists an indexj0 such that |hj0(x*)|>0. Due to the continuity of hj0, we must have |hj0(xk)|>(1/2)|hj0(x*)|>0 for k is large enough. Therefore,
(2.13)L(xk,λk,μk,ck)≥f*+∑j=1lμjkhj(xk)+ck2∑j=1lhj2(xk)-12ck∑i=1m(λik)2=f*+ck2∑j=1l{(hj(xk)+μjkck)2-(μjk)2ck2}-12ck∑i=1m(λik)2≥f*+ck2(hj0(xk)+μj0kck)2-12ck∑i=1m(λik)2-12ck∑j=1l(μjk)2.
Noticing that μj0k/ck→0 as mentioned above and |hj0(xk)|≥(1/2)|hj0(x*)|, we know that as k is large enough
(2.14)|hj0(xk)+μj0kck|≥|hj0(xk)|-|μj0k|ck≥14|hj0(x*)|,
which further implying
(2.15)(hj0(xk)+μj0kck)2>116|hj0(x*)|2.
This, together with (2.13), yields v(0)=+∞ since ck approaches to ∞, which contradicts (2.12).
Case 2.
There exists an indexi0 such that gi0(x*)>0. Similarly, due to the continuity of gi0, we know that gi0(xk)≥(1/2)gi0(x*)>0 as k is sufficiently large. Therefore,
(2.16)L(xk,λk,μk,ck)≥f*+12ck∑i=1m{max2{0,ϕ(ckgi(xk))+λik}-(λik)2}-12ck∑i=1l(μjk)2≥f*+12ckmax2{0,ckgi0(xk)+λi0k}-12ck∑i=1m(λik)2-12ck∑j=1l(μjk)2≥f*+ck8gi02(x*)-12ck∑i=1m(λik)2-12ck∑j=1l(μjk)2,
where the second inequality comes from the fact ϕ(a)≥a for all a≥0 by (A1), and the last inequality follows from the nonnegativity of λik by (2.3), since ϕ′(α)≥0 for all α∈ℝ due to the monotonicity of ϕ. Invoking (2.5) and taking limits in the above inequality yields v(0)=+∞, which contradicts (2.12). So far, we have established the feasibility of x*.
It remains to show thatx* is an optimal solution. In fact, notice that
(2.17)f(xk)=L(xk,λk,μk,ck)-12ck∑i=1m{max2{0,ϕ(ckgi(xk))+λik}-(λik)2}-∑j=1lμjkhj(xk)-ck2∑j=1lhj2(xk)≤L(xk,λk,μk,ck)+12ck∑j=1l(μjk)2+12ck∑i=1m(λik)2≤v(0)+12ck∑j=1l(μjk)2+12ck∑i=1m(λik)2,
where the second inequality is due to (2.11). Taking limits in the above yields
(2.18)f(x*)=lim⁡k→∞f(xk)≤limsup⁡k→∞{v(0)+12ck∑j=1l(μjk)2+12ck∑i=1m(λik)2}=v(0),
where the last step comes from the fact that  (μjk)2/ck and (λik)2/ck converge to zero (see (2.5)). This means that x* must be an optimal solution of (P). The proof is complete.Finally, let us deal with the performance of multiplier sequences{λk} and {μk} generated by our proposed algorithm.Theorem 2.4.
Let{xk} be an iterative sequence generated by Algorithm 2.1, and let x* be one of its accumulate points. The following statements hold:(i)
if both{∑i=1mλik} and {∑j=1lμjk} are bounded, then we have
(2.19)-∇f(x*)-∑i=1mλi*∇gi(x*)-∑j=1lμj*∇hj(x*)∈NΩ(x*),
where λi* and μi* are accumulate points of {λik} and {μik};(ii)
if either{∑i=1mλik} or {∑j=1lμjk} is unbounded, then we have
(2.20)-∑j=1lμj*∇hj(x*)-∑i=1mλi*∇gi(x*)∈NΩ(x*),
where λi* and μj* are accumulate points of {λik/Tk}and {μk/Tk} with(2.21)Tk=∑i=1mλik+∑j=1lμjk.Proof.
The proof is divided into the following two cases.
Case 1.
Both{∑i=1mλik} and {∑j=1lμjk} are bounded. In this case, we can obtain that x* is a stationary point of (P) by following almost the same argument as in Lemma 2.2.
Case 2.
Either{∑i=1mλik} or {∑j=1lμjk} is divergent. According to the updated strategy on penalty parameter ck, we know that (2.5) must occur as k sufficiently large. Hence, {ck} is unbounded in this case. Since {xk} is a global optimal solution of L(x,λk,μk,ck) over Ω by Step 4 in Algorithm 2.1, applying the optimality condition to the augmented Lagrangian relaxation problem L(λk,μk,ck) yields
(2.22)-∇xL(xk,λk,μk,ck)∈NΩ(xk),
that is,
(2.23)-∇f(xk)-∑i=1mλik+1∇gi(xk)-∑j=1lμjk+1∇hj(xk)∈NΩ(xk).
Sinceλik+1/Tk+1∈[0,1] and μik+1/Tk+1∈[0,1] are bounded, we can assume without loss of generality that
(2.24)λik+1Tk+1⟶λi*,μjk+1Tk+1⟶μj*,i=1,…,m,j=1,…,l.
Clearly, λi*  and  μj* are not all zeros. Dividing by Tk+1 in both sides of (2.23) and using the fact that 𝒩Ω is cone, we have
(2.25)-1Tk+1∇f(xk)-∑i=1mλik+1Tk+1∇gi(xk)-∑j=1lμjk+1Tk+1∇hj(xk)∈NΩ(xk).
Taking limits in the above yields
(2.26)-∑i=1mλi*∇gi(x*)-∑j=1lμj*∇hj(x*)∈NΩ(x*).
This establishes the result as desired.Now, a natural question arises: how does the algorithm perform if{xk}  is divergent? The following theorem gives the answer.Theorem 2.5.
Let{xk} be an iterative sequence generated by Algorithm 2.1. If {ck} is unbounded, then the following statements are equivalent:(a)
f(xk) converges to the optimal value, that is,
(2.27)lim⁡k→∞f(xk)=v(0);(b)
v(α) is lower semicontinuous at α=0 from right, that is, liminfα→0+v(α)≥v(0);(c)
v(α) is continuous at α=0 from right, that is, limα→0+v(α)=v(0).Proof.
(a)⇒(b). Suppose on the contrary that v is not lower semicontinuous at zero from right, then there must exist δ0>0 and ξt→0+(ast→∞)  such that
(2.28)v(ξt)≤v(0)-δ0,∀t.
For any given k, since ξt→0 we can choose a subsequence tk satisfying
(2.29)ξtkck⟶0ask⟶∞,
which together with the continuity of ϕ implies that
(2.30)ϕ(ckξtk)⟶0ask⟶∞.
In addition, pickzk∈Ω(ξtk) with f(zk)≤v(ξtk)+(δ0/2). Then f(zk)≤v(0)-(δ0/2) by (2.28).
Therefore,(2.31)f(xk)=L(xk,λk,μk,ck)-12ck∑i=1m{max2{0,ϕ(ckgi(xk))+λik}-(λik)2}-∑j=1lμjkhj(xk)-ck2∑j=1lhj2(xk)≤inf⁡x∈ΩL(x,λk,μk,ck)-ck2∑j=1l{(hj(xk)+μjkck)2-(μjk)2ck2}+12ck∑i=1m(λik)2≤inf⁡x∈ΩL(x,λk,μk,ck)+12ck∑j=1l(μjk)2+12ck∑i=1m(λik)2≤f(zk)+∑j=1lμjkhj(zk)+ck2∑j=1lhj2(zk)+12ck∑i=1m{max2{0,ϕ(ckgi(zk))+λik}-(λik)2}+12ck∑j=1l(μjk)2+12ck∑i=1m(λik)2=f(zk)+ck2∑j=1l{(hj(zk)+μjkck)2-(μjk)2ck2}+12ck∑i=1m{max2{0,ϕ(ckgi(zk))+λik}-(λik)2}+12ck∑j=1l(μjk)2+12ck∑i=1m(λik)2≤v(0)-δ02+ck2∑j=1l(ξtk+|μjk|ck)2+12ck∑i=1m(ϕ(ckξtk)+λik)2,
where the last step is due to the fact |hj(zk)|≤ξtk and gi(zk)≤ξtk since zk∈Ω(ξtk) and ϕ is nondecreasing by (A1). Taking the limits in both sides of (2.31) and using (2.27) and (2.30) yield
(2.32)v(0)=lim⁡k→∞f(xk)≤v(0)-δ02,
which leads to a contradiction.
(b)⇒(a). Since xk converges to x*, it then follows from Theorem 2.3 that x* is an optimal solution of (P). Hence, for any arbitrary ε, we have f(xk)≤f(x*)+ε=v(0)+ɛ and xk∈Ω(ε) (since gi(xk)→gi(x*)≤0<ε and |hj(xk)|→|hj(x*)|=0<ε as k is  sufficiently large). The latter further implies that v(ε)≤f(xk) by definition (2.6), that is,
(2.33)v(ε)≤f(xk)≤v(0)+ε.
Since v(ε) is lower semicontinuous at ε=0 from right by hypothesis, taking the lower limitation in (2.33) yields
(2.34)v(0)≤liminf⁡ε→0+v(ε)≤liminf⁡k→∞f(xk)≤limsup⁡k→∞f(xk)≤v(0),
that is,
(2.35)lim⁡k→∞f(xk)=v(0).
(c)⇒(b). It is clear.
(b)⇒(c). It follows by invoking the fact  that v(α)≤v(0), since Ω(0)⊂Ω(α) for all α≥0. Then limsupα→0+v(α)≤v(0), that is, v is upper semicontinuous at origin from right. This together with the lower semicontinuous of v at origin by hypothesis yields the desired result.


## 3. Preliminary Numerical Reports

To give some insight into the behavior of our proposed algorithm presented in this paper, we solve two problems by lettingϕ take the following different functions:(1)
y1=ϕ1(α)=α;(2)
y2=ϕ2(α)=eα-1;(3)
y3=ϕ3(α)={α2+α,α≥0,αotherwise;(4)
y4=ϕ4(α)={α3+α,α≥0,-α2otherwise.The computer codes were written in Matlab 7.0, and the test was done at a PC of Pentium 4 with 2.8 GHz CPU and 1.99 GB memory. We implement our algorithm to solve the following programming problems, where Example3.1 is obtained by adding an equality constraint in Example  6.3 in [19], and Example 3.2 comes from [20]. The corresponding numerical results are reported below, where k is the number of iterations, ck is the penalty parameter, xk is iterative point, and f(xk) is the objective function value.Example 3.1.
Consider the following nonconvex programming:(3.1)min⁡(x1+x2-2)2+(x1-x2)2+30(min⁡{0,x1-x2})2s.t.x1+x2-2≤0,x12+5x1x2-3=0.Example 3.2.
(3.2)min⁡f(x)=-5(x1+x2)+7(x4-3x3)+x12+x22+2x32+x42s.t.∑i=14xi2+x1-x2+x3-x4-8≤0,x12+2x22+x32+2x42-x1-x4-10≤0,2x12+x22+x32+2x1-x2-x4-5=0.In the implementation of our algorithm, we use the BFGS quasi-Newton method with a mixed quadratic and cubic line search procedure to solve the Lagrangian subproblem:(i)
for Example3.1, the initial data are x0=(1,1)T, M=1000, ϵk=1/2k, λ0=1, and  μ0=1, c0=1; {ck} remains unchanged because M is taken large enough to ensure the validity of (2.4); the obtained solution is a stationary point with the corresponding multipliers λ*=1.172 and μ*=0, and the inequality is active;(ii)
for Example3.2, the result starts from the nonfeasible point x0=(2,0,-2,1)T, M=100, ϵk=1/2k, λ0=(1,1)T, μ0=1, and c0=1; the obtained solution is also a stationary point with the corresponding multipliers λ*=(1,0)T and μ*=2, and the first inequality constraint is active.The above preliminary numerical result as shown in Tables1 and 2 and Figures 1 and 2 indicates that Algorithm 2.1 may have better numerical performance, when ϕ is nonlinear (see ϕ2 and ϕ3), and ϕ is nonconvex (see ϕ4) because the decreasing steep is more “quickly” in the beginning of the iterative, which leads to the requirement of fewer iterative steps. Our next research topic is to further study the performance of our algorithm for solving various practical questions.Table 1
Result of Example3.1.
ϕi(s)kckxkf(xk)ϕ1(s)12.5147(0.6722, 0.6722)0.402942.5147(0.6950, 0.6950)0.367562.5147(0.7071, 0.7071)0.3431ϕ2(s)110.4139(0.6612, 0.6612)0.3924310.4139(0.6981, 0.6981)0.3526510.4139(0.7071, 0.7071)0.3432ϕ3(s)117.5462(0.6517, 0.6517)0.3676317.5462(0.6992, 0.6992)0.3450517.5462(0.7071, 0.7071)0.3432ϕ4(s)117.5462(0.6388, 0.6388)0.3033317.5462(0.7273, 0.7273)0.3230517.5462(0.7071, 0.7071)0.3431Table 2
Result of Example3.2.
ϕi(s)kckxkf(xk)ϕ1(s)18.4614(−0.0072, 0.9963, 1.9907, −1.0058)−43.8610324.2390(−0.0011, 0.9997, 1.9998, −1.0033)−44.0076948.5455(−0.0012, 0.9997, 1.9998, −1.0036)−44.0083ϕ2(s)18.4430(0.0017, 1.0017, 2.0063, −1.0073)−44.1314324.7705(−0.0028, 0.9992, 1.9994, −1.0086)−44.0186948.7133(−0.0014, 0.9996, 1.9997, −1.0044)−44.0097ϕ3(s)13.8876(0.0025, 1.0017, 2.0054, −1.0027)−44.101839.8546(−0.0027, 0.9992, 1.9994, −1.0085)−44.0190748.2683(0.0005, 1.0002, 2.0006, −0.9987)−44.0050ϕ4(s)17.7614(0.0025, 1.0007, 2.0032, −1.0019)−44.0657324.6380(−0.0023, 0.9993, 1.9995, −1.0072)−44.0186748.5943(−0.0012, 0.9997, 1.9998, −1.0036)−44.0081Figure 1
Result of Example3.1.Figure 2
Result of Example3.2.


## 4. Conclusions

We introduce a new class of generalized quadratic augmented Lagrangian function with nonconvex functions. Global convergence properties of the corresponding multiplier algorithm are proposed without requiring the boundedness of Lagrangian multiplier sequence. Penalty parameter is improved only if multiplier sequences are too large and the iterative point is far away from feasible region. Necessary and sufficient conditions for the convergence of{f(xk)} to the optimal value are established as well.



---

*Source: 181629-2012-06-12.xml*