<figure><img src="image_1.png" /></figure>

HAL
open science

# The Exact l1 Penalty Function Method for Constrained Nonsmooth Invex Optimization Problems

Tadeusz Antczak

► To cite this version:

Tadeusz Antczak. The Exact l1 Penalty Function Method for Constrained Nonsmooth Invex Optimization Problems. 25th System Modeling and Optimization (CSMO), Sep 2011, Berlin, Germany. pp.461-470, 10.1007/978-3-642-36062-6_46. hal-01347568

HAL Id: hal-01347568

https://inria.hal.science/hal-01347568

Submitted on 21 Jul 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# The Exact $l_1$ Penalty Function Method for Constrained Nonsmooth Invex Optimization Problems

Tadeusz Antczak

Faculty of Mathematics and Computer Science, University of Łódź
Banacha 22, 90-238 Łódź, Poland

## Abstract

The exactness of the penalization for the exact $l_1$ penalty function method used for solving nonsmooth constrained optimization problems with both inequality and equality constraints are presented. Thus, the equivalence between the sets of optimal solutions in the nonsmooth constrained optimization problem and its associated penalized optimization problem with the exact $l_1$ penalty function is established under locally Lipschitz invexity assumptions imposed on the involved functions.

**Key Words:** exact $l_1$ penalty function method; absolute value penalty function; penalized optimization problem; locally Lipschitz invex function; Generalized Karush-Kuhn-Tucker optimality conditions.

# 1 Introduction

Considerable attention has been given in recent years to devising methods for solving nonlinear programming problems via unconstrained minimization techniques. One class of methods which has emerged as very promising is the class of exact penalty function methods. Methods using exact penalty function transform a constrained extremum problem into a single unconstrained optimization problem. The constraints are placed into the objective function via a penalty parameter $c$ in a way that penalizes any violation of the constraints.

One important property that distinguishes exact penalty functions is the exactness of the penalization. The concept of exact penalization is sometimes ambiguous, or at least varies from author to author. One of the definitions of the exactness of the penalization is the following: there is an appropriate penalty parameter choice such that a single unconstrained minimization of the penalty function yields a solution of the constrained optimization problem.

Nondifferentiable exact penalty functions were introduced for the first time by Eremin [6] and Zangwill [17]. In almost all of the introduced penalized approaches the notion of convexity plays a dominant role. In 1970, Luenberger [13]
showed that, under convex assumptions, there is a lower bound for a penalty pa-
rameter c, equal to the largest Lagrange multiplier in absolute value, associated
to the nonlinear optimization problem. Later, Charalambous [3] generalized
the result of Luenberger for the absolute value penalty function, assuming the
second-order sufficient conditions. Under the assumptions that the minimiza-
tion problem is solvable and that it satisfies the relaxed Slater constraint qual-
ification, Mangasarian [14] characterized solutions of the convex optimization
problem in terms of minimizers of the exact penalty function for a single value
of the penalty parameter exceeding some treshold. Bazaraa et al. [2] also used
the exact l₁ penalty function method to solve nonlinear convex optimization
problems with both inequality and equality constraints. They assumed that the
objective function and the inequality constraints are convex and the equality
constraints are affine functions to prove that a Karush-Kuhn-Tucker point in
the original optimization problem is a minimizer of the exact l₁ penalty function
in the associated penalized optimization problem with sufficiently large value
of a penalty parameter. In the mentioned above works, the lower bound of the
penalty parameter above which, for all penalty parameters, any optimal solution
of the original nonlinear optimization problem is also a minimizer of the penal-
ized problem has been given for differentiable optimization problems involving
convex functions. However, from the practical point of view, the converse result
is also important.

In recent years, some numerous generalizations of convex functions have
been derived which proved to be useful for extending optimality conditions and
some classical duality results, previously restricted to convex programs, to larger
classes of nonconvex optimization problems. One of them is the invexity notion
introduced by Hanson [10] for differentiable scalar functions and later general-
ized from different points of view, also in the case of nondifferentiable functions
(see [1], [5], [8], [11], [12], [16], [18], and others).

Now, we show that there is the equivalence between the set of optimal solu-
tions in a nondifferentiable nonconvex optimization problem and the set of mini-
mizers in its associated exact penalized problem with the absolute value penalty
function. It turns out that this property is not true only for (differentiable) con-
vex optimization problems, but it still holds for nonlinear optimization problems
involving locally Lipschitz invex functions with respect to the same function η
(with the exception of those equality constraint functions for which the asso-
ciated Lagrange multipliers are negative – these functions should be assumed
to be incave with respect to the same function η). The result established here
shows that there does exist a lower bound for a penalty parameter c, equal to
the largest Lagrange multiplier in absolute value, associated to a Karush-Kuhn-
Tucker point in the original nonlinear optimization problem, above which this
equivalence holds. Further, in the case when at least one of the functions con-
stituting the nondifferentiable constrained optimization problem is not locally
Lipschitz invex and in the case when the objective function is coercive but not
invex, then the equivalence in the sense discussed here might not hold between
these optimization problems.
## 2 Preliminaries and Problem Formulation

Throughout this section, $X$ is a nonempty subset of $\mathbb{R}^n$. A real-valued function $f: X \to R$ is said to be locally Lipschitz on $X$ if, for any $x \in X$, there exist a neighborhood $U$ of $x$ and a positive constant $K_x > 0$ such that, for every $y, z \in U$, it holds $|f(y) - f(z)| \le K_x \|y - z\|$. The Clarke generalized directional derivative [4] of a locally Lipschitz function $f: X \to R$ at $x \in X$ in the direction $v \in \mathbb{R}^n$, denoted $f^0(x; v)$, is given by $f^0(x; v) = \limsup_{y \to x \atop \lambda \downarrow 0} \frac{f(y+\lambda v)-f(y)}{\lambda}$.

**Definition 1** The Clarke generalized subgradient [4] of $f$ at $x \in X$, denoted $\partial f(x)$, is defined by $\partial f(x) = \{\xi \in \mathbb{R}^n : f^0(x; v) \ge \xi^T v \text{ for all } v \in \mathbb{R}^n\}$.

The following definition is a generalization of the definition of a class of differentiable convex functions to the case of a class of locally Lipschitz invex functions (see [10]).

**Definition 2** [10] Let a function $f: X \to R$ be a locally Lipschitz function on $X$ and $u \in X$. If there exists a vector-valued function $\eta: X \times X \to \mathbb{R}^n$ such that, for each $x \in X$, the inequality $f(x) - f(u) \ge \xi^T \eta(x, u)$ holds for any $\xi \in \partial f(u)$, then $f$ is said to be a locally Lipschitz invex function at $u$ on $X$ with respect to $\eta$. If the inequality above is satisfied at any point $u$, then $f$ is said to be a locally Lipschitz invex function on $X$ with respect to $\eta$.

In order to define an analogous class of (strictly) Lipschitz incave functions with respect to $\eta$, the direction of the inequality in the definition of invex functions should be changed to the opposite one.

**Definition 3** [15] A continuous function $f: \mathbb{R}^n \to R$ is said to be coercive if $\lim_{\|x\| \to \infty} f(x) = \infty$.

Consider the following constrained optimization problem:

$$ \begin{array}{ll} \text{minimize} & f(x) \\ \text{subject to} & g_i(x) \le 0, \quad i \in I = \{1, \dots, m\}, \\ & h_j(x) = 0, \quad j \in J = \{1, \dots, s\} \\ & x \in X, \end{array} \qquad (P) $$

where $f: X \to R$ and $g_i: X \to R$, $i \in I$, $h_j: X \to R$, $j \in J$, are locally Lipschitz functions on a nonempty set $X \subset \mathbb{R}^n$.

Let $D := \{x \in X : g_i(x) \le 0, i \in I, h_j(x) = 0, j \in J\}$ be the set of all feasible solutions of problem (P). Further, we denote a set of active inequality constraints at point $\bar{x} \in X$ by $I(\bar{x}) = \{i \in I : g_i(\bar{x}) = 0\}$.

**Theorem 4** [4], [18] (Generalized Karush-Kuhn-Tucker necessary optimality conditions). Let $\bar{x} \in D$ be an optimal solution in problem (P) and some suitable
constraint qualification be satisfied at $\bar{x}$. Then, there exist $\bar{\lambda} \in R^m$, $\bar{\mu} \in R^s$ such that

$$0 \in \partial f(\bar{x}) + \sum_{i=1}^{m} \bar{\lambda}_i \partial g_i(\bar{x}) + \sum_{j=1}^{s} \bar{\mu}_j \partial h_j(\bar{x}), \quad (1)$$

$$\bar{\lambda}_i g_i(\bar{x}) = 0, i \in J, \quad (2)$$

$$\bar{\lambda}_i \in R_+, \quad i \in J. \quad (3)$$

We will assume that a suitable constraint qualification is satisfied at any optimal point in problem (P).

**Definition 5** The point $\bar{x} \in D$ is said to be Karush-Kuhn-Tucker point (a KKT point, for short) if there exist the Lagrange multipliers $\bar{\lambda} \in R^m$, $\bar{\mu} \in R^s$ such that the conditions (1)-(3) are satisfied at $\bar{x}$.

### 3 The Exactness of the Exact $l_1$ Penalty Function Method

The most popular nondifferentiable exact penalty function is the absolute value penalty function also called the exact $l_1$ penalty function. Its definition, for the considered optimization problem (P), is the following

$$\text{minimize } P(x, c) = f(x) + c \left[ \sum_{i \in I} g_i^+(x) + \sum_{j \in J} |h_j(x)| \right], \quad (P(c)) \quad (4)$$

where, for a given constraint $g_i(x) \le 0$, the function $g_i^+$ is defined by

$$g_i^+(x) = \begin{cases} 0 & \text{if } g_i(x) \le 0, \\ g_i(x) & \text{if } g_i(x) > 0. \end{cases} \quad (5)$$

The unconstrained optimization problem defined above, we call the penalized optimization problem with the absolute value penalty function.

It is known (see, for example, [2]) that under suitable convexity assumptions and a constraint qualification, there exists a finite value $c$ that will recover an optimal solution in the constrained optimization problem (P) via the minimization of the exact penalty function being the objective function in the exact penalized optimization problem (P(c)). Now, we generalize this result by weakening the convexity assumption imposed on the functions constituting the considered nonsmooth optimization problem (P).

**Theorem 6** Let $\bar{x} \in D$ be a Karush-Kuhn-Tucker point in the constrained optimization problem (P), at which the Generalized Karush-Kuhn-Tucker conditions (1)-(3) are satisfied with the Lagrange multipliers $\bar{\lambda} \in R^m$ and $\bar{\mu} \in R^s$. Let $J^+(\bar{x}) = \{j \in J : \bar{\mu}_j > 0\}$ and $J^-(\bar{x}) = \{j \in J : \bar{\mu}_j < 0\}$. Furthermore,
assume that the functions $f, g_i, i \in I, h_j, j \in J^+(\bar{x})$, are locally Lipschitz invex at $\bar{x}$ on $X$ with respect to the same function $\eta$ and the functions $h_j, j \in J^-(\bar{x})$, are locally Lipschitz incave at $\bar{x}$ on $X$ with respect to the same function $\eta$. If $c$ is assumed to be sufficiently large (it is sufficient to set $c \ge \max\{\bar{\lambda}_i, i \in I, |\bar{\mu}_j|, j \in J\}$, where $\bar{\lambda}_i, i = 1, \dots, m, \bar{\mu}_j, j = 1, \dots, s$, are the Lagrange multipliers associated with the constraints $g_i$ and $h_j$, respectively), then $\bar{x}$ is also a minimizer of its penalized optimization problem (P(c)) with the absolute value penalty function.

**Proof.** By assumption, $\bar{x}$ is a Karush-Kuhn-Tucker point in the constrained optimization problem (P), at which the Generalized Karush-Kuhn-Tucker conditions (1)-(3) are satisfied with the Lagrange multipliers $\bar{\lambda} \in R^m$ and $\bar{\mu} \in R^s$. Since $c \ge \max\{\bar{\lambda}_i, i \in I, |\bar{\mu}_j|, j \in J\}$, then, by definition of the objective function in the penalized optimization problem (P(c)), it follows that

$$P(x, c) = f(x) + c \sum_{i=1}^{m} g_{i}^{+}(x) + c \sum_{j=1}^{s} |h_{j}(x)| \geq f(x) + \sum_{i=1}^{m} \bar{\lambda}_{i} g_{i}^{+}(x) + \sum_{j=1}^{s} |\bar{\mu}_{j} h_{j}(x)|. \quad (6)$$

Thus, (5) gives

$$f(x) + \sum_{i=1}^{m} \bar{\lambda}_{i} g_{i}^{+}(x) + \sum_{j=1}^{s} |\bar{\mu}_{j} h_{j}(x)| \geq f(x) + \sum_{i=1}^{m} \bar{\lambda}_{i} g_{i}(x) + \sum_{j=1}^{s} \bar{\mu}_{j} h_{j}(x). \quad (7)$$

By assumption, the inequality constraints $g_i, i \in I$, and the equality constraints $h_j, j \in J^+(\bar{x})$, are locally Lipschitz invex at $\bar{x}$ on $X$ and the equality constraints $h_j, j \in J^-(\bar{x})$, are locally Lipschitz incave at $\bar{x}$ on $X$. Hence, by the Generalized Karush-Kuhn-Tucker conditions (2) and (3) together with the feasibility of $\bar{x}$ in problem (P), it follows that the inequality

$$f(x) + \sum_{i=1}^{m} \bar{\lambda}_{i} g_{i}(x) + \sum_{j=1}^{s} \bar{\mu}_{j} h_{j}(x) \geq f(x) + \sum_{i=1}^{m} \bar{\lambda}_{i} \zeta_{i}^{T} \eta(x, \bar{x}) + \sum_{j=1}^{s} \bar{\mu}_{j} \gamma_{j}^{T} \eta(x, \bar{x}) \quad (8)$$

holds for any $\zeta_i \in \partial g_i(\bar{x})$, $i = 1, \dots, m$, and for any $\gamma_j \in \partial h_j(\bar{x})$, $j = 1, \dots, s$. Then, using the Generalized Karush-Kuhn-Tucker condition (2), we get

$$\begin{aligned} & f(x) + \sum_{i=1}^{m} \bar{\lambda}_{i} [g_{i}(\bar{x}) + \zeta_{i}^{T} \eta(x, \bar{x})] + \sum_{j=1}^{s} \bar{\mu}_{j} [h_{j}(\bar{x}) + \gamma_{j}^{T} \eta(x, \bar{x})] \\ & = f(x) + \sum_{i=1}^{m} \bar{\lambda}_{i} \zeta_{i}^{T} \eta(x, \bar{x}) + \sum_{j=1}^{s} \bar{\mu}_{j} \gamma_{j}^{T} \eta(x, \bar{x}). \end{aligned} \quad (9)$$

Thus, by the Generalized Karush-Kuhn-Tucker necessary optimality condition (1), it follows that

$$f(x) + \sum_{i=1}^{m} \bar{\lambda}_{i} \zeta_{i}^{T} \eta(x, \bar{x}) + \sum_{j=1}^{s} \bar{\mu}_{j} \gamma_{j}^{T} \eta(x, \bar{x}) = f(x) - \xi^{T} \eta(x, \bar{x}), \quad (10)$$

where $\xi \in \partial f(\bar{x})$. By assumption, $f$ is locally Lipschitz invex at $\bar{x}$ on $X$ also with respect to the function $\eta$. Using Definition 2 together with the feasibility
of $\bar{x}$ in problem (P), we get

$$f(x) - \xi^T \eta(x, \bar{x}) \geq f(\bar{x}) = f(\bar{x}) + c \sum_{i=1}^{m} g_i^+(\bar{x}) + c \sum_{j=1}^{s} |h_j(\bar{x})| = P(\bar{x}, c). \quad (11)$$

Then, by (6)-(11), we conclude that the inequality $P(x, c) \geq P(\bar{x}, c)$ holds for all $x \in X$. This means that $\bar{x}$ is a minimizer of the penalized optimization problem (P(c)) with the absolute value penalty function and the proof of theorem is complete. ■

**Corollary 7** Let $\bar{x}$ be an optimal point in the considered optimization problem (P). Furthermore, assume that all hypotheses of Theorem 6 are fulfilled. Then $\bar{x}$ is also a minimizer in the penalized optimization problem (P(c)) with the absolute value penalty function.

**Theorem 8** Let the point $\bar{x}$ be a minimizer of the penalized optimization problem (P(c)) with the absolute value penalty function. Furthermore, assume that the functions $f, g_i, i \in I, h_j, j \in J^+(\bar{x})$, are locally Lipschitz invex at $\tilde{x}$ on $X$ with respect to the same function $\eta$, and the functions $h_j, j \in J^-(\bar{x})$, are locally Lipschitz incave at $\tilde{x}$ on $X$ with respect to the same function $\eta$, where $\tilde{x}$ is any Karush-Kuhn-Tucker point in problem (P), at which the Karush-Kuhn-Tucker necessary optimality conditions (1)-(3) are satisfied with the Lagrange multipliers $\tilde{\lambda} \in R^m$ and $\tilde{\mu} \in R^s$. If the set of all feasible solutions in the constrained optimization problem (P) is compact and the penalty parameter $c$ is sufficiently large (it is sufficient if $c$ satisfies the following condition $c > \max\{\tilde{\lambda}_i, i \in I, |\tilde{\mu}_j|, j \in J\}$), then $\bar{x}$ is also optimal in problem (P).

**Proof.** We assume that $\bar{x}$ is a minimizer in the penalized optimization problem (P(c)) with the absolute value penalty function. Then, by the definition of the penalized optimization problem (P(c)) and (5), the following inequalities $f(x)+c(\sum_{i=1}^m g_i^+(x) + \sum_{j=1}^s |h_j(x)|) \geq f(\bar{x})+c(\sum_{i=1}^m g_i^+(\bar{x}) + \sum_{j=1}^s |h_j(\bar{x})|) \geq f(\bar{x})$ hold for all $x \in X$. Thus, for all $x \in D$, the following inequality

$$f(x) \geq f(\bar{x}) \qquad (12)$$

holds. The inequality above means that values of the function $f$ are bounded below on the set $D$ of all feasible solutions in the constrained optimization problem (P). Since $f$ is a continuous function bounded below on the compact set $D$, therefore, by Weierstrass' theorem, $f$ admits its minimum $\tilde{x}$ on $D$.

Now, we prove that $\bar{x}$ is also optimal in the considered optimization problem (P). First, we show that $\bar{x}$ is feasible in problem (P). By means of contradiction, suppose that $\bar{x}$ is not feasible in problem (P). As we have established above, the given constrained optimization problem (P) has an optimal solution $\tilde{x}$. Since a constraint qualification is satisfied at $\tilde{x}$, then there exist the Lagrange multipliers $\tilde{\lambda} \in R^m$ and $\tilde{\mu} \in R^s$ such that the Generalized Karush-Kuhn-Tucker necessary optimality conditions (1)-(3) are satisfied at $\tilde{x}$. By assumption, the functions $f$,
$g_i, i \in I, h_j, j \in J^+(\tilde{x})$, are invex at $\tilde{x}$ on $X$ with respect to the same function $\eta$ and the functions $h_j, j \in J^-(\tilde{x})$, are incave at $\tilde{x}$ on $X$ with respect to the same function $\eta$. Therefore, by Definition 2, respectively, it follows that the inequalities

$$f(\bar{x}) - f(\tilde{x}) \geq \xi^T \eta(\bar{x}, \tilde{x}), \quad (13)$$

$$g_i(\bar{x}) - g_i(\tilde{x}) \geq \zeta_i^T \eta(\bar{x}, \tilde{x}), \quad i \in I, \quad (14)$$

$$h_j(\bar{x}) - h_j(\tilde{x}) \geq \gamma_j^T \eta(\bar{x}, \tilde{x}), \quad j \in J^+(\tilde{x}), \quad (15)$$

$$h_j(\bar{x}) - h_j(\tilde{x}) \leq \gamma_j^T \eta(\bar{x}, \tilde{x}), \quad j \in J^-(\tilde{x}) \quad (16)$$

hold for each $\xi \in \partial f(\tilde{x})$, $\zeta_i \in \partial g_i(\tilde{x})$, $i = 1, \dots, m$, and $\gamma_j \in \partial h_j(\tilde{x})$, $j = 1, \dots, s$. Multiplying (14), (15) and (16) by the associated Lagrange multiplier and then adding both sides of the obtained inequalities and both sides of (13), we get

$$f(\bar{x}) - f(\tilde{x}) + \sum_{i=1}^{m} \tilde{\lambda}_i g_i(\bar{x}) - \sum_{i=1}^{m} \tilde{\lambda}_i g_i(\tilde{x}) + \sum_{j=1}^{s} \tilde{\mu}_j h_j(\bar{x}) - \sum_{j=1}^{s} \tilde{\mu}_j h_j(\tilde{x}) \\ \geq \left[ \xi^T + \sum_{i=1}^{m} \tilde{\lambda}_i \zeta_i^T + \sum_{j=1}^{s} \tilde{\mu}_j \gamma_j^T \right] \eta(\bar{x}, \tilde{x}).$$

Using (5) with the Karush-Kuhn-Tucker necessary optimality conditions (1), (2) and the feasibility of $\tilde{x}$ in problem (P), we get

$$f(\bar{x}) + \sum_{i=1}^{m} \tilde{\lambda}_i g_i^{+}(\bar{x}) + \sum_{j=1}^{s} \tilde{\mu}_j |h_j(\bar{x})| \geq f(\tilde{x}). \quad (17)$$

By assumption, the penalty parameter $c$ is sufficiently large (it is sufficient that $c > \max\{\tilde{\lambda}_i, i \in I, |\tilde{\mu}_j|, j \in J\}$). Since $\bar{x}$ is assumed to be not feasible in the given optimization problem (P), therefore, at least one of $g_i^+(\bar{x})$ and $|h_j(\bar{x})|$ must be nonzero. Therefore, (17) yields

$$f(\bar{x}) + c \left[ \sum_{i=1}^{m} g_i^{+}(\bar{x}) + \sum_{j=1}^{s} |h_j(\bar{x})| \right] > f(\tilde{x}). \quad (18)$$

Then, by $\tilde{x} \in D$ and (2), we get

$$f(\bar{x}) + c \left[ \sum_{i=1}^{m} g_i^{+}(\bar{x}) + \sum_{j=1}^{s} |h_j(\bar{x})| \right] > f(\tilde{x}) + c \left[ \sum_{i=1}^{m} g_i^{+}(\tilde{x}) + \sum_{j=1}^{s} |h_j(\tilde{x})| \right].$$

Then, by the definition of the exact $l_1$ penalty function (see (4)), it follows that the following inequality $P(\bar{x}, c) > P(\tilde{x}, c)$ holds, which is a contradiction to the assumption that $\bar{x}$ is a minimizer in the penalized optimization problem (P(c)) with the absolute value penalty function. Thus, we have proved that $\bar{x}$ is feasible in the given constrained optimization problem (P). Hence, the optimality of $\bar{x}$ in problem (P) follows directly from (12). ■
**Corollary 9** Let the hypotheses of Corollary 7 and Theorem 8 are fulfilled. Then, the set of optimal solutions in the considered extremum problem (P) and the set of minimizers in its associated exact penalized optimization problem (P(c)) with the absolute value penalty function coincide.

**Example 10** Consider the following nonsmooth optimization problem

$$ 
\begin{aligned}
f(x) &= \arctan(|x|) \rightarrow \min \\
g(x) &= \frac{1}{2}(e^{|x|-x} - 1) \leq 0.
\end{aligned} 
\quad (P1) $$

Note that $D = \{x \in R : x \geq 0\}$ and $\bar{x} = 0$ is an optimal solution in the considered nonsmooth optimization problem (P1). Since we use the exact $l_1$ penalty method for solving problem (P1), then we construct the following unconstrained optimization problem

$$ P(x, c) = \arctan(|x|) + c \max \left\{ 0, \frac{1}{2} (e^{|x|-x} - 1) \right\} \rightarrow \min. \quad (P1(c)) $$

Note that $\bar{x} = 0$ is feasible in problem (P1) and the Generalized Karush-Kuhn-Tucker necessary optimality conditions (1)-(3) are fulfilled at $\bar{x}$ with the La-grange multiplier $\bar{\lambda}$ satisfying the following condition: $0 \in \partial f(\bar{x}) + \bar{\lambda}\partial g(\bar{x})$, where $\partial f(\bar{x}) = [-1, 1]$ and $\partial g(\bar{x}) = [-1, 0]$. Further, it can be established by Definition 2 that the objective function $f$ and the constraint function $g$ are locally Lipschitz invex at $\bar{x}$ on $R$ with respect to the same function $\eta$ defined by $\eta(x, \bar{x}) = \frac{1}{2}(\arctan(|x|) - \arctan(|\bar{x}|))$. Then, by Theorems 6 and 8, it follows that, for any penalty parameter $c$ satisfying $c > \bar{\lambda}$, there is the equivalence between the sets of optimal solutions in optimization problems (P1) and (P1(c)). Further, note that not all functions involved in problem (P1) are differentiable and convex. Therefore, in order to show that the point $\bar{x} = 0$, being optimal in (P1), is also a minimizer in the unconstrained optimization problem (P1(c)), we can not use the conditions for convex smooth optimization problem (see, for instance, Theorem 9.3.1 [2]).

**Example 11** Consider the following nonsmooth constrained optimization problem

$$ 
\begin{aligned}
\min f(x) &= \begin{cases} -x+4 & \text{if } x < -4, \\ \frac{1}{2}x+10 & \text{if } -4 \leq x < 0, \\ -5x+10 & \text{if } 0 \leq x < 2, \\ x-2 & \text{if } x \geq 2, \end{cases} \\
g(x) &= x - \frac{1}{4} \leq 0,
\end{aligned} 
\quad (P2) $$

in which not all functions are locally Lipschitz invex. Note that $D = \{x \in R : x \leq \frac{1}{4}\}$ and $\bar{x} = -4$ is an optimal solution in the considered optimization problem (P2). Since $0 \in \partial f(0) = [-5, \frac{1}{2}]$, then $\tilde{x} = 0$ is a stationary point of $f$. It is not difficult to show that $\tilde{x}$ is not a global minimizer of $f$. Then the objective function $f$ is not locally Lipschitz invex on $R$ with respect to any function $\eta$ defined by $\eta : R \times R \rightarrow R$ (see, for example, [16]). However, we use the exact $l_1$
penalty method to solve the considered optimization problem (P2). Therefore, we construct the following unconstrained optimization problem

$$P(x, c) = f(x) + c \max \left\{ 0, x - \frac{1}{4} \right\} \rightarrow \min \quad (P2(c))$$

Note that $\bar{x} = -4$, being an optimal solution in problem (P2), is not a global minimizer in the associated penalized optimization problem (P2(c)) for all values of the penalty parameter c satisfying the condition $c > \bar{\lambda} = 0$, (where $\bar{\lambda}$ is the Lagrange multiplier associated with the inequality constraint g satisfying the Karush-Kuhn-Tucker necessary optimality conditions (1)-(3)). However, for every penalty parameter c ∈ (0, 32/7), the point $\bar{x} = 2$ is a global minimizer in the above penalized optimization problem (P2(c)). Therefore, there is no the equivalence between the sets of optimal solutions in problems (P2) and (P2(c)) for any penalty parameter c satisfying the condition $c > \bar{\lambda}$. This follows from the fact that not all functions constituting the considered optimization problem (P2) are locally Lipschitz invex on R.

**Remark 12** Peressini et al. [15] considered differentiable convex optimization problems and solved them by using the exact $l_1$ penalty function method. Under assumption that the objective function in the constrained optimization problem is coercive (see Definition 3), they proved that, for sufficiently large values of the penalty parameter c, the constrained optimal solution in (P) is also a minimizer in its associated penalized optimization problem (P(c)) with the exact $l_1$ penalty function. But the finite value of the penalty parameter c, above which this result holds, was not given in [15]. Note that the objective function in the optimization problem (P2) considered in Example 11 is coercive. However, for not all values of the penalty parameter c satisfying the condition $c > \bar{\lambda}$, an optimal solution in the considered optimization problem (P2) yields a minimizer in its associated penalized optimization problem (P(c)) with the exact $l_1$ penalty function. But the result proved in the paper shows that, under invexity assumptions imposed on the functions constituting the constrained nonsmooth optimization problem (P), for every value of the penalty parameter c satisfying the condition $c > \max\{\bar{\lambda}_i, i \in I, |\bar{\mu}_j|, j \in J\}$, the sets of optimal solutions in problems (P) and (P(c)) coincide. Hence, this example shows that in the case when the objective function is coercive but not invex the result established in the paper might not be true for such optimization problems.

## References

[1] Antczak, T.: Lipschitz r-invex functions and nonsmooth programming, Numerical Functional Analysis and Optimization 23, 265-283 (2002)

[2] Bazaraa, M.S., Sherali, H.D., Shetty, C.M.: Nonlinear programming: theory and algorithms, John Wiley and Sons, New York, (1991)
[3] Charalambous, C.: A lower bound for the controlling parameters of the exact penalty functions, Mathematical Programming 15, 278-290 (1978)

[4] Clarke, F.H.: Optimization and nonsmooth analysis, A Wiley-Interscience Publication, John Wiley&Sons, Inc., (1983)

[5] Craven, B.D.: Invex functions and constrained local minima, Bulletin of the Australian Mathematical Society 24, 357-366. (1981)

[6] Eremin, I.I.: The penalty method in convex programming, Doklady Akad. Nauk SSSR 143, 748-751 (1967)

[7] Di Pillo, G., Grippo, L.: Exact penalty functions in constrained optimization, SIAM Journal of Control and Optimization 27, 1333-1360 (1989)

[8] Egudo, R.R., Hanson, M.A.: On sufficiency of Kuhn-Tucker conditions in nonsmooth multiobjective programming, FSU Report No. M-888, (1993)

[9] Han, S.P., Mangasarian, O.L.: Exact penalty functions in nonlinear programming, Mathematical Programming 17, 251-269 (1979)

[10] Hanson, M.A.: On sufficiency of the Kuhn-Tucker conditions, Journal of Mathematical Analysis and Applications 80, 545-550 (1981)

[11] Kaul, R.N., Suneja, S.K., Lalitha, C.S.: Generalized nonsmooth invexity, Journal of Information and Optimization Sciences 15, 1-17. (1994)

[12] Kim, M.H. Lee, G.M.: On duality theorems for nonsmooth Lipschitz optimization problems, Journal of Optimization Theory and Applications 110, 669-675 (2001)

[13] Luenberger, D.: Control problem with kinds, IEEE Transaction on Automatic Control 15, 570-574 (1970)

[14] Mangasarian, O.L.: Sufficiency of exact penalty minimization, SIAM Journal of Control and Optimization 23, 30-37 (1985)

[15] Peressini, A.L., Sullivan, F.E., Uhl, Jr., J.J.: The mathematics of nonlinear programming, Springer-Verlag New York Inc., (1988)

[16] Reiland, T.W.: Nonsmooth invexity, Bulletin of the Australian Mathematical Society 42, 437-446 (1990)

[17] Zangwill, W.I.: Nonlinear programming via penalty functions, Management Science 13, 344-358 (1967)

[18] Zhao, F.: On sufficiency of the Kuhn-Tucker conditions in nondifferentiable programming, Bulletin of the Australian Mathematical Society 46, 385-389 (1992)