# Clarke's tangent cones and the boundaries of closed sets in $R^n$

Ralph Tyrrell Rockafellar

► To cite this version:

Ralph Tyrrell Rockafellar. Clarke's tangent cones and the boundaries of closed sets in $R^n$. Nonlinear Analysis: Theory, Methods and Applications, 1979, 3, pp.145-154. 10.1016/0362-546X(79)90044-0. hal-01552475

HAL Id: hal-01552475

https://hal.science/hal-01552475

Submitted on 3 Jul 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# CLARKE'S TANGENT CONES AND THE BOUNDARIES OF CLOSED SETS IN $\mathbf{R}^n$

R. T. ROCKAFELLAR

Dept. of Mathematics, University of Washington, Seattle, WA 98195, U.S.A.

*Key words:* Tangent cones, normal cones, generalized gradients, sub-differential calculus, Lipschitzian functions, optimality conditions.

## 1. INTRODUCTION

Let $C$ be a nonempty closed subset of $\mathbf{R}^n$. For each $x \in C$, the tangent cone $T_C(x)$ in the sense of Clarke consists of all $y \in \mathbf{R}^n$ such that, whenever one has sequences $t_k \downarrow 0$ and $x_k \to x$ with $x_k \in C$, there exist $y_k \to y$ with $x_k + t_k y_k \in C$ for all $k$. This is not Clarke's original definition in [1, 2], but it is equivalent to it by his Proposition 3.7 in [2] (see also [3, Remark 2.1] and more recently Hiriart-Urruty [4, Chapter VII; 5]).

It is obvious that $0 \in T_C(x)$ and that $T_C(x)$ really is a cone (i.e. $y \in T_C(x)$ implies $\lambda y \in T_C(x)$ for all $\lambda > 0$). Moreover $T_C(x)$ is closed. What is remarkable, however, is that $T_C(x)$ is always convex (cf. Clarke [1, 2]; a direct proof is also provided below). This property is surprising, because it is obtained without any convexity or smoothness assumptions on $C$. In the absence of such assumptions (and related 'constraint qualifications'), the other local cones that have been studied in optimization theory (cf. [6]) are typically not convex, and this has always posed difficulties. If $C$ is a "differentiable submanifold" of $\mathbf{R}^n$, $T_C(x)$ is the classical tangent space (as a subspace of $\mathbf{R}^n$), while if $C$ is convex $T_C(x)$ is the usual closed tangent cone of convex analysis [7].

Tangent cones in this sense have a natural role in the theory of flow-invariant sets and ordinary differential equations (and inclusions), see Clarke [2] and Clarke-Aubin [3]. They are fundamental in the study of optimization problems through duality with the normal cones

$$N_C(x) = T_C(x)^0 = \{z \in \mathbf{R}^n | y \in T_C(x), \langle y, z \rangle \le 0\} \quad (1.1)$$

and through their consequent close connection with the generalized gradient sets Clarke has defined for any lower semicontinuous function $f: \mathbf{R}^n \to (-\infty, \infty]$ by

$$\partial f(x) = \{z \in \mathbf{R}^n | (z, -1) \in N_{\text{epif}}(x, f(x))\} \quad (1.2)$$

(where epif is the (closed) epigraph set $\{(x, \alpha) \in \mathbf{R}^{n+1} | \alpha \ge f(x)\}$; if $f(x) = \infty$, $\partial f(x)$ is taken to be empty). Clarke has shown in [8-13] that these notions provide the means for extending to the nonconvex case the kinds of necessary conditions for optimality that have been developed for nonsmooth variational problems of convex type (cf. [7-16]).

<footnote>* Research sponsored by the Air Force Office of Scientific Research, Air Force Systems Command, USAF, under grant number 77-3204.</footnote>
The main purpose of this article is to establish a strong property of interior tangent vectors (Theorem 2) which implies that the boundary of C must be Lipschitzian around any boundary point x where int $T_c(x) \neq \emptyset$. The condition int $T_c(x) \neq \emptyset$ is equivalent to $N_c(x)$ being a pointed cone, in the sense that $0 \neq z \in N_c(x)$ implies $-z \notin N_c(x)$. This result is used to derive a rule for estimating the tangent cones and normal cones to the intersection of two sets and the inverse image of a set under a differentiable mapping (Theorem 5). A strengthened convexity property of C is also proved (Theorem 1). It is shown that $\partial f(x)$ cannot be a nonempty bounded set unless f is actually Lipschitzian around x (Theorem 4).

## 2. CONVEXITY OF THE TANGENT CONE

Clarke's original approach to the definition of $T_c(x)$ in [1, 2] is based on special properties of Lipschitzian functions (for which he initially defined $\partial f(x)$ in another manner). It is of some interest to know that the convexity of $T_c(x)$ can also be deduced straight from the equivalent definition adopted here. For the record we provide a proof which also shows how the convexity is approached 'uniformly' in the limit, a property that will be needed in deriving the fundamental theorem in the next section.

Let B denote the closed unit ball in $\mathbb{R}^n$, so that $x + \delta B$ is the closed ball of radius $\delta$ about x. One has $y \in T_c(x)$ if and only if for every $\varepsilon > 0$ there exist $\delta > 0$ and $\lambda > 0$ such that

$$ C \cap [x' + t(y + \varepsilon B)] \neq \emptyset \quad \text{for all } x' \in C \cap (x + \delta B), \quad t \in [0, \lambda]. \quad (2.1) $$

In what follows, the convex hull of a set D is denoted by co D.

**THEOREM 1.** Let D be any nonempty compact subset of $T_c(x)$. Then for every $\varepsilon > 0$ there exist $\delta > 0$ and $\lambda > 0$ such that (2.1) is valid simultaneously for all $y \in \text{co} D$. (Thus in particular $\text{co} D \subset T_c(x)$, so $T_c(x)$ is convex.)

*Proof.* Let $\varepsilon > 0$. Since D is compact, it can be covered by a finite family of balls $y_i + \varepsilon B$, where $y_i \in T_c(x)$ for $i = 1, \dots, m$. Then

$$ \text{co} D \subset \text{co} \bigcup_{i=1}^{m} (y_i + \varepsilon B) = \text{co}\{y_1, \dots, y_m\} + \varepsilon B. \quad (2.2) $$

It will suffice to show that (2.1) holds for all y in co{$y_1, \dots, y_m$}, because this will imply via (2.2) that for $2\varepsilon$ in place of $\varepsilon$ it holds for all $y \in \text{co} D$ (a property equivalent to the desired conclusion, since $\varepsilon$ is arbitrary anyway).

For $i = 1, \dots, m$, (2.1) holds for $y_i$ and certain $\delta_i > 0$, $\lambda_i > 0$. Taking $\bar{\delta}$ and $\bar{\lambda}$ to be the smallest of these values, one has

$$ C \cap [x' + t(y_i + \varepsilon B)] \neq \emptyset \quad \text{for all } x' \in C \cap (x + \delta B), \quad t \in [0, \bar{\lambda}], \quad \text{and } i = 1, \dots, m. \quad (2.3) $$

Choose $\delta \in (0, \bar{\delta}]$ and $\lambda \in (0, \bar{\lambda}]$ small enough that

$$ \delta + \lambda(\rho + \varepsilon) \le \bar{\delta}, \quad \text{where } \rho = \max\{|y_1|, \dots, |y_m|\}. \quad (2.4) $$

The assertion

$$ C \cap [x' + t(y + \varepsilon B)] \neq \emptyset \quad \text{for all } x' \in C \cap (x + \delta B), \quad t \in [0, \lambda] \text{ and all } y \in \text{co}\{y_1, \dots, y_k\} \quad (2.5) $$
holds trivially for $k = 1$, in view of (2.3); make the induction hypothesis that it holds for $k = m - 1$. For any $x' \in C \cap (x + \delta B)$, $t \in (0, \lambda)$, $y \in \text{co}\{y_1, \dots, y_m\}$. Write $y = \alpha y' + (1-\alpha)y_m$, where $y' \in \text{co}\{y_1, \dots, y_{m-1}\}$ and $\alpha \in [0, 1]$. Since $\alpha t \in [0, \lambda]$, we have by induction that $C$ meets $x' + \alpha t(y' + \varepsilon B)$. Let $x''$ be any point in the intersection. Then in particular

$$x'' \in [(x + \delta B) + \alpha t(|y'|B + \varepsilon B)] = x + [\delta + \alpha t(|y'| + \varepsilon)]B,$$

where $|y'| \le \max\{|y_1|, \dots, |y_{m-1}|\} \le \rho$ in (2) and consequently

$$\delta + \alpha t(|y'| + \varepsilon) \le \delta + \lambda(\rho + \varepsilon) \le \bar{\delta}.$$

Thus $x'' \in C \cap (x + \delta B)$, and since also $(1-\alpha)t \le \lambda \le \bar{\lambda}$ it follows from (2.3) that $C$ meets $x'' + (1-\alpha)t(y_m + \varepsilon B)$. Hence $C$ meets

$$[x' + \alpha t(y' + \varepsilon B)] + (1 - \alpha)t(y_m + \varepsilon B) = x' + t(y + \varepsilon B).$$

This verifies (2.5) for $k = m$ and completes the proof.

*Remark.* The proof of Theorem 1 is easily extended to infinite-dimensional spaces and thereby demonstrates that the convexity of Clarke's tangent cone (under the corresponding extension of the present form of the definition) is a far more general phenomenon than has been realized. For applications of this approach to the study of generalized directional derivatives of lower semi-continuous functions on locally convex spaces, see [17].

### 3. INTERIORS OF TANGENT CONES

The following theorem will be fundamental to the rest of this paper. (C still denotes a closed subset of $\mathbf{R}^n$, and $x$ is a point of C.)

**THEOREM 2.** One has $y \in \text{int } T_C(x)$ if any only if there exist $\varepsilon > 0$, $\delta > 0$, $\lambda > 0$ such that

$$x' + ty' \in C \quad \text{for all } x' \in C \cap (x + \delta B), \quad t \in [0, \lambda], \quad y' \in (y + \varepsilon B). \quad (3.1)$$

*Proof. Sufficiency.* Suppose the condition holds, and consider arbitrary $y' \in (y + \varepsilon B)$. For any sequences $x_k \in C \to x$, $t_k \downarrow 0$, one has $x_k \in (x + \delta B)$ and $t_k \in (0, \lambda)$ for all $k$ sufficiently large, and consequently $x_k + t_k y_k \in C$ for $y_k = y'$. Thus by definition $y' \in T_C(x)$. This proves $T_C(x) = (y + \varepsilon B)$.

*Necessity.* Given $y \in \text{int } T_C(x)$, choose $\varepsilon > 0$ small enough that $y + 3\varepsilon B \subset T_C(x)$. Apply Theorem 1 to $D = y + 3\varepsilon B$ to obtain $\delta' > 0$ and $\lambda' > 0$ such that

$$C \cap [x' + t(y' + \varepsilon B)] \ne \emptyset \quad \text{whenever } x' \in C \cap (x + \delta' B), \quad t \in [0, \lambda'], \quad y' \in (y + 3\varepsilon B). \quad (3.2)$$

Next choose $\delta > 0$ and $\lambda > 0$ small enough that

$$\lambda \le \lambda' \quad \text{and} \quad \delta + 2\lambda(|y| + \varepsilon) \le \delta'. \quad (3.3)$$

It will be demonstrated that (3.1) holds for this choice of $\varepsilon$, $\delta$, $\lambda$.

Suppose (3.1) does not hold. Then there exist

$$\bar{x} \in C \cap (x + \delta B), \quad \bar{\lambda} \in [0, \lambda], \quad \bar{y} \in (y + \varepsilon B), \quad (3.4)$$

such that $\bar{x} + \bar{\lambda}\bar{y} \notin C$. Choose any $\rho > 0$ small enough that

$$C \cap (\bar{x} + \bar{\lambda}\bar{y} + \rho B) = \emptyset \quad (\text{hence } \rho < |\bar{x} - (\bar{x} + \bar{\lambda}\bar{y})| = \bar{\lambda}|\bar{y}|). \quad (3.5)$$
Define

$$
\tilde{\lambda} = \max\{s \in [0, \tilde{\lambda}] | C \cap (\bar{x} + s\bar{y} + \rho B) \neq \emptyset\}; \quad (3.6)
$$

this maximum is attained, because C is closed and B is compact. Since (3.5) holds but $\bar{x} \in C$, one
has

$$
0 < \tilde{\lambda} < \bar{\lambda} \le \lambda \le \lambda'. \tag{3.7}
$$

Select any $\tilde{x} \in C \cap (\bar{x} + \tilde{\lambda}\bar{y} + \rho B)$, as exists by (3.6). The interior of the ball $\bar{x} + \tilde{\lambda}\bar{y} + \rho B$ cannot meet $C$, in view of (3.6), so actually

$$
\tilde{x} = \bar{x} + \tilde{\lambda}\bar{y} + \rho e \text{ with } |e| = 1. \quad (3.8)
$$

Then by (3.4), (3.5), (3.7), one has

$$
|\tilde{x} - x| \le |\bar{x} - x| + |\tilde{x} - \bar{x}| \le \delta + |\tilde{\lambda}\bar{y} + \rho e| < \delta + \tilde{\lambda}|\bar{y}| + \bar{\lambda}|\tilde{y}| \le \delta + 2\lambda(|y| + \epsilon).
$$

It follows from (3.3) that for this $\tilde{x}$ and for $\tilde{y} = \bar{y} - 2\epsilon\epsilon$ one has

$$
\tilde{x} \in C \cap (x + \delta'B) \text{ and } \tilde{y} \in (y + 3\epsilon B),
$$

and therefore by (3.2)

$$
C \cap [\tilde{x} + t(\tilde{y} + \varepsilon B)] \neq \emptyset \quad \text{for all } t \in [0, \lambda'.] \tag{3.9}
$$

However, consider any $t$ small enough that

$$
0 < t < \min\{\rho/2\epsilon, \tilde{\lambda} - \tilde{\lambda}'\} \text{ (hence } t < \tilde{\lambda} \le \tilde{\lambda}'\text{).} \quad (3.10)
$$

It will be shown that

$$
C \cap [\tilde{x} + t(\tilde{y} + \varepsilon B)] = \emptyset \quad (\text{even though } t \in [0, \lambda')). \tag{3.11}
$$

The contradiction between this and (3.9) will finish the proof. Since $t < \rho/2\epsilon$ in (3.10), one has
$0 < \rho - 2\epsilon t < \rho - \epsilon t$, so that

$$
t(\tilde{y} - \bar{y}) = \rho e = (\rho - 2\epsilon t)e \in (\rho - \epsilon t)B.
$$

Then $\tilde{x} + t\tilde{y} \in (\tilde{x} - \rho e + t\bar{y} + (\rho - \epsilon t)B)$. Using (3.8) one obtains

$$
\tilde{x} + t(\tilde{y} + \varepsilon B) \subset \bar{x} + (\tilde{\lambda} + t)\bar{y} + \rho B,
$$

where $\tilde{\lambda} < \tilde{\lambda} + t < \tilde{\lambda}$ (since $t < \tilde{\lambda} - \tilde{\lambda}'$ in (3.10)). This yields (3.11), because

$$
C \cap (\bar{x} + (\tilde{\lambda} + t)\bar{y} + \rho B) - \emptyset
$$

by the definition (3.6) of $\tilde{\lambda}$.

*Counterexample* 1. Theorem 2 is no longer true when $\mathbf{R}^n$ is replaced by an infinite-dimensional Banach space, even in the case of a convex set. Let $C$ be the closed convex subset of the Hilbert space $l^2 \times \mathbf{R}$ which is the epigraph of the function

$$
f(\xi) = \sum_{j=1}^{\infty} j \xi_j^2, \quad \text{where} \quad \xi = (\xi_1, \xi_2, \dots).
$$
It can be verified that $T_c(0, 0)$ is the upper half-space $\{(\xi, \alpha)|\alpha \ge 0\}$ and hence has nonempty interior containing $y = (0, 1)$. But the interior of $C$ is empty, because $f$ is not bounded above on any neighborhood of $0$. Hence the property in Theorem 2 cannot hold at $x = (0, 0)$.

*Remark.* Hiriart-Urruty [5, Theorem 4] has proved for Banach spaces a result somewhat akin to Theorem 2 but involving the interior of $U_c(x) = T_c(x) \cap (-T_c(x))$, where $C'$ is the closure of the complement of $C$. He assumes $x$ is a 'regular' boundary point (an 'angularity' property) and proves that for $y \in \text{int } U_c(x)$ there then exist $\varepsilon > 0$ and $\lambda > 0$ with $x + ty' \in C$ and $x - ty' \in C'$ for all $y' \in (y + \varepsilon B), t \in (0, \lambda)$. His argument is based heavily on the 'regularity' assumption and is very different from ours. In the finite-dimensional case, one obtains from Theorem 2 (cf. also the remarks in the next section) that the same conclusion is valid not only for $x$ but all neighboring boundary points $x'$, whether or not $x$ is 'regular', and assuming merely that $y \in \text{int } T_c(x)$.

**COROLLARY 1.** One has $x \in \text{int } C$ if and only if $x$ is a point of $C$ such that $T_c(x)$ is all of $\mathbf{R}^n$ (i.e. $N_c(x) = \{0\}$). Thus $C$ has at least one nonzero 'normal vector' at each of its boundary points.

*Proof.* This is the case of Theorem 2 where $y = 0$.

**COROLLARY 2.** Let $x$ be a point of $C$ where $\text{int } T_c(x) \neq \emptyset$ (i.e. $N_c(x)$ is pointed). Then the multi-function $N_c$ is closed at $x$, in the sense that

$$x_k \in C \rightarrow x, \quad z_k \in N_c(x_k), \quad z_k \rightarrow z \quad \Rightarrow \quad z \in N_c(x). \qquad (3.12)$$

*Proof.* To prove (3.12), consider first any $y \in \text{int } T_c(x)$. The property in Theorem 2 implies $y \in T_c(x')$ for all $x' \in C \cap (x + \delta B)$, and hence $y \in T(x_k)$ for all $k$ sufficiently large. Then $\langle y, z_k \rangle \le 0$ by the definition (1.1) of $N_c$, so that $0 \ge \lim_{k \to \infty} \langle y, z_k \rangle = \langle y, z \rangle$. This shows that

$$\langle y, z \rangle \le 0 \quad \text{for all } y \in \text{int } T_c(x). \qquad (3.13)$$

Since $T_c(x)$ is convex with nonempty interior, it is the closure of this interior, and the inequality in (3.13) therefore carries over to all $y \in T_c(x)$. Thus $z \in N_c(x)$.

**COROLLARY 3.** Let $f: \mathbf{R}^n \rightarrow \mathbf{R} \cup \{+\infty\}$ be lower semicontinuous, and let $x$ be a point where $f$ is finite, $\partial f(x)$ is nonempty, and $\partial f(x)$ does not contain any entire line (i.e. is a convex set of linearity zero [7, p. 65]). Then the multifunction $\partial f$ is closed at $x$, in the relative sense that

$$x_k \rightarrow x, \quad z_k \in \partial f(x_k), \quad z_k \rightarrow z, \quad f(x_k) \rightarrow f(x) \quad \Rightarrow \quad z \in \partial f(x). \qquad (3.14)$$

*Proof.* In view of Corollary 2 and the definition (1.2) of $\partial f$, it is enough to show that the cone $N_{\text{epi}_f}(x, f(x))$ is pointed under these assumptions. Since this cone is by nature always contained in the lower half-space $\{(x, \alpha)|\alpha \le 0\}$, it fails to be pointed if and only if for some $u \neq 0$ it contains both $(u, 0)$ and $(-u, 0)$. For an arbitrary element of $N_c(x)$ of the form $(z, -1)$ (or any other form, for that matter) this property of $u$ is equivalent to having

$$(z, -1) + t(u, 0) \in N_c(x) \quad \text{for all } t \in \mathbf{R}.$$
Thus if $\partial f(x)$ contains an element $z$, the property of $u$ is equivalent to the line $\{z + tu | t \in \mathbf{R}\}$ being contained in $\partial f(x)$.

*Counterexample 2.* The fact that (3.12) and (3.14) can fail without the assumptions in Corollaries 2 and 3 is illustrated by the set

$$C = \{(x_1, x_2, x_3) \in \mathbf{R}^3 | x_3 = x_1 x_2 \text{ or } x_3 = -x_1 x_2\}.$$

For all $t \neq 0$ the cone $N_c(t, 0, 0)$ is the $x_2x_3$-plane, and similarly $N_c(0, t, 0)$ is the $x_1x_3$-plane, yet $N_c(0, 0, 0)$ is just the $x_3$-axis. This is a counterexample to (3.12), and by taking $f$ to be the indicator of $C$, so that $\partial f(x) = N_c(x)$, one obtains a counterexample to (3.14).

Of course (3.12) and (3.14) do always hold in the *convex* case [7, Section 24]. Furthermore, (3.14) always holds when $f$ is Lipschitzian, cf. [2] (this also follows from Corollary 3 and Theorem 4 below).

## 4. LIPSCHITZIAN PROPERTIES

A set $C \subset \mathbf{R}^n$ is *epi-Lipschitzian* at a point $x \in C$ if it can be represented near $x$ as the epigraph of a Lipschitzian (Lipschitz continuous) function. This means that for some neighborhood $U$ of $x$ there is a nonsingular linear transformation $A: \mathbf{R}^n \to \mathbf{R}^{n-1} \times \mathbf{R}$ such that $C \cap U = C \cap A^{-1}(\text{epi } \phi)$, where $\phi$ is a function on $\mathbf{R}^{n-1}$ that is finite and Lipschitzian around the point $\xi$ which is the $\mathbf{R}^{n-1}$-component of $A(x)$. The part of the boundary of $C$ in $U$ is represented correspondingly by the graph of $\phi$ and is a 'Lipschitzian surface.'

**THEOREM 3.** A closed set $C \subset \mathbf{R}^n$ is epi-Lipschitzian at a point $x \in C$ if and only if $\text{int } T_c(x) \neq \emptyset$ (i.e. $N_c(x)$ is pointed).

The fact that a closed set $C$ is epi-Lipschitzian at $x$ if and only if (3.1) holds, is of course all that is needed in deriving Theorem 3 from Theorem 2. This fact is 'well known', but an explicit statement is hard to find. Recently Caffarelli [20, proof of Theorem 2] used it without supplying an argument. A proof for a similar situation, involving boundaries of 'star-shaped' regions, has been given by Friedman and Kinderlehrer [21, Lemma 4.1]. While the fact certainly is elementary it is trickier to establish than might be supposed, since the boundary is not already given as a 'surface' but must be shown to be such (locally) under (3.1). This is the crux of the proof of Theorem 3 that is furnished below.

**THEOREM 4.** A lower semicontinuous function $f: \mathbf{R}^n \to (-\infty, +\infty]$ is (finite and) Lipschitzian in a neighborhood of a point $x \in \mathbf{R}^n$ if and only if $\partial f(x)$ is nonempty and bounded.

These results, which are closely related, will be derived as consequences of Theorem 2, and it is convenient to deal with Theorem 4 first. The necessity of Theorem 4 has already been established by Clarke [1, 2] (in proving that his general definition for $\partial f(x)$ reduces to his first-stage definition in terms of limits in the case where $f$ is Lipschitz continuous).
*Proof of Theorem 4.* Lipschitz continuity of $f$ around $x$ means the existence of $\mu \ge 0$ and a neighborhood $U$ of $x$ such that $f$ is finite on $U$ and

$$|f(x'') - f(x')| \le \mu|x'' - x'| \quad \text{for all } x', x'' \in U.$$

This property can be expressed equivalently, although somewhat oddly, as follows: there exist $\delta > 0$, $\varepsilon \in (0, 1)$ and $\lambda > 0$ such that

$$f(x' + ty') \le f(x') + t(\varepsilon^{-1} - 1) \quad \text{for all } t \in [0, \lambda] \text{ when} \\ x' \in (x' / x + \delta B), \quad f(x') \le f(x) + \delta, \quad y' \in \varepsilon B. \qquad (4.1)$$

(Here $\varepsilon$ corresponds to $(1 + \mu)^{-1}$; note that the condition applies in particular when $x' = x$, so $f$ must be finite on a neighborhood of $x$.) The virtue of (4.1) is that it can be restated in epigraph terms as ($f$ finite at $x$ and)

$$(x', \alpha) + t(y', \beta) \in \text{epi } f \quad \text{for all } t \in [0, \lambda] \text{ when} \\ (x', \alpha) \in (\text{epi } f) \cap [(x, f(x)) + \delta(B \times [-1, 1])], \qquad (4.2) \\ (y', \beta) \in [(0, 1) + (B \times [-1, 1])].$$

But this is just the condition in Theorem 2 in the case of the set $C = \text{epi } f$, the point $(x, f(x))$ in $C$, and the vector $(0, 1)$. (If course $C$ is closed, since $f$ is l.s.c.).

It follows that $f$ is (finite and) Lipschitzian in a neighborhood of $x$ if and only if $f$ is finite at $x$ and $(0, 1) \in \text{int } T_C(x, f(x))$. When this is true $T_C(x, f(x))$ cannot be the whole of $\mathbb{R}^{n+1}$ (for then Corollary 1 of Theorem 2 would yield the impossible conclusion that $(x, f(x)) \in \text{int } C$). Now for a convex cone $K$ in $\mathbb{R}^m$, $0 \in K \ne \mathbb{R}^m$, the condition $a \in \text{int } K$ is dual to a property of the polar cone $K^\circ$, namely that the cross-section $M = \{u \in K^\circ | \langle u, a \rangle = -1\}$ is nonempty and compact $[18, \text{Corollary 7F}]$. Applying this to $a = (0, 1)$ and the cone $T_C(x, f(x))$, whose polar is the normal cone $N_C(x, f(x))$, one sees that $f$ is (finite and) Lipschitzian in a neighborhood of $x$ if and only if $f$ is finite at $x$ and the set $\{z \in \mathbb{R}^n | (z, -1) \in N_C(x, f(x))\}$ is nonempty and bounded. But this is just $\partial f(x)$ by Definition (1.1) when $f(x) < \infty$. (When $f(x) = \infty$, $\partial f(x) = \emptyset$.)

*Proof of Theorem 3. Necessity.* The proof of Theorem 4 shows that the epigraph of a Lipschitzian function has tangent cones with nonempty interior. Hence if $C$ can be represented in a neighborhood of $x$ as the epigraph of such a function, in the sense defined, it must be true that $\text{int } T_C(x) \ne \emptyset$.

*Sufficiency.* If $x \in \text{int } C$, the conclusion that $C$ is epi-Lipschitzian near $x$ is trivial. Suppose therefore that $x$ is a boundary point of $C$. Then $T_C(x)$ is not all of $\mathbb{R}^n$ (cf. Corollary 1 to Theorem 2). Let $y \in \text{int } T_C(x)$, $y \ne 0$, and let $H$ be the hyperplane through the origin orthogonal to $y$. Each $x' \in \mathbb{R}^n$ can be expressed uniquely in the form $\xi' + \alpha'y$, where $\xi' \in H$, $\alpha' \in \mathbb{R}$; the mapping $A: x' \to (\xi', \alpha')$ is a nonsingular linear transformation from $\mathbb{R}^n$ onto $H \times \mathbb{R}$. Let $(\xi, \alpha) = A(x)$. Since $y \in \text{int } T_C(x)$, the property in Theorem 2 holds, and in this the ball $B$ can be replaced equally
well by the product of its intersection $B'$ with $H$ and the interval $\{ty| -1 \le t \le 1\}$. In terms of $A(C)$, the property is that for some $\varepsilon > 0$, $\delta > 0$, $\lambda > 0$, one has

$$ (\xi', \alpha') + t(\eta, \beta) \in A(C) \quad \text{for all } t \in [0, \lambda] \text{ when} \\ (\xi', \alpha') \in A(C) \cap [(\xi, \alpha) + \delta(B' \times [-1, 1])], \quad (4.3) \\ (\eta, \beta) \in [(0, 1) + \varepsilon(B' \times [-1, 1])]. $$

For all $\xi' \in H$ define

$$ \phi(\xi') = \inf\{\alpha'|\alpha' \ge \alpha - \delta, (\xi', \alpha') \in A(C)\} \ge \alpha - \delta. \quad (4.4) $$

(where the convention $\inf \emptyset = +\infty$ is implicit). Since $C$ is closed, $\phi$ is a lower semicontinuous function with values in $(-\infty, +\infty]$. Taking $(\xi' \alpha') = (\xi, \alpha)$ in (4.3), one sees that

$$ \phi(\xi + t\eta) \le \alpha + t(1 - \varepsilon) \quad \text{when } \eta \in \varepsilon B', \quad t \in [0, \lambda]. $$

Hence there exists $\delta' \le \delta$ such that for all $\xi' \in (\xi + \delta' B)$ one has $\phi(\xi') \le \alpha + \delta$ and (consequently)

$$ (\xi', \phi(\xi')) \in A(C) \cap [(\xi, \alpha) + \delta(B' + [-1, 1])]. $$

Then (4.3) implies (with $\beta = 1$)

$$ (\xi' + t\eta, \phi(\xi') + t) \in A(C) \quad \text{when } t \in [0, \lambda], \quad \eta \in \varepsilon B'. \quad (4.5) $$

Therefore, for all $\xi' \in (\xi + \delta' B)$ one has

$$ \phi(\xi' + t\eta) \le \phi(\xi') + t \quad \text{when } t \in [0, \lambda], \quad \eta \in \varepsilon B', $$

so that $\phi$ is Lipschitzian on a neighborhood of $\xi$. Furthermore

$$ (\xi', \phi(\xi') + t) \in A(C) \quad \text{for all } t \in [0, \lambda] $$

by (4.3). For $t < 0$, of course, one has $(\xi', \phi(\xi') + t) \notin A(C)$ by the definition of $\phi$. Thus there is a neighborhood of $(\xi, \alpha) = (\xi, \phi(\xi))$ in which $A(C)$ coincides with the epigraph of $\phi$. This proves that $C$ is epi-Lipschitzian at $x$.

**Remark.** Hiriart-Urruty [4, Chapter VII] has introduced the symmetrized tangent cone to $C$ at a boundary point $x$ as the intersection of $T_c(x)$ and $-T_c(x)$, where $C'$ is the complement of $\text{int } C$. Substituting this for $T_c(x)$ in the definitions of $N_c(x)$ and $\partial f(x)$, he has defined the symmetrized normal cone and symmetrized generalized gradient sets. He has noted that the symmetrized tangent and normal cones reduce to $T_c(x)$ and $N_c(x)$ if $C$ is either convex or epi-Lipschitzian at $x$. It follows now from Theorem 3 that the symmetrized cones can differ from Clarke's cones only in rather "degenerate" cases, where in particular $\text{int } T_c(x) = \emptyset$ and $N_c(x)$ is not pointed (and hence contains some entire line through the origin). As for the symmetrized generalized gradient set, this likewise has to reduce to $\partial f(x)$ except perhaps in certain cases where $\partial f(x)$ is empty or is unbounded and contains some entire line. (As seen in the proof of Corollary 3 in Section 3, this condition is implied by the 'nonpointedness' of $N_{\text{epi}} f(x, f(x))$.)
5. AN INCLUSION FOR TANGENT AND NORMAL CONES

A rule for estimating tangent and normal cones will now be derived from Theorem 2. This rule
can be used in the computation of necessary conditions for optimality in problems where the
feasible set is the intersection of other sets corresponding to various constraints.

THEOREM 5. Let $u \in E = D \cap F^{-1}(C)$, where $C \subset \mathbf{R}^n$ and $D \subset \mathbf{R}^p$ are closed sets and $F: \mathbf{R}^p \to \mathbf{R}^n$ is continuously differentiable. Let $J$ be the Jacobian of $F$ at $u$, and suppose that

$$
T_D(u) \cap J^{-1} \operatorname{int} T_C(F(u)) \neq \emptyset. \tag{5.1}
$$

Then

$$
T_E(u) \supset T_D(u) \cap J^{-1} T_C(F(u)), \quad (5.2)
$$

$$
N_E(u) \subset N_D(u) + J^* N_C(F(u)) \text{ (closed).} \tag{5.3}
$$

Two cases of Theorem 5 are of particular note. The first is where $E = D \cap C$ (thus $\mathbf{R}^p = \mathbf{R}^n$, $F$ is the identity transformation, $I = J = J^{-1} = J^*$). The second is where $E = F^{-1}(C)$ (thus $D = \mathbf{R}^p = T_D(u)$, $N_D(u) = \{0\}$).

*Proof*. Since the tangent cones are closed convex sets, condition (5.1) implies that

$$
\mathrm{cl}[T_D(u) \cap J^{-1} \mathrm{int} \, T_C(F(u))] = T_D(u) \cap J^{-1} T_C(F(u))
$$

(cf. [7, Theorems 6.3, 6.3, 6.7]). To establish (5.2), therefore, it will be enough to show that $T_E(u)$ includes the set in (5.1). Then (5.3) will follow immediately by passing to the polar cones (cf. [7, Corollaries 16.3.2, 16.4.2]).

Let $v$ be an element of the intersection in (5.1), and let $y = Jv$, $x = F(u)$; then $v \in T_D(u)$,
$y \in \text{int } T_C(x)$. Suppose $t_k \downarrow 0$, $u_k \in E$, $u_k \to u$. In particular $u_k \in D$, and since $v \in T_D(u)$ there must
exist $v_k \to v$ with $u_k + t_k v_k \in D$. Also $u_k \in F^{-1}(C)$, so that points $x_k = F(u_k)$ belong to $C$ and
$x_k \to x$. For

$$
y_k = \frac{[F(u_k + t_k v_k) - F(u_k)]}{t_k}
$$

one also has $y_k \to Jv = y$, because $F$ is continuously differentiable. Note that $F(u_k + t_k v_k) = x_k + t_k v_k$; the property in Theorem 2 implies therefore that $F(u_k + t_k v_k) \in C$ for all $k$ sufficiently large, i.e. $u_k + t_k v_k \in F^{-1}(C)$. Thus $u_k + t_k v_k \in E$ for all $k$ sufficiently large, and it follows that $v \in T_E(u)$.

*Remark.* The dual form of condition (5.1) is that, for some $v \in \mathbf{R}^p$, one has

$$
\begin{align*}
\langle v, w \rangle &\le 0 \text{ for all } w \in N_D(u), \\
\langle Jv, z \rangle &< 0 \text{ for all nonzero } z \in N_C(F(u)).
\end{align*}
$$

The proof of the theorem does not really require $F$ to be continuously differentiable on $\mathbf{R}^m$,
just strongly (strictly) differentiable at $u$ in the sense that

$$
[F(u' + tv') - F(u')]/t \to J(u)v \text{ when } u' \to u, v' \to v, t \downarrow 0.
$$

For applications of Theorem 5 to the computation of generalized gradients, see [19].
REFERENCES

1. CLARKE F. H., *Necessary conditions for nonsmooth problems in optimal control and the calculus of variations*, Thesis, University of Washington, Seattle (1973).

2. CLARKE F. H., *Generalized gradients and applications*, *Trans. Am. math. Soc.* **205**, 247–262 (1975).

3. CLARKE F. H. & AUBIN J. P., *Monotone invariant solutions to differential inclusions*, to appear.

4. HIRIART-URRUTY J. B., *Contributions à la programmation mathématique: cas déterministe et stochastique*, Thèse, Université de Clermont-Ferrand II (1977).

5. HIRIART-URRUTY J.-B., *Tangent cones, generalized gradients and mathematical programming in Banach spaces*, *Math. Op. Research* (1978).

6. HESTENES M., *Optimization Theory: The Finite-Dimensional Case*. Wiley, New York (1975).

7. ROCKAFELLAR R. T. *Convex Analysis*. Princeton Univ. Press (1970).

8. CLARKE F. H., *The Euler-Lagrange differential inclusion*, *J. diff. Eqns* **19**, 80–90 (1975).

9. CLARKE F. H. *Necessary conditions for a general control problem*, in *Calculus of Variations and Control Theory* (D. Russell, ed.), pp. 257–278. Academic Press (1976).

10. CLARKE F. H., *The maximum principle under minimal hypotheses*, *S.I.A.M. J. Control Opt.* **14**, 1078–1091 (1976).

11. CLARKE F. H., *The generalized problem of Bolza*, *S.I.A.M. J. Control Opt.* **14**, 682–699 (1976).

12. CLARKE F. H., *Extremal arcs and extended Hamiltonian systems*, *Trans. Am. math. Soc.* **231**, 349–367 (1977).

13. CLARKE F. H., *A new approach to Lagrange Multipliers*, *Math. Op. Research* **1**, 165–174 (1976).

14. ROCKAFELLAR R. T., *Conjugate convex functions in optimal control and the calculus of variations*, *J. math. Analysis Applic.* **32**, 174–222 (1970).

15. ROCKAFELLAR R. T., *State constraints in convex problems of Bolza*, *S.I.A.M. J. Control* **10**, 691–715 (1972).

16. ROCKAFELLAR R. T., *Dual problems of Lagrange for arcs of bounded variation*, in *Calculus of Variations and Control Theory* (D. Russell, ed.), pp. 155–192. Academic Press (1976).

17. ROCKAFELLAR R. T., *Generalized directional derivatives and subgradients of nonconvex functions*, *Can. J. Math.*, to appear.

18. ROCKAFELLAR R. T., *Level sets and continuity of conjugate convex functions*, *Trans. Am. math. Soc.* **123**, 46–63 (1966).

19. ROCKAFELLAR R.T., *Directionally Lipschitzian functions and subdifferential calculus*, *Proc. Lond. math. Soc.*, to appear.

20. CAFFARELLI L. A., *The regularity of free boundaries in higher dimensions*, *Acta math.*, *Stockh.* **139** (3-4), 155–184 (1977).

21. FRIEDMAN A. & KINDERLEHRER D., *A one phase Stephan problem*, *Ind. Univ. Math. J.* **24**, 1005–1036 (1975).