数理解析研究所講究録
第 829 巻 1993 年 107-116

107

On an \mbox{\boldmath $\varepsilon$}-optimal Policy in Dynamic
Programming with a Discount Factor
新潟大学理学部 田中謙輔 (Kensuke Tanaka)

1.

序論

かも知れないので、任意の指定された政策$f_{0}$

が任意の\epsilon $>0$ に対して \epsilon -最適にこれに関連すると思われる内容は、少し古くなるが inverse optimal problem と呼ばれる D. 問題であろう 参考文献$[1],[2|,[3|,[4|)_{0}$ このような問題に対して、 いろいろな接近方法が考えられるかも知れませんが、 ここでは一つの接近方法として epigraph の概念より -劣微分の性質を用いる方法を展開したい、即ち
に対して、
$P$

$($

$f^{*}\in\partial_{\epsilon}F(f_{0})$$\epsilon$$F(f)-F(f_{0})\geq<f^{*},$ $f-f_{0}>-\epsilon$$\forall f$.

この時に、 損失関数を$G(f)=F(f)-<f^{*})f>$に修正すればは修正 D. $P$ システムの -最適政策となるだろう o
ここで、上の -最適政策のもとで以下の Ekeland’s theorem が適用可能となり、この指定された政策の近くにある、或政策がこの修正 $D.P$ システムの$f_{0}$$\epsilon$$\epsilon$$f_{0}$$f_{0}$

最適政策になるように更に修正出来る事になるだろう oEkeland’s theorem
Let (X, d) be a complete metric space, and $G:Xarrow R\cup\{+\infty\}$ , a $l.s.c$ . function,, bounded from below. Let
and a point $f_{0}\in X$ such that$\not\equiv+\infty$$\epsilon>0_{f}$$G(f_{0}) \leq\inf_{f\in X}G(f)+\epsilon$.

108
Then there exists some point $f_{*}\in X$ such that
$G(f_{*})\leq G(f_{0})$$d(f_{*}, f_{0})\leq 1$$\forall f\neq f_{*}$$G(f)>G(f_{*})-\epsilon d(f_{*}, f)$.

Formulation of Markov decision problem

2.

A dynamic decision model is specified by a set of six elements$(S, A, F, q, r, \beta)$

(2.1)

,

where
(i)

is a non-empty Borel subset of a Polish(i.e., complete, separable, metric), the set of states of the decision system.
space with the Borel -field$S$$\sigma$

(ii)

$A$

$\beta(S)$is a Polish space with the Borel -field$\sigma$$\beta(A)$, namely, the action space.

(iii) $F$ is a multifunction which assigns to each state $s\in S$ a non-empty permissible set of actions $F(s)\subset A$ . We assume that $GrF=\{(s\rangle a)|a\in F(s),$ $s\in$
is a Borel subset in $S\cross A$ with the Borel $\sigma-field\beta(S)\cross\beta(A)$ .
$S\}$(iv) qisatransition probability measure q $(\cdot|s, a)ontheBorelsubsetsofSgiven$each $(s, a)\in GrF$ , i.e., $q(B|s, a)$ is a probability of a Borel subset $B\in\beta(S)$for each $(s, a)\in GrF$ and a Borel measurable function of $(s, a)\in GrF$ foreach Borel subset . The law of motion of the decision system is given by
.
$B$

$q$

(v) $r(s, a)$ is a real-valued Borel measurable function, $GrFarrow R$ , the one-steploss function.

(vi)$\beta$is a discount factor, $0<\beta<1$ .

In the specification, we should note that the permissible set of actions $F(s)$depends on a state $s\in S$ andis independent of the time.
$q(\cdot|s, a)$109,Then, a policy is defined as a sequence of infinite decision functions...,of which is a Borel measurable selection for the), each function
is a Borel measurable mapping from into
constraint multifunction , i.e.,
A such that $f_{t}(s)\in F(s)$ for each state $s\in S$ . Thus, such a decision function
indicates an action to use for each state $s\in S$ . We assume that we use only suchpolicies, which are called Markov policies, on the decision system. Especially, if
any decision function in a Markov policy is independent of the stage number
and dependent only on the present state, that is, $f_{t}=f$ for all , this policy issaid to be stationary and is written as instead of . The class of all Markov
policies is denoted by in the paper,
Now, the dynamic decision system is interpreted as follows. If a policy,
is employed, at the successive decision time $t=1,2,3,$
we observe the state of the decision system and classify it to a possible state $s_{t}\in S$ .
So, we choose an action $a_{t}\in F(s_{t}),$ $a_{t}=f_{t}(s_{t})$ , by the decision function . As a.
result of the state and the choice at the time , we will incur a loss
Then, the decision system moves to a new state $s_{t+1}\in S$ according to transition
probability. After that, the process of the dynamic decision system isanalogously developed from. So, given an initial state $s_{1}=x$ on , any
policy together with the transition probability , gives a probability measureon the state space at each time in the decision system.
Thus, the expected loss at each time is given by$(f_{1},$$\pi$$f_{t},$$\cdots$$f_{2}$$f_{t}$$F$$S$$f_{t}$$f_{t}$

$\pi$

$t$

$f$

$\pi$

$\pi$

$\Pi$

$7\Gamma=$$(f_{1}, f_{2}, f_{3}, \cdots, f_{t}, \cdots)$$t,$$\cdots$$f_{t}$$s_{t}$$r(s_{t}, a_{t})$$t$$\alpha_{t}$$q(\cdot|s_{t}, a_{t})$$S$$s_{t+1}$

$\pi$

$q$

$S$

$p_{t}^{\pi}$

$t$

$t$

$E_{\pi}[r(s_{t} \rangle a_{t})|s_{1}=x]=\int_{S}r(s, f_{t}(s))p_{t}^{\pi}(ds|s_{1}=x)$

.

(2.2)

So, if a policy $\pi=(f_{1}, f_{2}, \cdots, f_{t}, \cdots)$ is employed under the discount factor
the total expected loss is given by$I(\pi)(x)$$=$$\sum_{t=1}^{\infty}\beta^{t-1}E_{\pi}[r(s_{t}, a_{t})|s_{1}=x]$$=$$\sum_{t=1}^{\infty}\beta^{t-1}\int_{S}r(s, f_{t}(s))p_{t}^{\pi}(ds|s_{1}=x)$.

$\beta$,(2.3)

Then, assuming that $\inf_{\pi\in\Pi}I(\pi)(x)>-\infty$ under an initial state $x\in S$ , weconsider a basic minimization problem (P) for the dynamic decision system:(P)minimize $I(\pi)(x)$In this problem (P), if there exists a policy$I(\pi)(x)\geq I(\overline{\pi})(x)-\epsilon$$\overline{\pi}$subject to$\Pi$such that, forfor all$’/\ulcorner\in\Pi$,.

$\epsilon\geq 0$

110
the policy is said to be an -optimal one. Especially, if
an optimal policy.
$\overline{\pi}$$\epsilon$$\epsilon=0,$$\overline{\pi}$is said to be

An g-optimality of a given stationary
in the modified dynamic system

3.

$p$

olicy

In order to show that some specified stationary policy becomes an optimal
one in the Markov decision system with modified loss functions, let $M(S)$ be theset of all real-valued Borel measurable and bounded functions on . Further, let$V(S)$ be the set of all extended real-valued Borel measurable functions on, each. We impose some assumptions on $F,$ ,of which is function from into
$S$

$S$

$S$

and
(A1)

$r$

$R\cup\{\infty\}$$q$

as follows.
is a convex, closed-valued multifunction from into , that is, $F(s)$ is aconvex and closed nonempty subset in for each $s\in S$ .
$F$

$S$

$A$

$A$

(A2) The loss function is a real-valued Borel measurable, bounded function,$GrFarrow R$ and, for each $s\in S,$
is convex, and lower semi-continuous(l.s. ) with respect to $a\in F(s)$ ,$r$$r(s, \alpha)$$c.$(A3) For any $u\in V(S)$ and $s\in S$ ,$\int_{S}u(y)q(dy|s, a)$

is a convex and l.s. . function with respect to $a\in F(s)$ and$c$$\inf_{a\in s)}\int_{S}u(y)q(dy|s, a)>-\infty$.

REMARK 3.1 If stochastic Markov policies are only used in the decision sys$tem$ , it will be reasonable that the integral in $(A3)$ is a convex and continuousfunction.
Now, let $D$ denote the set of all permissible decision functions : $Sarrow A$ , inwhich each is Borel measurable selection and $f(s)\in F(s)$ for each $s\in S$ . Inview of a selection theorem [8],if is lower measurable set-valued function,
$f$

$f$

$D\neq\emptyset$

$F$

111
that is, for every open set in the action space , the set $\{s\in S|F(s)\cap O\neq$. We define, for each $f\in D$ , an operator $T(f)$ on $V(S)$ as follows: foreach $u\in V(S)$ and $s\in S$ ,
$A$

$O$

$\emptyset\}\in\beta\cdot(S)$$T(f)u(s)=r(s, f(s))+ \beta\int_{S}u(y)q(dy|s, f(s))$ .

Further, we define an operator$T_{0}$(3.1)

on $V(S)$ by$T_{0}u(s)= \inf_{f\in D}T(f)u(s)$

.

(3.2)

Evidently, from (A3), $T_{0}u(s)\in V(S)$ , whenever $u\in V(S)$ . If the domain ofis a contraction operator on
the operator is limited to $M(S)$ , the operator
in $M(S)$ ,has a unique fixed point
Banach space $M(S)$ with supnorm. So,
that is, $u^{*}=T_{0}u^{*}$ . It is well known thatis an optimal value for the problem(P), that is,(3.3)$u^{*}(x)= \inf_{\pi\in\Pi}I(\pi)(x)$ .
$T$$T_{0}$$u^{*}$$T_{0}$$u^{*}(x)$See E.B.Dynkin and A.A.Yushkevich [5] in detail.
Then, if a policy $\pi=(f_{1}, f_{2)}f_{3}, \cdots, f_{t}, \cdots),$ $f_{t}\in D,t=1,2$ .
, is employed,for any time , the total expected loss $I(\pi)(x)$ with the initial state $x\in S$ can be
rewritten as$\cdots$$k$$I(\pi)(x)$$=$$\sum_{t=1}^{\infty}\beta^{t-1}E_{\pi}[r(s_{t}, a_{t})|s_{1}=x]$$=$$\sum_{t=1}^{k}\beta^{t-1}E_{\pi}[r(s_{t}, a_{t})|s_{1}=x]+\beta^{k}E_{\pi}[I(\pi^{k+1})(s_{k+1})|s_{1}=x]$$=T(fi)T(f_{2})\cdots T(f_{k})I(\pi^{k+1})(x)$

,

(3.4)

where
denotes a policy constructed by a sequence of decision functions after
the time $t=k+1$ in the policy , i.e., $\pi^{k+1}=(f_{k+1}, f_{k+2}, f_{k+3}, \cdots)$ .
Now, we need the notations in the convex analysis to prove the main theorem
in this paper. So, firstly we define the extended function $G(\cdot)u(s)$ of $T(\cdot)u(s)$ on$F(s)\subset A$ as follows:$\pi^{k+1}$$\pi$$G(a)u(s)=\{T(a)u(s)\infty$$if\alpha ifa\not\in\in F(s)F(s)$and $a\in A$ .

Secondly, we define the epigraph of $G(a)u(s)$ as follows:$epiG(\cdot)u(s)=\{(a, r)|r\geq G(a)u(s), a\in A\}$.

Then, the following lemma in the convex analysis plays an important role.

112
LEMMA 3.1 Let
be a Banach space and any function : $Barrow R\cup\{\infty\}$ ,, convex, and $l.s.c.$ . Then, for any $\epsilon>0$ , the -subdifferential of at
is a nonempty, convex
dom(g)(dom(g) is the set where is finite),denotes the dual space of and, whereand weak*-closed subset in
$B$

$g$

$\not\equiv\infty$$g$$\epsilon$$b_{0}\in$$\partial_{\epsilon}g(b_{0})$$g$$\partial_{\epsilon}g(b_{0})=$$B$$B^{*}$$B^{*}${ $b^{*}\in B^{*}|g(b)\geq g(b_{0})+<b^{*},$$b-b_{0}>-\epsilon$

for all $b\in B$ }.

(3.5)

This lemma is proved by using the properties of epigraph of , epi , in [11].
$g$int(dom(g)), for any
REMARK 3.2 If
convex,compact and locally bounded.
$b_{0}\in$$\epsilon>0,$$\partial_{e}g(b_{0})$$g$

is a nonempty,$weak^{*}$REMARK 3.3 In [10], the subdifferential
in detail.

$\partial g(b_{0})$of for$g$$\epsilon=0$

is discussed

is
LEMMA 3.2 Suppose that $G(\cdot)u(s)$ is finite at $f_{0}\in D$ andlower measurable on S. Then, if
is separable, for any $u\in V(S)$ and $s\in S$ ,there exists a Borel measurable function: $Sarrow A^{*}$ such that for all $f\in D$$\partial_{\epsilon}G(f_{0})u(\cdot)$$A^{*}$$f^{*}$$T(f)u(s)\geq T(f_{0})u(s)+<f^{*}(s),$ $f(s)-f_{0}(s)>-\epsilon$ ,(3.6)$T(f)u(s)-<f^{*}(s),$ $f(s)>\geq T(f_{0})u(s)-<f^{*}(s),$ $f_{0}(s)>-\epsilon$ ,(3.7)

that is,

Proof. From (A1), (A2), (A3), and conditions of the lemma, for each $s\in$, the extended function $G(\cdot)u(s)$ : $Aarrow R\cup\{\infty\}$ , is convex, and l.s. . at$f_{0}(s)\in F(s)\subset A$ . Then, since $G(f_{0})u(s)<\infty$ , it follows from Lemma 3.1 thatis a nonempty, convex and weak*-closed subset offor each $s\in S$ .
So, we get for all
$S$

$c$

$A^{*}$$\partial_{\epsilon}G(f_{0})u(s)$$a^{*}\in\partial_{\epsilon}G(f_{0})u(s)$$T(f)u(s)\geq T(f_{0})u(s)+<\alpha^{*}\rangle f(s)-f_{0}(s)>-\epsilon$

for all $f\in D$

(3.8)

113
is separable. Further,is a Polish space since it is assumed that
we assume thatis lower measurable on . In view of a selection
theorem in [8], there is a Borel measurable selection. Thus,: $Sarrow A^{*}$ satisfying (3.6) for the
we constract a Borel measurable function
given policy $f_{0}\in D$ . Thus, the proof of the lemma is completed.

Then,$A^{*}$$A^{*}$$S$$\partial_{\epsilon}G(f_{0})u(s)$$f^{*}(s)\in\partial_{\epsilon}G(f_{0})u(s)$$f^{*}$

Then, in order to show that a given stationary policy, , is an -optimal
one for a dynamic decision model with the loss functions modified by using the: $Sarrow A^{*}$ as follows:
measurable function$f_{0}$$\epsilon$$f^{*}$$(S, A, F, q, r-<f^{*}, \cdot>, \beta)$

(3.9)

,

we introduce, for each decision function $f\in D$ , a modified operator $T(f^{*}, f)$ of$T(f)$ on $V(S)$ as follows: for each $u\in V(S)$ and $s\in S$$T(f^{*}, f)u(s)=T(f)u(s)-<f^{*}(s),$ $f(s)>$ ,(3.10)

that is,$T(f^{*}, f)u(s)=r(s, f(s))-<f^{*}(s),$ $f(s)>+ \beta\int_{S}u(y)q(dy|s, f(s))$ .

(3.11)REMARK 3.4 Since $G(\cdot)u(s)$ is convex, and 1. $s.c.,$ $G(\cdot)u(s)$ is locally Lipschitzian at $f_{0}(s)\in int(domG(\cdot)u(s))$ . So, there exis $tsM(s)>0$ and a neighborhood ofsuch that$U$$f_{0}(s)$(3.12)$|G(f)u(s)-G(f_{0})u(s)|\leq M(s)\Vert f(s)-f_{0}(s)||$ ,whenever $f(s)\in U$ , where . Il denotes the metric on A. Since $f_{0}(s)\in U$ and, we have, for all $f(s)\in U$$||$$f^{*}(s)\in\partial_{\epsilon}G(f_{0})u(s)$$<f^{*}(s),$ $f(s)-f_{0}(s)>\leq M(s)\Vert f(s)-f_{0}(s)||+\epsilon$

which shows that 1 $f^{*}(s)||_{*}\leq M(s)+\epsilon$ , where$||\cdot||_{*}$,

denotes the norm on

Thus, using Lemma 3.2, we can prove the main theorem.

(3.13)$A^{*}$.

114
THEOREM 3.1 Let everything be as in Lemma 3.2 and suppose that, foroo, there exists
witha specified stationary policyII:a sequence of infinite Borel measurable functions$Sarrow A^{*}$ , andthat is, bounded on .
II,
Then, we have for any policy$f_{0}$$\sup_{s\in S}$$f_{0}(s)\Vert=\Vert f_{0}\Vert<$$\pi^{*}=(f_{1^{*}}, f_{2^{*}}, \cdots, f_{t^{*}}, \cdots))f_{t^{*}}$$\sup_{s\in S}$$S$$f_{t^{*}}(s)||_{*}\leq M_{f}$$\pi\in\Pi$,$I^{*}(\pi^{*}, \pi)(x)\geq I^{*}(\pi^{*}, f_{0})(x)-\epsilon$

where, for a policy $\pi=(f1, f_{2}, \cdots, f_{t}, \cdots)$ ,$I^{*}(\pi^{*}, \pi)(x)$$|s_{1}=x$]$=$$\sum_{t=1}^{\infty}$$=$$\sum_{t=1}^{\infty}\beta^{t-1}\int_{S}[r(s, f_{t}(s))-<f_{t^{*}}(s), f_{t}(s)>]p_{t}^{\pi}(ds|s_{1}=x)$).

, the initial state $x\in S$ and theProof. For any policy
optimal value $u^{*}(x)(see(3.3))$ , we define a notation as follows$\pi=(f_{1}, f_{2}, \cdots, f_{t)}\cdots)$$V^{k}(\pi^{*}, \pi)(x)=T(f_{1^{*}}, f_{1})T(f_{2^{*}}, f_{2})\cdots T(f_{k^{*}}, f_{k})u^{*}(x)$,(3.14)

Since, from (A2), the loss function is bounded, i.e., $|r|\leq N$ for some positive
number , on $GrF$ , we have $|u^{*}(s)|\leq\cdot N/(1-\beta)$ for any state $s\in S$ . Further,from the condition of the theorem, we have for each $s\in S$ and $t=1,2,$ ,
$r$

$N$

$\cdots$$\sup_{s\in S}|<f_{t}^{*}(s),fo(s)>|\leq\sup_{s\in S}$I$f_{t}^{*}(s)||_{*}||fo(s)||\leq M$II$f_{0}||$.

(3.15)

So it follows that for each $s\in S$$|I^{*}( \pi^{*(k+1)}, f_{0})(s)|\leq\frac{N+M||f_{0}||}{1-\beta}$,(3.16)

where
denotes a sequence of infinite functions after the time $t=k+1$
in , i.e.,. Thus, we need to show the result of
the theorem for $I^{*}(\pi^{*}, \pi)(\cdot)\in V(S)$ . Then, if $I^{*}(\pi^{*}, \pi)(x)=\infty$ , the result of thetheorem is obvious. So, it is sufficient to show that the result holds only when
$I^{*}(\pi^{*}, \pi)(x)<\infty$ . If $I^{*}(\pi^{*}, \pi)(x)<\infty$ , for sufficiently small $\eta>0$ , from (3.4), it
follows that there exists a sufficiently large integer $k>0$ such that for any state$x\in S$ ,$\pi^{*(k+1)}$$\pi^{*}$$\pi^{*(k+1)}=(f_{k^{*}+1}, f_{k^{*}+2}, f_{k^{*}+3)}\cdots)$$|I^{*}(\pi^{*}, \pi)(x)-V^{k}(\pi^{*}, \pi)(x)|<\eta$.

115
Thus, to prove the theorem, it is sufficient to show the result of the theorem for$V^{k}(\pi^{*}; \pi)(\cdot)\in V(S),$. From (3.14),
is successively constructedby the modified operators $T(f^{*}, f)$ . So, from Lemma 3.2 and the conditions of
the theorem, it follows that, for , there exists a function: $Sarrow A^{*}$ , suchthat, for each $s\in S$ and $f\in D$ ,$V^{k}(\pi^{*}, \pi)(x)$$\neq\infty$$f_{0}$$f_{k^{*}}$$T(f_{k}^{*}, f)u^{*}(x)\geq T(f_{k}^{*}, f_{0})u^{*}(x)-\epsilon$.

(3.17)

So, applying Lemma 3.2 to $T(f_{k^{*}}, f_{k})u^{*}(.\cdot)\in V(S),$with theinstead of in (3.17), we obtain a function
functionsuch that, for each $x\in S$ and $f\in D$$k$$\neq\infty$$f_{k}in\pi$th decision$f_{k-1}\in\partial_{\epsilon}G(f_{0})T(f_{k^{*}}, f_{k})u^{*}(s)$$f$$T(f_{k^{*}-1}, f)T(f_{k^{*}}, f_{k})u^{*}(x)\geq T(f_{k^{*}-1}, f_{0})T(f_{k^{*}}, f_{k})u^{*}(x)-\epsilon$.

(3.18)

Then, since
is a monotone operator on $V(S)$ , combining (3.17) with (3.18)
and using the $(k-1)$ th decision function $f_{k-1}in\pi$ instead of in (3.18), we
obtain for the policy $\pi=(f_{1}, f_{2}, \cdots, f_{t}, \cdots)$ ,$T(\cdot, \cdot)$$f$$T(f_{k-1}^{*}, f_{k-1})T(f_{k}^{*}, f_{k})u^{*}(x)\geq T(f_{k-1}^{*}, f_{0})T(f_{k}^{*}, f_{k})u^{*}(x)-\epsilon$$\geq T(f_{k-1}^{*}, f_{0})T(f_{k}^{*}, f_{0})u^{*}(x)-\beta\epsilon-\epsilon$.

(3.19)

Further, applying Lemma 3.2 to (3.19) repeatedly, we arrive at$T(f_{1^{*}}, f_{1})T(f_{2^{*}}, f_{2}) \cdots T(f_{k^{*}-1}, f_{k-1})T(f_{k^{*}}, f_{k})u^{*}(x)-\sum_{i=1}^{k-1}\beta^{i}\epsilon$$\geq T(f_{1}^{*}, f_{0})T(f_{2}^{*}, f_{0})\cdots T(f_{k-1}^{*}, f_{0})T(f_{k}^{*}, f_{0})u^{*}(x)-\sum_{i=1}^{k}\beta^{i}\epsilon$

Thus, from (3.20), we get for sufficiently large$\epsilon(1-\beta)^{-1}$as$\epsilon$

and

$k$

as

$\infty$(3.20)$k$$V^{k}( \pi^{*}, \pi)(x)\geq V^{k}(\pi^{*}, f_{0})(x)-\frac{\epsilon}{1-\beta}$

So, taking
completed.

.

.

(3.21)in (3.21), the proof of the theorem is

THEOREM 3.2 Let everything be as in $L^{\cdot}emma3.2$ and assume that, foreach $u\in M(S)$ , the zero vector,
belongs tofor all $s\in S$ , that is,for all $u\in M(S)$ and $s\in S$ .
Then, for any policy
we have$\theta^{*}$$\partial_{\epsilon}G(f_{0})u(s)$$\theta^{*}\in\partial_{\epsilon}G(f_{0})u(s)$$\pi_{f}$$I(\pi)(x)\geq I(f_{0})(x)-\epsilon$.

116
for all $u\in M(S)$ and $s\in S$ , we can choose, as each functionof in Theorem 3.1. So, for any policy$t=1,2.,$is equal to $I(\pi)(x)$ . Thus, the proof is completed.

Proof. Since$f_{t^{*}}=\theta^{*},$$\pi,$$\theta^{*}\in\partial_{\epsilon}G(f_{0})u(s)$$\cdots$$f_{t^{*}}$$\pi^{*}$$I^{*}(\pi^{*}, \pi)(x)$

参考文献[1] R.Bellman (1970), Dynamic programming and inverse optimal problems in
Mathematical economics, J. Math. Anal. Appl. 29, 424-428.
[2] R. Bellman, H. Kagiwada and R. Kalaba (1967), Dynamic programming and an
inverse problem in transport theory, Computing, 2, 5-16.
[3] R.Bellman and R.Kalaba (1963), An inverse problem in dynamic programming and automatic control, J. Math. Anal. Appl. 7, 322-325.

[4] R.Bellman and J.M.Richardson (1961), A note on an inverse problem inmathematical Physics, Quart. Appl. Math. 19, 269-271.

[5] E.B.Dynkin and A.A.Yushkevich (1979), Controlled Markov Process,Springer-Verlag, New York.
[6] I.Ekeland (1979), Nonconvex minimization problems, Bull. Amer. Math. Soc.
3, No. 1, 443-474.

[7] I.Ekeland (1974), On the variational principle, J. Math. Anal. Appl. 47, 324353.

[8] K.Kuratowaski and Ryll-Nardzewski (1965), A general theorem in selectors,
Bull. Acad. Polon. Sci. 13, 379-403.

[9] D.G.Luenberger (1968), optimization by Vector Space Methods, John Wileyand Sons, Inc., New York.
[10] R.R.Phelps (1989), Convex Functions, Monotone Operators and Differentiability, Lecture notes in Mathematics 1364, Springer-Verlag, Berlin.

[11] J.-B.Hiriart-Urruty (1980),
mont.

$\mathcal{E}$-subdifferential Calculus, Universit\^e de Cler-[12] K.Tanaka and K.Yokoyama (1991), On -equilibrium point in a noncooperative n-person game, J. Math. Anal. Appl. Vol.160, No.2, 413-423.
$\epsilon$