# Value Function and Optimal Rule on the Optimal Stopping Problem for Continuous-Time Markov Processes

Authors: Lu Ye
Journal: Chinese Journal of Mathematics
 (2017)

---

## Abstract

This paper considers the optimal stopping problem for continuous-time Markov processes. We describe the methodology and solve the optimal stopping problem for a broad class of reward functions. Moreover, we illustrate the outcomes by some typical Markov processes including diffusion and Lévy processes with jumps. For each of the processes, the explicit formula for value function and optimal stopping time is derived. Furthermore, we relate the derived optimal rules to some other optimal problems.


---

## Body

## 1. Introduction

LetΩ,F,P be a complete probability space; the problem studied in this paper is to find the optimum (1)vx=supτ⁡Exe-rτ-tfXτ∣Ft,where r>0 is the discount rate and τ≥t is a stopping time. The process Xt is a continuous-time Markov process with starting state Xt=x. When Xt denotes the stock price and f· is the payoff function, vx is the pricing expression for American option (e.g., see Wong [1]). When f· is an investor’s utility function about the stock price Xt, the optimum solution indicates the best time to buy or sell the stock (e.g., see McDonald and Siegel [2]).Because of the Markov property, the value functionvx can be written as (2)vx=supτ⁡Exe-rτ-tfXτ∣Xt=x≜supτ⁡Exe-rτ-tfXτ,where Ex· denotes the conditional expectation Ex·∣Xt=x. The function f· is called the reward function. The possibility that τ=∞ is allowed, and making the convention that (3)e-rτfXτ=0ifτ=∞,it means that if an option is never exercised, then its reward payment is valueless for the investor.In the paper, we are interested in determining both an optimal stopping timeτ∗ and value function vx for a large class of reward functions. We show the conditions for reward functions and deduce the explicit optimal rules for general continuous-time Markov processes including diffusion and Lévy processes with jumps.The study of optimal stopping time for stochastic processes, especially geometric Brownian motion, has a long history in finance literature. Under the assumption thatXt is geometric Brownian motion, the seminal paper by McDonald and Siegel [2] puts forward the problem with the reward function fx=1-x as a model to illustrate the financial decision making. Hu and Øksendal [3] solved the problem in multidimensional cases, when f(x)=∑i=1n‍xi-∑i=n+1m‍xi, but they restricted the stopping time τ in a bounded interval. Recently, Nishide and Rogers [4] extended the problem by relaxing the restriction on the stopping time. For the other forms of value functions, Pedersen and Peskir [5] solved the problem by taking some special diffusion processes.The purpose of our work is threefold. Firstly, we intend to make clear the assumptions on reward functionfx, in such a way that the explicit value can be generalized to a larger class of issues. For this purpose, throughout this article we assume that the function fx is nonincreasing, concave and twice continuous differentiable. These properties are very powerful in the following proof, as we will see. Secondly, we pay attention to general Markov processes including diffusion and Lévy processes with jumps. With the help of the infinitesimal generator, we obtain an explicit formula for the value function and the stopping time. Thirdly, we find that the optimal problem (2) is equivalent to other optimal problems like (4)wx=supτ⁡Ex∫tτe-rs-tfXsds,(5)w^x=infτ⁡Ex∫tτe-rs-tfXsds.This work is inspired by Pedersen and Peskir [5], who verified the equivalence of problems (2) and (4) for fx=x, where Xt is an Ornstein-Uhlenbeck process. Note that we do not consider the case fx=x, so our approach to deal with the optimal problem is different from that of Pedersen and Peskir. Moreover, our work naturally explore the explicit solutions of the new optimal problem (4) for a larger class of reward functions and underlying processes Xt.The paper is organized as follows. In Section2, the explicit value function and optimal stopping time are derived for a general Markov process along with the condition for the reward functions. Section 3 discusses some applications to diffusion, which include Brownian motion with drift, geometric Brownian motion, and the Ornstein-Uhlenbeck process. Section 4 displays some concrete examples of Lévy processes with jumps. In Section 5, we will link the outcomes with other optimal problems such that explicit solutions for the new problems can also be feasible to a general Markov process with a large class of reward functions. Finally, concluding remarks are given in Section 6.


## 2. Optimal Rule for Continuous-Time Markov Processes

For a Markov processXt, the infinitesimal generator of Xt is defined as (6)Ggx=limt↘0ExgXt-gxt,where gx is twice continuous differentiable. In the diffusion case, namely, (7)dXt=aXtdt+σXtdWt,where Wt is a standard Brownian motion, a(x)>0,σ(x)≠0, the infinitesimal generator is equivalent to (8)Ggx=axdgdx+σ2x2d2gdx2.In the case of Lévy process with jumps, driven by the equation (9)dXt=aXt-dt+σXt-dWt+cXt-dNt,where Nt is a homogeneous Poisson process, ax>0,σx≠0,cx>0, the infinitesimal generator is (10)Ggx=axdgdx+σ2x2d2gdx2+∫gx+cy-gxΠdy,where Π· is the Lévy measure (for jump diffusion and its generators, e.g., we can refer to Gihman and Skorohod [6]).First, we make the assumption about the reward function.Assumption 1.
The reward functionfx is nonincreasing, concave, and twice continuous differentiable; that is, dfx/dx≤0, d2fx/dx2≤0, and fx is C2.Under Assumption1, we present the explicit solutions for general continuous-time Markov processes.Theorem 2.
For a Markov process with infinitesimal generatorG, let gx be the solution of (11)Ggx=rgx,satisfying (12)limx→∞⁡gx=0,dgxdx≤0,d2gxdx2≥0.Given the reward function fx in Assumption 1, if there exists a point x∗ such that (13)dfxdxx=x∗=fx∗gx∗dgxdxx=x∗,fx∗>0,then the optimal problem (14)vx=supτ⁡Exe-rτ-tfXτhas an explicit expression for the value function (15)vx=fx,x≤x∗,fx∗gx∗gx,x≥x∗.The optimal stopping time is (16)τ∗=inf⁡s≥t:Xs≤x∗.Proof.
Define a function(17)hx=fx∗gx∗gx-fx,and then we get (18)hx=0,dhxdx=fx∗gx∗dgxdx-dfxdx=0iffx=x∗,(19)d2hxdx2=fx∗gx∗d2gxdx2-d2fxdx2≥0.Hence, hx attains the minimum at x=x∗, fx∗/gx∗g(x)≥fx. So we have the inequality (20)Exe-rτ-tfXτ≤Exe-rτ-tvXτ.
The value functionvx is C1, v′′x exists, and is continuous except at x∗. By Itô’s formula, (21)e-ruvXu=e-rtvXt+local  martingale+∫tue-rsG-rvXsds,for any time u>t, where G is the infinitesimal generator of the process Xt.
Because the value functionv(x) is bounded, the local martingale term on the right-hand side of  (21) is also bounded (all the other terms in (21) are clearly bounded), implying that it is in fact a martingale with zero expectation. Hence, by the optimal sampling theorem, we have, for any stopping time τ, (22)Exe-rτ-tvXτ=vX0+Ex∫tτe-rs-tG-rvXsds.
(a) Whenx≤x∗, v(x)=f(x), we claim that (23)G-rvx=G-rfx≤0.
For diffusion,(24)G-rfx=axdfdx+σ2x2d2fdx2-rfx,ax>0,dfx/dx≤0, d2f(x)/dx2≤0, fx≥fx∗>0, and the inequality in (23) holds naturally. For Lévy processes with jumps, (25)G-rfx=axdfdx+σ2x2d2fdx2+∫fx+cy-fxΠdy-rfx.As cy>0 and fx is decreasing on states, then ∫fx+cy-fxΠdy<0.
(b) Whenx≥x∗, vx=fx∗/gx∗gx, the function gx is the solution of (26)Ggx=rgx,so G-rvx=0.
Therefore, from (a) and (b), we haveG-rvx≤0, and the equality holds for x≥x∗. It turns to be (27)Exe-rτ-tfXτ≤Exe-rτ-tvXτ=vX0+Ex∫tτe-rs-tG-rvXsds≤vx.From which we can see that v(x) is an upper bound for the value starting from X0=x. This bound is achieved when τ=τ∗. When the starting state x is smaller than x∗, the optimal stopping time is τ∗=t. That is, supτExe-r(τ-t)f(Xτ)=Exf(Xt)=v(x) as it reaches its upper bound. If the starting state is greater than the point x∗, it must wait until τ∗, which is the first hitting time to the point x∗. At the stopping time τ∗, f(Xτ∗)=v(Xτ∗), and ∫tτ∗e-rs-tG-rvXsds=0, so Exe-rτ∗-tfXτ∗=vx, reaching its upper bound.Remark 3.
Actually, for all diffusion, Theorem2 holds for the drift term satisfying ax<0,a′(x)<r. We calculate that (28)axdfxdx-rfx′=a′xf′x+axf′′x-rf′x≥0,so for x≤x∗, (29)axf′x-rfx≤ax∗f′x∗-rfx∗=fx∗gx∗ax∗g′x∗-rgx∗.As gx∗ satisfies ax∗g′x∗+σ2x∗/2g′′x∗-rgx∗=0, we can arrive at (30)G-rfx=axf′x+σ2x2f′′x-rfx≤-σ2x∗2fx∗gx∗g′′x∗+σ2x2f′′x≤0.The rest of the proof is the same as that in Theorem 2.However, the condition for Lévy process will be more complicated. In order to have a uniform style, we restrict to the caseax>0. Moreover, we can see from (19) that if the point x∗ exists, then it is unique. Next we present results on some classical Markov processes as applications of Theorem 2.


## 3. Diffusion

### 3.1. Brownian Motion with Drift

For Brownian motion with driftμ>0, and variance σ2σ≠0, namely, the process is driven by the SDE (31)dXt=σdWt+μdt,X0=x>0,by using Theorem 2, we get the following proposition.Proposition 4.
Letfx be the function in Assumption 1; if there exists a point x∗ such that (32)dfxdxx=x∗=-μ2σ4+2rσ2+μσ2fx∗,fx∗>0,then the value function v(x) has the form (33)vx=fx,x≤x∗,fx∗exp⁡μ2σ4+2rσ2+μσ2x∗-x,x≥x∗.
The optimal stopping time is(34)τ∗=inf⁡s≥t:Xs≤x∗.Proof.
The infinitesimal generator for Brownian motion with drift isG=μd/dx+σ2/2d2/dx2. It is well known that the ordinary differential equation (ODE) of Ggx=rgx has two linearly independent solutions (35)ϕx=exp⁡xμ2σ4+2rσ2-μσ2,φx=exp⁡-xμ2σ4+2rσ2+μσ2.So gx=c1ϕx+c2φx (c1 and c2 are constants). Considering the boundary condition limx→∞gx=0 and g′x≤0,g′′x≥0, then c1 must be equal to zero, g(x)=c2φx, and c2>0. Equation (13) in Theorem 2 tells us that the point x∗ is determined by (36)f′x∗=fx∗gx∗g′x∗=-μ2σ4+2rσ2+μσ2fx∗,since g′x/gx=-μ2/σ4+2r/σ2-μ/σ2. Thus, the expression for the value function v(x) is easily obtained by (15).As the simplest example, we take the reward functionf(x)=1-x and we take the parameters μ=1, σ=2, and r=1. Then, the problem v(x)=supτ⁡Exe-τ-t1-Xτ has the solution (37)vx=1-x,x≤32-52,52-12exp⁡-52+12x+52-12,x≥32-52.The optimal stopping time is (38)τ∗=inf⁡s≥t:Xs≤32-52.We draw the value function and the point x∗ in Figure 1.Figure 1
Plot of the value function and the pointx∗x∗≃0.3820 for Brownian motion with parameters μ=1, σ=2, and r=1.


### 3.2. Geometric Brownian Motion

As for geometric Brownian motion, that is, the processXt is driven by (39)dXt=XtσdWt+μdt,X0=x>0,we derive the following proposition.Proposition 5.
Letfx be the function in Assumption 1; if there exists a point x∗ such that (40)dfxdxx=x∗=βfx∗x∗,fx∗>0,where β=1/2-μ/σ2-1/2-μ/σ22+2r/σ2, then the value function vx has the form (41)vx=fx,x≤x∗,fx∗xx∗β,x≥x∗.
The optimal stopping time is(42)τ∗=inf⁡s≥t:Xs≤x∗.Proof.
The infinitesimal generator for geometric Brownian motion isG=μxd/dx+σ2x2/2d2/dx2. The associated ODE of Ggx=rgx has two linearly independent solutions ϕxβ′ and φxβ, where β′=1/2-μ/σ2+1/2-μ/σ22+2r/σ2 and β=1/2-μ/σ2-1/2-μ/σ22+2r/σ2. By using the boundary conditions of Theorem 2, we find that gx=cφxβ, where c is a constant bigger than zero. Then, by substituting gx in Theorem 2, we deduce the value function and the point x∗.Remark 6.
If the volatilityσ is equal to zero, then the drift μ should be smaller than zero. Or the problem cannot be solved by Theorem 2. We take the reward function fx=1-x as an illustration. For σ=0 and μ<0, from Theorem 2, the solution for the optimal value function is (43)vx=1-x,x≤rr-μ,-μr-μrx-μxrr/μ,x≥rr-μ.
The optimal stopping time is(44)τ∗=inf⁡t:Xt≤rr-μ.If σ=0 and μ>0, dXs=μXsds, that is, Xs=xeμs-t, for any time s≥t. By direct computation, we find that (45)vx=supτ⁡Exe-rτ-t1-xeμτ-t=0,x≥1,1-x,x≤1,with the optimal stopping time (46)τ∗=inf⁡t:Xt≤1.When the starting state is smaller than 1, it should take action at once. When the starting state is greater than 1, it will never invest, because the process is increasing, and it never reaches its optimal invest time. But, in this case, the point x∗=1 is not determined by (13). Because the left side is -1 and the right side is 0 if we replace fx=1-x and x∗=1 in (13).
Taking parametersμ=1, σ=2, and r=1, the value function for the reward function f(x)=1-x is (47)vx=1-x,x≤12,14x,x≥12.
The optimal stopping time is(48)τ∗=inf⁡t:Xt≤12.
This problem’s solution is the key to finding the optimum(49)vx=supτ⁡Exe-rτXτ1-Xτ2,which has many applications in finance, and it has been considered for geometric Brownian motion from different points of view in a variety of articles. Let us just mention [1–4, 7–9].


### 3.3. Ornstein-Uhlenbeck Process

In this subsection we consider the optimal stopping problem whenX is an Ornstein-Uhlenbeck process, (50)dXt=σdWt-μXtdt,X0=x.Proposition 7.
Letfx be as in Assumption 1; if there exists a point x∗ such that (51)dfxdxx=x∗=-fx∗Gr/μ+1x∗Gr/μx∗,fx∗>0,then the value function vx has the form (52)vx=fx,x≤x∗,fx∗Gr/μxGr/μx∗,x≥x∗,where (53)Gr/μx=∫0∞sr/μ-1exp⁡-xs-σ2s24μds.
The optimal stopping time is(54)τ∗=inf⁡s≥t:Xs≤x∗.Proof.
For the Ornstein-Uhlenbeck process, the infinitesimal generator isG=-μxd/dx+σ2/2d2/dx2. The ODE of Ggx=rgx has two linearly independent solutions (55)Fr/μx=∫0∞sr/μ-1exp⁡xs-σ2s24μds,Gr/μx=∫0∞sr/μ-1exp-xs-σ2s24μds.The solution satisfying the boundary conditions in Theorem 2 is gx=cGr/μx, where c is a constant bigger than zero. Thanks to the relations (56)Gr/μx=μxrGr/μ+1x+σ22rGr/μ+2x,dGr/μxdx=-Gr/μ+1x,d2Gr/μxdx=Gr/μ+2x,we obtain the value function and the point x∗.Regarding the reward functionfx=1-x, if there exists a point x∗ such that fx∗=1-x∗>0 and Gr/μ+1x∗1-x∗=Gr/μx∗, the problem vx has the form (57)vx=1-x,x≤x∗,1-x∗Gr/μxGr/μx∗,x≥x∗.The optimal stopping time is (58)τ∗=inf⁡s≥t:Xs≤x∗.When parameters are chosen as μ=1,σ=2, and r=1, the point x∗ is the solution of (59)G2x1-x=G1x.Thanks to(60)G1x=∫0∞exp⁡-xs-s22ds=exp⁡x22∫x∞exp⁡-s22ds,G2x=1-xG1x,G1x and G2x can be reduced to expressions of gammaincx2/2,1/2 in Matlab. By numerical calculation, we get the point x∗≃-0.1628 and we show the value function in Figure 2.Figure 2
The value function and the pointx∗≃-0.1628 for the Ornstein-Uhlenbeck process with parameters μ=1, σ=2, and r=1, with regard to the reward function fx=1-x.


## 3.1. Brownian Motion with Drift

For Brownian motion with driftμ>0, and variance σ2σ≠0, namely, the process is driven by the SDE (31)dXt=σdWt+μdt,X0=x>0,by using Theorem 2, we get the following proposition.Proposition 4.
Letfx be the function in Assumption 1; if there exists a point x∗ such that (32)dfxdxx=x∗=-μ2σ4+2rσ2+μσ2fx∗,fx∗>0,then the value function v(x) has the form (33)vx=fx,x≤x∗,fx∗exp⁡μ2σ4+2rσ2+μσ2x∗-x,x≥x∗.
The optimal stopping time is(34)τ∗=inf⁡s≥t:Xs≤x∗.Proof.
The infinitesimal generator for Brownian motion with drift isG=μd/dx+σ2/2d2/dx2. It is well known that the ordinary differential equation (ODE) of Ggx=rgx has two linearly independent solutions (35)ϕx=exp⁡xμ2σ4+2rσ2-μσ2,φx=exp⁡-xμ2σ4+2rσ2+μσ2.So gx=c1ϕx+c2φx (c1 and c2 are constants). Considering the boundary condition limx→∞gx=0 and g′x≤0,g′′x≥0, then c1 must be equal to zero, g(x)=c2φx, and c2>0. Equation (13) in Theorem 2 tells us that the point x∗ is determined by (36)f′x∗=fx∗gx∗g′x∗=-μ2σ4+2rσ2+μσ2fx∗,since g′x/gx=-μ2/σ4+2r/σ2-μ/σ2. Thus, the expression for the value function v(x) is easily obtained by (15).As the simplest example, we take the reward functionf(x)=1-x and we take the parameters μ=1, σ=2, and r=1. Then, the problem v(x)=supτ⁡Exe-τ-t1-Xτ has the solution (37)vx=1-x,x≤32-52,52-12exp⁡-52+12x+52-12,x≥32-52.The optimal stopping time is (38)τ∗=inf⁡s≥t:Xs≤32-52.We draw the value function and the point x∗ in Figure 1.Figure 1
Plot of the value function and the pointx∗x∗≃0.3820 for Brownian motion with parameters μ=1, σ=2, and r=1.


## 3.2. Geometric Brownian Motion

As for geometric Brownian motion, that is, the processXt is driven by (39)dXt=XtσdWt+μdt,X0=x>0,we derive the following proposition.Proposition 5.
Letfx be the function in Assumption 1; if there exists a point x∗ such that (40)dfxdxx=x∗=βfx∗x∗,fx∗>0,where β=1/2-μ/σ2-1/2-μ/σ22+2r/σ2, then the value function vx has the form (41)vx=fx,x≤x∗,fx∗xx∗β,x≥x∗.
The optimal stopping time is(42)τ∗=inf⁡s≥t:Xs≤x∗.Proof.
The infinitesimal generator for geometric Brownian motion isG=μxd/dx+σ2x2/2d2/dx2. The associated ODE of Ggx=rgx has two linearly independent solutions ϕxβ′ and φxβ, where β′=1/2-μ/σ2+1/2-μ/σ22+2r/σ2 and β=1/2-μ/σ2-1/2-μ/σ22+2r/σ2. By using the boundary conditions of Theorem 2, we find that gx=cφxβ, where c is a constant bigger than zero. Then, by substituting gx in Theorem 2, we deduce the value function and the point x∗.Remark 6.
If the volatilityσ is equal to zero, then the drift μ should be smaller than zero. Or the problem cannot be solved by Theorem 2. We take the reward function fx=1-x as an illustration. For σ=0 and μ<0, from Theorem 2, the solution for the optimal value function is (43)vx=1-x,x≤rr-μ,-μr-μrx-μxrr/μ,x≥rr-μ.
The optimal stopping time is(44)τ∗=inf⁡t:Xt≤rr-μ.If σ=0 and μ>0, dXs=μXsds, that is, Xs=xeμs-t, for any time s≥t. By direct computation, we find that (45)vx=supτ⁡Exe-rτ-t1-xeμτ-t=0,x≥1,1-x,x≤1,with the optimal stopping time (46)τ∗=inf⁡t:Xt≤1.When the starting state is smaller than 1, it should take action at once. When the starting state is greater than 1, it will never invest, because the process is increasing, and it never reaches its optimal invest time. But, in this case, the point x∗=1 is not determined by (13). Because the left side is -1 and the right side is 0 if we replace fx=1-x and x∗=1 in (13).
Taking parametersμ=1, σ=2, and r=1, the value function for the reward function f(x)=1-x is (47)vx=1-x,x≤12,14x,x≥12.
The optimal stopping time is(48)τ∗=inf⁡t:Xt≤12.
This problem’s solution is the key to finding the optimum(49)vx=supτ⁡Exe-rτXτ1-Xτ2,which has many applications in finance, and it has been considered for geometric Brownian motion from different points of view in a variety of articles. Let us just mention [1–4, 7–9].


## 3.3. Ornstein-Uhlenbeck Process

In this subsection we consider the optimal stopping problem whenX is an Ornstein-Uhlenbeck process, (50)dXt=σdWt-μXtdt,X0=x.Proposition 7.
Letfx be as in Assumption 1; if there exists a point x∗ such that (51)dfxdxx=x∗=-fx∗Gr/μ+1x∗Gr/μx∗,fx∗>0,then the value function vx has the form (52)vx=fx,x≤x∗,fx∗Gr/μxGr/μx∗,x≥x∗,where (53)Gr/μx=∫0∞sr/μ-1exp⁡-xs-σ2s24μds.
The optimal stopping time is(54)τ∗=inf⁡s≥t:Xs≤x∗.Proof.
For the Ornstein-Uhlenbeck process, the infinitesimal generator isG=-μxd/dx+σ2/2d2/dx2. The ODE of Ggx=rgx has two linearly independent solutions (55)Fr/μx=∫0∞sr/μ-1exp⁡xs-σ2s24μds,Gr/μx=∫0∞sr/μ-1exp-xs-σ2s24μds.The solution satisfying the boundary conditions in Theorem 2 is gx=cGr/μx, where c is a constant bigger than zero. Thanks to the relations (56)Gr/μx=μxrGr/μ+1x+σ22rGr/μ+2x,dGr/μxdx=-Gr/μ+1x,d2Gr/μxdx=Gr/μ+2x,we obtain the value function and the point x∗.Regarding the reward functionfx=1-x, if there exists a point x∗ such that fx∗=1-x∗>0 and Gr/μ+1x∗1-x∗=Gr/μx∗, the problem vx has the form (57)vx=1-x,x≤x∗,1-x∗Gr/μxGr/μx∗,x≥x∗.The optimal stopping time is (58)τ∗=inf⁡s≥t:Xs≤x∗.When parameters are chosen as μ=1,σ=2, and r=1, the point x∗ is the solution of (59)G2x1-x=G1x.Thanks to(60)G1x=∫0∞exp⁡-xs-s22ds=exp⁡x22∫x∞exp⁡-s22ds,G2x=1-xG1x,G1x and G2x can be reduced to expressions of gammaincx2/2,1/2 in Matlab. By numerical calculation, we get the point x∗≃-0.1628 and we show the value function in Figure 2.Figure 2
The value function and the pointx∗≃-0.1628 for the Ornstein-Uhlenbeck process with parameters μ=1, σ=2, and r=1, with regard to the reward function fx=1-x.


## 4. Lévy Processes with Jumps

### 4.1. The Jump Diffusion

Jump diffusion processes are processes of the form(61)dXt=σdWt+μdt+d∑i=1NtYi,X0=x.They are studied in Kou and Wang [10], as regards first passage times. Here, Nt is a homogeneous Poisson process with intensity rate λ>0, drift μ>0, and volatility σ>0, and the jump sizes Y1,Y2,… are independent and identically distributed random variables. We also assume that the standard Brownian motion Wt is independent of Y1,Y2,…. The common density of Y is given by (62)pYy=ηe-ηy,where η>0.Proposition 8.
Regarding the reward functionfx in Assumption 1, if there exists a point x∗ such that (63)dfxdxx=x∗=βfx∗,fx∗>0,where β<0 is the negative root of (64)σ22y2+μy+λyη-y-r=0,then the value function vx has the form (65)vx=fx,x≤x∗,fx∗exp⁡βx-βx∗,x≥x∗.The optimal stopping time is (66)τ∗=inf⁡s≥t:Xs≤x∗.Proof.
In this case(67)G-rvx=μdvxdx+σ22d2vxdx2+λ∫0∞vx+y-vxηe-ηydy-rvx.(a)
Whenx≤x∗, vx+y-vx≤0 for every y>0, dfx/dx≤0, d2f(x)/dx2≤0, for μ>0, and G-rvXs≤0.(b)
Whenx≥x∗, G-rvx=1/2σ2β2+μβ+λβ/η-β-rfx∗exp⁡βx-βx∗=0. Since G-rvx≤0 for all x, the rest of the proof is the same as in Theorem 2.Remark 9.
We assume thatμ>0 as Kou and Wang [10] do, and we do not know whether the results hold for μ≤0.
LetH(y) be defined as H(y)=1/2σ2y2+μy+λy/η-y, and it has three roots, y1=0, y2=η/2-μ/σ2+η/2+μ/σ22+2λ/σ2>0, and y3=η/2-μ/σ2-η/2+μ/σ22+2λ/σ2<0.
Next, we prove that the equationH(y)=r for all r>0 has exactly a negative root β. (68)H′y=σ2y+μ+ληη-y2,H′′y=σ2+2ληη-y3,so Hy is increased and convex on the interval 0,η with H0=0 and Hη-=∞, and there is exactly one root for Hy=r. Furthermore, since Hη+=-∞, H(∞)=∞, there is at least one root on (η,∞). Similarly, there is at least one root on -∞,0, as H-∞=∞ and H(0)=0. But the equation H(y)=r is actually a polynomial equation with degree three; therefore, it can have at most three real roots. It follows that, on each interval, (-∞,0) and η,∞, there is exactly one root.As an example, we take the reward functionfx=1-x again. From Proposition 8, x∗=1+1/β, where β is the negative root of (69)12σ2y2+μy+λyη-y-r=0,and the value function vx has the form (70)vx=1-x,x≤x∗,1-x∗exp⁡βx-βx∗,x≥x∗.The optimal stopping time is(71)τ∗=inf⁡s≥t:Xs≤x∗.If the parameters areμ=1, σ=2, r=1, λ=1, and η=1.5, we show the value function and the point x∗ in Figure 3. Numerical calculation gets the negative solution for β to be β≃-1.8421, and the point x∗≃0.4571. Compared with the case of Brownian motion with drift, the decision point x∗ results larger after adding the jump process. Intuitively, as risk in the system is increasing, the investors are intent to wait and observe for a longer time, so that the optimal stopping occurs later and the decision point gets greater.Figure 3
The value function and the pointx∗x∗≃0.4571 for the jump diffusion model with parameters μ=1, σ=2, r=1, λ=1, and η=1.5.The more general problem, when the common density ofY is given by (72)pYy=∑i=1mpiηie-ηiy,where 0<η1<η2<⋯<ηm<∞, and ∑i=1mpi=1, has the following result.Proposition 10.
Letfx be as in Assumption 1; if there exists a point x∗ such that (73)dfxdxx=x∗=βfx∗,fx∗>0,where β<0 is the negative root of (74)σ22y2+μy+λ∑i=1mpiyηi-y-r=0,then the value function vx has the form (75)vx=fx,x≤x∗,fx∗exp⁡βx-βx∗,x≥x∗.The optimal stopping time is (76)τ∗=inf⁡s≥t:Xs≤x∗.Remark 11.
The equationσ2/2y2+μy+λ∑i=1m‍piy/ηi-y-r=0 has m+2 roots, which are all real and distinct. Indeed, it has m+1 positive roots, β1,…,βm+1, and a negative root β, as follows: (77)-∞<β<0<β1<⋯<βm+1<∞.
DefineLy as (78)Ly≜σ22y2+μy+λ∑i=1mpiyηi-y,and we have L0=0, Lη1-=∞, Lη1+=-∞, Lη2-=∞,…,Lηm+=-∞, L∞=∞, and Ly is continuous and increasing in every interval 0,η1-, η1+,η2-,…,ηm+,∞. So it has m+1 positive roots, if it also has a negative root β then the negative root is unique as it has at most m+2 real roots. The detailed proof can be found in Theorem 3.1 of Kou and Cai [11].


### 4.2. The Exponential Lévy-Type Stochastic Integral

The exponential Lévy-type stochastic integralXt is given by (79)dXtXt-=σdWt+μdt+d∑i=1NteYi-1,X0=x>0.Here, Nt is a Poisson process with intensity rate λ>0, and μ and σ are positive constants. The jump sizes Y1,Y2,… are independent and identically distributed random variables and independent of Wt. The common density of Y is given by (80)pYy=ηe-ηy,where η>1 ensures that X has a finite expectation.Proposition 12.
Given the reward functionfx in Assumption 1, if there exists a point x∗ such that (81)dfxdxx=x∗=βfx∗x∗,fx∗>0,where β<0 is the negative root of (82)σ22y2y-1+μy+λyη-y-r=0,then the value function vx has the form (83)vx=fx,x≤x∗,fx∗xx∗β,x≥x∗.
The optimal stopping time is(84)τ∗=inf⁡s≥t:Xs≤x∗.Proof.
In this case(85)G-rvx=μxdvxdx+σ2x22d2vxdx2+λ∫0∞vxey-vxηe-ηydy-rvx.(a)
Whenx≤x∗, because y≥0 and ey≥1, then xey≥x and fxey≤fx. Moreover, thanks to μ>0, df(x)/dx≤0, d2f(x)/dx2≤0, we have (G-r)v(Xs)≤0.(b)
Whenx≥x∗, (86)G-rvx=12σ2β2-β+μβ+λβη-β-rfx∗xx∗β=0.Taking (a) and (b) together, we conclude that G-rvx≤0 for every x>0. Hence, vx is the upper bound for Exe-rτ-tfXτ. At the optimal stopping time τ∗, Exe-rτ∗-tfXτ∗=vx, attaining the upper bound.Remark 13.
The equationσ2/2(y-1)+μy+λy/η-y-r=0 has a unique negative root β. This can be obtained proceeding as in Remark 11.
For the reward functionfx=1-x, β is the negative root of (87)σ22yy-1+μy+λyη-y-r=0,we find x∗=β/β-1, and the value function is (88)vx=1-x,x≤x∗,1-x∗xx∗β,x≥x∗.
The optimal stopping time is(89)τ∗=inf⁡s≥t:Xs≤x∗.For the choice of parameters μ=1, σ=2, r=1, λ=1, and η=1.5, we obtain β≃-1.2020 and the point x∗≃0.5459, and we draw the graph of the value function in Figure 4. The point x∗ is bigger than the value concerning geometric Brownian motion, after adding the jump risk.Figure 4
The value function and the pointx∗(x∗≃0.5459) for μ=1, σ=2, r=1, λ=1, and η=1.5.


## 4.1. The Jump Diffusion

Jump diffusion processes are processes of the form(61)dXt=σdWt+μdt+d∑i=1NtYi,X0=x.They are studied in Kou and Wang [10], as regards first passage times. Here, Nt is a homogeneous Poisson process with intensity rate λ>0, drift μ>0, and volatility σ>0, and the jump sizes Y1,Y2,… are independent and identically distributed random variables. We also assume that the standard Brownian motion Wt is independent of Y1,Y2,…. The common density of Y is given by (62)pYy=ηe-ηy,where η>0.Proposition 8.
Regarding the reward functionfx in Assumption 1, if there exists a point x∗ such that (63)dfxdxx=x∗=βfx∗,fx∗>0,where β<0 is the negative root of (64)σ22y2+μy+λyη-y-r=0,then the value function vx has the form (65)vx=fx,x≤x∗,fx∗exp⁡βx-βx∗,x≥x∗.The optimal stopping time is (66)τ∗=inf⁡s≥t:Xs≤x∗.Proof.
In this case(67)G-rvx=μdvxdx+σ22d2vxdx2+λ∫0∞vx+y-vxηe-ηydy-rvx.(a)
Whenx≤x∗, vx+y-vx≤0 for every y>0, dfx/dx≤0, d2f(x)/dx2≤0, for μ>0, and G-rvXs≤0.(b)
Whenx≥x∗, G-rvx=1/2σ2β2+μβ+λβ/η-β-rfx∗exp⁡βx-βx∗=0. Since G-rvx≤0 for all x, the rest of the proof is the same as in Theorem 2.Remark 9.
We assume thatμ>0 as Kou and Wang [10] do, and we do not know whether the results hold for μ≤0.
LetH(y) be defined as H(y)=1/2σ2y2+μy+λy/η-y, and it has three roots, y1=0, y2=η/2-μ/σ2+η/2+μ/σ22+2λ/σ2>0, and y3=η/2-μ/σ2-η/2+μ/σ22+2λ/σ2<0.
Next, we prove that the equationH(y)=r for all r>0 has exactly a negative root β. (68)H′y=σ2y+μ+ληη-y2,H′′y=σ2+2ληη-y3,so Hy is increased and convex on the interval 0,η with H0=0 and Hη-=∞, and there is exactly one root for Hy=r. Furthermore, since Hη+=-∞, H(∞)=∞, there is at least one root on (η,∞). Similarly, there is at least one root on -∞,0, as H-∞=∞ and H(0)=0. But the equation H(y)=r is actually a polynomial equation with degree three; therefore, it can have at most three real roots. It follows that, on each interval, (-∞,0) and η,∞, there is exactly one root.As an example, we take the reward functionfx=1-x again. From Proposition 8, x∗=1+1/β, where β is the negative root of (69)12σ2y2+μy+λyη-y-r=0,and the value function vx has the form (70)vx=1-x,x≤x∗,1-x∗exp⁡βx-βx∗,x≥x∗.The optimal stopping time is(71)τ∗=inf⁡s≥t:Xs≤x∗.If the parameters areμ=1, σ=2, r=1, λ=1, and η=1.5, we show the value function and the point x∗ in Figure 3. Numerical calculation gets the negative solution for β to be β≃-1.8421, and the point x∗≃0.4571. Compared with the case of Brownian motion with drift, the decision point x∗ results larger after adding the jump process. Intuitively, as risk in the system is increasing, the investors are intent to wait and observe for a longer time, so that the optimal stopping occurs later and the decision point gets greater.Figure 3
The value function and the pointx∗x∗≃0.4571 for the jump diffusion model with parameters μ=1, σ=2, r=1, λ=1, and η=1.5.The more general problem, when the common density ofY is given by (72)pYy=∑i=1mpiηie-ηiy,where 0<η1<η2<⋯<ηm<∞, and ∑i=1mpi=1, has the following result.Proposition 10.
Letfx be as in Assumption 1; if there exists a point x∗ such that (73)dfxdxx=x∗=βfx∗,fx∗>0,where β<0 is the negative root of (74)σ22y2+μy+λ∑i=1mpiyηi-y-r=0,then the value function vx has the form (75)vx=fx,x≤x∗,fx∗exp⁡βx-βx∗,x≥x∗.The optimal stopping time is (76)τ∗=inf⁡s≥t:Xs≤x∗.Remark 11.
The equationσ2/2y2+μy+λ∑i=1m‍piy/ηi-y-r=0 has m+2 roots, which are all real and distinct. Indeed, it has m+1 positive roots, β1,…,βm+1, and a negative root β, as follows: (77)-∞<β<0<β1<⋯<βm+1<∞.
DefineLy as (78)Ly≜σ22y2+μy+λ∑i=1mpiyηi-y,and we have L0=0, Lη1-=∞, Lη1+=-∞, Lη2-=∞,…,Lηm+=-∞, L∞=∞, and Ly is continuous and increasing in every interval 0,η1-, η1+,η2-,…,ηm+,∞. So it has m+1 positive roots, if it also has a negative root β then the negative root is unique as it has at most m+2 real roots. The detailed proof can be found in Theorem 3.1 of Kou and Cai [11].


## 4.2. The Exponential Lévy-Type Stochastic Integral

The exponential Lévy-type stochastic integralXt is given by (79)dXtXt-=σdWt+μdt+d∑i=1NteYi-1,X0=x>0.Here, Nt is a Poisson process with intensity rate λ>0, and μ and σ are positive constants. The jump sizes Y1,Y2,… are independent and identically distributed random variables and independent of Wt. The common density of Y is given by (80)pYy=ηe-ηy,where η>1 ensures that X has a finite expectation.Proposition 12.
Given the reward functionfx in Assumption 1, if there exists a point x∗ such that (81)dfxdxx=x∗=βfx∗x∗,fx∗>0,where β<0 is the negative root of (82)σ22y2y-1+μy+λyη-y-r=0,then the value function vx has the form (83)vx=fx,x≤x∗,fx∗xx∗β,x≥x∗.
The optimal stopping time is(84)τ∗=inf⁡s≥t:Xs≤x∗.Proof.
In this case(85)G-rvx=μxdvxdx+σ2x22d2vxdx2+λ∫0∞vxey-vxηe-ηydy-rvx.(a)
Whenx≤x∗, because y≥0 and ey≥1, then xey≥x and fxey≤fx. Moreover, thanks to μ>0, df(x)/dx≤0, d2f(x)/dx2≤0, we have (G-r)v(Xs)≤0.(b)
Whenx≥x∗, (86)G-rvx=12σ2β2-β+μβ+λβη-β-rfx∗xx∗β=0.Taking (a) and (b) together, we conclude that G-rvx≤0 for every x>0. Hence, vx is the upper bound for Exe-rτ-tfXτ. At the optimal stopping time τ∗, Exe-rτ∗-tfXτ∗=vx, attaining the upper bound.Remark 13.
The equationσ2/2(y-1)+μy+λy/η-y-r=0 has a unique negative root β. This can be obtained proceeding as in Remark 11.
For the reward functionfx=1-x, β is the negative root of (87)σ22yy-1+μy+λyη-y-r=0,we find x∗=β/β-1, and the value function is (88)vx=1-x,x≤x∗,1-x∗xx∗β,x≥x∗.
The optimal stopping time is(89)τ∗=inf⁡s≥t:Xs≤x∗.For the choice of parameters μ=1, σ=2, r=1, λ=1, and η=1.5, we obtain β≃-1.2020 and the point x∗≃0.5459, and we draw the graph of the value function in Figure 4. The point x∗ is bigger than the value concerning geometric Brownian motion, after adding the jump risk.Figure 4
The value function and the pointx∗(x∗≃0.5459) for μ=1, σ=2, r=1, λ=1, and η=1.5.


## 5. Connection with the Other Optimal Problems

As the reward functionfx is C2 in Assumption 1, by Itô’s formula, (90)e-ru-tfXt=fx+∫tue-rs-tGfXs-rfXsds+local  martingale.Assume that (91)GfXs-rfXs=γfXs,where γ≠0 is a constant. Taking expectation in (90), (92)Exe-ru-tfXu=fx+Ex∫tue-rs-tγfXsds+Exlocal  martingale.As assumed in the paper, e-ru-tfXt is bounded, so Exlocalmartingale=0. For a stopping time τ, (93)Exe-rτ-tfXτ=fx+Ex∫tτe-rs-tγfXsds,from which we can see that if γ>0 the problem supτ⁡Exe-r(τ-t)f(Xτ) is equivalent to (94)supτ⁡Ex∫tτe-rs-tfXsds,and if γ<0 the problem supτ⁡Exe-rτ-tfXτ is equivalent to (95)infτ⁡Ex∫tτe-rs-tfXsds.We conclude the solution for the problems supτ⁡Ex∫tτ‍e-r(s-t)f(Xs)ds and infτ⁡Ex∫tτe-rs-tfXsds in the following theorem.Theorem 14.
For a given reward functionfx in Assumption 1, if Gfx-rfx=γfx and γ>0, the solution for (96)wx=supτ⁡Ex∫tτe-rs-tfXsdshas the form (97)wx=0,x≤x∗,fx∗gx∗gxγ-fxγ,x≥x∗.The optimal stopping time is (98)τ∗=inf⁡s≥t:Xs≤x∗.
Ifγ<0, then the solution for (99)w^x=infτ⁡Ex∫tτe-rs-tfXsdshas the form (100)wx=0,x≤x∗,fx∗gx∗gxγ-fxγ,x≥x∗.
The optimal stopping time is(101)τ∗=inf⁡s≥t:Xs≤x∗.
The functiongx is given by (102)Ggx=rgx,satisfying (103)limx→∞gx=0,dgxdx≤0,d2gxdx2≥0,and the point x∗ is determined by (104)dfxdxx=x∗=fx∗gx∗dgxdxx=x∗,fx∗>0.Since the expression ofEx∫tτe-rs-tfXsds can be represented as valuation of equity (see A.3 in Duffie [12]), we will consider the practical application in the stock market in a future work.


## 6. Conclusions

In this paper, we have studied the optimal stopping problems for a continuous-time Markov process, and we have presented the explicit value function and optimal rule.Three main contributions of this paper are as follows. First, we constructed the value functions and optimal rules for the problems with a large class of reward functions, which are different from the previous researches using some specific functions. With simple conditions for the reward functions, we gave a rigorous mathematical proof to deduce the optimal rule based on variational inequalities. Second, we derived the solution of the problem for a general Markov process. Meanwhile, we showed the explicit value functions and optimal stopping times for some concrete Markov processes including diffusion and Lévy processes with jumps. Finally, we linked our results to some optimal problems. Also, we extended the explicit solution of the new optimal problem to a broad class of reward functions and to a general continuous-time Markov process.



---

*Source: 3596037-2017-10-09.xml*