# Optimal Oil-Owner Behavior in Piecewise Deterministic Models

Authors: Atle Seierstad
Journal: Journal of Calculus of Variations
 (2013)

---

## Abstract

Six simple piecewise deterministic oil production models from economics are discussed by using solution tools that are available in the
theory of piecewise deterministic optimal control problems.


---

## Body

## 1. Introduction

Six simple piecewise deterministic models for optimal oil-owner behavior are presented. Their central property is sudden jumps in states. The aim of this paper is to show in admittedly exceedingly simple models how available tools for piecewise deterministic models, namely, the HJB equation and the maximum principle, can be used to solve these models analytically. We are looking for solutions given by explicit formulas. That can only be obtained if the models are simple enough. The models may be too simple to be of much interest in themselves, but they can provide some intuition about features optimal solutions may have in more complicated models.Piecewise deterministic models have been used a number of times in economic problems in the literature; some few scattered references are given that contain such applications [1–4]. I have not been able to find references directly concerned with piecewise deterministic oil production problems. For different probability structures, and for discrete time, a host of related problems has been discussed in the literature; references to such literature have been left out, with one exception. Problems of control of jump diffusions, see [5], encompass piecewise deterministic problems, and some problems appearing in [5] are related to the ones discussed below. A classic reference to piecewise deterministic control problems is [2].In all models below, an unbounded number of jumps in the state can occur at times0<τ1<τ2<τ3<⋯, and, when τj is given, τj+1 is exponentially distributed in [τj,∞) (all τi-τi-1 independent). Sometimes, the size of the jumps is influenced by stochastic variables Vj. Let ω=(τ1,V1,τ2,V2,…). At time t, we imagine that the control values chosen can be allowed to depend on what has happened, that is, on τi, Vi for τi<t, but not on future events, that is, τi, Vi for which τi>t. Such controls (written u(t,ω)) are called nonanticipative. Corresponding state solutions denoted by x(t,ω) are then also nonanticipative. (A general set-up, with further explanations, is given in Appendix.) Frequently below, x(t,ω) will be the wealth of the oil owner. In infinite horizon economic models, the weakest terminal condition that is natural to apply is an expectational no-Ponzi-game condition; namely, liminfT→∞E[e-kTx(T,ω)]≥0, where k is the discount factor. Note that some stronger conditions will be used in some of the models presented in the sequel.


## 2. Model 1

Consider the optimization problem(1)maxu(·)E[∫0Tγ-1e-δtuγdt+ax(T)]a,δ>0,γ∈(0,1),a,δ,γfixed,
where u is the control, subject to the state dynamics,
(2)x˙=kx-u,u≥0,x(0)=x0>0,
where k and x0 are fixed positive constants,
(3)x(τi+)-x(τi-)=αi,αi>0,∑αi<∞,αi fixed. (Formally, only supiαi<∞ is needed.) Let τ0=0. There is a fixed intensity λ of jumps; that is, for i=1,2,…, given τi-1,τi~λe-λ(τi-τi-1), τi≥τi-1 (all τi-τi-1 independent), and we assume δ>k.The interpretation of the model is thatu is consumption rate, αi is the size (in dollars) of an oil field found at time τi, x is wealth, and kx is interest. Oil fields are sold immediately after discovery.Let us solve problem (1)–(3) by using the extremal method (see the appendix). Now, with H=e-δtuγ/γ+p(kx-u), solving the first-order condition for maximum of H, we get that u=eδt/(γ-1)p1/(γ-1) maximizes the Hamiltonian.The adjoint equation (see(A.8) in Appendix) is
(4)p˙(t,j)=-kp(t,j)+λp(t,j)-λp(t,j+1),p(T)=a;
we guess that we do not need the fact that p(t,j) depends on arbitrary initial points (s,y), as in appendix. Equation (4) is satisfied by p(t)=a^e-kt, a^:=aekT (we try the possibility that p(t,j) is even independent of j).Now, forb:=a^1/(γ-1), ρ:=(k-δ)/(γ-1)>0, we get u(t)=eδt/(γ-1)(p(t))1/(γ-1)=be-ρt. Because H is concave in (x,u), ax is linear, and αi is independent of x, sufficient conditions based on concavity (see Theorem 3 in Appendix) then give us that the control u=be-ρt is an optimal control for problem (1)–(3). The optimal control is independent of λ and the αi’s.Writeβ:=b/(k+ρ). For u=be-ρt, we have that x˙*=kx*-be-ρt. The solution of this equation equals, for t∈(τi,τi+1),
(5)x*(t)=βe-ρt+(x0-β)ekt+∑j=1iαjek(t-τj),
To see this, find x˙*(t) and simply check that the differential equation is satisfied. Moreover, evidently x*(·) satisfies x*(τi+)=x*(τi-)+αi. (To ensure that x*(t)≥0 a.s., we could have postulated at the outset that a is so large that x0≥β.)Writex*(t)=ektΦi(t,ω), for t∈(τi,τi+1), where Φi(t,ω)=(e-(ρ+k)t-1)β+∑j=1i‍αje-kτj+x0.We are now going to replace the bequest functionax(T) by the terminal constraint Ex(T,ω)≥0. For this purpose, we are now going to vary a and hence also a^ and β. For t=T, consider the expectation of Φi(t,ω). We will not give an explicit formula for this expectation, but we mention that it can be calculated in two steps, first given that i jumps have happened in [0,T] and then the expectation with i being stochastic (the rule of double expectations is then used). The expectation of the sum containing αj as well as x0 is positive, and the term in front of β is negative. There thus exists a unique positive value βT of β and, hence, of a (denoted by aT) such that EΦi(T,ω)=0; that is, Ex*(T,ω)=0.If we drop the termax(T) in the criterion but add the terminal condition Ex(T,ω)≥0, the free end optimality obtained in the original problem for a=aT evidently means optimality of u=be-ρt=a^T1/(γ-1)e-ρt=(aTekT)1/(γ-1)e-ρt in the end constrained problem (the found value aT appears in u, though now not in the criterion). (Alternatively, we could use the sufficient conditions for end constrained problems in Theorem 3 in Appendix to obtain optimality in the present end constrained problem. This would require us to check condition (A.13), which is easily done.)Let us discuss a little more the value ofβ for large values of T. Now,
(6)E[e-kτj]=E[E[e-kτj∣τj-1]]=E[e-kτj-1∫0∞‍λe-kxe-λxdx]=EE[[λk+λ]e-kτj-1∣τj-2]=E[[λk+λ]2e-kτj-2].
And, continuing, we get E[e-kτj]=[λ/(k+λ)]j.Then,EΦi(∞,ω)=x0+∑j=1i‍αj[λ/(k+λ)]j-β.Now, with probability 1 an infinite number of jumps occur in[0,∞), and
(7)EΦ∞(∞,ω)=x0+∑j=1∞αj[λk+λ]j-β.
Denote by β∞ the value of β for which EΦ∞(∞,ω)=0, and define b∞:=β∞(ρ+k). For T being large, EΦi(T,ω)≈EΦi(∞,ω)=EΦ∞(∞,ω), so βT is close to β∞ and bT:=βT(ρ+k) is close to b∞. So for T being large, the optimal control is approximately b∞e-ρt. Note that ρ belongs to (δ-k,∞) and both ρ and b∞ are increasing in γ∈(0,1), so a larger γ means we consume more in the beginning. In the problem where ax(T) is replaced by the terminal constraint Ex(T,ω)≥0, the optimal control depends on αi, λ. It is immediately seen that β∞ is increasing in λ and in each αi, so also b∞ has these properties, indicating, for T being large, that even βT and bT have these properties. This conclusion actually follows for any T, because E[∑j=1iαje-kτj] is evidently increasing in λ and in each αi.We may assume that the jumps are stochastic, that is, thatx(τi+)-x(τi-)=Vi, where {Vi}i take values in a common bounded set and are independent, and are independent of the τi’s. If we then assume that ∑E(Vi)<∞, the solution in this problem is the same as the one above for αi=E(Vi).Note that we must assume that we have a deal with the bank in which our wealth is placed that it accepts the above behavior. That is, before time 0, we have got an acceptance for the possibility of operating with this type of admissible solutions, which means that only in expectation we leave a wealth in the bank≥0. In actual runs, sometimes we leave in the bank a positive wealth (that the bank gets), and for other runs a negative wealth (debt) that the bank has to cover.Consider now the case whereT=∞, a=0 in (1). Then, for a^=[β∞(ρ+k)]γ-1,
(8)limT→∞E[p(T)x*(T,ω)]=limT→∞Ea^Φi(T,ω)=a^EΦ∞(∞,ω)=0,
when β=β∞. In this case, u=b∞e-ρt is optimal in the problem where liminfT→∞E[p(T)x(T,ω)]≥0 is added as an end constraint. (See the appendix, (A.14).) The latter condition is equivalent to the so-called no-Ponzi-game condition.


## 3. Model 2

(This model Is related to exercise 4.1 in Øksendal and Sulem [5].)Consider the following problem:(9)maxu(·)∫0∞e-δtuγdt,whereδ,γarefixedpositiveconstantsγ∈(0,1),u≥0,
where u is the control, subject to
(10)x˙=kx-u,k,x0aregivenpositivenumbers,x(τi+)-x(τi-)=αx(τi-),x(t)>0∀t,x(0)=x0,
where α is a given positive number, δ>γk, τi, i=1,2,…,  is exponentially distributed in (τi-1,∞) with intensity λi and  λi is decreasing towards zero (τ0=0, all τi-τi-1 independent).In contrast to problem (1)–(3), now the jumps are linearly dependent on x(τi-). To defend such a feature, one might argue that the richer we are, the more we are able to generate large jumps (the jump may actually represent a collection of oil finds). On the other side, we will assume that such jumps occur with smaller and smaller intensities.It is easy to see that the current value functionJ~(y,j) is of the form A~jyγ when we replace x0 by an arbitrary start value y. For any pair (x(t,ω),u(t,ω)) satisfying (10), we can write u(t,ω)=v(t,ω)x(t,ω), v(t,ω)≥0. Then, for t∈(τi,τi+1),
(11)x(t,ω)=(1+α)x(τi-)ek(t-τi)-∫τitv(s,ω)ds=(1+α)2x(τi-1-)ek(t-τi-1)-∫τi-1tv(s,ω)ds=⋯(1+α)iyekt-∫0tv(s,ω)ds,
so u(t)=y times some function w(t,ω), t∈(τi,τi+1). Then, the value of the criterion for this v(·), denoted by J^(y,v(·,)), is proportional to yγ and so is also supv(,·,)J^(y,v(·,)). A similar argument works for J~(y,j), the optimal value of the criterion when j≥1 jumps have already occurred at time t=0. So J~(y,j)=A~jyγ, j=0,1,…. Writing J and Aj instead of J~ and A~j, the current value function J satisfies the following HJB equation:
(12)δJ(x,j)=maxu{uγ+Jx(x,j)(kx-u)}-λi+1(J(x,j)-J((1+α)x,j+1))=maxu{uγ+γAjxγ-1(kx-u)}-λi+1(Ajxγ-(1+α)γAj+1xγ).
The first order condition for maximum is γuγ-1-γAjxγ-1=0. So u=Bjx, for Bj=(Aj)1/(γ-1). Then,
(13)δAjxγ=Bjγxγ+γkAjxγ-γAjxγBj-λi+1(Ajxγ-(1+α)γAj+1xγ),
or  dividing by xγ and rearranging,
(14)0=-δAj+Ajγ/(γ-1)(1-γ)+γkAj-λj+1Aj+λj+1(1+α)γAj+1.
Dividing by Aj gives
(15)0=-δ+γk-λj+1+λj+1(1+α)γAj+1Aj+Aj1/(γ-1)(1-γ).
Let A(λ) satisfy
(16)δ-γk+λ-λ(1+α)γ=A(λ)1/(γ-1)(1-γ).
Evidently, relationship (16) would be obtained from (15) if all λj=λ(⇒Aj=Aj+1); in that case, the optimal value function would be A(λ)yγ.Now, assume first, for somek*, that λk*+1=λk*+2=⋯=0. Then, Ak*yγ is the optimal value in the problem with no jumps, so Ak*=A(0). Given Ak*, (14) determines Ak*-1, Ak*-2,…,A0. It must be the case that Aj is decreasing. Given the same start point x, the optimal value when j+1 jumps have occurred already at time 0 must be smaller than the optimal value when j jumps have occurred at time 0; in the former problem, prospective jumps have smaller probabilities for happening. (Let us show by backwards induction that Aj, j=0,1,2,…,k, is nonincreasing. Evidently, Ak=Ak+1. Assume by induction that Aj≥Aj+1. If Aj-1<Aj, then, by (15), (γ-1)(Aj-1)1/(γ-1)=-δ+γk-λj + λj(1+α)γAj/Aj-1>-δ+γk-λj+1+λj+1(1+α)γAj/Aj-1>-δ+γr-λj+1+λj+1(1+α)γAj+1/Aj = (γ-1)(Aj)1/(γ-1). Because γ-1<0 and z→(z)1/(γ-1) is decreasing, a contradiction is obtained.) As we will let k* vary, denote the sequence A0,…,Ak* defined by (14) for Ak*=A(0) by A0k*,…,Ak*k*.Now,A(0)=Ak*+1k*+1<Ak*k*+1, so A(0)=Ak*k*<Ak*k*+1. In fact, as is easily seen, this holds for all j: Ajk*<Ajk*+1. (Compare the optimal values in the case where jump j occurs at (before) t=0 in the two problems where λm=0, m≥k*+1, and where λm=0, m≥k*+2.) Assume now that λj>0 for all j and let  Aj=limk*→∞Ajk*. (It is shown below that, for some A0*, A0*≥Ajk*, for all k*, j.) Then, limj→∞Aj=A(0), and the Aj’s satisfy (14).Letx*(t,ω) be the solution for u≡0. By (11), x*(t)=∑ix0ekt(1+α)i1[τi,τi+1)(t)=ektΓ, where Γ:=(1+α)i1[τi,τi+1)(t). Choose the  smallest j* such that αλj*<δ-kγ. If j* jumps occur at once at 0(λj=∞,j≤j*), while further jumps occur with intensity λ=λj*, then EΓ would equal (1+α)j*∑i=1∞(1+α)i[(λt)i/i!]e-λt=(1+α)j*eαλj*t. Hence, EΓγ≤(1+α)j*eαλj*t. For any admissible solution x(t,ω), x(t,ω)≤x*(t,ω), for all (t,ω); hence, for t=T, 0≤E[e-δTAj(x(T,ω))γ] ≤ E[e-δTA0ekγTΓγ]≤e-(δ-kγ)TA0(1+α)j*eαλj*T→0 when T→∞. So (A.7) in the appendix is satisfied. Hence, sufficient conditions hold (see Theorem 2 in Appendix) and uj=x(Aj)1/(γ-1), j=0,1,…, are optimal. That is, if v*(t,ω)=(Aj)1/(γ-1)1[τj,τj+1)(t) and x*(t,ω)=x0(1 + α)iekt-∑1≤j≤i‍Aj-11/(γ-1)(τj-τj-1)-Ai1/(γ-1)(t-τi) for t∈(τi,τi+1) (see (11)), then u*(t,ω)=v*(t,ω)x*(t,ω) is optimal (x*(t,ω) evidently satisfies x*(t,ω)>0, for all t). Hence, Aj=A~j. (In the appendix, for the sufficiency of the HJB equation to hold, it is required that u*(t,ω) is bounded if T<∞ (for T=∞, we need boundedness for t in all bounded intervals). This is not the case here. But we could have replaced u by vx, v being the control. Assume that we require v(t,ω) to be bounded in the above manner. Now, v*(t,ω) is so bounded, and it is then optimal in the set of such bounded v(·,·)’s.)We can show that theAj’s are increasing in α and in each λi, i>j. The simplest argument is that this must be so, when we now know that Ajyγ is the optimal value function after j jumps at (before) t=0.If allλj=λ<(δ-kγ)/α, then, using (16), x*(t)=x0(1+α)iekt-A(λ)1/(γ-1)t=x0(1+α)ie-ρ*t when t∈(τi,τi+1), where 0<ρ*=ρ+[λ-λ(1+α)γ]/(1-γ)<ρ=(δ-k)/(1-γ). (The inequality αλ<δ-kγ implies δ-kγ+λ-λ(1+α)γ>δ-kγ+λ-λ(1+α)>0, so A(λ)>0. In particular, A(λj*)>0. The above calculations show that in the problem where λj=λj*, for j>j*, the optimal value functions are Aj*yγ, where Aj*=A(λj*) for j≥j*, Aj* given by backward induction using (14) for j<j*, again Aj-1*≥Aj*. Evidently, Aj*≥Aj. Now, Aj≤A(λj) for j≥j* and, from (16), it follows that A(λj)→A(0) when j→∞.) Now, relate the present case to what happened in Model 1. As u*(t)=A(λ)1/(1-γ)x*(t) and ρ*<ρ, in each interval (τi,τi+1), (d/dt)u*(t)>d/dt(be-ρt) (be-ρt is the optimal control in Model 1); moreover, when t passes each τiu*(t) changes by a factor (1+α).


## 4. Model 3

In this model, the physical volume of oil production is constant=1, but the oil price z jumps up and down at Poisson distributed time points τi, i=1,2,…. Let τ0=0. Consider the problem
(17)maxu(·)∫0Tγ-1e-δtuγdt,γ,δgivenpositiveconstants,γ∈(0,1),u≥0,
where u is the control, subject to the differential equations below and
(18)z(τi+)=Vi,
where Vi is taking a finite number of values {Vj}j with given probabilities pj, all Vi  are identically distributed, and  τi is exponentially distributed in (τi-1,∞) with intensity λ (all the random variables τi-τi-1, Vi independent). Let v=EV. The two states y and z are governed by (18) (no jumps in y) and
(19)y˙=ay+z-u,δ>a>0,z˙=0,whereaisagivenconstant;y(0)=y0, z(0)=z0, where y0 and z0 are given constants; in fact, it will be convenient to assume that z0=v. The end constraint E[y(T)]≥0 is required to hold. Here, y is wealth and ay is interest earned.Let us use the extremal method (see the appendix) for solving this problem. In what follows,  it is guessed that adjoint functions do not depend on arbitrarily given initial points (which in the appendix are denoted by(s,y)). The adjoint function corresponding to y is hence denoted by p(t,i). We make the guess that p(t,i) simply equals Ke-at (independent of i), K being an unknown. It does satisfy the adjoint equation
(20)p˙(t,i-1)=-ap(t,i-1)+λp(t,i-1)-λEp(t,i).
(Epi(t), the expectation with respect to Vi, simply equals p(t,i), since p(t,i) does not depend on Vi.) The adjoint function pz(t,i) corresponding to the state variable z satisfies p˙z(t,i)=-p(t,i)+λpz(t,i), with pz(T,i)=0 (z(T) is free), we do not need the formula for pz(t,i).Next, maximum of the HamiltonianH=e-δtuγ/γ+p(ay+z-u) is obtained for u satisfying the first order condition uγ-1e-δt=p, so u=(eδtp)1/(γ-1)=Kβe-κt, where κ=(a-δ)/(γ-1)>0, β=1/(γ-1).Now,y˙=ay+z(t)-u=ay+z(t)-Kβe-κt, so
(21)y(t)=Ceat+eat∫0tz(s)e-asds+Kβe-κta+κ,
where C=y0-Kβ/(a+κ).Note that, for anys, Ez(s)=v. We want y(t) to satisfy the condition Ey(T)=0; that is,
(22)0=E{y0eaT+eaT∫0Tz(s)e-asds+Kβe-κTa+κ-eaTKβa+κ}=eaT{y0+v(1-e-aT)a+Kβe-(κ+a)Ta+κ-Kβa+κ},
which determines K. From now on, denote the solutions y(t) and z(t) by y*(t,ω) and z*(t,ω), with u*(t)=Kβe-κt. Sufficient conditions (see Theorem 3 in the appendix) are easily seen to be satisfied. We need to check the end condition E[p(T,ω)(y(T,ω)-y*(T,ω))]≥0(p(·,ω)=p(·)) for all admissible solutions y(t,ω) (see (A.13) in the appendix). The terminal condition is Ey(T,ω)≥0, for any admissible y(t,ω), and by construction y*(t,ω) satisfies Ep(T)y*(T,ω)=0, so the just-mentioned end condition is satisfied. We hence have optimality of u*(t),y*(t,ω), and z*(t,ω) among admissible triples (u(t,ω),y(t,ω),z(t,ω)) for which Ey(T,ω)≥0(z(t,ω)=z*(t,ω)).IfT=∞, the end condition required for using the sufficient condition in Theorem 2 in the appendix is liminfT→∞E[p(T,ω)(y(T,ω)-y*(T,ω))]≥0. The condition limTE[p(T)y*(T,ω)]=0 determines K; it now reduces to the condition y0+v/a=Kβ/(a+κ). Moreover, we now have optimality among all triples u(t,ω),y(t,ω),z(t,ω) for which liminfT→∞E[p(T)y(T,ω)]≥0. (This inequality means that the next-to-last inequality is satisfied.) Note that Kβ and so the control, quite expectably, are increasing in v=EV.


## 5. Model 4

In this example,y is the rate of production (production per unit of time) of oil. The oil price z jumps up or down between two values at Poisson distributed time points. The income the owner does not consume, that is, zy-u, can be used to increase the rate of production, with the change y˙ being proportional to zy-u. (If zy-u is negative, it means that he takes money out of his business and runs it down.) For simplicity, the proportionality factor is put equal to 1, and, simply, z jumps between the values 1 and 2. So consider
(23)maxu(·)E[∫0∞e-δtuγdt],whereδandγaregivenpositiveconstants,γ∈(0,1),δ>2γ, u≥0 the control, subject to
(24)y˙=zy-u,y(t)≥0a.s.,z˙=0,z(0)=1,z(T)free,z(τi+)=Vi,Vi∈{1,2},Pr[Vi=1]=12,
where τi is exponentially distributed in (τi-1,∞) with intensity λ (all Vi,τi-τi-1 independent V0=1, z0=0). There are no jumps in y and y(0)=y0>0, y0 being fixed.LetJ(y,j),j=1,2, be the current optimal value function sought, where j=z∈{1,2}. The current value HJB equation is as follows
(25)δJ(y,j)=max{uγ+Jy(y,j)(jy-u)}+(12)λJ(y,1)+(12)λJ(y,2)-λJ(y,j).
Let us try J(y,1)=A1yγ, J(y,2)=A2yγ. Then, the maximizing u equals u=Bjy, Bj=(Aj)1/(γ-1). In the two cases j=1 and j=2, we get the two equations
(26)0=-δA1+A1γ/(γ-1)+γA1-γA1(A1)1/(γ-1)+(A12+A22)λ-λA1,(27)0=-δA2+A2γ/(γ-1)+2γA2-γA2(A2)1/(γ-1)+(A12+A22)λ-λA2.
The equations can be solved for positive A1 and A2 (shown in a moment). Automatically, the solution y*(t,ω) corresponding to u(y,z)=u(y,j)=Bjy satisfies y*(t,ω)≥0 a.s., for all t.To show the existence of a solution of (26) and (27), note that if we put A2=A1 in (26), then by dividing by A1, we get 0=-δ+γ+(1-γ)A11/(γ-1)=:κ1(A1), and κ1(A1)=0 yields A1=A1*:=[(δ-γ)/(1-γ)]γ-1. Similarly, if we put A1=A2 in (27), we get 0=-δ+2γ+(1-γ)A21/(γ-1)=:κ2(A2), and κ2(A2)=0 yields A2=A2*:=[(δ-2γ)/(1-γ)]γ-1. Now, A1*/A2*=[(δ-2γ)/(δ-γ)]1-γ<1. Denote the right-hand sides in (26) and (27) by ϕ(A1,A2) and ψ(A2,A1), respectively. Both are increasing in the second place and decreasing in the first place. When A1∈[A1*,A2*], ψ(A2*,A1)≤ψ(A2*,A2*)=0, and ψ(A1*,A1)≥ψ(A1*,A1*)>ϕ(A1*,A1*)=0 (the only difference in ψ(A,A) and ϕ(A,A) is the third term in the formulas). Thus, ψ(A2,A1)=0 can be uniquely solved with respect to A2 for A1 in [A1*,A2*], denote the solution A2(A1), it is evidently increasing in A1 and A2(A1)∈[A1*,A2*], A2(A2*)=A2*. Moreover, A1<A2*,⇒0<κ2(A1) = ψ(A1,A1)⇒A2(A1)>A1. Consider the function β(A1):=ϕ(A1,A2(A1))>ϕ(A1,A1). Now, β(A1*)>ϕ(A1*,A1*)=0 and β(A2*)=ϕ(A2*,A2*)<ψ(A2*,A2*)=0, so A1∈(A1*,A2*) exists for which 0=β(A1)=ϕ(A1,A2(A1)). So this A1∈(A1*,A2*) and A2=A2(A1)∈(A1*,A2*), A2>A1, satisfy both (27) and (26).For any admissibley(t,ω), 0≤E[e-δTAj(y(T,ω))γ]≤A2*e-δT(y*(T))γ, where y*(t) is the solution of y˙=2y (i.e., y*(T)=y0e2t), so, for all admissible y(t,ω), 0≤limT→∞E[e-δTAj(y(T,ω))γ] ≤ limT→∞e-δTA2*(y*(T))γ=limT→∞A2*y0γe(2γ-δ)T=0. Hence, (A.7) in the appendix holds; the sufficient conditions in Theorem 2 are satisfied, and if u=y(Aj)1/(γ-1) when Vi=j∈{1,2}, then u is optimal. (Hence, if v(t,ω):=∑i=0∞(A11/(γ-1)1{V=1}(Vi)+A21/(γ-1)1{V=2}(Vi))1[τi,τi+1)(t) and y*(t,ω) is the solution of y˙=z(t,ω)y-v(t,ω)y, then u*(t,ω)=v(t,ω)y*(t,ω) is optimal.)DefineA*=[(δ-3γ/2)/(1-γ)]γ-1 (i.e., A* satisfies 0=-δA+Aγ/(γ-1)+3γA/2-γAA1/(γ-1)). When λ=0 then Ai=Ai*, and one may show that when λ>0 decreases, A2 increases towards A2* while A1 decreases towards A1*. When λ increases to infinity, A1 increases to A*, and A2 decreases to A*. In fact, A1∈(A1*,A*) and A2∈(A*,A2*). To prove convergence to A*, note first that A1, A2 are bounded uniformly in λ. If A1 and A2 did not converge to a common number, say A*, when λ→∞, the two last terms in (27) and (26) would blow up to infinity for (certain) large values of λ, while the remaining terms would be bounded, a contradiction. Summing the two equations in (27) and (26) gives 0=-δA1+A1γ/(γ-1)+γA1-γA1A11/(γ-1)-δA2+A2γ/(γ-1)+2γA2-γA2A21/(γ-1). Hence, A* satisfies 0=-2δA*+2A*γ/(γ-1)+3γA*-2γA*A*1/(γ-1); that is, A*=A*.Two very simple models discussed below contain the feature that the owner can influence the chance of discovery but that it is costly to do so. In the first one, the intensity of discoveriesλ is influenced by how much money is put into search at any moment in time; in the second one, it is costly buildup of expertise that matters for the intensity of discoveries.


## 6. Model 5

Consider the problem(28)maxu(·)E[∑i=1∞e-δτig0-∫0∞e-δtudt],whereδisafixedpositivenumber,x(τi+)-x(τi-)=α,whereαisafixedpositivenumber,x(0)=0,x˙=0,
and λ=μu1-κ/(1-κ), where μ>0, κ are given constants, κ∈(0,1), u≥0 is the control, and λ is the intensity of jumps. All fields found are of the same size α. We imagine that fields are sold immediately when they are found or that they are produced over a fixed period of time, with a fixed production profile. In both cases, we let income from a field discounted back to the time of discovery be equal to g0:=βα, for a given β>0. The oil owner wants to maximize the sum of discounted incomes earned over [0,∞).Let us try the proposal that the current value optimal value functionJ=a, where a, an unknown. The current value HJB equation is
(29)δa=maxu{-u+μg0u1-κ1-κ},
which implies that the maximizing u satisfies μg0u-κ=1, so u=(μg0)1/κ, -u+μg0u1-κ/(1-κ) = -(μg0)1/κ+μg0((μg0)1/κ)1-κ/(1-κ)=(κ/(1-κ))(μg0)1/κ=:a^, and a=a^/δ. Trivially,  (A.7) in the appendix holds, so the sufficient conditions in Theorem 2 are satisfied and u=(μg0)1/κ is optimal. Because a>0, in expectation (but not in all runs), over [0,∞), a positive sum of discounted incomes (=a) has been earned.Often, it is the case that the best fields are exploited first hence,x(τi+)-x(τi-)=αi, where αi is decreasing and even αi↓0. Then, when j jumps are imagined to have happened already at time 0, we guess J(j)=aj. Define g0i=βαi. Then, the HJB equation is
(30)δaj=maxu{-u+μ(g0j+1+aj+1-aj)u1-κ1-κ}=(κ1-κ)(μg0j+1+aj+1-aj)1/κ=:νj(aj),
and uj=(μg0j+1+aj+1-aj)1/κ yields maximum. Postulate for the moment that, for some j, aj+1∈[aj+2,g0j+2+aj+2). Then, δaj+1=νj+1(aj+1)≤(κ/(1-κ))(μg0j+2)1/κ<(κ/(1-κ))(μg0j+1)1/κ = νj(aj+1). Thus, if we try aj=aj+1, we get δaj<νj(aj). If we try aj=g0j+1+aj+1, we get δaj>νj(aj)=0. So a correct value of aj∈(aj+1,g0j+1+aj+1) exists such that δaj=ν(aj). This continues backwards and gives a0>a1>⋯>aj+1. If, for some j*, αj=0, for j≥j*+1, then aj*=aj*+1=aj*+2=⋯=0, so in this case, (30) gives 0<aj*-1<aj*-2<⋯<a0. If all αj>0, letting j*→∞, it is easily seen that there exists an infinite decreasing sequence aj↓0 when j→∞ satisfying (30) (compare a similar argument in Model 2).Still, trivially condition(A.7) below holds, sufficient conditions in Theorem 2 in the appendix are satisfied and uj=(μg0j+1+aj+1-aj)1/κ on [τj,τj+1), j=0,1,2,…, are optimal. (Uniqueness of the aj’s follows from optimality.)


## 7. Model 6

Consider the problem(31)maxu(·)E[∑i=1∞e-δτig0-∫0∞e-δtudt],whereδisafixedpositivenumber,x(τi+)-x(τi-)=αwhereαisafixedpositivenumber,x(0)=0,x˙=0,y˙=u1-κ1-κ,whereu≥0,uisthecontrol,y(0)=0,
no jumps in y, λ=μy, μ, κ  are fixed positive numbers, κ<1, λ is the intensity of jumps in x.All fields found are of the same sizeα. We again imagine that fields are sold immediately when they are found or that they are produced over a fixed period of time, with a fixed production profile. Let income from a field discounted back to the time of discovery be equal to g0:=βα, for a given β>0. The state y(t) is the amount of expertise available for finding new fields, built up over time according to y˙=u1-κ/(1-κ), where u is money per unit of time spent on building expertise.Let us try the proposal that the current value functionJ equals a+ky, a and k being unknowns. The current value HJB equation is
(32)δ(a+ky)=maxu{-u+ku1-κ1-κ+μyg0}.
Then, kδ=μg0, so k=μg0/δ. Moreover, the maximizing u equals u=k1/κ, and -u+ku1-κ/(1-κ) = (κ/(1-κ))k1/κ=:aˇ, a=aˇ/δ. The optimal solution y*(t) equals tb, where b=k(1-κ)/κ/(1-κ). Evidently, (A.7) below is satisfied, because J=e-δT(a+ky(T))≥0 for any admissible x(·), y(·) and e-δT(a+ky*(T))→0 when T→∞. Hence, u=k1/κ=(μg0/δ)1/κ is optimal.


## 8. Comparisons

In Models 1, 2, 5, and 6, oil finds are made at stochastic points in time; in Models 3 and 4, it is the price of oil that changes at stochastic points in time. In Model 1, we operate with the constraintEx(∞,ω)≥0 (for T=∞, a=0), where x is the oil-owners’ wealth. Here, for some runs, x(∞,ω) can be negative (β>x0) and, for other runs, positive. In Model 2, we required x(t,ω)>0 for all t, all ω. (The results in that model would have been the same if we had required only x(∞,ω)≥0 for all ω.) In Model 2, the optimal control comes out as stochastic and not deterministic as in Model 1. Moreover, as a comment in Model 2 says, as a function of time, the optimal control decreases more rapidly in Model 1 as compared to Model 2. The latter feature stems from the fact that, in Model 2, we enhance future income prospects by not decreasing x too fast, because the jump term (the right-hand side of the jump equation) depends positively on x, which is not the case in Model 1.In Models 3 and 4, the oil price exhibits sudden stochastic jumps. In Model 3, the rate of oil production is constant, but income earned (as well as interest) is placed in a bank after subtraction of consumption. In Model 4, income earned, after subtraction of consumption, is reinvested in the oil firm to increase production. In Model 4, the optimal control is stochastic; it depends on whether the current price is high or low. In Model 3, the control is deterministic, and it depends only on the expectationv of the stochastic price. Consider the case where, in Model 3, the expected price v is zero and a=3/2. Then, u=Kβe-κt, where κ=(3/2-δ)/(γ-1) and Kβ=y0(3/2+κ). We saw in Model 4 that when the intensity of jumps λ is very high, A1≃A2≃A*, where A*=[δ-3γ/2)/(1-γ)]γ-1; we hardly pay attention to what the current price is, because it changes so frequently. Now u=y(Aj)1/(γ-1) when the current price is j(=z), or u≃y(A*)1/(γ-1) when λ is large. When λ is large (so z switches very frequently between 1 and 2), the stochastic path of the equation y˙=zy-u=zy-y(A*)1/(γ-1) most often is very close to the deterministic path of y˙=(Ez)y-u=(3/2)y-y(A*)1/(γ-1) = (3/2)y-[(δ-3γ/2)/(1-γ)]y=-κy. The latter equation has the solution y=y0e-κt, and the corresponding u equals (3/2)y-y˙ = (3/2)y0e-κt+κy0e-κt=κy0e-κt = y0(3/2+κ)e-κt, the same control as that obtained in Model 3 in the case a=3/2, v=0.In the extremely simple Models 5 and 6, the frequency of oil finds is not fixed but influenced by a control. In Model 5, the current frequency (or intensity) is determined by how much money is put into search at that moment in time. In the simplest case considered in Model 5, a find today does not influence the possibility of making equally sized discoveries tomorrow. Then it, is not unreasonable that the optimal control (which equals(μg0)1/κ) is independent of the discount rate δ but dependent on the fixed value of the finds g0=βα. (Here, actually α could be the expected size of a find, in both Model 5 and Model 6; we could have had the sizes of the finds being independently stochastic, with α being the expected value of the sizes.) In Model 6, it is the ability to discover oil that is built up over time, so with greater impatience (higher δ), we should expect less willingness of devoting money to increase this ability, and this shows up in the formula u=(μg0/δ)1/κ.



---

*Source: 368096-2013-11-28.xml*