# Harvesting in Stochastic Environments: Optimal Policies in a Relaxed Model

Richard H. Stockbridge, Chao Zhu

► To cite this version:

Richard H. Stockbridge, Chao Zhu. Harvesting in Stochastic Environments: Optimal Policies in a Relaxed Model. 25th System Modeling and Optimization (CSMO), Sep 2011, Berlin, Germany. pp.197-206, 10.1007/978-3-642-36062-6_20. hal-01347538

HAL Id: hal-01347538

https://inria.hal.science/hal-01347538

Submitted on 21 Jul 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Harvesting in Stochastic Environments:
Optimal Policies in a Relaxed Model

Richard H. Stockbridge and Chao Zhu

Department of Mathematical Sciences
University of Wisconsin-Milwaukee
Milwaukee, WI 53201
{stockbri,zhu}@uwm.edu

**Abstract.** This paper examines the objective of optimally harvesting a single species in a stochastic environment. This problem has previously been analyzed in [1] using dynamic programming techniques and, due to the natural payoff structure of the price rate function (the price decreases as the population increases), no optimal harvesting policy exists. This paper establishes a relaxed formulation of the harvesting model in such a manner that existence of an optimal relaxed harvesting policy can not only be proven but also identified. The analysis imbeds the harvesting problem in an infinite-dimensional linear program over a space of occupation measures in which the initial position enters as a parameter and then analyzes an auxiliary problem having fewer constraints. In this manner upper bounds are determined for the optimal value (with the given initial position); these bounds depend on the relation of the initial population size to a specific target size. The more interesting case occurs when the initial population exceeds this target size; a new argument is required to obtain a sharp upper bound. Though the initial population size only enters as a parameter, the value is determined in a closed-form functional expression of this parameter.

**Key Words.** Singular stochastic control, linear programming, relaxed control.

**AMS subject classification.** 93E20, 60J60.

## 1 Introduction

This paper examines the problem of optimally harvesting a single species that lives in a random environment. Let $X$ be the process denoting the size of the population and $Z$ denote the cumulative amount of the species harvested. We assume $X(0−) = x_0 > 0$, $Z(0−) = 0$, and $X$ and $Z$ satisfy

$$dX(t) = b(X(t))dt + \sigma(X(t))dW(t) - dZ(t), \quad (1)$$

in which $W(·)$ is a 1-dimensional standard Brownian motion that provides the random fluctuations in the population's size, and $b$ and $\sigma$ are real-valued continuous functions. We assume that $b$ and $\sigma$ are such that in the absence of harvesting
the population process X takes values in $\mathbb{R}_+$ and that $\infty$ is a natural boundary so that the population will not explode to $\infty$ in finite time. The boundary 0 may be an exit or a natural boundary point but may not be an entrance point; this indicates that the species will not spontaneously reappear following extinction. Note that $X(0)$ may not equal $X(0^-)$ due to an instantaneous harvest $Z(0)$ at time 0 and the process $Z$ is restricted so that $\Delta Z(t) := Z(t) - Z(t^-) \le X(t^-)$ for all $t \ge 0$. This latter condition indicates that one cannot harvest more of the species than exists. Let $r > 0$ denote the discount rate and $f$ denote the marginal yield for harvesting. The objective is to select a harvesting strategy $Z$ so as to maximize the expected discounted revenue

$$J(x_0, Z) := \mathbf{E}_{x_0} \left[ \int_0^\tau e^{-rs} f(X(s^-)) dZ(s) \right], \quad (2)$$

where $\tau = \inf\{t \ge 0 : X(t) = 0\}$ denotes the extinction time of the species.

As a result of developments in stochastic analysis and stochastic control techniques, there has been a resurgent interest in determining the optimal harvesting strategies in the presence of stochastic fluctuations (see, e.g., [1, 6]). In particular, [1] examines the current problem using dynamic programming techniques and determines the value function. The paper indicates the lack of an optimal policy in the admissible class of (strict) harvesting policies by commenting that a “chattering” policy will be optimal. The problem of optimal harvesting of a single species in a random environment is also studied in [8] in which the model is extended to regime-switching diffusions so as to capture different dynamics such as for drought and non-drought conditions. The paper also adopts a dynamic programming solution approach to determine the value function while at the same time exhibiting $\epsilon$-optimal harvesting policies since, as in the static environment of [1], no optimal harvesting policy exists. In light of the complexities of the regime-switching model, it further identifies a condition under which the value function is shown to be continuous and a viscosity solution to the variational inequality.

The focus of this paper is on developing a relaxed formulation for the harvesting problem under which an optimal harvesting control exists and on establishing optimality using a linear programming formulation instead of dynamic programming. In addition, it is sufficient to have a weak solution to (1) rather than placing Lipschitz and polynomial growth conditions on the coefficients $b$ and $\sigma$ that guarantee existence of a strong solution. Intuitively, relaxation completes the space of admissible harvesting rules by allowing measure-valued policies. A benefit of the linear programming solution methodology is the analysis concentrates on the optimal value for a single, fixed initial condition, rather than seeking the value function and thus no smoothness properties need to be established about the value as a function of the initial position.
To set the stage for the relaxed singular control formulation of the model, let $\mathcal{D} = C_c^2(\mathbb{R}_+)$ and for a function $g \in \mathcal{D}$, define the operators $A$ and $B$ by

$$Ag(x) = \frac{1}{2} \sigma^2(x) g''(x) + b(x) g'(x), \quad \text{and} \quad (3)$$

$$Bg(x, z) = \begin{cases} \frac{g(x-z)-g(x)}{z}, & \text{if } z > 0, \\ -g'(x), & \text{if } z = 0, \end{cases} \quad (4)$$

where $x, z \in \mathbb{R}_+$. Itô's formula then implies

$$g(X(t)) = g(x_0) + \int_0^t Ag(X(s)) ds + \int_0^t Bg(X(s), \Delta Z(s)) dZ(s) \\ + \int_0^t \sigma(X(s))g'(X(s)) dW(s), \quad \forall g \in \mathcal{D}.$$

It therefore follows that for any $g \in \mathcal{D}$

$$g(X(t)) - g(x_0) - \int_0^t Ag(X(s)) ds - \int_0^t Bg(X(s), \Delta Z(s)) dZ(s) \quad (5)$$

is a mean 0 martingale. In fact, requiring (5) to be a martingale for a sufficiently large collection of functions $g$ is a way to characterize the processes $(X, Z)$ which satisfy (1). We turn now to a precise formulation of the model in which the processes are relaxed solutions of a controlled martingale problem for the operators $(A, B)$.

## 1.1 Formulation of the Relaxed Model

For a complete and separable metric space $S$, we define $M(S)$ to be the space of Borel measurable functions on $S$, $B(S)$ to be the space of bounded, measurable functions on $S$, $C(S)$ to be the space of continuous functions on $S$, $\bar{C}(S)$ to be the space of bounded, continuous functions on $S$, $M(S)$ to be the space of finite Borel measures on $S$, and $P(S)$ to be the space of probability measures on $S$. $M(S)$ and $P(S)$ are topologized by weak convergence.

Recall, the amount of harvesting is limited by the size of the population. Define $\mathcal{R} = \{(x, z) : 0 \le z \le x, x \ge 0\}$; $\mathcal{R}$ denotes the space on which the paired process $(X, Z)$ evolves when considering solutions of (1).

The formulation of the population model in the presence of “relaxed” harvesting policies adapts the relaxed formulation for singular controls given in [5] to the particulars of the harvesting problem. This adaptation sets the state space $E$ to be $\mathbb{R}_+$ and the control space $U = \mathbb{R}_+$, with $\mathcal{U} = \mathcal{R} \subset \mathbb{R}_+ \times \mathbb{R}_+$.

Let $X$ be an $\mathbb{R}_+$-valued process and $\Gamma$ be an $\mathcal{L}(\mathcal{R})$-valued random variable. Let $\Gamma_t$ denote the restriction of $\Gamma$ to $\mathcal{R} \times [0, t]$. Then $(X, \Gamma)$ is a relaxed solution of the harvesting model if there exists a filtration $\{\mathcal{F}_t\}$ such that $(X, \Gamma_t)$ is $\{\mathcal{F}_t\}$-progressively measurable, $X(0-) = x_0$, and for every $g \in \mathcal{D}$,

$$g(X(t)) - g(x_0) - \int_0^t Ag(X(s)) ds - \int_{\mathbb{R} \times [0, t]} Bg(x, z) \Gamma(dx \times dz \times ds) \quad (6)$$
is an {$\mathcal{F}_t$}-martingale, in which the operators $A$ and $B$ are given by (3) and (4), respectively. Throughout the paper we assume that a relaxed solution $(X, \Gamma)$ exists and is strong Markov. Let $\mathcal{A}$ denote the set of measures $\Gamma$ for which there is some $X$ such that $(X, \Gamma)$ is a relaxed solution of the harvesting model.

We turn now to the extension of the reward criterion (2) to the relaxed framework. Specifically, $f: \mathbb{R}_+ \mapsto \mathbb{R}_+$ represents the instantaneous marginal yield accrued from harvesting. Assume $f$ is continuous and non-increasing with respect to $x$. Thus $f(x) \ge f(y)$ whenever $x \le y$; this assumption indicates that the price when the species is plentiful is smaller than when it is rare. Moreover, we assume $0 < f(0) < \infty$. Let $(X, \Gamma)$ be a solution to the harvesting model (6). Let $S = (0, \infty)$ denote the survival set of the species. Then the expected total discounted value from harvesting is

$$J(x_0, \Gamma) := \mathbf{E} \left[ \int_{\mathcal{R} \times [0, \tau]} e^{-rs} f(x) \Gamma(dx \times dz \times ds) \right]. \quad (7)$$

The goal is to maximize the expected total discounted value from harvesting over relaxed solutions $(X, \Gamma)$ of the harvesting model and to find an optimal harvesting strategy $\Gamma^*$. Thus, we seek

$$V(x_0) = J(x_0, \Gamma^*) := \sup_{\Gamma \in \mathcal{A}} J(x_0, \Gamma). \quad (8)$$

We emphasize that the initial position $x_0$ is merely a parameter in the problem and that $V$ is not to be viewed as a function with any particular properties but merely is the value of the harvesting problem when the initial population size is $x_0$. We do, however, obtain the value in functional form for $x_0$ in two regions.

## 2 Linear Programming Formulation and Main Result

Throughout this paper, we assume the equation $(A - r)u(x) = 0$ has two fundamental solutions $\psi$ and $\phi$, where $\psi$ is strictly increasing and $\phi$ is strictly decreasing; without loss of generality we may assume $\psi(0) = 0$ (see [1]).

The main result of this paper is summarized in the following theorem.

**Theorem 1** Assume that there exists some $\tilde{b} \ge 0$ such that

$$(i) \quad \frac{f(x)}{\psi'(x)} \le \frac{f(\tilde{b})}{\psi'(\tilde{b})}, \quad \forall x \ge 0,$$

(ii) the function $f/\psi'$ is nonincreasing on $[\tilde{b}, \infty)$, and

(iii) the function $f$ is continuously differentiable on $(\tilde{b}, \infty)$.

Put $b^* = \inf\{\tilde{b} \ge 0 : \tilde{b} \text{ satisfies (i)-(iii)}\}$. Then the value is given by

$$V(x_0) = \frac{f(b^*)}{\psi'(b^*)} \psi(x_0 \wedge b^*) + \int_{b^*}^{x_0 \vee b^*} f(y)dy \quad (9)$$
and an optimal relaxed harvesting policy is given by

$$ \Gamma^*(dx \times dz \times dt) = I_{(b^*, \infty)}(x_0)\lambda_{[b^*, x_0]}(dx)\delta_{\{0\}}(dz)\delta_{\{0\}}(dt) + \Gamma_{b^*}(dx \times dz \times dt), \quad (10) $$

where $\lambda_{[b^*, x_0]}(\cdot)$ denotes Lebesgue measure on $[b^*, x_0]$ and $\Gamma_{b^*}$ is defined in Proposition 6.

We begin the task of reformulating the harvesting problem with the following observation. Let $\tilde{\tau}$ be any ${\mathcal{F}_t}$-stopping time. The optional sampling theorem along with the requirement that (6) be a mean 0 martingale for each $g \in \mathcal{D}$ implies

$$ e^{-r(t \wedge \tilde{\tau})} g(X(t \wedge \tilde{\tau})) - g(x_0) - \int_0^{t \wedge \tilde{\tau}} e^{-rs} [A - r] g(X(s)) ds - \int_{\mathcal{R} \times [0, t \wedge \tilde{\tau}]} e^{-rs} B g(x, z) \Gamma(dx \times dz \times ds) $$

is also a martingale. Recall $g \in \mathcal{D}$ means $g$ has compact support and hence is bounded. So taking expectations and letting $t \to \infty$ yields

$$ g(x_0) = \mathbf{E} [e^{-r\tilde{\tau}} I_{\{\tilde{\tau} < \infty\}} g(X(\tilde{\tau}))] - \mathbf{E} \left[ \int_0^{\tilde{\tau}} e^{-rs} [A - r] g(X(s)) ds \right] - \mathbf{E} \left[ \int_{\mathcal{R} \times [0, \tilde{\tau}]} e^{-rs} B g(x, z) \Gamma(dx \times dz \times ds) \right]. \quad (11) $$

The initial analysis takes $\tilde{\tau} = \tau$; later we will need (11) for a different stopping time.

The measures involved in the infinite-dimensional linear program are expected discounted occupation measures corresponding to relaxed solutions $(X, \Gamma)$ of the harvesting model. Indeed, for any Borel measurable $G_1 \subset S$ and $G \subset \mathcal{R}$, we define

$$ \mu_{\tau}(G_1) = \mathbf{E} [e^{-r\tau} I_{G_1}(X(\tau)) I_{\{\tau < \infty\}}], \quad \mu_0(G_1) = \mathbf{E} \left[ \int_0^{\tau} e^{-rs} I_{G_1}(X(s)) ds \right], \\ \mu_1(G) = \mathbf{E} \left[ \int_{\mathcal{R} \times [0, \tau]} e^{-rs} I_G(x, z) \Gamma(dx \times dz \times ds) \right]. \quad (12) $$

Using these measures, the singular control problem of maximizing (7) over relaxed solutions of the harvesting problem (6) can be written in the form

$$ \left\{ \begin{array}{ll} \displaystyle \text{Maximize} & \int f d\mu_1, \\ \text{subject to} & \displaystyle \int g d\mu_{\tau} - \int (A-r) g d\mu_0 - \int B g d\mu_1 = g(x_0), \quad \forall g \in \mathcal{D}, \\ & \mu_{\tau}, \mu_0, \mu_1 \in \mathcal{M}(S), \mu_{\tau}(S) \le 1, \mu_0(S) \le \frac{1}{r}. \end{array} \right. \quad (13) $$

Since each relaxed solution $(X, \Gamma)$ defines measures $\mu_{\tau}$, $\mu_0$ and $\mu_1$ by (12), the harvesting problem is embedded in (13). There might be feasible measures
which do not arise in this manner. Consequently, letting $V_{lp}(x_0)$ denote the value of the LP problem (13) with initial condition $X(0-) = x_0 > 0$, we have $V(x_0) \le V_{lp}(x_0)$.

# 3 The Proof of Theorem 1

This section is devoted to the proof of Theorem 1 and involves two steps.

## 3.1 Step 1: Universal Upper Bound

The proof follows along the lines of the arguments used in [4]. The general argument involves finding an upper bound for $V_{lp}(x_0)$ by reducing the number of constraints in the linear program (13). We state the results and leave the proofs to the reader.

**Proposition 2** Let $b^*$ be defined as in Theorem 1. Then for every $x_0 \ge 0$,

$$V(x_0) \le \frac{f(b^*)}{\psi'(b^*)} \psi(x_0). \quad (14)$$

Notice the bound in (14) holds for all initial positions $x_0$. The following result shows that this bound is sharp for $x_0 \le b^*$.

**Proposition 3** For $x_0 \le b^*$, let $L_{b^*}$ denote the local time process of $X$ at $b^*$. Define the random measure $\Gamma_{b^*}$ for Borel measurable $G \subset \mathcal{R}$ and $t \ge 0$ by

$$\Gamma_{b^*}(G \times [0, t]) = \int_0^t I_G(X(s-), \Delta L_{b^*}(s)) dL_{b^*}(s). \quad (15)$$

$$\text{Then } J(x_0, L_{b^*}) = J(x_0, \Gamma_{b^*}) = \frac{f(b^*)}{\psi'(b^*)} \psi(x_0).$$

Since $\Delta L_{b^*}(s) = 0$ for every $s \ge 0$, an optimal strategy is to harvest just enough of the population (using the local time of $X^*$ at $b^*$) so that the population size “reflects” at $b^*$.

The value function has been determined for initial population sizes $x_0$ that are smaller than $b^*$. It therefore remains to prove the validity of (9) when $x_0 > b^*$.

## 3.2 Step 2: Return of Stochasticity for a Refined Upper Bound

This step is the more interesting of the two and requires a new argument and also a different type of harvesting policy than appears in the literature.

When dealing with singular control problems, one usually takes the so-called reflection strategy, namely, $Z(t) = (x_0 - b^*)^+ + L_{b^*}(t)$, where one follows an immediate jump from $x_0$ to $b^*$ by using the local time process $L_{b^*}$ at $b^*$. Such a reflection strategy is used in [2], [7] and others. The corresponding income is

$$J(x_0, Z) = f(x_0)(x_0 - b^*) + \frac{f(b^*)}{\psi'(b^*)} \psi(b^*).$$
When $f$ is strictly decreasing, the reflection strategy is not optimal. Our purpose is to find an optimal relaxed harvesting strategy.

To develop a sharp upper bound, it is beneficial to revisit the definitions of the occupation measures in (12) so that the connection between the measures, the initial position and the harvesting strategy is more clearly displayed. Let $x_0 \in \mathbb{R}_+$ and $(X, \Gamma)$ be a relaxed solution of the harvesting model. Modify the notations of the measures to indicate their dependence on $x_0$ and $\Gamma$ by writing $\mu_\tau(G; x_0, \Gamma)$, $\mu_0(G; x_0, \Gamma)$ and $\mu_1(G; x_0, \Gamma)$.

**Proposition 4** For $x_0 > b^*$,

$$V(x_0) \le \int_{b^*}^{x_0} f(y) dy + \frac{f(b^*)}{\psi'(b^*)} \cdot \psi(b^*). \quad (16)$$

*Proof.* The proof of (16) is broken into two parts, with a technical lemma between the parts.

*Part 1:* Define the stopping time $\tau_{b^*} = \inf\{t \ge 0 : X(t) \le b^*\}$ to be the first time the process $X$ takes value at most $b^*$ and note that $\tau_{b^*} \le \tau$. For the harvesting measure $\Gamma$, define $\Gamma_{\tau_{b^*}}$ by

$$\Gamma_{\tau_{b^*}}(G \times [0, t]) = I_{\{\tau_{b^*} < \tau\}} \Gamma(G \times [\tau_{b^*}, \tau_{b^*} + t]), \quad G \in \mathcal{B}(\mathcal{R}), t \ge 0.$$

Notice that $\Gamma_{\tau_{b^*}}$ captures all harvesting using the measure $\Gamma$ from time $\tau_{b^*}$ onwards. Also define the measures $\mu_{0,\tau_{b^*}}$ and $\mu_{1,\tau_{b^*}}$ by

$$\mu_{0,\tau_{b^*}}(G; x_0, \Gamma) = \mathbf{E}_{x_0} \left[ \int_{0}^{\tau_{b^*}} e^{-rs} I_G(X(s)) ds \right], \\ \mu_{1,\tau_{b^*}}(G; x_0, \Gamma) = \mathbf{E}_{x_0} \left[ \int_{\mathcal{R} \times [0, \tau_{b^*})} e^{-rs} I_G(x, z) \Gamma(dx \times dz \times ds) \right].$$

Note carefully that any harvesting at the time $\tau_{b^*}$ is excluded from the measure $\mu_{1,\tau_{b^*}}$. Also observe that the total mass of $\mu_{0,\tau_{b^*}}$ equals $r^{-1}(1 - \mathbf{E}_{x_0}[e^{-r\tau_{b^*}}])$.

Using the strong Markov property of $(X, \Gamma)$, for each $G \in \mathcal{B}(\mathcal{R})$ it follows that

$$\mathbf{E}_{x_0} \left[ \int_{\mathcal{R} \times [0, \tau]} e^{-rs} I_G(x, z) \Gamma(dx \times dz \times ds) \right] \\ = \mathbf{E}_{x_0} \left[ \int_{\mathcal{R} \times [0, \tau_{b^*})} e^{-rs} I_G(x, z) \Gamma(dx \times dz \times ds) \right] \\ + \mathbf{E}_{x_0} \left[ \mathbf{E}_{x_0} \left[ I_{\{\tau_{b^*} < \tau\}} \int_{\mathcal{R} \times [\tau_{b^*}, \tau]} e^{-rs} I_G(x, z) \Gamma(dx \times dz \times ds) \right] \middle| \mathcal{F}_{\tau_{b^*}} \right] \\ = \mathbf{E}_{x_0} \left[ \int_{\mathcal{R} \times [0, \tau_{b^*})} e^{-rs} I_G(x, z) \Gamma(dx \times dz \times ds) \right] \\ + \mathbf{E}_{x_0} \left[ e^{-r\tau_{b^*}} I_{\{\tau_{b^*} < \tau\}} \mathbf{E}_{X(\tau_{b^*})} \left[ \int_{\mathcal{R} \times [0, \tau]} e^{-rs} I_G(x, z) \Gamma_{\tau_{b^*}}(dx \times dz \times ds) \right] \right].$$
As a result, for each $G \in \mathcal{B}(\mathcal{R})$, this identity can be written in terms of the
measures as

$$
\mu_1(G; x_0, \Gamma) = \mu_{1, \tau_{b^*}}(G; x_0, \Gamma) + \mathbf{E}_{x_0} \left[ e^{-r\tau_{b^*}} I_{\{\tau_{b^*} < \tau\}} \mu_1(G; X(\tau_{b^*}), \Gamma_{\tau_{b^*}}) \right].
$$

Notice, in particular, that the expectation term involves the measure $\mu_1$ evalu-
ated at the random initial position $X(\tau_{b^*})$. Hence

$$
\begin{align}
& \int f(y) \mu_1(dy; x_0, \Gamma) \nonumber \\
&= \int f(y) \mu_{1, \tau_{b^*}}(dy; x_0, \Gamma) + \mathbf{E}_{x_0} \left[ e^{-r\tau_{b^*}} I_{\{\tau_{b^*} < \tau\}} \int f(y) \mu_1(dy; X(\tau_{b^*}), \Gamma_{\tau_{b^*}}) \right] \nonumber \\
&\le \int f(y) \mu_{1, \tau_{b^*}}(dy; x_0, \Gamma) + \mathbf{E}_{x_0} \left[ e^{-r\tau_{b^*}} I_{\{\tau_{b^*} < \tau\}} \right] \frac{f(b^*)}{\psi'(b^*)} \cdot \psi(b^*), \tag{17}
\end{align}
$$

in which the inequality follows from Step 1.

This concludes Part 1 of the proof. Part 2 concentrates on estimating the
first term of the right-hand side of (17); a technical lemma is required.

**Lemma 5** Assume the conditions in Theorem 1. Define the function $h$ by $h(x) := \int_{b^*}^{x} f(y)dy$ for $x \ge 0$. Then the following estimates hold:

$$
(A - r)h(x) \leq r \frac{f(b^*)}{\psi'(b^*)} \psi(b^*), \quad \text{for } x \geq b^* \text{ and} \tag{18}
$$

$$
-Bh(x, z) \geq f(x), \quad \text{for all } (x, z) \in \mathcal{R}. \tag{19}
$$

Proof. Since by assumption the function $f/\psi'$ is nonincreasing and differentiable on $(b^*, \infty)$, we have

$$
0 \geq \frac{d}{dx} \left( \frac{f(x)}{\psi'(x)} \right) = \frac{f'(x)\psi'(x) - f(x)\psi''(x)}{(\psi'(x))^2}, \quad x > b^*.
$$

But $\psi$ is strictly increasing and so $\psi'(x) > 0$. Hence it follows that $f'(x)\psi'(x) - f(x)\psi''(x) \le 0$, or equivalently $f'(x) \le \frac{f(x)}{\psi'(x)}\psi''(x)$, for $x > b^*$. It then follows that for each $x > b^*$

$$
\begin{align*}
(A - r)h(x) &\le \frac{1}{2}\sigma^2(x)\frac{f(x)}{\psi'(x)}\psi''(x) + b(x)f(x) - r\frac{f(x)}{\psi'(x)}(\psi(x) - \psi(b^*)) \\
&= \frac{f(x)}{\psi'(x)}\left[\frac{1}{2}\sigma^2(x)\psi''(x) + b(x)\psi'(x) - r\psi(x)\right] + r\frac{f(x)}{\psi'(x)}\psi(b^*) \\
&= r\frac{f(x)}{\psi'(x)}\psi(b^*) \le r\frac{f(b^*)}{\psi'(b^*)}\psi(b^*).
\end{align*}
$$

Turning to a consideration of (19), since $f$ is nonincreasing, for any $0 \le x_1 < x_2$, we have

$$
f(x_2)[x_2 - x_1] \le \int_{x_1}^{x_2} f(y)dy = h(x_2) - h(x_1).
$$
Hence it follows that for $(x, z) \in \mathcal{R}$, we have

$$ -Bh(x, z) = \begin{cases} h'(x) = f(x), & \text{if } z = 0 \\ \frac{h(x)-h(x-z)}{z} \geq f(x), & \text{if } z > 0. \end{cases} $$

The relation (19) is therefore established.

Part 2: The goal is of this part of the proof is to estimate $\int f(y) \mu_{1,\tau_{b^*}}(dy \times dz)$ of (17). Using Itô's formula, one obtains for each $t > 0$,

$$ \begin{aligned} & - \mathbf{E} \left[ \int_{\mathcal{R} \times [0, t \wedge \tau_{b^*})} e^{-rs} Bh(x, z) \Gamma(dx \times dz \times ds) \right] \\ &= h(x_0) - \mathbf{E} \left[ e^{-r(t \wedge \tau_{b^*})} h(X((t \wedge \tau_{b^*})-)) \right] + \mathbf{E} \left[ \int_0^{t \wedge \tau_{b^*}} e^{-rs} [A-r] h(X(s)) ds \right], \end{aligned} $$

in which the deliberate choice of the half-open interval $[0, t \wedge \tau_{b^*})$ in the integral with respect to $\Gamma$ leads to the use of $X((t \wedge \tau_{b^*})-)$ for the location of the process just before any harvest occurs at time $\tau_{b^*}$. This is extremely important since $h(X((t \wedge \tau_{b^*})-)) \ge 0$ and hence the right-hand side is not decreased by dropping the first expectation. Letting $t \to \infty$ yields

$$ \begin{aligned} & - \mathbf{E} \left[ \int_{\mathcal{R} \times [0, \tau_{b^*})} e^{-rs} Bh(x, z) \Gamma(dx \times dz \times ds) \right] \\ & \leq h(x_0) + \mathbf{E} \left[ \int_0^{\tau_{b^*}} e^{-rs} [A-r] h(X(s)) ds \right]. \end{aligned} $$

Using the estimates (18) and (19) and the definition of the measures $\mu_{1,\tau_{b^*}}$ and $\mu_{0,\tau_{b^*}}$, we obtain

$$ \begin{aligned} \int_{\mathcal{R}} f(y) \mu_{1, \tau_{b^*}}(dy \times dz; x_0, \Gamma) & \leq - \int_{\mathcal{R}} Bh(y, z) \mu_{1, \tau_{b^*}}(dy \times dz; x_0, \Gamma) \\ & \leq h(x_0) + \int r \cdot \frac{f(b^*) \psi(b^*)}{\psi'(b^*)} \mu_{0, \tau_{b^*}}(dx; x_0, \Gamma) \\ & = h(x_0) + (1 - \mathbf{E}_{x_0} [e^{-r \tau_{b^*}}]) \frac{f(b^*)}{\psi'(b^*)} \psi(b^*), \quad (20) \end{aligned} $$

in which the last equality follows from the mass of $\mu_{0,\tau_{b^*}}$. Combining (17) and (20) produces the desired relation

$$ \int f(y) \mu_1(dy \times dz; x_0, \Gamma) \leq \int_{b^*}^{x_0} f(y) dy + \frac{f(b^*)}{\psi'(b^*)} \psi(b^*). $$

We have derived an upper bound for the value $V(x_0)$ in Proposition 4. The following proposition exhibits an optimal relaxed harvesting policy. The proof is left to the reader.
**Proposition 6** Let $\lambda_{[b^*,x_0]}(\cdot)$ denote Lebesgue measure on $[b^*,x_0]$. Also let $L_{b^*}$ denote the local time process of Proposition 3 with $x_0$ taken to be $b^*$ and denote by $\Gamma_{b^*}$ the random measure defined in (15). Finally, define the relaxed harvesting strategy by

$$ \Gamma^*(dx \times dz \times dt) = \lambda_{[b^*,x_0]}(dx)\delta_{\{0\}}(dz)\delta_{\{0\}}(dt) + \Gamma_{b^*}(dx \times dz \times dt). $$

Then

$$ V(x_0) = J(x_0, \Gamma^*) = \int_{b^*}^{x_0} f(y)dy + \frac{f(b^*)}{\psi'(b^*)} \psi(b^*). \quad (21) $$

We observe that the manner in which this optimal harvesting policy differs from the typical “reflection” strategy occurs at the initial time. Whereas the reflection strategy has the process $X$ instantaneously jump from $x_0$ to $b^*$, the optimal relaxed harvesting policy obtains this relocation in an instantaneous but continuous manner.

Finally we note that the combination of Propositions 2 and 6 establishes Theorem 1. Moreover, the optimal relaxed harvesting policy in (10) unifies the two cases.

**Acknowledgements.** This research was supported in part by the U.S. National Security Agency under Grant Agreement Number H98230-09-1-0002, the National Science Foundation under DMS-1108782, a grant from the UWM Research Growth Initiative, and City University of Hong Kong (SRG) 7002677. The United States Government is authorized to reproduce and distribute reprints notwithstanding any copyright notation herein.

## References

1. Alvarez, L. H. R.: Singular stochastic control in the presence of a state-dependent yield structure. *Stochastic Process. Appl.*, 86(2):323–343, 2000.

2. Choulli, T., Taksar, M., and Zhou, X. Y.: A diffusion model for optimal dividend distribution for a company with constraints on risk control. *SIAM J. Control Optim.*, 41(6):1946–1979, 2003.

3. Ethier, S. N. and Kurtz, T. G.: *Markov processes: Characterization and Convergence*. John Wiley & Sons Inc., New York, 1986.

4. Helmes, K. L. and Stockbridge, R. H.: Thinning and harvesting in stochastic forest models. *J. Econom. Dynam. Control*, 35(1):25–39, 2011.

5. Kurtz, T. G. and Stockbridge, R. H.: Stationary solutions and forward equations for controlled and singular martingale problems. *Electron. J. Probab.*, 6:1–52, 2001. Paper No. 14.

6. Lungu, E. and Øksendal, B.: Optimal harvesting from a population in a stochastic crowded environment. *Math. Biosci.*, 145(1):47–75, 1997.

7. Pham, H.: *Continuous-time stochastic control and optimization with financial applications*, Springer-Verlag, Berlin, 2009.

8. Song, Q. S., Stockbridge, R. H., and Zhu, C.: On optimal harvesting problems in random environments. *SIAM J. Control Optim.*, 49(2):859–889, 2011.