\[ \left( v ^ { \alpha _ { n } } ( x ) - v ^ { \alpha _ { n } } ( x _ { 0 } ) \right) \to v ( x ) \]

for all \( x \in \mathcal { X } \), for some \( \lambda \in \mathbb { R } \) and some function \( v : \mathcal { X } \rightarrow \mathbb { R } \). Note that \|λ\|and \|v(·)\| are both bounded by \( C ^ { \prime } \). Finally, we notice that, for any \( x \in \mathcal { X } \),

\[ \alpha _ { n } v ^ { \alpha _ { n } } ( x ) = \alpha _ { n } v ^ { \alpha _ { n } } ( x _ { 0 } ) + \alpha _ { n } \bigl ( v ^ { \alpha _ { n } } ( x ) - v ^ { \alpha _ { n } } ( x _ { 0 } ) \bigr ) \to \lambda , \]

so the convergence of this sequence to \( \lambda \) holds for all x.

We are now in a position to prove existence of solutions to our Ergodic BSDE.

Theorem 4.4. Let v and \( \lambda \) be as constructed in Lemma 4.3. The triple \( ( Y , Z , \lambda ) \), where

\[ Y _ { t } : = v ( X _ { t } ) , \qquad e _ { k } ^ { * } Z _ { t } : = v ( e _ { k } ) . \]

is the unique bounded, stationary (i.e. does not depend on t), Markovian solution, with \( v ( x _ { 0 } ) = 0 \), to the Ergodic BSDE

\[ Y _ { T } = Y _ { t } - \sum _ { t \le u < T } \left( f ( X _ { u } , Z _ { u } ) - \lambda \right) + \sum _ { t \le u < T } Z _ { u } ^ { * } M _ { u + 1 } . \eqno ( 4 . 4 ) \]

Any other bounded solution \( ( Y ^ { \prime } , Z ^ { \prime } , \lambda ^ { \prime } ) \) satisfies \( \lambda \ = \ \lambda ^ { \prime } \), and any other bounded, stationary, Markovian solution \( ( \dot { Y } ^ { \prime } , Z ^ { \prime } , \lambda ^ { \prime } ) \) satisfies \( Y _ { t } \, = \, Y _ { t } ^ { \prime } + c \) for some \( c \in \mathbb { R } \), and \( Z \sim _ { M } Z ^ { \prime } \) 0.

Proof. Let \( \{ \alpha _ { n } \} _ { n > 1 } \) be the sequence constructed in Lemma 4.3. We have that \( Y _ { t } ^ { \alpha _ { n } } = v ^ { \alpha _ { n } } ( X _ { t } ) \) and \( e _ { k } ^ { * } Z _ { t } ^ { \alpha _ { n } } = v ^ { \alpha _ { n } } ( e _ { k } ) \) solve the discounted BSDE

\[ Y _ { T } ^ { \alpha _ { n } } = Y _ { t } ^ { \alpha _ { n } } - \sum _ { t \leq u < T } \left( f ( X _ { u } , Z _ { u } ^ { \alpha _ { n } } ) - \alpha _ { n } Y _ { u } ^ { \alpha _ { n } } \right) + \sum _ { t \leq u < T } \left( Z _ { u } ^ { \alpha _ { n } } \right) ^ { * } M _ { u + 1 } . \]

However, since \( \| \mathbf { 1 } \| _ { M _ { t + 1 } } = 0 \), and f does not distinguish between values of \( Z _ { u } \) up to equivalence \( { \sim } { M } _ { u + 1 } \) , we can equally write \( Z _ { u } ^ { \alpha _ { n } } - v ^ { \alpha _ { n } } ( x _ { 0 } ) \mathbf { 1 } \) in the place of \( Z _ { u } ^ { \alpha _ { n } } \) in the above. Note that \( e _ { k } ^ { * } Z _ { u } ^ { \alpha _ { n } } - v ^ { \alpha _ { n } } ( x _ { 0 } ) \to e _ { k } ^ { * } Z _ { u } \) as \( n \to \infty \) for each \( e _ { k } \in \mathcal { X } \), and that, by the bound established in Lemma 4.3, \|e∗kZuαn   −vαn(x0)\|is uniformly bounded. Since f is Lipschitz in z, we deduce that

\[ f ( X _ { u } , Z _ { u } ^ { \alpha _ { n } } - v ^ { \alpha _ { n } } ( x _ { 0 } ) \mathbf { 1 } ) \to f ( X _ { u } , Z _ { u } ) \quad { \mathrm { a s } } \quad n \to \infty \quad { \mathrm { a . s . } } \]

It follows that

\[ \begin{array} { l } { { \displaystyle Y _ { T } = \operatorname* { l i m } _ { n \to \infty } \left( v ^ { \alpha _ { n } } ( X _ { T } ) - v ^ { \alpha _ { n } } ( x _ { 0 } ) \right) } } \\ { { \displaystyle \quad = \operatorname* { l i m } _ { n \to \infty } \left( v ^ { \alpha _ { n } } ( X _ { t } ) - v ^ { \alpha _ { n } } ( x _ { 0 } ) \right) - \operatorname* { l i m } _ { n \to \infty } \displaystyle \sum _ { t \leq u < T } \left( f ( X _ { u } , Z _ { u } ^ { \alpha _ { n } } ) - \alpha _ { n } v ^ { \alpha _ { n } } ( X _ { u } ) \right) } } \\ { { \displaystyle \qquad + \operatorname* { l i m } _ { n \to \infty } \displaystyle \sum _ { t \leq u < T } \left( Z _ { u } ^ { \alpha _ { n } } \right) ^ { * } M _ { u + 1 } } } \\ { { \displaystyle \qquad = v ( X _ { t } ) - \operatorname* { l i m } _ { n \to \infty } \displaystyle \sum _ { t \leq u < T } \left( f ( X _ { u } , Z _ { u } ^ { \alpha _ { n } } - v ^ { \alpha _ { n } } ( x _ { 0 } ) \mathbf { 1 } ) - \alpha _ { n } v ^ { \alpha _ { n } } ( X _ { u } ) \right) } } \\ { { \displaystyle \qquad + \operatorname* { l i m } _ { n \to \infty } \displaystyle \sum _ { t \leq u < T } \left( Z _ { u } ^ { \alpha _ { n } } - v ^ { \alpha _ { n } } ( x _ { 0 } ) \mathbf { 1 } \right) ^ { * } M _ { u + 1 } } } \\ { { \displaystyle \qquad = Y _ { t } - \displaystyle \sum _ { t \leq u < T } \left( f ( X _ { u } , Z _ { u } ) - \lambda \right) + \displaystyle \sum _ { t \leq u < T } Z _ { u } ^ { * } M _ { u + 1 } , } } \end{array} \]

and we see that \( ( Y , Z , \lambda ) \) is indeed a solution of the EBSDE (4.4).

Suppose that \( ( Y ^ { \prime } , Z ^ { \prime } , \lambda ^ { \prime } ) \) is another bounded solution. Let \( \tilde { Y } \, = \, Y \, - \, Y ^ { \prime } \),\( \tilde { Z } = Z - Z ^ { \prime } \) and \( \tilde { \lambda } = \lambda - \lambda ^ { \prime } \). Then

\[ \tilde { Y } _ { T } = \tilde { Y } _ { 0 } - \sum _ { 0 \leq u < T } \left( f ( X _ { u } , Z _ { u } ) - f ( X _ { u } , Z _ { u } ^ { \prime } ) - \tilde { \lambda } \right) + \sum _ { 0 \leq u < T } \tilde { Z } _ { u } ^ { \ast } M _ { u + 1 } . \qquad ( 4 . 5 ) \]

By Proposition 2.2, there exists a measure Q such that

\[ - \sum _ { 0 \leq u < t } \left( f ( X _ { u } , Z _ { u } ) - f ( X _ { u } , Z _ { u } ^ { \prime } ) \right) + \sum _ { 0 \leq u < t } \tilde { Z } _ { u } ^ { \ast } M _ { u + 1 } \]

is a martingale under Q. Taking an \( \mathbb { E } _ { \mathbb { Q } } \) expectation in (4.5), we obtain

\[ \begin{array} { r } { \tilde { \lambda } = T ^ { - 1 } \mathbb { E } _ { \mathbb { Q } } \big [ \tilde { Y } _ { T } - \tilde { Y } _ { 0 } \big ] . } \end{array} \]

Since \( \tilde { Y } \) is uniformly bounded, taking the limit \( T \to \infty \) gives \( \tilde { \lambda } \) = 0, so that \( \lambda = \lambda ^ { \prime } \). Substituting back into (4.5) and taking an \( \mathbb { E } _ { \mathbb { O } } [ \, \cdot \, | \, \mathcal { F } _ { 0 } ] \)] expectation gives

\[ \mathbb { E } _ { \mathbb { Q } } { \left[ \tilde { Y } _ { T } \mid \mathcal { F } _ { 0 } \right] } = \tilde { Y } _ { 0 } . \eqno ( 4 . 6 ) \]

Suppose further that \( Y ^ { \prime } \) and \( Z ^ { \prime } \) are Markovian and do not depend on t, so that in particular there exists a function \( v ^ { \prime } : \mathcal { X } \rightarrow \mathbb { R } \) such that \( Y _ { t } ^ { \prime } \, = \, v ^ { \prime } ( X _ { t } ) \). Then the measure Q may be taken to be the measure given by Lemma 4.1, so that X is still a uniformly ergodic Markov chain under Q. Writing \( \tilde { \pi } \) for the ergodic measure of X under Q, it follows from (4.6) that, for any \( x \in \mathcal { X } \),

\[ v ( x ) - v ^ { \prime } ( x ) = \operatorname* { l i m } _ { T \to \infty } \mathbb { E } _ { \mathbb { Q } } \big [ \tilde { Y } _ { T } \, \big | \, X _ { 0 } = x \big ] = \int _ { \mathcal { X } } \big ( v ( y ) - v ^ { \prime } ( y ) \big ) d \tilde { \pi } ( y ) . \]

Since the right hand side is independent of x, we see that \( v ( x ) = v ^ { \prime } ( x ) + c \) for all x, and hence that \( Y _ { t } = Y _ { t } ^ { \prime } + c \), for some \( c \in \mathbb { R } \). In particular, if \( v ^ { \prime } ( x _ { 0 } ) = 0 \), then \( c \) = 0, and hence \( Y = Y ^ { \prime } \) up to indistinguishability.

With \( \tilde { \lambda } \) = 0 and \( \tilde { Y } _ { t } = c = \tilde { Y } _ { t + 1 } \), we deduce from the one-step dynamics of \( \tilde { Y } \) that \( \tilde { Z } _ { t } ^ { * } M _ { t + 1 } = 0 , \) and hence that \( Z \sim _ { M } Z ^ { \prime } \).