Computing Hitting Probabilities of Markov Chains: Structural Results with regard to the Solution Space of the Corresponding System of Equations In a previous paper, we have shown that forward use of the steadystate difference equations arising from homogeneous discretestate space Markov chains may be subject to inherent numerical instability. More precisely, we have proven that, under some appropriate assumptions on the transition probability matrix P, the solution space S of the difference equation may be partitioned into two subspaces , where the stationary measure of P is an element of , and all solutions in  are asymptotically dominated by the solutions corresponding to . In this paper, we discuss the analogous problem of computing hitting probabilities of Markov chains, which is affected by the same numerical phenomenon. In addition, we have to fulfill a somewhat complicated side condition which essentially differs from those conditions one is usually confronted with when solving initial and boundary value problems. To extract the desired solution, an efficient and numerically stable generalizedcontinuedfractionbased algorithm is developed.

On local stability of stochastic delay nonlinear discrete systems with statedependent noise Abstract   We examine the local stability of solutions of a delay stochastic nonlinear difference equation with deterministic and statedependent Gaussian perturbations. We apply the degenerate Lyapunovxe2x80x93Krasovskii functional technique and construct a sequence of events, each term of which is defined by a bound on a normally distributed random variable. Local stability holds on the intersection of these events, which has probability at least     1  xe2x88x92  xcexb3  ,     xcexb3xe2x80xafxe2x88x88xe2x80xaf(0, 1). This probability can be made arbitrarily high by choosing the initial value sufficiently small. We also present a generalization to systems where a condition for stability is expressed in terms of the diagonal part of the unperturbed system, and computer simulations which illustrate our results.