  Dissipativity theory is central to discussing the stability of policies
resulting from minimzing economic stage costs. In its current form, the
dissipativity theory applies to problems based on deterministic dynamics, and
does not readily extends to Markov Decision Processes, where the dynamics are
stochastic. In this paper, we clarify the core reason for this difficulty, and
propose a generalization of the dissipativity theory that circumvents it. This
generalization is based on nonlinear stage cost functionals, allowing one to
discuss the Lyapunov asymptotic stability of policies for Markov Decision
Processes in the set of probability measures. This theory is illustrated in the
stochastic Linear Quadratic Regulator case, for which a storage functional can
be provided analytically. For the sake of brevity, we limit our discussion to
undiscounted MDPs.
