We study an extension of optimal transport techniques to stationary Markov chains from a computational perspective. In this context, naively applying optimal transport to the stationary distributions of the Markov chainsof interest would not capture the Markovian dynamics. Instead, we study a new problem, called the optimal transition coupling problem, in which the optimal transport problem is constrained to the set of stationary Markovian couplings satisfying a certain transition matrix condition. After drawing a connection between this problem and Markov decision processes, we prove that solutions can be obtained via the policy iterationalgorithm . For settings with large state spaces, we also define a regularized problem, propose a faster, approximate algorithm , and provide bounds on the computational complexityof each iteration. Finally, we validate our theoretical results empirically  , demonstrating that the approximate algorithm exhibits faster overall runtime with low error in a simulation study .
We study the optimal transport problem for pairs of stationary finite-state Markov chains, with an emphasis on the computation of optimal transition couplings. Transition couplings are a constrained family of transport plans that capture the dynamics of Markov chains . Solutions of the optimal transition coupling (OTC) problem correspond to alignments of the two chains that minimize long-term average cost. We establish a connection between the OTC problem and Markov decision processes, and show that solutions of the OTC problem can be obtained via an adaptation of policy iteration . For settings with large state spaces, we develop a fast approximate algorithm based on an entropy-regularized version of the OTC problem , and provide bounds on its per-iteration complexity. We establish a stability result for both the regularized and unregularized algorithms, from which a statistical consistency result follows as a corollary. We validate our theoretical results empirically through a simulation study , demonstrating that the approximate algorithm exhibits faster overall runtime with low error . Finally, we extend the setting and application of our methods to hidden Markov models, and illustrate the potential use of the proposed algorithms in practice with an application to computer-generated music .
R
an extension of optimal transport techniques to stationary
the optimal transport problem for pairs of stationary finite-state Markov chains, with an emphasis on the computation of optimal transition couplings. Transition couplings are a constrained family of transport plans that capture the dynamics of
R
from a computational perspective. In this context, naively applying optimal transport to the stationary distributions of the Markov chainsof interest would not capture the Markovian dynamics. Instead, we study a new problem, called the
. Solutions of the
R
problem, in which the optimal transport problem is constrained to the set of stationary Markovian couplings satisfying a certain transition matrix condition. After drawing
(OTC) problem correspond to alignments of the two chains that minimize long-term average cost. We establish
R
this
the OTC
R
we prove that solutions
and show that solutions of the OTC problem
R
the policy iterationalgorithm
an adaptation of policy iteration
R
also define a regularized problem, propose a faster, approximate algorithm
develop a fast approximate algorithm based on an entropy-regularized version of the OTC problem
R
the computational complexityof each iteration. Finally, we
its per-iteration complexity. We establish a stability result for both the regularized and unregularized algorithms, from which a statistical consistency result follows as a corollary. We
A
through a simulation study
R
in a simulation study
. Finally, we extend the setting and application of our methods to hidden Markov models, and illustrate the potential use of the proposed algorithms in practice with an application to computer-generated music