We compute the finite-sample minimax (modulo logarithmic factors) sample complexity of learning the parameters of a finite Markov chain from a single long sequence of states. Our error metric is a natural variant of total variation. The sample complexity necessarily depends on the spectral gap and minimal stationary probability of the unknown chain, for which there are known finite-sample estimators with fully empirical confidence intervals. To our knowledge, this is the first PAC-type result with nearly matching (up to logarithmic factors) upper and lower bounds for learning, in any metric, in the context of Markov chains. Our definitions and notation are mostly standard, and are given in Section 3. Since the focus of this paper is on statistical rather than computational complexity, we defer the (straightforward) analysis of the runtime of our learner to the Appendix, Section A.
REF derives a PAC-type bound for learning the transition matrix of an ergodic Markov chain with respect to the total variation loss.
Minimax Learning of Ergodic Markov Chains
ArXiv
ArXiv
Computer Science Mathematics