We prove the first Chernoff-Hoeffding bounds for general (irreversible) finite-state Markov chains based on the standard L 1 (variation distance) mixing-time of the chain. Specifically, consider an ergodic Markov chain M and a weight function f : , where π is the stationary distribution of M . A t-step random walk (v 1 , . . . , v t ) on M starting from the stationary distribution π has expected total weight We show that the probability of X deviating from its mean by a multiplicative factor of δ, i.e., Pr [|X − µt| ≥ δµt], is at most exp(−Ω δ 2 µt/T ) for 0 ≤ δ ≤ 1, and exp(−Ω (δµt/T )) for δ > 1. In fact, the bounds hold even if the weight functions f i 's for i ∈ [t] are distinct, provided that all of them have the same mean µ. We also obtain a simplified proof for the Chernoff-Hoeffding bounds based on the spectral expansion λ of M , which is the square root of the second largest eigenvalue (in absolute value) of MM , whereM is the time-reversal Markov chain of M . We show that the probability Pr [|X − µt| ≥ δµt] is at most exp(−Ω δ 2 (1 − λ)µt ) for 0 ≤ δ ≤ 1, and exp(−Ω (δ(1 − λ)µt)) for δ > 1. Both of our results extend to continuous time Markov chains, and to the case where the walk starts from an arbitrary distribution ϕ, at a price of a multiplicative factor depending on the distribution ϕ in the concentration bounds.
REF obtain rates that depend on the mixing time of the chain rather than the spectral gap, but which are still suboptimal.
Chernoff-Hoeffding Bounds for Markov Chains: Generalized and Simplified
STACS
Computer Science Mathematics