Fast discrete-time Markov chains