A Markov chain implementation