Markov machine simulator