markov test code