markov hw