Written all the markov code.