Stuff from the markov chain model used in chapters 2-4 in phd