Fixed all issues in markov.py.