Implementation of Markov Models