Markov process
We define a Markov process by defining a transition probability formula_37 where the numbers formula_38 define a (countable) stochastic matrix formula_39 i.e.