The Markov property is about independence, specifically about the future development of a process being independent of anything that has happened in the past, given its present value. The Markov property also holds for SRW with barriers. But there are many more examples. Sunny or Cloudy. Today's weather affects tomorrow's, but yesterday's is already irrelevant to tomorrow's forecast.
Any random process has a fixed set of values which it can take.
The collection of possible values is called the state space , S , and the possible values are termed states . The elements of a transition matrix must lie between 0 and 1. In addition, since each transition must take the chain somewhere, the row sums must be equal to 1. 4.3 Multi-step transitions Knowing today's weather, the transition probabilities give information about tomorrow's. In principle we only need to find the powers of P to evaluate the distribution of X n for all n . Within each column the values are getting closer.