this page contains examples of markov chains in action. == board games played with dice == a game of snakes and ladders or any other game whose moves are determined entirely by dice is a markov chain, indeed, an absorbing markov chain. this is in contrast to card games such as blackjack, where the cards represent a 'memory' of the past moves. to see the difference, consider the probability for a certain event in the game. in the above-mentioned dice games, the only thing that matters is the current state of the board. the next state of the board depends on the current state, and the next roll of the dice. it doesn't depend on how things got to their current state. in a game such as blackjack, a player can gain an advantage by remembering which cards have already been shown (and hence which cards are no longer in the deck), so the next state (or hand) of the game is not independent of the past states. == a center-biased random walk == consider a random walk on the number line where, at each step, the position (call it x) may change by +1 (to the right) or â1 (to the left) with probabilities: p_ = \tfrac{1}{2} + \tfrac{1}{2} \left( \tfrac{x}{c+|x|} \right) p_ = 1 - p_ (where c is a constant greater than 0) for example if the constant, c, equals 1, the probabilities of a move to the left at positions x = â2,â1,0,1,2 are given by \tfrac{1}{6},\tfrac{1}{4},\tfrac{1}{2},\tfrac{3}{4},\tfrac{5}{6} respectively. the random walk has a centering effect that weakens as c increases. since the probabilities depend only on the current position (value of x) and not on any prior positions, this biased random walk satisfies the definition of a markov chain. == a very simple weather model == the probabilities of weather conditions (modeled as either rainy or sunny), given the weather on the preceding day, can be represented by a transition matrix: : p = \begin{bmatrix} 0.9 & 0.1 \\ 0.5 & 0.5 \end{bmatrix} the matrix p represents the weather model in which a sunny day is 90% likely to be followed by another sunny day, and a rainy day is 50% likely to be followed by another rainy day. the columns can be labelled "sunny" and "rainy", and the rows can be labelled in the same order. thumbnail|the above matrix as a graph. (p)i j is the probability that, if a given day is of type i, it will be followed by a day of type j. notice that the rows of p sum to 1: this is because p is a stochastic matrix. === predicting the weather === the weather on day 0 is known to be sunny. this is represented by a vector in which the "sunny" entry is 100%, and the "rainy" entry is 0%: : \mathbf{x}^{(0)} = \begin{bmatrix} 1 & 0 \end{bmatrix} the weather on day 1 can be predicted by: : \mathbf{x}^{(1)} = \mathbf{x}^{(0)} p = \begin{bmatrix} 1 & 0 \end{bmatr
Markov_models Mathematical_examples