Markov model

A Markov model is when things are predicted from other things that came immediately before. For example, if it usually snows after it rains, you might guess that it will snow tomorrow if it is raining today. A Markov model is part of probability theory. Mostly, it tries to predict "states" from previous states. Some Markov models include the Markov chain and the Markov decision process.

 Markov chain 

The Markov chain is a simple Markov model. It has "the Markov property," which means that future states of the variable are because of the previous states of the variable.

 Hidden Markov model 
A hidden Markov model is a type of Markov chain. This is very similar to a basic Markov model, but when the state is only partially observable or noisily observable. In other words, past states are related to the state of the system, but they don't tell us enough to perfectly know the state we are trying to predict. This is used commonly. For example, it helps computers automatically turn speech audio into words.

 Markov decision process 

A Markov decision process is a type of Markov chain. A Markov decision process goes from one state to another. It uses a vector to do this.

 Uses 
Markov chains are used to predict the future. They can predict prices and wind power.

 References 



Category:Probability theory