Bayesian inference

**Bayesian inference** (  or  ) is a type of statistical inference. In Bayesian inference, evidence or information is available, Bayes' theorem is used to change (or update) the probability of a hypothesis. Bayesian inference uses a prior distribution to estimate posterior probabilities. Bayesian inference is an important to statistics, mathematical statistics, decision theory, and sequential analysis. Bayesian inference is used in science, engineering, philosophy, medicine, sport, and law.

## Bayes' rule
Bayesian inference figures out the posterior probability from prior probability and the "likelihood function". The likelihood function comes from a statistical model of the data.
P(H \mid E) = \frac{P(E \mid H) \cdot P(H)}{P(E)},
where
* H is a *hypothesis* that is changed by data (or *evidence*). There are usually many hypotheses. The point of the test is to see which hypothesis is more likely.
* P(H) is the *prior probability*. It estimates the probability  of a hypothesis before there is any evidence.
* E is the *evidence*, or data. It is any new data that is found.
* P(H \mid E) is the *posterior probability.* This is what we want to know.
* P(E \mid H) is the  *likelihood function*. 
* P(E) is the marginal likelihood. It is the same for all possible hypotheses that are being tested. P(E) has to be greater than 0. If P(E) is 0, then you divide by zero.

## Related pages
* Epistemology
* Free energy principle

## Further reading
* 
* 

* Stone, JV (2013), "Bayes' Rule: A Tutorial Introduction to Bayesian Analysis",   Download first  chapter here, Sebtel Press, England.
* 
* 
* 
*
* Bolstad, William M. (2007) *Introduction to Bayesian Statistics*: Second Edition, John Wiley 
* Updated classic textbook. Bayesian theory clearly presented.
* Lee, Peter M. *Bayesian Statistics: An Introduction*. Fourth Edition (2012), John Wiley 
* 
* 

* 
*
* DeGroot, Morris H., *Optimal Statistical Decisions*. Wiley Classics Library. 2004. (Originally published (1970) by McGraw-Hill.) .
* 
* Jaynes, E. T. (1998). *Probability Theory: The Logic of Science*.
* O'Hagan, A. and Forster, J. (2003). ''Kendall's Advanced Theory of Statistics*, Volume 2B: *Bayesian Inference*. Arnold, New York. .
* 
* Pearl, Judea. (1988). *Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference*, San Mateo, CA: Morgan Kaufmann.
* Pierre Bessière et al. (2013). "Bayesian Programming". CRC Press. 
* Francisco J. Samaniego (2010). "A Comparison of the Bayesian and Frequentist Approaches to Estimation". Springer. New York, 

## References
## Other websites
* Bayesian Statistics from Scholarpedia.
* Introduction to Bayesian probability  from Queen Mary University of London
* Mathematical Notes on Bayesian Statistics and Markov Chain Monte Carlo
* Bayesian reading list , categorized and annotated by Tom Griffiths
* A. Hajek and S. Hartmann: Bayesian Epistemology, in: J. Dancy et al. (eds.), A Companion to Epistemology. Oxford: Blackwell 2010, 93–106.
* S. Hartmann and J. Sprenger: Bayesian Epistemology, in: S. Bernecker and D. Pritchard (eds.), Routledge Companion to Epistemology. London: Routledge 2010, 609–620.
* *Stanford Encyclopedia of Philosophy*: "Inductive Logic"
*Bayesian Confirmation Theory (PDF)
* *Data, Uncertainty and Inference'' — Informal introduction with many examples, ebook (PDF) freely available at causaScientia
* Broad Introduction to Bayesian Statistics for Machine Learning