Bayesian inference

**Bayesian inference** (  or  ) is a type of statistical inference. In Bayesian inference, evidence or information is available, Bayes' theorem is used to change (or update) the probability of a hypothesis. Bayesian inference uses a prior distribution to estimate posterior probabilities. Bayesian inference is an important to statistics, mathematical statistics, decision theory, and sequential analysis. Bayesian inference is used in science, engineering, philosophy, medicine, sport, and law.

## Bayes' rule

||Satisfies hypothesis H !! Violates hypothesis ¬H !! rowspan="5" style="padding:0;"| !! Total||
| --- | --- | --- |
||**P(H|E)·P(E)** = P(E|H)·P(H) || |**P(¬H|E)·P(E)** = P(E|¬H)·P(¬H) || **P(E)**|
|nowrap|**P(H|¬E)·P(¬E)** = P(¬E|H)·P(H) || nowrap|**P(¬H|¬E)·P(¬E)** = P(¬E|¬H)·P(¬H) || nowrap|**P(¬E) = 1−P(E)**|
|colspan="5" style="padding:0;"|||
|P(H)|P(¬H) = 1−P(H)|1|
Bayesian inference figures out the posterior probability from prior probability and the "likelihood function". The likelihood function comes from a statistical model of the data.

where
- is a _hypothesis_ that is changed by data (or _evidence_). There are usually many hypotheses. The point of the test is to see which hypothesis is more likely.
- is the _prior probability_. It estimates the probability  of a hypothesis before there is any evidence.
- is the _evidence_, or data. It is any new data that is found.
- is the _posterior probability._ This is what we want to know.
- is the  _likelihood function_.
- is the marginal likelihood. It is the same for all possible hypotheses that are being tested.  has to be greater than 0. If  is 0, then you divide by zero.

## Related pages

- Epistemology
- Free energy principle

## Further reading

- Stone, JV (2013), "Bayes' Rule: A Tutorial Introduction to Bayesian Analysis",   Download first  chapter here, Sebtel Press, England.
- Bolstad, William M. (2007) _Introduction to Bayesian Statistics_: Second Edition, John Wiley
- Updated classic textbook. Bayesian theory clearly presented.
- Lee, Peter M. _Bayesian Statistics: An Introduction_. Fourth Edition (2012), John Wiley

- DeGroot, Morris H., _Optimal Statistical Decisions_. Wiley Classics Library. 2004. (Originally published (1970) by McGraw-Hill.) .
- Jaynes, E. T. (1998). _Probability Theory: The Logic of Science_.
- O'Hagan, A. and Forster, J. (2003). _Kendall's Advanced Theory of Statistics_, Volume 2B: _Bayesian Inference_. Arnold, New York. .
- Pearl, Judea. (1988). _Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference_, San Mateo, CA: Morgan Kaufmann.
- Pierre Bessière et al. (2013). "Bayesian Programming". CRC Press.
- Francisco J. Samaniego (2010). "A Comparison of the Bayesian and Frequentist Approaches to Estimation". Springer. New York,

## Other websites

- Bayesian Statistics from Scholarpedia.
- Introduction to Bayesian probability  from Queen Mary University of London
- Mathematical Notes on Bayesian Statistics and Markov Chain Monte Carlo
- Bayesian reading list , categorized and annotated by Tom Griffiths
- A. Hajek and S. Hartmann: Bayesian Epistemology, in: J. Dancy et al. (eds.), A Companion to Epistemology. Oxford: Blackwell 2010, 93–106.
- S. Hartmann and J. Sprenger: Bayesian Epistemology, in: S. Bernecker and D. Pritchard (eds.), Routledge Companion to Epistemology. London: Routledge 2010, 609–620.
- _Stanford Encyclopedia of Philosophy_: "Inductive Logic"
- Bayesian Confirmation Theory (PDF)
- _Data, Uncertainty and Inference_ — Informal introduction with many examples, ebook (PDF) freely available at causaScientia
- Broad Introduction to Bayesian Statistics for Machine Learning