Bayesian inference

Bayesian inference (  or  ) is a type of statistical inference. In Bayesian inference, evidence or information is available, Bayes' theorem is used to change (or update) the probability of a hypothesis. Bayesian inference uses a prior distribution to estimate posterior probabilities. Bayesian inference is an important to statistics, mathematical statistics, decision theory, and sequential analysis. Bayesian inference is used in science, engineering, philosophy, medicine, sport, and law.

Bayes' rule

+ Contingency table   SatisfieshypothesisH  Violateshypothesis¬H   Total Has evidenceEP(H|E)·P(E)= P(E|H)·P(H) P(¬H|E)·P(E)= P(E|¬H)·P(¬H)  P(E) No evidence¬EP(H|¬E)·P(¬E)= P(¬E|H)·P(H) P(¬H|¬E)·P(¬E)= P(¬E|¬H)·P(¬H) P(¬E) =1−P(E) Total    P(H) P(¬H) = 1−P(H) 1
Bayesian inference figures out the posterior probability from prior probability and the "likelihood function". The likelihood function comes from a statistical model of the data.

where
  is a hypothesis that is changed by data (or evidence). There are usually many hypotheses. The point of the test is to see which hypothesis is more likely.
  is the prior probability. It estimates the probability  of a hypothesis before there is any evidence.
  is the evidence, or data. It is any new data that is found.
  is the posterior probability. This is what we want to know.
  is the  likelihood function. 
  is the marginal likelihood. It is the same for all possible hypotheses that are being tested.  has to be greater than 0. If  is 0, then you divide by zero.

Related pages
 Epistemology
 Free energy principle

Further reading




 Stone, JV (2013), "Bayes' Rule: A Tutorial Introduction to Bayesian Analysis",   Download first  chapter here, Sebtel Press, England.




 Bolstad, William M. (2007) Introduction to Bayesian Statistics: Second Edition, John Wiley 
 Updated classic textbook. Bayesian theory clearly presented.
 Lee, Peter M. Bayesian Statistics: An Introduction. Fourth Edition (2012), John Wiley 





 DeGroot, Morris H., Optimal Statistical Decisions. Wiley Classics Library. 2004. (Originally published (1970) by McGraw-Hill.) .

 Jaynes, E. T. (1998). Probability Theory: The Logic of Science.
 O'Hagan, A. and Forster, J. (2003). Kendall's Advanced Theory of Statistics, Volume 2B: Bayesian Inference. Arnold, New York. .

 Pearl, Judea. (1988). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, San Mateo, CA: Morgan Kaufmann.
 Pierre Bessière et al. (2013). "Bayesian Programming". CRC Press. 
 Francisco J. Samaniego (2010). "A Comparison of the Bayesian and Frequentist Approaches to Estimation". Springer. New York, 

References

Other websites

 Bayesian Statistics from Scholarpedia.
 Introduction to Bayesian probability  from Queen Mary University of London
 Mathematical Notes on Bayesian Statistics and Markov Chain Monte Carlo
 Bayesian reading list , categorized and annotated by Tom Griffiths
 A. Hajek and S. Hartmann: Bayesian Epistemology, in: J. Dancy et al. (eds.), A Companion to Epistemology. Oxford: Blackwell 2010, 93–106.
 S. Hartmann and J. Sprenger: Bayesian Epistemology, in: S. Bernecker and D. Pritchard (eds.), Routledge Companion to Epistemology. London: Routledge 2010, 609–620.
 Stanford Encyclopedia of Philosophy: "Inductive Logic"
Bayesian Confirmation Theory (PDF)
 Data, Uncertainty and Inference — Informal introduction with many examples, ebook (PDF) freely available at causaScientia
 Broad Introduction to Bayesian Statistics for Machine Learning

interference