page_title: History of statistics
section_title: Bayesian statistics
hierarchical_section_title: History of statistics / Bayesian statistics
context_page_description: Statistics, in the modern sense of the word, began evolving in the 18th century in response to the novel needs of industrializing sovereign states. The evolution of statistics is, in particular, intimately connected with the development of European states following the peace of Westphalia, and with the development of probability theory, which put statistics on a firm theoretical basis.
  In early times, the meaning was restricted to information about states, particularly demographics such as population. This was later extended to include all collections of information of all types, and later still it was extended to include the analysis and interpretation of such data. In modern terms, "statistics" means both sets of collected information, as in national accounts and temperature records, and analytical work which requires statistical inference. Statistical activities are often associated with models expressed using probabilities, hence the connection with probability theory. The large requirements of data processing have made statistics a key application of computing; see history of computing hardware.
context_section_description: The term Bayesian refers to Thomas Bayes (1702–1761), who proved that probabilistic limits could be placed on an unknown event.  However it was Pierre-Simon Laplace (1749–1827) who introduced (as principle VI) what is now called Bayes' theorem. and applied it to celestial mechanics, medical statistics, reliability, and jurisprudence.  When insufficient knowledge was available to specify an informed prior, Laplace used uniform priors, according to his "principle of insufficient reason". Laplace assumed uniform priors for mathematical simplicity rather than for philosophical reasons. Laplace also introduced primitive versions of conjugate priors and the theorem of von Mises and Bernstein, according to which the posteriors corresponding to initially differing priors ultimately agree, as the number of observations increases. This early Bayesian inference, which used uniform priors following Laplace's principle of insufficient reason, was called "inverse probability" (because it infers backwards from observations to parameters, or from effects to causes).
  After the 1920s, inverse probability  was largely supplanted  by a collection of methods that were developed by Ronald A. Fisher, Jerzy Neyman and Egon Pearson. Their methods came to be called frequentist statistics. Fisher rejected the Bayesian view, writing that "the theory of inverse probability is founded upon an error, and must be wholly rejected". At the end of his life, however, Fisher expressed greater respect for the essay of Bayes, which Fisher believed to have anticipated his own, fiducial approach to probability; Fisher still maintained that Laplace's views on probability were "fallacious rubbish". Neyman started out as a "quasi-Bayesian", but subsequently developed confidence intervals (a key method in frequentist statistics) because "the whole theory would look nicer if it were built from the start without reference to Bayesianism and priors".
  The word Bayesian appeared around 1950, and by the 1960s it became the term preferred by those dissatisfied with the limitations of frequentist statistics.
  In the 20th century, the ideas of Laplace were further developed in two different directions, giving rise to objective and subjective currents in Bayesian practice. In the objectivist stream, the statistical analysis depends on only the model assumed and the data analysed. No subjective decisions need to be involved. In contrast, "subjectivist" statisticians deny the possibility of fully objective analysis for the general case.
  In the further development of Laplace's ideas, subjective ideas predate objectivist positions. The idea that 'probability' should be interpreted as 'subjective degree of belief in a proposition' was  proposed, for example, by John Maynard Keynes in the early 1920s. This idea was taken further by Bruno de Finetti in Italy (Fondamenti Logici del Ragionamento Probabilistico, 1930) and Frank Ramsey in Cambridge (The Foundations of Mathematics, 1931). The approach was devised to solve problems with the  frequentist definition of probability but also with the earlier, objectivist approach of Laplace. The subjective Bayesian methods were further developed and popularized in the 1950s by L.J. Savage.
  Objective Bayesian inference was further developed by Harold Jeffreys at the University of Cambridge. His seminal book "Theory of probability" first appeared in 1939 and played an important role in the revival of the Bayesian view of probability. In 1957, Edwin Jaynes promoted the concept of maximum entropy for constructing priors, which is an important principle in the formulation of objective methods, mainly for discrete problems. In 1965, Dennis Lindley's 2-volume work "Introduction to Probability and Statistics from a Bayesian Viewpoint" brought Bayesian methods to a wide audience. In 1979, José-Miguel Bernardo introduced reference analysis, which offers a general applicable framework for objective analysis. Other well-known proponents of Bayesian probability theory include I.J. Good, B.O. Koopman, Howard Raiffa, Robert Schlaifer and Alan Turing.
  In the
caption: Pierre-Simon, marquis de Laplace, the main early developer of Bayesian statistics.
exif: {}