 solomonoff's theory of universal inductive inference is a theory of prediction based on logical observations, such as predicting the next symbol based upon a given series of symbols. the only assumption that the theory makes is that the environment follows some unknown but computable probability distribution. it is a mathematical formalization of occam's razor.jj mccall. induction: from kolmogorov and solomonoff to de finetti and back to kolmogorov â metroeconomica, 2004 â wiley online library.d stork. foundations of occam's razor and parsimony in learning from ricoh.com â nips 2001 workshop, 2001a.n. soklakov. occam's razor as a formal basis for a physical theory from arxiv.org â foundations of physics letters, 2002 â springerm hutter. on the existence and convergence of computable universal priors arxiv.org â algorithmic learning theory, 2003 â springer ==origin== ===philosophical=== the theory is based in philosophical foundations, and was founded by ray solomonoff around 1960.samuel rathmanner and marcus hutter. a philosophical treatise of universal induction. entropy, 13(6):1076â1136, 2011 it is a mathematically formalized occam's razor. all computable theories which perfectly describe previous observations are used to calculate the probability of the next observation, with more weight put on the shorter computable theories. marcus hutter's universal artificial intelligence builds upon this to calculate the expected value of an action. ===mathematical=== the proof of the "razor" is based on the known mathematical properties of a probability distribution over a denumerable set. these properties are relevant because the infinite set of all programs is a denumerable set. the sum s of the probabilities of all programs must be exactly equal to one (as per the definition of probability) thus the probabilities must roughly decrease as we enumerate the infinite set of all programs, otherwise s will be strictly greater than one. to be more precise, for every \epsilon > 0, there is some length l such that the probability of all programs longer than l is at most \epsilon. this does not, however, preclude very long programs from having very high probability. fundamental ingredients of the theory are the concepts of algorithmic probability and kolmogorov complexity. the universal prior probability of any prefix p of a computable sequence x is the sum of the probabilities of all programs (for a universal computer) that compute something starting wi
Inductive_reasoning Inference Machine_learning Statistical_inference