Unlike Kolmogorov complexity, information theory is only concerned with the average information of a random source. Universal Induction and Data Compression. Traditional wisdom has it that the better a theory compresses the learning data concerning some phenomenon under investigation, the better we learn, generalize, and the better the theory predicts unknown data. This belief is vindicated in practice but before the advent of Kolmogorov complexity has not been rigorously proved in a general setting. Ray Solomonoff invented the notion of universal prediction using the Kolmogorov complexity based universal distribution.
Universal prediction is related to optimal effective compression.
Whereas the single best hypothesis does not necessarily give the best prediction, it can be demonstrated that nonetheless compression is almost always the best strategy in prediction methods in the style of R. Solomonoff. A short biography of Kolmogorov is available. Also Gregory Chaitin proposed this idea in a paper in J. ACM in 1969. This paper continues a paper by G. Chaitin in J.ACM in 1966, which however was concerned with a complexity measure based on Turing machine state-symbol product following ideas of C.E. Shannon . After we and others pioneered several successful applications of Kolmogorov complexity in the theory of computation, the general pattern of the incompressibility method emerged.