Still Minding the Gap? Reflecting on Transitions between Concepts of Information in Varied Domains This conceptual paper, a contribution to the tenth anniversary Special Issue of Information, gives a crossdisciplinary review of general and unified theories of information. A selective literature review is used to update a 2013 article on bridging the gaps between conceptions of information in different domains, including material from the physical and biological sciences, from the humanities and social sciences including library and information science, and from philosophy. A variety of approaches and theories are reviewed, including those of Brenner, Brier, Burgin and Wu, Capurro, CardenasGarcia and Ireland, Hidalgo, Hofkirchner, Kolchinsky and Wolpert, Floridi, Mingers and Standing, Popper, and Stonier. The gaps between disciplinary views of information remain, although there has been progress, and increasing interest, in bridging them. The solution is likely to be either a general theory of sufficient flexibility to cope with multiple meanings of information, or multiple and distinct theories for different domains, but with a complementary nature, and ideally boundary spanning concepts.

Generalizing Information to the Evolution of Rational Belief Information theory provides a mathematical foundation to measure uncertainty in belief. Belief is represented by a probability distribution that captures our understanding of an outcomexe2x80x99s plausibility. Information measures based on Shannonxe2x80x99s concept of entropy include realization information, Kullbackxe2x80x93Leibler divergence, Lindleyxe2x80x99s information in experiment, cross entropy, and mutual information. We derive a general theory of information from first principles that accounts for evolving belief and recovers all of these measures. Rather than simply gauging uncertainty, information is understood in this theory to measure change in belief. We may then regard entropy as the information we expect to gain upon realization of a discrete latent random variable. This theory of information is compatible with the Bayesian paradigm in which rational belief is updated as evidence becomes available. Furthermore, this theory admits novel measures of information with welldefined properties, which we explored in both analysis and experiment. This view of information illuminates the study of machine learning by allowing us to quantify information captured by a predictive model and distinguish it from residual information contained in training data. We gain related insights regarding feature selection, anomaly detection, and novel Bayesian approaches.