A B ayesian Framework for Information-Theoretic Probing (2021)

Abstract: Pimentel et al. (2020) recently analysed probing from an information-theoretic perspective. They argue that probing should be seen as approximating a mutual information. This led to the rather unintuitive conclusion that representations encode exactly the same information about a target task as the original sentences. The mutual information, however, assumes the true probability distribution of a pair of random variables is known, leading to unintuitive results in settings where it is not. This paper proposes a new framework to measure what we term Bayesian mutual information, which analyses information from the perspective of Bayesian agents—allowing for more intuitive findings in scenarios with finite data. For instance, under Bayesian MI we have that data can add information, processing can help, and information can hurt, which makes it more intuitive for machine learning applications. Finally, we apply our framework to probing where we believe Bayesian mutual information naturally operationalises ease of extraction by explicitly limiting the available background knowledge to solve a task.

## background: information theory information theory @xcite provides us with a number of tools to analyse data and their associated probability distributions-among which are the entropy and mutual information. these are traditionally defined according to a "true" probability distribution, i.e. p(x) or p(x, y) foot_0 which may not be known, but dictates the behaviour of random variables x and y . the atomic unit of information theory is the surprisal, which is defined as follows: arguably, the most important information-theoretic definition is its expected value, termed entropy: unfortunately, information theory has a few properties which do not conform to our intuitions about the mechanics of information in machine learning: ## a bayesian approach to information the primary motivation for this paper is developing a series of tools that help us overcome the limitations of traditional information theory as applied to machine learning. specifically, probing representations requires a data-dependent information theory. we thus formulate analogues of surprisal, entropy and mi in terms of bayesian agents-using a framework heavily inspired by bayesian experimental design @xcite . we then prove this framework does not suffer the same infelicities as standard information theory in this context. ## a framework for incremental probing the proposed bayesian framework for information allows us to take into account the amount of data we have for probing. crucially, previous work @xcite failed to adequately account for the observation of data. in doing so, they only analysed the limiting behaviour of information, under which the probing enterprise is not fully sensible-given unlimited data and computation, there is no point in using pre-trained functions. indeed, the higher-level motivation of this work is to find an information-theoretic framework which serves machine learning, and under which the goal of probing is inherently clear. to that end, we propose a relatively simple experimental design. we compute bayesian mutual information, which is a function of the amount of data, to create several learning curves. notation. we define a sentence-level random variable s, with instances s taken from v * , the kleene closure of a potentially infinite vocabulary v . we further define a representation-valued random variable r and a task-valued random variable t , each with instances @xmath0, where t is the set of possible values for the analysed task (e.g. the set of parts of speech in a language). ## an intuitive decomposition we now present some basic results about our framework which, although not strictly necessary for the present study, help motivate it. they also serve as a justification for our choice of cross-entropy when formalising bayesian entropy. with this in mind, we analyse information from the perspective of a fully bayesian agent with a well-formed belief. 12 a classic decomposition of the cross-entropy is the following: we posit a new interpretation for this equality. theorem 6. let θ be a parameter-valued random variable. the entropy of a consistent bayesian agent with well-formed beliefs decomposes as proof. see app. i. in other words, the cross-entropy is composed of the sum between the entropy itself-i.e. the "true" information the data source provides, or its inherent uncertainty-and how much information the data provides about its distribution itself. relation to sdl. the minimum description length (mdl; voita and titov, 2020) is a probing metric defined as h θ (d n ). in its online coding interpretation, it is rewritten as @xcite : h θ (d n ) = n @xmath1. the surplus description length (sdl; @xcite is defined as the difference between a dataset's cross-entropy and its entropy: h θ (d n ) -h(d n ). using theorem 6, we derive a new interpretation for sdl: while closely related to one another, the bayesian mi, mdl and sdl converge to different values in the limit of infinite dataset sizes: it is easy to see that mdl goes to infinity as the dataset size grows-h θ (d n ) grows at least linearly with the data size. the reasons behind sdl also exploding as the data increases are less straightforward, though, but become clear from its bayesian mi interpretation. if the parameter space is continuous, and if the bayesian belief converges at the limit of infinite data (as per theorem 4), the bayesian mutual information in eq. ( 32 ) will naturally go to infinity. 13 we thus argue that bayesian mutual information is a better measure for probing ## conclusion in this paper we proposed an information-theoretic framework to analyse mutual information from the perspective of a bayesian agent; we term this bayesian mutual information. this framework has intuitive properties (at least from a machine learning perspective), which traditional information theory does not, for example: data can be informative, processing can help, and information can hurt. in the experimental portion of our paper, we use bayesian mutual information to probe representations for both part-of-speech tagging and dependency arc labelling. we show that albert is the most informative of the analysed representations in english; and high dimensional representations can provide negative information on low data scenarios.