# Active Data Selection and Information Seeking
## 2 Bayesian Inference, Generative Models, and Expected Information Gain  
Bayesian inference is the process of inverting a model of how data (\(y\)) are generated to obtain two things [18]. The first is the probability of observed data under our model--sometimes referred to as marginal likelihood. This is a ubiquitous measure of the evidence our data affords the hypothesis expressed in our model. Second is the probability of the random variables (\(\theta\)) in the model given our data--known as a posterior probability. These two things can be obtained by specifying the a priori plausible distributions of model variables and the likelihood of generating patterns of data given the values those variables might take. These prior and likelihood beliefs form our generative model. The  
Figure 1: The active sampling cycle. This graphic illustrates the basic idea behind this paper. Analogous with action–perception cycles of the sort found in biological systems, it shows reciprocal interactions between the process of sampling data (in biology, through acting upon the world to solicit new sensations) and drawing inferences about these data (akin to perceptual inference). relationship between the generative model and the associated marginal likelihood and posterior probability is given by [PERSON] theorem, which can be expressed as:  
\[\underbrace{p(y|\theta)p(\theta)}_{\text{Generative}} =\underbrace{p(\theta|y)}_{\text{Posterior}}p(y)\] (1) Model \[\underbrace{p(y)}_{\text{Marginal}} =\ \mathbb{E}_{p(\theta)}[p(y|\theta)]\] \[\text{Likelihood}\]  
Equation (1) shows the generative model, its inversion, and the relationship between a model and its marginal likelihood. The second line shows that the marginal likelihood is obtained directly from the prior and likelihood beliefs simply by taking the expectation (or average) of the latter, assuming our \(\theta\) variables are distributed according to the prior (also known as marginalisation). This leaves only one unknown in the first line--the posterior--which can then be obtained from rearrangement of the other three terms. In practice, \(\theta\) may include many different parameters or hidden states, making exact computation impractical [19]. While we will touch upon the topic of approximate inference briefly in one of the examples, this is of limited importance for the primary message of this paper. However, the question of how we deal with complex models with multiple parameters is important to address. Bayesian networks, and related graphical notations, offer a visualisation of a generative model. They tell us which variables depend upon which other variables. This is useful in that it suggests efficient message passing schemes that may be used to perform an inference (see [20; 21] for overviews). Such representations have broad applicability, ranging from clinical reasoning [22] to environmental conservation [23]. For our purposes, message passing in graphical models is useful in that it helps us to find the quantities required for computing expected information gain. The information we gain on making an observation can be formalised as the change in our beliefs as a consequence of that observation. A change in probabilities of this sort is quantified using a KL-Divergence (also known as relative entropy)--the average difference between the log probabilities before and after that observation (i.e., between the prior and posterior of Equation (1)). The value this KL-Divergence takes, averaged over the data we would predict using our marginal likelihood, is our expected information gain [14]:  
\[