# Active Data Selection and Information Seeking
## 4 Function Approximation
An answer to this question can be drawn from behavioural neuroscience and the so-called exploitation-exploration dilemma [27; 28]. This deals with the situation in which certain parts of our environment may be particularly aversive or attractive--perhaps due to the presence of food or predators--but we do not know where these locations are to begin with. Initially, we must explore, seeking information about our environment, and then at some stage switch to exploiting the knowledge acquired, to ensure we occupy preferred locations. If all locations were equally preferrable, we could continue exploring forever, developing ever more precise beliefs about our world. The same is true when sampling data or performing experiments if there is no cost to performing this sampling. However, once we acknowledge the computational and resource costs to sampling, we conclude that if it were not for the potential information gain, it would be preferable to stop acquiring new data. To accommodate this cost of sampling, we can simply augment the expected information gain with a prior belief about our propensity to stop sampling:  
\[G=[I[\pi],0]+C \tag{11}\]  
Equation (11) treats the information gain as a vector of all the possible sampling locations and concatenates this with a zero element, which reflects the information gained if we were to stop sampling. The relative preference for sampling or not is then expressed in the \(C\) vector, which assigns a prior (in the form of a log probability) to each choice of sampling location. In neurobiology, this expression is sometimes referred to as an expected free energy--reflecting its analogous form (KL-Divergence and log marginal) to the variational free energy functionals used in approximate inferences [29; 30; 31]. It allows us to combine information-seeking and cost-aversive imperatives into the same objective function. Figure 5 shows what happens when we select our data according to \(G[\pi]\), where the preference for stopping sampling takes the value \(1/4\) (i.e., \(C=[0,\ldots,0,1/4]\) includes zeros for all but the'stop sampling' option, which is assigned a value of \(1/4\)). This means that when the expected information gain is sufficiently low the chance of deciding to stop sampling increases. In Figure 5, we see that the same first 15 choices are made as in Figure 4. However, at this point, a decision is made to stop sampling. This is because sufficient uncertainty has been resolved that the cost of sampling is greater than the potential information gain by sampling further. Depending upon how costly the sampling is assumed to be, this decision to stop will move earlier or later, and the quality of the function approximation will vary. A reasonable question to ask at this stage is why bother with the full information-seeking objective? It is clear from Figures 4 and 5 that all we need to do is choose to sample from the regions with greatest predicted variance, sequentially reducing that variance until it is minimal throughout the domain of the function being approximated [32]. This suggests that the predictive entropy from Equation (2) is sufficient on its own (c.f., maximum entropy sampling [33]). However, Figure 6 offers a rebuttal to this with a process whose measurement noise increases in variance from the centre of the function domain. This means the amount of unresolvable uncertainty is heterogeneous throughout the domain of potential sampling. Were we to ignore this, the uncertainty left to resolve will always be greater than the cost of sampling. Using the full expected information gain tells us that, although there is uncertainty left, there is no information to gain, and allows for earlier termination of our sampling procedure. Furthermore, it leads to a methodical sampling strategy: sampling starts from the minimally ambiguous central location and fans out as local uncertainty is reduced until the capacity to resolve further uncertainty drops below the cost of acquiring further samples.