  We develop BatchEvaluationBALD, a new acquisition function for deep Bayesian
active learning, as an expansion of BatchBALD that takes into account an
evaluation set of unlabeled data, for example, the pool set. We also develop a
variant for the non-Bayesian setting, which we call Evaluation Information
Gain. To reduce computational requirements and allow these methods to scale to
larger acquisition batch sizes, we introduce stochastic acquisition functions
that use importance-sampling of tempered acquisition scores. We call this
method PowerEvaluationBALD. We show in first experiments that
PowerEvaluationBALD works on par with BatchEvaluationBALD, which outperforms
BatchBALD on Repeated MNIST (MNISTx2), while massively reducing the
computational requirements compared to BatchBALD or BatchEvaluationBALD.
