  Epistemic uncertainty is the part of out-of-sample prediction error due to
the lack of knowledge of the learner. Whereas previous work was focusing on
model variance, we propose a principled approach for directly estimating
epistemic uncertainty by learning to predict generalization error and
subtracting an estimate of aleatoric uncertainty, i.e., intrinsic
unpredictability. This estimator of epistemic uncertainty includes the effect
of model bias and can be applied in non-stationary learning environments
arising in active learning or reinforcement learning. In addition to
demonstrating these properties of Direct Epistemic Uncertainty Prediction
(DEUP), we illustrate its advantage against existing methods for uncertainty
estimation on downstream tasks including sequential model optimization and
reinforcement learning. We also evaluate the quality of uncertainty estimates
from DEUP for probabilistic classification of images and for estimating
uncertainty about synergistic drug combinations.
