Before revision:  With this design , NOMU can provide model uncertainty for any given (previously trained) NN by plugging it into the framework as the sub-network used for model prediction. NOMU is designed to yield uncertainty bounds (UBs) that satisfy four important desiderata regarding model uncertainty , which established methods often do not satisfy. Furthermore, our UBs are themselves representable as a single NN, which leads to computational cost advantages in applications such as Bayesian optimization. We evaluate NOMU experimentally in multiple settings. For regression, we show that NOMU performs as well as or better than established benchmarks.
After revision:  With this design , NOMU can provide model uncertainty for any given (previously trained) NN if given access to its training data. We first experimentally study noiseless regression with scarce training data to highlight the deficiencies of the established benchmarks. Finally, we study the important task of Bayesian optimization (BO) with costly evaluations, where good model uncertainty estimates are essential. Our results show that NOMU performs as well as or better than established benchmarks.