Trust Your Robots! Predictive Uncertainty Estimation of Neural Networks with Sparse Gaussian Processes
Authors: Jongseok Lee
Jianxiang Feng
Matthias Humt
Marcus Gerhard Müller
Rudolph Triebel
Venue: CoRL2021 Poster (2021)
Keywords: Robotic Introspection
Bayesian Deep Learning
Gaussian Processes

Abstract: This paper presents a probabilistic framework to obtain both reliable and fast uncertainty estimates for predictions with Deep Neural Networks (DNNs). Our main contribution is a practical and principled combination of DNNs with sparse Gaussian Processes (GPs). We prove theoretically that DNNs can be seen as a special case of sparse GPs, namely mixtures of GP experts (MoE-GP), and we devise a learning algorithm that brings the derived theory into practice. In experiments from two different robotic tasks -- inverse dynamics of a manipulator and object detection on a micro-aerial vehicle (MAV) -- we show the effectiveness of our approach in terms of predictive uncertainty, improved scalability, and run-time efficiency on a Jetson TX2. We thus argue that our approach can pave the way towards reliable and fast robot learning systems with uncertainty awareness.

Trust Your Robots! Predictive Uncertainty Estimation of Neural Networks with Sparse Gaussian Processes
Jongseok Leel.2 Jianxiang Feng!. Matthias Humt! Marcus G. Müller Rudolph Triehel
Institute of Robotics and Mechatronics, German Aerospace Center (DLR)
High Performance Humanoid Technologies, Karlsruhe Institute of Technology (KIT)
Chair of Computer Vision and Artificial Intelligence, Technical University of Munich (TUM)
Autonomous Systems Laboratory, ETH Zürich (ETHZ)
Abstract: This paper presents probabilistic framework to obtain both reliable and fast uncertainty estimates for predictions with Deep Neural Networks (DNNs). Our main contribution is practical and principled combination of DNNs with sparse Gaussian Processes (GPs). We prove thcoretically that DNNs can be seen special case of sparse GPs, namely mixtures of GP experts (MoE-GP), and we devise a learning algorithm that brings the derived theory into practice. experiments from two different robotic tasks inverse dynamics of manipulator and object detection on a micro-aerial vehicle (MAV) - we show the effectiveness of our approach in terms of predictive uncertainty, improved scalability, and run- time efficiency on a Jetson TX2. We thus argue that our approach can pave the way towards reliable and fast robot learning systems with uncertainty awareness.
Keywords: Robotic Introspection, Bayesian Deep Learning, Gaussian Processes
Introduction
This work aims to provide an algorithm that can estimate uncertainty of DNN predictions reliably and fast, and at the same time, is suited for integration into large range of robotic systems. Generie solutions to this problem are crucial for safe robot learning and introspection [1. 2]. giving the robots with an ability to assess their own failure probabilities and to alter their behaviors towards safety. While the state of the art is advancing [3, 4, 5), we still face practical difficulties in two main domains. First, the current methods methods require multiple predictions from several copies of a model [4] or samples from the model's distribution [3]. And second, in general these methods do not provide reliable uncertainty estimates when compared to GPs - often known as the golden standard of probabilistic machine learning [6. 7J.
often not efficient at test-time, e.g. for single input, the
Therefore, we propose to estimate the predictive uncertainty of DNNs using MoE-GPs [8, 9] sparse variant of GPs that divides the input space into smaller local regions using a gating function, where individual GPs called experts learn and make predictions (see figure 1). To do so, we provide both thcoretic foundations and a practical learning algorithm. First, we formally derive a connection between DNNs and MoE-GPs. As a result, we reveal how MoE-GPs with DNN-based kernel [10] can provably approximate uncertainty in DNNs. Moreover, we devise a learning algorithm that brings the derived theory into practice. Our solution involves gating function that strictly divides the input data into smaller subsets by performing clustering in kernel space, and we propose the concept of a patchwork prior, mitigating the problem of discontinuity between local GP experts at their boundaries. For efficiency, we further propose to exploit active learning and model compression techniques.
Our approach has several favorable features for many classification and regressions tasks in robotics. At run-time, it maintains the predictive power of a DNN, and its uncertainty estimates do not require combining multiple predictions of DNNs. Moreover, inherit the benefits of MoE-GPs for uncertainty estimates. These include an improved scalability when compared to a GP with the neural tangent kernel [10] (NTK), well natural support for distributed training. We contend that such probabilistic methods must run efficiently on a robot and as applicable as the sparse GPs. Therefore, not only provide ablation studies and evaluations against the oatte-of-the-are (SOTA), but also show that our method (i) scales to approximately 2 million data points for the task of learning inverse
5th Conference on Robot Learning (CoRI. 2021), London, UK.

---

Method
MAV
Lander 48 0%
Deterministic Probabilisti
Lander: 82.0%
Our method (left) computes uncertainty & of Neural Network predictions y using MoE- GPs with the Neural Tangent Kernels (NTK). With this, a MAV [11] (right) performs probabilistic object detection at an interactive frame rate, reducing overcontidence of an object detector. Only feedforward network is shown but our method also applies to convolution and recurrent DNNs.
Figure I:
dynamics, and (ii) Jetson TX2 of MAV, runs more than 12 times faster than the widely used MC-dropout [3], while performing object detection within a scenario for planetary exploration.
Contributions In summary, our main contribution is a novel method for estimating predictive uncertainty of DNNs with sparse GPs (section 3.1), backed up by (i) a theoretical connection between DNNs and MoE-GPs (section 3.2), (ii) a learning algorithm for its applicability in practice (section 3.3), and (iii) an exhaustive empirical evaluation that shows the benefits of our approach (section 4).
2 Related Work
Many researchers explored the idea of robotic introspection [2], i.e. robots that reason about "when they don't 't know" in order to avoid catastrophic failures. So far, many proposed methods used the robots' past experiences of failures [12, 13, 14], or detect the inconsistencies within the predictors [15, 16]. These works provide strong evidence that introspection improves the reliability of the robots. Other methods are based uncertainty estimation [1], where learning algorithms such as DNNs use probability theory to express their own reliability [17, 18, 19]. Commonly, these works technique called MC-dropout [3]. However, the intense computational burden in estimating uncertainty via sampling or ensembles [20] limits their applicability in real-time systems [11, 21, 22].
Several techniques without sampling or ensembles have been presented for run-time efficiency. These methods often use either Dirichlet distributions or functional Bayesian Neural Networks [23, 24]. Unfortunately, the applicability of Dirichlet distributions is limited to classification, while functional Bayesian Neural Networks face hurdles in scaling to large datasets without approximations [24]. We recommend the recent works of He et al. [25] and Adlam et al. [26], who use NTK-based GPs for uncertainty estimation of DNNs and provide useful insights. However, we focus on a technique that MAV. Hence, we take different path to ensembles of infinite width DNNs [25, 26]. We find uncertainty propagation techniques [27, 28, 29] to be practical these methods are training- free, i.c. assume existing and already well-trained DNNs.
can be readily deployed a real-time system such
In addition, our theoretic contribution (section 3.2) extends on how DNNs are related to GPs (pioneered by Neal [30] and others [31, 10, 32]), where we derive a relationship between DNNs and MoE-GPs as opposed to single GP. Theoretic insights that connect DNNs and sparse GPs may pave the way towards their application. as GPs alone do not scale to big data required by DNNs. Lastly, we extend previous works on MoE-GPs (section 3.3). MoE-GPs are efficient but suffer from non-smooth uncertainty estimates [33]. Park and Apley [34] show an elegant solution called patchwork kriging. By augmenting data at the boundaries between GP experts, the experts are forced to produce similar predictions at their boundaries. Yet, as patchwork kriging is limited to low-dimensional inputs [34), we propose several techniques that extend to higher dimensions, like images (assuming the NTK).
3 Materials and Methods
We describe method that addresses the problem of quantifying uncertainty estimates in DNN predictions, without sampling or ensembles. First, we introduce the main concept of estimating predictive uncertainty with sparse GPs, followed by theoretic results and our learning algorithm.

---

3.1 Main Idea: Fast Uncertainty Estimation with Sparse Gaussian Processes
Figure I visualizes our approach. The proposed predictive model uses an existing, and already well-trained DNN for accurate predictions. At the same time, the method offers the possibility to obtain the predictors' reliable uncertainty in a closed form solution, using DNN-based MoE-GPs. Intuitively, we can linearize and transform DNNs around a mode to obtain a linear model. As any linear models are GP regressors from a function space view, we can get GP-based representation of DNNs with a DNN-based kernel, the NTK. We describe below how the obtained GPs can estimate the uncertainty of DNNs, but cannot replace the DNN predictions. Moreover, as full GPs do not scale to big data, we choose to use MoE-GPs, which divides the data into smaller subsets and form many smaller GPs per these subsets. This results in the proposed combination of DNNs and sparse GPs.
Fundamentals First, we introduce our notation. Considering a supervised learning task on input- output pairs = [X.3] = ((0r).2)))
'N j=1' where 2; ER", y/ E ER we describe DNN as a parametrized function fo RD - RK where O E R... Here, learning typically seeks to obtain an empirical risk minimizer of the loss function, i.c. ming
[D] (Laea)cc ((f((x)))))))))))))))))))()))))) 5 0 where ō is an L2- (MAP) solution
regularizer, and mini-batches B c D are used to find a local maximum-a-posterinnn We assume a twice differentiable and strictly convex loss function L. e.g. mean squared error (MSE), and piece-wise linear activations in fe (e.g. RELU). For a clear exposition, we drop the indices i.
To set the scene for the paper, the Neural Linear Models (NLMs) [27, 35, 36, 32] are defined, which can estimate a DNNs' predictive uncertainty. To do so, consider a transformed dataset D = (x.J) with the pseudo-output ÿ := Jf(z)) ( g)) This transformation uses the model derivatives of the underlying DNN around 0 namely the Jacobian f := 0fo(z)/000 e RKXP, the Hessian H H((z,)) := :=
((fo@.) fe())' (f)()) E Rxx and the residuals Rg(w,
( E RK Assuming a white noise and an uninformative prior, the NLMs are:
== J with e~ C~N(O0,g(a.gg)
and p(0) ~ N(O,0-
(1)
The NLMs can be thought as a Bayesian linear model with learned features from a DNN, which is obtained via linearization around the DNNs' last layer [27]. Thus, the isotropic prior is governed by the parameter ö that corresponds to the L2 regularizer [37] while the white noise is governed by the term H R& in the described data transformation, given that the DNN fit the train data well, e.g. R = O. The term H is independent of the input if the 2nd derivative of a loss is a constant, e.g. MSE. Moreover, for commonly used loss functions such as cross entropy and MSE, the covariance Σ of DNN predictions fo is equal to that the NIMs' y [27, 32]. This means that for the same input æ, the NI.Ms can estimate the predictive uncertainty of DNNs. Yet, cannot be used to replace fo. due to their differences in fe and y. the Appendix, we provide concrete examples and their derivations.
Main idea Let's now apply a kernel trick [38] to the NLMs, so that we can obtain an effective combination of DNNs with GPs. To explain, we can further map the NLM function space, as opposed to working in the weight space. Doing so, a well known function space formulation of linear models [38] turns the NLM into a GP with the kernel K = J J which is also known as the Network Tangent Kernel (NTK). Thus, GPs with NTK can be used to estimate the predictive uncertainty of DNNs, as their equivalent NLMs can. Here, what motivates the formulation is the idea of deep kernel machines [39], i.c. we learn the kernel representations as oppose to hand designing the features and the kernels, and in our case, the kernel representations are the tangents of DNNs' Jacobians. Importantly, doing so, we get training-free free combination of DNNs with GPs, as we keep DNN predictions the same as the MAP, while estimating their uncertainty using GPs with NTK.
Unfortunately, such combinations of DNNs with GPs are restricted by a prominent weakness of standard GPs [40]: the cubic time complexity O(N2) that grows with the dataset size N. So, the computational costs are prohibitive for large scale problems that DNNs assume (typically referred to N > 10000 in [40, 38J). While we leave the theoretic motivation to section 3.2, we thus propose MoE-GPs with NTK, order advance the scalability of the proposed combination. A MoE-GP consists of M experts of GPs or learners F = =[GGP, /// f.....))) ). and a gating function g: RD - M-1 that maps any input xn to g(æ) = (a). (=)( [8, 9]. Each expert of the MoE learns and predicts within a subset of the input domain, and gating function generates these subsets.
3

---

1. Duality on MoE-DNNs and MoE-GPs
MoE-DNN Neural Networks
3. Approximation Step
NN
Local Noural Network Experta
AN M
MNM
CP
Approstmation
Midure of aoparts with NS
Mistures of oxperts Local GP
Michone of waperrs with neural notworks (NN)
Neuual Networka
Matures csperts with GP-NKK
Figure 2: We visualize theoretic results as a roadmap of the proof path. Detailed theorems and their proof's are in the Appendix. (1) Assuming a strict division of data, duality between mixtures of experts (MoE) with DNNs and GPs are established first. (2) To expand the applicability of the results beyond MoE-DNN,
show that the MoE-DNN are equivalent with an assumption on the identical local DNN experts. (3) This results in provable approximation error between the proposed MoE-GPs and the true GPs with the NTK.
relationships of single DNN and
Specifically, we choose a gating function (m) I in just one coordinate for each input [41]. Such strict partition enables the use of local model for each input [34], avoiding the combinations of multiple predictions of model for each input, e.g. model averaging as in Meier et al. [42]. The subscripts m 1,2.... denote the mh expert throughout the paper. Then, we write a MoE-GP as,
M Σ
_ (x) Jn
y =
Sm (m(x))FP. (x) Em with (x) ~ GP(O,
(2)
5m
As depicted in figure 1, the gating function (x) assigns the input data x to the mh subset, and individual GPs (x) learn and make predictions for the assigned data within the m subset. Consequently, the generative model and the predictions N(Y Ex,(x") ))
on the new test datum xx are:
ym
((0.
Km + Jo.m Ao
km. kmn,.
Tm k (Km m + 0. () ymm
and
SaP
Σn kmmos ki (Km + cr0 T)- kim +0.m.
where, K,, (n,,,X), =im, and kim,ss = (mm""" ,,"). This posterior predictive distribu- tion Σm is an indicator of total uncertainty, i.e. the kernel function captures the model uncertainty while the constant term T corresponds to the aleatoric uncertainty [43]. GPs, as we not have access to exact function values, the aleatoric uncertainty often relies on the assumption of additive i.id white noise, and in MoE-GPs, the terms d are inferred from the data per subset, leading to a non-stationary kernel. This is achieved by optimizing the marginal likelihood [38].
The benefits of the MoE- formulation are two-folds. First, as shown above, the generative model of a GP is defined per subset, and therefore, the GP experts are smaller than the one on the entire dataset. This improves the computational complexity of GPs [34]. Second, the covariance computations are in a closed form. Thus, we neither need sampling nor ensemble [3, 4J. Using the Lanczos approximation [44), which approximates the matrix inversion with the multiplications (suited for GPUs), we can compute the uncertainty estimates from GPs, with constant time complexity [44].
For classification, we leverage the framework of Lu et al. [45], which can perform classification with the regression models via confidence calibration [5]. While we refer to Lu et al. [45] for the details, this steps enables us obtain the classification uncertainty in closed form, i.e, for class c:
p(d softmax
Zm
with Tm vI Am. (x=")
T
where 2m is the activation, Tm is the temperature scaling factor, which is function of scaling constant Am,,- and GP regression uncertainty (x"x)), Intuitively, high prediction uncertainty will reduce the corresponding p(c)zm)) by increasing the Tm. thereby calibrating the confidences [5, 46, 45).
3.2 On Theory: Neural Networks as Sparse Gaussian Processes
So far, we have outlined our main idea using the function space view of NLMs and dividing the input space into smaller subsets, we can form smaller GPs per division. These GPs then estimate the

---

mi GP expert
Aparcc.
subsets
X61
Active leamning
N parch
Mpscc
Vk
centroid
Model pruning
Inputs
Division Step
Conquer Step
Memory reduction
Figure 3: The proposed algorithm involves (i) the division of input data using Jacobians of already trained DNNs. (ii) training of GP experts within their partitions and neighboring regimes, and (iii) combining active learning and model pruning techniques reduce space complexity.
predictive uncertainty of DNNs. Now, we discuss the theoretical foundations on how DNNs can be cast MoF-GPs with NTK. Due to space constraints, we refer the reader the Appendix for in depth treatment, where we provide various background materials, our theorems and their proofs. In section 3.3 we explain our learning algorithm while here, briefly summarize the main results.
Main result Figure summarizes our findings. Consider mixtures of experts model, where the experts are DNNs and gating function strictly divides the input space, i.e only one local DNN expert per division
Then, instead of the MAP parameter estimates Q we consider Bayesian DNNs with Gaussian distributions p() ie representing DNNs as probability distributions over their parameters (obtained using [47, 48, 49, 50]). As first step, specific instance of Khan et al. [32], we show that the DNN experts have mathematically equivalent Gaussian distributions with GPs using NTK. We further show that MoE-DNNs and MoE-GPs have equivalent Gaussian distributions, establishing the duality of the two models in a Bayesian sense. Main insight to the former is the probabilistic independence between the local experts due to strict division of the input space.
Next, we establish a connection between DNN, and a MoE-GP as shown figure in order to increase the applicability of our theoretic results. To do relationships a single DNN and a MoE-DNN are equivalent, if all the local DNN experts are the same the single DNN. Under the given conditions, cast as a MoE-GP, with an assumption that the data stationary. The resulting step yields the approximation error of J (X,X) (X, = Σij (x;; x x), >M Lijen K(x,, x), x)²²² where K
point out that the input- ppuut-predictinn
derive that single DNN can be
Ks for all M GP experts (while is still possible keep different 6m and am) and Kirue is the kernel matrix of the true DNu-equivalent GP with NTK. This means that less approximation error occurs when less correlated data points are assumed to be independent by MoE-GP, while strongly correlated points by NTK are within the same GP experts.
3.3 From Theory to Practice: A Practical Learning Algorithm
Now, we attempt to bring our theoretical results into practical tool for robotics. As shown in figure 3 our solution involves the division of data into several partitions. Then, conquer step, we train GP experts within the partitions. Moreover, we propose concepts to reduce the memory complexity.
Division step We design gating function that divides the input data into smaller regimes s.t. highly correlated data points are grouped together whilst the points with weak correlations are separated apart. To do so, perform NTK PCA, i.c. kernel PCA that uses NTK to reduce the dimensions of the data. Then, K-means is applied to form the clusters. Concretely, NTK PCA reduces the input data 2; for = I.....N into the low dimensional principal components U2 = ΣN ( Jy(xi), where is the eigenvector of K Kay Aaax with centered kernel matrix K K - 21 + 1rK for the matrix of ones 1y E NxV. Then, the centroid μi and division labels e are computed, iterating:
ΣM 1 = i] Σ 1(c, =
G = arg min ΣΣ Iu Hill
and
The proposed technique gives solution for Kernel K- K-means [51], which minimizes the derived error bound (X.X) X0II (section 3.2) with a balancing normalization [52]. As kernel PCA
5

---

Without Petch
Time (hours]
Division Step
Exccutionn Time (hours]
Step
10
5.
0.0
75
-25
10
Number of GPUs
Number of GPUs
Figure 4: (Left) We visualize predictive uncertainty of DNNs with and without the proposed patch- work prior. The black dots are train data points, and the red line shows the predictions of a determin- istic network. Blue shades show up to three standard deviations. Without the patch prior, sharp peaks are observed in uncertainty estimates. (Right) Training time for approximately million data points is shown for a varying the number of GPUs. We learned 512 GP experts per joint torques.
does not scale to large datasets, we use 2-step kernel K-means [53]. which uses randomly sampled and less data to compute the cluster centroids. We refer to Chitta al. [53] for more details.
Conquer step Next, we form the individual GP experts per divided local regime. A naive strategy is to form single GP per regime. Unfortunately, such model suffers from discontinuity in predictions at the boundaries [33]. For example, input points at the cluster boundaries may yield different predictions from the surrounding GPs, causing sharp peaks in predictions. Therefore, we propose:
K m..putch = =IJ (Jm)). (261). ... J..J (obL)) J (2T) (2b1), Jubl (IbL))
for the mh GP expert. We include the neighboring GP experts J Jmbi-.... J Juaz into the prior of the Th expert. This includes the information of neighboring GPs into the prior of the mh GP expert. Intuitively, the discontinuities occur at the shared boundary regimes of two GP experts. Our GP prior removes such boundary regimes locally by taking into account the neighboring regimes.
patch.m) where Npatehum = Nm ND1 + the ... Nox, resulting in added costs Nm < N patehk.- To mitigate, we perform active learning neighboring GP experts, and thus choose fewer, but the most informative points, e.g. IVMs [54, 55]. Intuitively, as the neighboring data are only included for reducing the discontinuity problem, we may select fewer data points. Concretely, we use the following steps: (i) for the neighboring GPs of the mh expert, we draw an initial subset and train the GPs. (ii) Using the trained GP models, rank the remaining data (stored as pool) by uncertainty (queries generated by uncertainty sampling). (iii) the GP expert is then updated with the most uncertain point. These steps repeat until Nox is reduced to desired, smaller value. In this way, while forming a patchwork prior, the neighboring GPs actively choose the neighboring data they want to learn from. As result, we obtain: Npatch,m < pattch.n-
Active uncertainty learning The complexity of each GP expert is bounded to O(N3
the number of data points pattck. includes the data points of neighboring GPs:
Gaussian process compression We further reduce the complexity of the algorithm by exploiting model compression techniques. Note that the Jacobians (a)) are paptch. by matrices where is the total number of parameters in DNNs. As (x) represents the sensitivity of each parameter to the output, the elements of (x) that belong to an unimportant DNN parameter can be removed. To do so, we propose a two-staged compression technique: (i) rank the DNN parameters by their importance using existing pruning methods and remove the corresponding elements of J ( for all m. (ii) Per GP expert, rank the elements of J (x) by metric IT, (x)), as smaller absolute values contribute less the kernel. The first step targeted at removing redundant parameters in DNNs (for the division step) while the second step is targeted at each individual expert, resulting in P < P.
4 Experiments and Evaluations
We provide sets of experiments to not only validate the method, but also to show the benefits of the proposed formulation namely performance, scalability and run-time. Implementation details can be found in the Appendix, and importantly, the accompanying video shows the experiments with a MAV.
Toy illustration With toy regression on the Snelson dataset, we show (i) on how MoE-GPs can estimate the predictive uncertainty of DNNs, and (ii) the ability of the patchwork prior to produce smoother uncertainty estimates. For this, we train single hidden layer Multi-layer perceptron

---

8
1634
2086
1000
2600
3600
4680
Variations
Variations
Variations (lng? scale)
Variatinss
Figure 5: The effects of each hyperparameter is shown with normalized test NLL, by varying them in different steps (Variations), and fixing the others to default settings. Lower the better.
Table 1: The results of inverse dynamic experiments are reported using NLL. Lower the better.
Train
Test
Ensemble
MC-dropout
rMC-dropout
Laplace
rLLaplace
Ours
sarcos
sarcos
1.267=0.108
1.398:0.012
1.398=0.012
1.392=0.014
1.392=0.001
1.035:0.0.39
sarcos
kukal
17.86+2.995
16.21=0.866
20.58:1.144
25.88=1.354
24.55=2.746
21.53+2.268
sarcos
kuka2
17.50+2.895
15.63+0.858
20. 30.17+0.988
25.59+1.079
24.42+2.257
20.92+2.236
sarcos
kukasim
23.06:2.649
51.09=11.14
61.40:12.922
77.13=15.40
74.03=13.23
68.9544.003
kukal
kukal
2.013+0.020
+0.07
+,620=0.006
1.676=0.013
1.719=0.071
+..17
kukal
kuka2
2.085=0.005
1.349=0.019
1.330=0.006
1.310=0.005
1.310=0.008
1.315+0.00 003
kukal
kukasim
60.44=11108
22.
48. 7640.566
60.37=0.799
59.74=1.571
1.348=00012
kukal
sarcos
8.128=0.169
22.24
42.36:22.398
122.68+13.32
1837=2431.3
0,05
kuka2
kuka2
2,042=0.009
1.443+0.005
1.43=0.000 006
1.429=0.006
1.423=0.006
1.354=0.013
kuka2
kukasim
63.07=0.741
38. 04-0.
48. 0379777
60.36=1.005
19,05=1.158
1.333=+0012
kuka2
sarcos
8.509=0.461
18. 18.24x1.5999
33.24:6.287
141.7=17.14
211.3=180.2
0.015
kuka2
kukal
2.106=0.0100
1.461=0.004
1.395=0.004
1.373=0.005
1.374=0.004
1.331+0.011
(MLP) with 200 units and a tanh activation. Evaluating for out-of-distributio (OOD) data (points far from train data), and domain-shiff (DS) data (removed in- betweeewn points), as shown in figure the produced uncertainty estimates are high test points move away from the train data. Moreover, when DNN predictions match the train data, the produced uncertainty estimates are close to the data noise. We also observe that the patchwork prior mitigates the discontinuity problem of local GPs.
On hyperaarameters Next, we examine (i) the influences of hyperparameter choices, and (ii) provide a simple tuning recipe. For this, we examine regression task using 5-layered, 200 units MLP on SARCOS robot arm data-set [38]. Here, we vary each hyprrparreeter while fixing the remaining ones, and we compute the Negative Log Likelihood (NLL). The results are shown in figure 5, with 4 hyperparmeters namely, (i) the number of GP experts, (ii) the subset sizes for clustering, (iii) the pruning level, and (v) the size of the informative data points. We observe the following: (i) the NLL is proportional to the number of GP experts, (ii) the subset size do not significantly influence the NLL, (iii) pruning steps have a tipping point where the NLL increases, and (iv) the active learning parameter has also a tipping point, where the NL. decrease is marginal. In contrary, by the design, the computational complexity increases with (i) lower number of experts and (ii) the size of actively selected points, and decreases with the pruning steps. We find these results confirm our intuition. For example, keeping 2048 GP experts for SARCOS results in 21 data-point per experts average. This setting is inferior to keeping 32 GP experts, which has 1390 data points on average. Reflecting this, our strategy is to assign more data points for each GP expert, while selecting only the needed active learning points. The pruning levels can then be varied within a range that does not deteriorate the performance in order to reduce the computational complexity of the method.
Comparison study Now, we evaluate the performance of our method using the datasets namely SARCOS, KUKAI and KUKA2 [42]. The baselines are MC-dropout [3], Laplace Approximation [47, 49], and their fast variants: rMC-dropout [28] and rLaplace [27, 29, 49]. The fast variants main competitors: principled Bayesian approaches that are training-freet i.c. works with pre-trained DNNs without modifications to the training procedures. We find decoupling DNN training to uncertainty estimates crucial, it ensures comparisons of uncertainty estimates only by keeping the accuracy of the predictors constant amongst the baselines (in this case, the same pretrained 5 5-layered MLP across the baselines). We also include the ensembles [4]. Lastly, we adopt zero-shot transfer to systematically evaluate in-domain, OOD and DS scenarios, following the recent insights, that is, we train dataset, and evaluate uncertainty estimates for all other datasets.
The results are in table 1, where we averaged over random initializations and report NLL as a metric (a standard for regression tasks). We find that our method often outperforms other baselines
7

---

GTX1080
JelsonTXX
McE-GP
MC-dropout
10
000 IN-DIST
25-
000
MaF-GP Mc-deopout
Counts
Counts
1b
12
167x327
336:254 6634500
1674127
3365254 FAAL50R Image Size
0.5 1.0
0.00
0.25
0.50 0.75
Image Size
Normalized Entropy
Normalized Entropy
Figure 6: The run- time analysis (left) shows efficiency of our approach when compared to MC- dropout [3] with 20 samples. Any difference to a deterministic DNN can also be seen. The entropy histograms (right) show novelty detection performance. The memory requirement is 8GB.
significantly. MC-dropout dominates in 3 experiments. Yet, MC-dropout requires sampling for uncertainty estimates one of the core problems addressed in our work. We note again, that all the compared methods are training-free making the comparisons meaningful. Importantly, rLaplace uses the NLM from section 3 to infer Gaussian uncertainty in weight space, in contrast to function space inference with GPs. Thus, the comparison of rLaplace and ours instantiates the comparison of inference between weight space and function space views. We believe that our results show the relevance of GP based uncertainty estimates for DNNs over uncertainty propagation methods.
On scalability Here, we show that (i) our approach scales to KUKA SIM dataset (contains 1984950 data points), and (ii) supports distributed training. Scalability is relevant, as DNNs operate in the regime of big data, and an exact GP does not scale to such settings. Moreover, a key benefit of MoE-GP over other sparse GPs lies in its distributed, local nature, and we can exploit it in practice. The results are depicted in figure 4, where we plot the training time in hours against the number of used GPUs. Using 512 GP experts and 100 iterations for MLL optimization, we find that our method scales to approximately 2 million data points within a day, and less than hours when using 6 GPUs. This experiment shows empirical evidence that our method scales to 2 million data points for the task of inverse dynamic learning, and validates the claim that MoE-GPss support distributed training.
On real robot To validate the applicability of our method to robot and evaluate the run-time on the hardware, we perform experiments on MAV within the context of an on-going space demo mission for future planetary exploration [56]. For this, we train an object detector, DETR [57] with EfficientNet backbone. For testing, we create two scenarios: (i) carry test, used for training and testing with 1008 manual labels, and (ii) flight tests with different configurations of the objects (e.g. space rover and lander). The later creates OOD samples, with a slight domain shift in data distribution. In field robotics, the assumption that the test set comes from the same training data distribution is routinely violated due to the changes in the environments, and we attempt to create such effects. Moreover, we also evaluate the complexity of our method on Jetson-TX2, which is used MAV. The quantitative results are shown in figure 6, which shows that (i) our method can be fast on Jetson- Jetson-TX,, and (ii) can clearly separate OOD samples within this scenario. The accompanying video explains this setup and the qualitative results. More details can be found in the Appendix.
5 Discussion
If the computational complexity of GPs can be tamed in practice, our work shows that the predictive uncertainty of DNNs can be obtained from the GP formulation. By advancing the scalability of full NTK, the advantages of our approach are demonstrated. Yet, the applicability to larger data regimes, e.g. imagenet, is confined to that of the sparse GPs, e.g. the non-parametric models require an access to the training data at run-time, which can require more memory than the parametric models such as DNNs. Sparse GPs may also face struggles when the output dimension is larger. The alternatives can be the weight space-based methods [58] such as Sharma et al. [59]. Here, various approximations have been devised so far, in order to cope with the high dimensional weight space. For robotics though, when the availability of data is limited, the dimensions of data space can be smaller than the weight space, and thus, our work can provide a reference that the Bayesian nonpaaraettric can be relevant tool in addressing the current challenges of the uncertainty estimation for neural networks.
In future works, we plan to apply the proposed methodology in field robotics setting, and study how the uncertainty estimates can be used to improve the robustness of the robotie systems.
8

---

Acknowledgments
We thank the anonymous reviewers and area chairs for their thoughtful feedback. Special thanks to Klaus Strobl, Maximilian Denninger and Antonin Raffin for proof reading the paper, and also Scok Joon Kim for the support during his internship at DLR. We also would like to acknowledge the support of Helmholtz Association, the project ARCHES (contract number ZTT-033), the Initiative and Networking Fund (INF) under the Helmholtz Al platform grant agreement (ID Z1-1-PF-5-1) and lastly, the EU Horizon 2020 project RIMA under the grant agreement number 824990. Jianxiang Feng is supported by the Munich School for Data Science (MUDS) and Rudolph Triebel is a member of MUDS.
References
[1] S. Thrun. Probabilistic robotics. Communications of the ACM, 45(3):52-57, 2002.
[2] H. Grimmett, R. Triebel, R. Paul, and I. Posner. Introspective Classification for Robot Perception. International Journal of Robotics Research (IJRR). 2015.
[3] Y. Gal and Z. Ghahramani. Dropout bayesian approximation: Representing model uncer- tainty in deep learning. In international conference on machine learning, pages 1050-1059, 2016.
[4] B. Lakshminarayanan, A. Pritzel, and C. Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in neural information processing systems, pages 6402-6413, 2017.
[5] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger. On calibration of modern neural networks. In International Conference on Machine Learning, pages 1321-1330, 2017.
[6] S. Sun, G. Zhang, J. Shi, and R. Grosse. Functional varitional bayesian neural networks. In International Conference Learning Representations, 2019.
[7] R. Paul, R. Triebel, D. Rus, and P. Newman. Semantic categorization of outdoor scenes with uncertainty estimates using multi-class gaussian process classification. In 2012 JEEE/RSJ International Conference on Intelligent Robots and Systems, pages 2404-2410. IEEE, 2012.
[8] R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton. Adaptive mixtures of local experts. Neural computation, 3(1):79-87, 1991.
[9] V. Tresp. Mixtures of gaussian processes. Advances in neural information processing systems, pages 664-660,22 2001.
[10] A. Jacot, F. Gabriel, and C. Hongler. Neural tangent kernel: Convergence and generalization in neural networks. In Advances neural information processing systems, pages 8571-8580, 2018.
[11] P. Lutz, M. G. Müller, M. Maier, S. Stoneman, T. Tomić, von Bargen, M. J. Schuster, F. Steidle, A. Wedler, W. Stürzl, and R. Triebel. ARDEA An MAV with skills for future planetary missions. Journal of Field Robotics (JFR), 2019.
[12] S. Daftry, S. Zeng. J. A. Bagnell, and M. Hebert. Introspective perception: Learning to predict failures in vision systems. In 2016 JEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pages 1743-1750. IEFF, 2016.
[13] M. S. Ramanagopal, C. Anderson, R. Vasudevan, and M. learn: autonomously identifying perception failures for self-driving cars. IEEE Robotics and Automation Letters, 3(4):3860-38)) 000000387777777 2018.
Failing to
[14] C. Gurău, D. Rao, C. H. Tong, and I. Posner. Learn from experience: probabilistic prediction of perception performance to avoid failure. The International Journal of Robotics Research, 37(9): 981-995, 2018
[15] C. Richter and Roy. Safe visual navigation via deep learning and novelty detection. In Robotics, Science and Systems (RSS), 2017.

---

[16] K. Wong, S. Wang, M. Ren, M. Liang, and R. Urtasun. Identifying unknown instances for autonomous driving. In L. P. Kaelbling, D. Kragic, and K. Sugiura, editors, Proceedings of the Conference on Robot Learning, volume 100 of Proceedings of Machine Learning Research, pages 383-393. PMIR, 30 Oct-01 Nov 2020.
[17] J. Feng, M. Durner, Z.-C. Marton, F. Balint-Bencee perception using smoothed predictions from bayesian neural networks. In 2019 International Symposium on Robotics Research (ISRR), 2019.
and R. Triebel. Introspective robot
[18] A. Harakeh, M. Smart, and S. L. Waslander. Bayesod: A bayesian approach for uncertainty estimation in deep object detectors. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 87-93. IEEE, 2020.
[19] N. Sünderhauf, F. Dayoub, D. Hall, J. Skinner, H. Zhang. G. Cameeio,, and Corke. A probabilistic challenge for object detection. Nature Machine Intelligence, 1(9):433 443, 2019.
[20] J. Gawlikowski, C. R. N. Tassi, M. Ali, J. Lee, M. Humt, J. Feng, A. Kruspe. R. Triebel, P. Jung. R. Roscher, et al. A survey of uncertainty in deep neural networks. arXiv preprint arXiv:2107.03342, 2021.
[21] Lee, R. Balachandran, Y. S. Sarkisov, M. De Stefano, A. Coelho, K. Shinde, M. J. Kim, R. Triebel, and K. Kondak. Visual-inertial telepresence for aerial manipulation. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 1222-1229. IEEE, 2020.
[22] J. Lee, T. Muskardin, C. R. Pacz, P. Oettershagen, T. Stastny, I. Sa, R. Siegwart, and K. Kondak. Towards autonomous stratospherie flight: A generic global system identification framework for fixed-wing platforms. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 0233-6240.. IEEE, 2018
[23] M. Sensoy, L. Kaplan, and M. Kandemir. Evidential deep learning to quantify classification uncertainty. In Advances Neural Information Processing Systems, pages 3179-3189, 2018.
[24] E. D. Carvalho, R. Clark, A. Nicastro, and P. H. Kelly, Scalable uncertainty for computer vision with functional variational inference. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pages 12003-12013 2020.
[25] B. He, B. Lakshminarayanan, and Y. W. Teh. Bayesian deep ensembles via the neural tangent kernel. arXiv preprint wwXiv:2007005860 2020.
[26] B. Adlam, Lee, L. Xiao, J. Pennington, and J. Snoek. Exploring the uncertainty properties of neural networks' implicit priors in the infinite-widih limit. In International Conference Learning Representationss 2021.
[27] D. J. MacKay. Information- based objective functions for active data selection. Neural computa- tion, 4(4):590 604, 1992.
[28] J. Postels, F. Ferroni, H. Coskun, N. Navab, and F. Tombari. Sampling-free epistemic uncertainty estimation using approximated variance propagation. In Proceedings of the IEEE International Conference on Computer Vision, pages 2931-2940, 2019.
[29] A. Y. Foong, Y. Li, J. M. Hernández-Lobato, and R. E. Turner. 'in-between' bebettwenn'unertainnn uncertainty in bayesian neural networks. arXiv preprint arXiv:1906.1 1537,, 2019.
[30] R. M. Neal. Priors for infinite networks. In Bayesian Learning for Neural Networks, pages 29-53. Springer, 1996.
[31] J. Lee. Y. Bahri, R. Novak, S. Schoenholz, J. Pennington, and Sohl-dickstein. Deep neural networks gaussian processes. In International Conference Learning Representations, 2018.
[32] M. E. E. Khan, A. Immer, E. Abedi, and M. Korzepa. Approximate inference turns deep networks into gaussian processes. Advances in Neural Information Processing Systems, pages 3088-3098. 2019.
10

---

[33] C. Park and J. Z. Huang. Efficient Computation of Gaussian Process Regression for Large Spatial Data Sets by Patching Local Gaussian Processes. Journal of Machine Learning Research, 17(174):1-29, 2016.
[34] C. Park and D. Apley. Patchwork kriging for large-scale gaussian process regression. The Journal of Machine Learning Research, (9(1)26693111 2018.
[35] S. W. Ober and C. E. Rasmussen. Benchmarking the neural linear model for regression. Third Symposium Advances in Approximate Bayesian Inference, 2020.
[36] J. Watson, J. A. Lin, P. Klink, and J. Peters. Neural linear models with functional gaussian process priors. In Third Symposium Advances in Approximate Bayesian Inference, 2020.
[37] C. M. Bishop. Pattern recognition and machine learning. springer, 2006.
[38] C. Rasmussen and C. Williams. Gaussian Processes for Machine Learning. Adaptive Computa- tion and Machine Learning. MIT Press, Cambridge, MA, USA, Jan. 2006.
[39] A. G. Wilson, Z. Hu, R. Salakhutdinov, and E. Xing. Deep kernel learning. In Artificial intelligence and statistics, pages 370-378. PMLR, 2016.
[40] H. Liu, Y.-.. Ong, X. Shen, and Cai. When gaussian process meets big data: A review of scalable gps. IEEE Transactions on Neural Networks and Learning Systems, 2020.
[41] S. Gross, M. Ranzato, and A. Szlam. Hard mixtures of experts for large scale weakly supervised vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6865-6873, 2017.
[42] F. Meier, P. Hennig, and S. Schaal. Incremental local gaussian regression. In Advances in Neural Information Processing Systems, pages 972-980, 2014.
[43] E. Hüllermeier and W. Waegeman. Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine Learning, 110(3):477-06, 2021.
[44] G. Pleiss, Gardner, K. Weinberger, and G. Wilson. Constant-time predictive distributions for gaussian processes. In International Conference on Machine Learning, pages 4114-4123 PMLR, 2018
[45] Z. Lu, E. Ie, and F. Sha. Uncertainty estimation with infinitesimal jackknife, its distribution and mean-field approximation. CoRR, abs/2006.07584 2020.
[46] J. Wenger, H. Kjellström, and R. Triebel. Non-parametric calibration for classification. In International Conference on Artificial Intelligence and Statistics, pages 178-190. PMLR, 2020.
[47] H. Ritter, A. Botev, and D. Barber. A scalable laplace approximation for neural networks. In 6th International Conference on Learning Representations, ICLR 2018- 2018-Conference Track Proceedings, volume 6. International Conference on Representation Learning, 2018.
[48] M. Humt, J. Lee, and R. Triebel. Bayesian optimization meets laplace approximation for robotic introspection. arXiv preprint (0ti:2000016144: 2020.
[49] J. Lee, M. Humt, J. Feng, and R. Triebel. Estimating model uncertainty of neural networks in sparse information form. In H. D. III and A. Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 5702-5713. PMLR, 13-18 Jul 2020.
[50] K. Shinde, J. Lee, M. Humt, A. Sezgin, and R. Triebel. Learning multiplicative interactions with bayesian neural networks for visual-inertial odometry. arXiv preprint arXiv:2007.00 07630, 2020.
[51] C. Ding and He. K-means clustering via principal component analysis. Proceedings of the twenty-first international conference Machine learning, page 29, 2004.
11

---

[52] I. S. Dhillon, Y. Guan, and B. Kulis. Kernel k-means: spectral clustering and normalized cuts. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 551-556 2004.
[53] R. Chitta, R. Jin, T. C. Havens, and A. K. Jain. Approximate kernel k-means: Solution to large scale kernel clustering. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 895-903, 2011.
[54] N. Lawrence, M. Seeger, and R. Herbrich. Fast sparse gaussian process methods: The infor- mative vector machine. In Proceedings of the 16th annual conference on neural information processing systems, number CONF, pages 609-616, 2003.
[55] R. Triebel, H. Grimmett, R. Paul, and Posner. Driven learning for driving: How introspection improves semantic mapping. In Robotics Research, pages 449-465. Springer, 2016.
[56] M. J. Schuster, M. G. Müller, S. G. Brunner, H. Lehner, P. Lehner, R. Sakagami, A. Dömel, L.. Meyer, B. Vodermayer, R. Giubilato, et al. The arches space-analogee demonstration mission: towards heterogeneous teams of autonomous robots for collaborative scientific sampling in planetary exploration. IEEE Robotics and Automation Letters, 5(4):5315-5322, 2020.
[57] N. Carion, Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko. End-to-end object detection with transformers. In European Conference Computer Vision, pages 213-229. Springer, 2020.
[58] D. Madras, J. Atwood, and A. D' Amour. Detecting extrapolation with local ensembles. arXiv preprint arXiv:I910.0940 2019.
[59] A. Sharma, N. Azizan, and M. Pavone. Sketching curvature for efficient out-of-diitribution detection for deep neural networks. The Conference on Uncertainty in Artificial Intelligence (UAI), 2021.
12


Reviews:
[{"review_id": "NT_wJBPJMwp", "reviewer": "Reviewer_mLeS", "summary": "This paper addresses the problem of introspection, an intrinsic understanding of how confident the model is in its own predictions. The paper provides a theoretical connection between DNNs and MoE-GPs and finally provides a practical learning application. Experiments are carried out on data sets derived from robotic sampling and inverse dynamics problems. \n", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- The paper addresses an important problem of uncertainty estimation in prediction. The work provides a theoretical foundation from a Bayesian DNN and GPs viewpoint which culminates in a practical learning algorithm. \n- The paper provides a scalable approach using a mixture of GPs approach and other techniques to reduce the online inference time making them suitable for robotics application. \nResults on real robotics data sets. \n\nWeaknesses\n- The approach in the current form does not distinguish aleatoric and epistemic uncertainty in predictions and hence is likely to be less resilient to model uncertainty. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "This paper contributes a theoretical framework for uncertainty estimation as well as a practical learning algorithm. The result is likely to have impact on real robotics applications. \n", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "reviewer_expertise": "Good: General knowledge of the area"}, "extra_text": {"issues": "Issues\n- I would recommend improvements in the technical exposition of the material. For example, Section 3 can provide a roadmap for how the proof will develop. \n- Does the current system has a way of forgetting points. In an online application, the incoming data may exceed available memory. Hence, is there a possibility for a principled approach for removing points as is done in Informative Vector Machines (IVMs)"}}, {"review_id": "FzxNX79sv_I", "reviewer": "Reviewer_hvPu", "summary": "This paper presents an approach to equipping trained DNNs with a measure of epistemic uncertainty. Leveraging a connection between the local liberalization of a DNN and the resulting equivalent GP (using the neural tangent kernel), the authors propose to use this GP to characterize the uncertainty on test points. The approach approximates this equivalent GP using a mixture-of-experts GP model, wherein a hard gating function divides the input into disjoint sets, each of which correspond to a separate GP. These GPs have a kernel with the same features equal to the Jacobian of the DNN. The approach requires (1) performing approximate kernel k-means to design the gating function and split the training data into subsets; (2) fitting a NTK GP to each of the subsets, by (2a) taking an active learning approach to include only a limited number of informative datapoints in the gram matrix for each GP; and (2b) pruning features by removing elements of the Jacobian which have small values and thus are less relevant to the kernel. To ensure smoothness across the separate GPs, each GP also includes the data from some of its neighbors. At test time, one can compute an uncertainty estimate by constructing evaluating the gating function to determine the expert GP to to evaluate, and subsequently evaluating the kernel for that GP on the test input and the selected training data for that expert, and finally, using this information to compute the posterior covariance.\n\nThe authors provide theoretical motivation for their approach justifying why the MoE GP is a good approximation of the DNN uncertainty.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper presents an interesting approach exploiting a connection between local linearization of a DNN and GPs. The presented approach takes this theoretical idea, and presents an algorithm that yields tractable inference times at test time. While the method is complex, the paper included helpful figures which communicated the ideas well. I especially liked Figure 3 in this regard. I also appreciated the practical results, demonstrating that the approach can be deployed in hardware, and offers good OOD detection performance over using mc-dropout. \n\nHowever, there are several areas for improvement.\n\nFirst, I found the paper to be occasionally hard to follow, especially in the theoretical derivations. The jump from DNN setup to the NLM was abrupt, without explanation providing intuition for the particular choice of covariances. The construction of the practical model also felt disconnected from the theoretical derivation, specifically, while the NLM treats the output as having additive gaussian noise with covariance equal to the inverse Hessian of the loss (eq. 1), a quantity that is input dependent, the predictions are made assuming equal noise for each sample of \\sigma_0. Furthermore, eq. 3, connecting the GP regression uncertainty to a classification task, lacked theoretical justification beyond an intuitive notion that it would have the right effect on the predictive uncertainty. This felt at odds with the fully general derivation of the NLM. \n\nThe theoretical results could be presented more clearly. While I appreciated the visual description of the theoretical results in Figure 2, I felt that the choice to start by viewing a single DNN as a mixture of expert DNNs with identical networks for each expert, and subsequently showing that each expert can be cast as a GP. In contrast the premise of the approach was to approximate the posterior predictive uncertainty of the full DNN, in which case it would have been clearer to consider the GP corresponding to the full DNN, and subsequently show how the MoE-GP approximates the full GP. Furthermore, the paper could be strengthened by including theoretical results bounding the errors made by using the various approximation techniques applied to make the approach practical: the active learning for data subsampling, and the feature pruning.\n\nSecond, the scalability of this approach is unclear. While the authors take several steps to make the approach more practical, including subsampling data and breaking the data into subsets, the kernel-GP approach taken requires storing significant amounts of data at test time (even if all of the data isn’t used in evaluation). In addition, the authors use separate GPs per output dimension, meaning the approach cannot capture correlations across output dimensions, or scale easily to higher-dimensional outputs (e.g. semantic segmentation tasks). Also, while the authors demonstrate the approach on a large dataset in simulation, the on-robot results use a much smaller dataset. Furthermore, the appendix suggests that the results in the paper also only consider the Jacobian of the last few layers of the network, rather than the whole DNN. This hinders the authors’ argument for the scalability and efficiency of the approach.\n\nFinally, the authors should compare to more standard benchmarks, in addition to MC-dropout. Experiments comparing to Deep Ensembles are crucial, as they are a commonly used tool for DNN uncertainty, and can give good results even with a low number of ensemble members, allowing for efficient real-time application. Furthermore, there are recent works considering different Laplace approximations that are closely related to a weight-space view of the DNN GP, that are worth discussing if not comparing against [1,2].\n\n\n[1] Madras et al., “Detecting Extrapolation with Local Ensembles,” ICLR 2019\n[2] Sharma et al., “Sketching Curvature for Efficient Out-of-Distribution Detection for Deep Neural Networks,” UAI 2021\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Overall, I think this paper presents some interesting, novel ideas, but the message is unclear, and the practical implementation of the idea has some shortcomings in its ability to scale. The paper would be improved with improved clarity in the presentation of the ideas and results, and a more thorough experimental comparisons highlighting the benefit of this approach over existing baselines for predictive uncertainty estimation. \n\n---\nHaving read the revisions, I believe the authors have addressed my issues and I have increased my score.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "2: Unsure, difficult to assess.", "reviewer_expertise": "Very good: Comprehensive knowledge of the area"}, "extra_text": {"issues": "- The authors should, if possible, refactor the theoretical results as suggested in the main review section, or provide more justification why the posterior predictive uncertainty of the MoE-DNN model is a useful analog to the posterior predictive uncertainty of a single GP.\n- Address the mismatch between the NLM derivation and the GP expressions used in practice (specifically, the choice of observation noise)\n- The main missing experiments are comparisons to Deep Ensembles.\n- More details on the stopping criterion for the active learning and Jacobian pruning strategies should be given.\n- The discussion should highlight the fact that this method requires access to training data during the inference procedure."}}, {"review_id": "9DbSvsOm0uN", "reviewer": "Reviewer_Q7n6", "summary": "The paper is motivated by predictive uncertainty that can be used on mobile robotic platforms (in this case MAVs). \nThe proposed model in a mixture-of-experts of BNNs, specfically the neural tangent kernel. \nThe model is evaluted on inverse dynamics learning and object detection. ", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Fast predictive models with useful uncertainty quantification would be a useful tool in robotics.\nImproving the uncertainty quantification of BNNs to be as good as GPs is an active topic of research.  \n\nMy central issue with this paper is that I couldn't tell if it was a BNN paper or robotics paper. \nThe real robot experiment and motivation is cool and relevant to CoRL, while the actual contribution of the paper is as a BNN.\nMoreover, w.r.t. to the robotic applications, it is not clear what uncertainty is needed for, i.e. what is the downstream task (active learning, Bayesian optimization, etc)?\nAs a BNN paper, the baselines and experiments were somewhat limited compared to standard studies.  \n\n1) The neural linear model is actually an established class of Bayesian neural network (i.e. [1,2]), so you need a different term. This family should be included in the related work.\n\n2) As I understand it, the NTK is more a tool for understanding the NN/GP connection rather than a practical model, since it comes with the kernel scaling issues and the need to compute Jacobians.\n\n3) The core issue with this paper is that key design decsisions are centered on making BNNs like GPs, but GPs are never evaluated as a baseline. Since so much work is done to make the NTK scale to the small partioned small data, why aren't GP's evaluated for regression? The paper does not cite [3], which is a GP version of the idea presented here.\n\n4) I was frequently confused by the memory requirements of your problem setting. Ensembles and GPs are described as being too memory intensive, and yet the MoE-GP model seems to cache an awful lot of memory (pre-computing various quanities and lots of models). I would like to see a comparison between the memory footprint of a MoE-GP vs ensembles and GPs to validate these claims. \n \n5). 'To our knowledge, we are the first to show that you can use GPs to estimate the uncertainty of DNNs on a Jetson TX2 of a MAV.' While this is probably true, this is not a contribution worth stating, as it is too specific to be noteworthy.   \n\n\n\n[1] Benchmarking the Neural Linear Model, Ober et al AABI\n\n[2] Neural Linear Models with Gaussian Process Priors, Watson et al, AABI\n\n[3] Incremental Local Gaussian Regression, Meier et. al NeurIPS 2014\n\n\nMinor\n\n'eigen-vector' should just be eigenvector \n\ntypo on theory in appedix line 194", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "This paper has lots of interesting ideas, but perhaps too many as the current draft does not carefully motivate and validate the many design choices that go into the model. \n\nI would be happy to increase the score if these design choices become more coherent. ", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Poor: Difficult to follow and understand.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "issues": "See above", "reviewer_expertise": "Very good: Comprehensive knowledge of the area"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes an approach to novelty detection for deep neural networks using Gaussian Processes (GPs). More specifically, the paper uses mixture of experts (MoE)-GPs with a neural network-based kernel to estimate predictive uncertainty associated with a neural network’s output. The resulting approach is demonstrated on two robotic tasks corresponding to learning the inverse dynamics of a manipulator and object detection on a micro-aerial vehicle. 

Strengths:
+ The reviewers are generally in agreement that the paper addresses an important problem (uncertainty estimation for DNNs) that is relevant to robotics.
+ The reviewers generally agree that there is novelty in the technical approach (using MoE-GPs to perform uncertainty estimation).
+ The reviewers appreciated the experimental results on real-world robotics datasets.

Weaknesses:
- The primary concern raised by the reviewers related to the technical presentation of the approach. In particular, the theoretical derivations in Section 3 are not clearly described; certain technical choices are not clearly motivated or explained (see the comments from Reviewers mLeS and hvPu for detailed comments and suggestions).
- There are also concerns regarding the scalability of the approach. The onboard robotic experiments use relatively small datasets and the Jacobian used for uncertainty estimation is only associated with the last few layers of the DNN. Reviewers also raised concerns regarding the memory footprint of the approach.
- Reviewers suggested additional comparisons with baselines, including Deep Ensembles, and GPs. 

Suggestions:
I urge the authors to consider the reviewers’ detailed feedback in order to improve the paper. The primary suggestions for improvement include:
- Improving the technical exposition and justifying theoretical derivations and choices in Section 3.
- Addressing the concerns regarding scalability of the approach.
- Providing additional comparisons with baselines (and including discussions of some of the papers that the reviewers mentioned; particularly the ones mentioned by Reviewer Q7n6).

----- Post rebuttal -----

The reviewers have made significant improvements to the paper in the rebuttal and revision period. In particular, they have addressed the primary concerns raised by reviewers by (i) adding an additional strong baseline (Deep Ensembles), (ii) addressing questions regarding the memory footprint, and (iii) improving the presentation of the work. Given the strengths of the work in terms of importance of the problem, novelty, and real-world robotics evaluations (and also taking into account the improvements made to the paper), I believe this work represents a strong contribution. 
Decision: Accept (Poster)