Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: This is an interesting paper which evaluates the calibration of NNGPs, including around OOD detection.

Major points:
1. It is worth noting that in the appendix for arXiv:1808.05587, they explicitly use the full categorical log-likelihood with softmax probabilities (albeit with inducing points), and they find that the resulting model is well-calibrated.  I would be surprised if the use of the Categorical-softmax did not appear in other papers.

2. I am a bit confused about the notion of an "ensemble" of NNGPs.  To my mind, that would imply a number of different NNGPs with different hyperparameters (e.g. depth).  I'm not sure what is being done here, but it appears to be equivalent to a single NNGP?

3. However, my primary concern is around the strength of the NN baselines. They find that the performance of their CNNs and C-NNGPs with CIFAR-10 is very similar, with around 60% accuracy.  For C-NNGPs,  60% accuracy is pretty standard, but for NNGPs, it is really, really bad: with very little effort it is possible to get 90% accuracy, and with a little further tuning, you can get to 95%.  Even when trying to do a fair comparison to infinite networks, other researchers have found 80% accuracy in finite networks (arXiv:1810.05148).  This poor performance brings into question the relevance of their results.  At the very least, the paper needs:
    * A detailed description of why the NN's performance is so poor.
    *Additional comparisons against stronger NN baselines.

4. It isn't clear what the infinite width layers give you that's better than either using Bayesian linear regression over the output weights, or deep kernel learning.

Minor points:
1. Figure 1 (left) is completely unreadable.  Either use separate plots, or use a line (rather than an area) to denote the density.
2. Quote NLL per datapoint, not for full dataset.
3. Exactly what is in table 3 is very unclear: does it aggregate over corruptions?  What are 1k etc. (all of this should be in the caption).
4. Table S1. "confidence on a specific test point being 1.0 to machine precision".  The entropy in that case is zero, and any standard, numerically stable implementation should give that.
review_title: Review

final_score: rating: 6: Marginally above acceptance threshold
review_content: Update: the authors have greatly improved the figures and tables, and expanded the captions, removing my major concern about clarity. I have improved my rating. I not not share the objection of reviewers 5 and 4 about the small size of the CNN, resulting in an inferior baseline in section 3, as the approach is still impractical and mostly a proof of concept that should encourage further research. What matters is that improvements in section 5 are built upon a SOTA baseline, inspired by section 3.

The stated goal of this paper is very ambitious: how NNGPs provide better confidence prediction, in terms of calibration, OOD data  and distributional shift.

After an abstract that is somehow confusing, Introduction and Background  are clear and comprehensive.
Next the authors “describe” their NNGP and try an impressive number of architectures over several datasets, however, assembling the paper seems to have been an hasty business where many clarity issues have been left unresolved, and these are severe, especially for reader such as myself who are not an expert in GP.
The key innovation presented in section 5: the NNGP is just added as a calibration layer on top of a pretrained NN.  It could be of high significance for a practitioner like myself interested in added methods improving the calibration of an existing model, but the authors set aside computational complexity issues and do not give an implementation. There are also serious clarity issues.
Results look excellent, but in the way they are reported in figures and tables, there are too many questions, inconsistencies and readability issues to be sure.

In summary, this seems to be an unfinished write-up where the rich material needs to be better organized with a clear presentation flow, and tables and figures need serious improvements. It has the potential for an excellent publication, proposing a novel solution to an important problem, with rich technical and experimental material.

Detailed comments:

For a reader such as myself who is not an expert in GP but interested in their application to calibration and OOD data, the general introduction in Eq.(1) and (2) makes no sense. I was not even able to parse them and I had to read section 2.4 of (Lee et al 2018) to fully understand how the posterior on a test point is computed. Further down, the authors mention in passing  two critical issues that make NNGP so expensive  without proper explanation:
-	Hyperparameter tuning using Vizier: what are the hyperparameters?
-	Computational cost: they mention it is O(N^3) In the number examples because of the Cholesky decomposition (within what operation? Inversion? reference?). This seems extreme as I see an O(N^2) complexity in other work (Lee et al.)
Note that section A in the appendix did not provide more clarity.

Standard RBF method: can you give a reference?

I spent some time looking for experiments that account for dataset shift before realizing ‘fog’ perturbations represent dataset shift. Same issue with the OOD data: it just appears in Figure 1 and table 1 without proper analysis.

Figure 1: the coloring scheme of the left diagram is not readable. Where is the figure showing entropy?

Table 1 is dropped without any description of the metrics and the datasets:
-	The reader is left guessing what the ‘Fog’ corruptions are, as they are neither introduced nor described, except for a mention of “CIFAR fog corruption” in a table caption. One later learns those are the same as Ovadia shift levels.
-	What is “accuracy”? The reported number for CIFAR10 is 0.487: is this 48.7%? SOTA is more than 90%.
-	Are SVHN and CIFAR100 introduced as OOD? Define mean confidence and entropy? No analysis of results here.

Figure 2 is not readable or comparable to Ovadia et al and should be made larger (maybe remove the Brier Score?):
-	Like in table 1, accuracy is in the 0.6 range when it should be 0.9
-	ECE should range between 0 and .2, not 0 and .8

Table 3: 
-	what is the number of examples in NNGP-LL last column? 50K?
-	I have never seen quartiles used for comparison and I do not understand why they are used, as the authors do not seem to exploit the added information. They should be either explained/exploited or removed (recommended to make space for better explanations elsewhere)


Appendix: lots of good material there, some of which could be moved to the main paper (and some material in the main paper should be moved to the appendix, in particular some rows in table 1 and 3)

review_title: Novel contribution with good results, but paper lacks clarity

final_score: rating: 7: Good paper, accept
review_content: update 1: I thank the authors for the rebuttal. My questions and concerns were appropriately addressed. However, other reviewers raised some concerns regarding the experimental set-up that I have missed. Thus, I will keep my score as is -- 7: good paper, accept.

Summary:

The submission studies Gaussian process models with the infinite-width neural network kernels. In particular, through a series of experiments, the paper investigates their uncertainty properties and answers the question “how calibrated are the predictive uncertainties for in-distribution/out-of-distribution inputs?”: i) a comparison between GP classification with the infinite-width neural network kernels and finite width neural network classification was provided to test the calibration, (ii) a study of these kernels on regression tasks where the GP posterior can be obtained exactly, (iii) a study of using GPs with these neural network kernels on features extracted from a pre-trained network, aka, deep kernel learning of Wilson et al with neural network kernels. Overall, the performance of these GP variants is promising and competitive to the best existing methods.


Assessment:

The paper is very clear and is an interesting read. The experiments are well-designed and the comparison to relevant state-of-the-art methods was provided. Whilst the theoretical contribution is on the light side (no new models or algorithms proposed in this submission), the results from the experiments outweighs this weakness and thus make this paper potentially relevant to the large Bayesian deep learning/GP community. I am leaning toward "accept" and willing to increase the score if my questions below could be clarified.

1. This point is rather philosophical. The calibration metrics are great and they can be used for out of distribution detection. Perhaps this is discussed in previous work but it is not clear to me how these scores can be used to compare methods. For log likelihood, Grosse et al pointed out that a difference of xx nats is significant. Can we quantitatively do the same for these metrics, i.e. the ideal classifier would produce xxx, or a useful threshold for practical use is xxx, or the differences between the whisker plots in figures 2 and 3 are significant. Additionally, are these uncertainties calibrated enough for downstream tasks such as active learning, continual learning, bandits/reinforcement learning etc?. At the risk of asking too much for a conference paper, perhaps since this submission is about understanding the uncertainty of these GP models, an experiment or two on this aspect could be included.

2. I think the difference between the models/inference schemes considered here could be made clearer so that readers could get a clear picture of what methods are being compared to what alternatives. My understanding is that there are the following configuration of models/inf methods -- top: inference, left: net width

net width  Bayesian   point (ML/MAP) 
---------------------------------------------------
finite               (1)                  (2)

infinite           (3)                  (4)

(1) is hard to compute exactly. Approximate Bayesian is not calibrated enough. Though there are approximate sampling methods that can scale to the network size considered in this submission but these were not considered.
(2) is cheap and thus often used in practice, but not calibrated.
(3) exact solution can be obtained for regression with Gaussian observation noise, but expensive. Approximate inference/sampling is often required. This submission shows this configuration is better calibrated than existing solutions for (1) and (2).
(4) is not considered in the experiments provided. It might be interesting to consider this.

3. Why not directly optimising the marginal likelihood (in the regression case)? If I understand correctly, Lee et al used this for their MNIST regression experiment?

4. It would be interesting to compare the infinite net (GP) performance with wide-net performance or narrow-net performance, to understand what regime of the networks is being used and what causes the performance difference across model/inf method configurations: is that because of the model, or is that because of the inference method?. Matthews et al had some comparisons like this using BNN with HMC and GP regression.

5. clarity: For figure 1 (left) and those in the appendix, it’s very hard to see the overlapping bars, maybe just plot the curves connecting the top of the bars? For figure 2 (middle) the limits of the y-axis could be changed. 
review_title: very clear and interesting experimental evaluation of Gaussian process models with infinite-width neural network kernels.

final_score: rating: 6: Marginally above acceptance threshold
review_content: After rebuttal: I am uneasy about the overstated claims made in section 2. That the architectures are small should really be mentioned more prominently. However reviewers #1 and #2 make a good case that what matters are the improvements presented in Section 5. Thus, I reluctantly recommend acceptance.

# Paper summary

The authors empirically investigate the calibration performance of NN-GPs in CIFAR10 and several UCI data sets, in three forms:
- Bayesian inference for the NN-GP function-space prior, through a softmax link function
- Heuristics to convert the classification-as-regression posterior into class probabilities
- As a "head" to a pre-trained network

Generally, they find the NN-GPs are competitive with the _examined_ alternatives.

# High-level comments

I think empirical work like this paper is important: we Bayesians like to justify ourselves using calibration, but unless the beautiful Bayesian methods are *actually* calibrated, they are not useful. This paper mainly explores some properties of existing algorithms that are not understood, which is great.

## Table 1's architectures are too small (2 layer CNN?)

However, I will criticise the methodology. The architecture of the NNs used in Table 1 is specified in the supplement, and it is not pretty: the CNN is restricted to have only 1 or 2 layers (Appendix B)q. The convolution filter shape is unspecified, so I'll assume it is 3x3 like most modern CNNs. Because these networks are so shallow, the receptive field of a given convolutional location is not even close to the full input image.

This architecture is simpler than even the 5-layer 1998 LeNet , and is very far from the 2015 ResNet (He et al.) that is a reasonable CIFAR10 baseline; let alone the current state of the art. It is also much simpler than the networks considered for kernel architectures by Shankar et al. (2020). 

Given how far these architectures are from things used in practice that have decent CIFAR10 performance, I'm not sure we can extrapolate much from the data in Table 1. Perhaps the good calibration and superior performance of the kernel will stop being true once we go to 18 or 20 layers, or once we add mean-pooling to the CNN.  We know that the superior performance of the kernel stops being true at a certain point for classification-as-regression, and I see no reason to think it would be different for "proper" Bayesian inference. (In fact, in my experience, variational inference for GP classification with these kernels has worse accuracy than classification-as-regression).

It is true that calculating the kernel matrix for more complicated CNNs is much more expensive. However, you could do that experiment without tuning the hyperparameters with Vizier. Since the complexity of MCMC once you have the kernel matrix is the same, you should be able to do it. I suspect the accuracy will be much better for the modern CNNs without tuning, than for the 2-layer CNNs you found (the calibration may be better or worse).

At the very least, the caveat that the CNNs are tiny should be clearly stated in Section 3 (and not hidden in the appendix).

## Ignores similar work doing Bayesian linear regression on top of a NN base

The paper "Deep Bayesian bandits showdown" (https://arxiv.org/abs/1802.09127) compares various Bayesian deep learning methods on a bandit task, where calibrated uncertainty is crucial. As a result, most of them fail. The one that seems to come out on top is to change the last layer of a network to a Bayesian linear regression layer. This is equivalent to the method shown in Section 5 of this paper (except this paper sometimes uses a kernel, that is not just the linear regression kernel). It should be at least acknowledged, and ideally it would be included in Table 3.

Table 3 should also acknowledge Bradshaw et al. (2017, https://arxiv.org/abs/1707.02476), which adds an RBF kernel on top of a neural network. All of these should perform similarly to NNGP-LL, especially given how close the RBF kernel is to the NNGP in Table 2.

## Unclear that NNGPs have better uncertainty in Table 2

And that's okay! But it should be a bit more emphasized than "competitive in NLL" just before Section 4.2. Also, the non-standard training for RBF kernels (selecting hyperparameters on a validation set, instead of maximising the train likelihood) may hurt their performance in Table 2.

That NN-GPs are good in accuracy at UCI data sets was noted by Arora et al. (https://openreview.net/forum?id=rkl8sJBYvH), though only for classification

## In conclusion: overstated claims

I think this paper is good, but it overstates how good the methods evaluated turn out to be. It paints a rosy picture of everything, like calibration in NNs is now solved, but that is not true. However, I do think this paper is a meaningful contribution, that puts together a few ideas that were lying around and shows that you can use them to make progress in calibration in NNs.

The authors should clearly caveat in section 3 that the results may not extrapolate to bigger CNNs or, ideally, add a data point with bigger CNNs. The authors should compare to similar algorithms that will be competitive in Table 3.

# Minor points

Top of pg. 2: SNR -> show acronym.

before section 1.1: "It is possible to disambiguate between the uncertainty properties of the NN prior and those due to the specific optimization decisions by performing Bayesian inference". What do you mean? Transforming the prior into a posterior using Bayesian updating is one such decision. Also, because you're approximating the prior, deciding which Bayesian inference algorithm to use matters.

Section 3: "by avoiding heuristic approaches to inference, we are able to directly evaluate the prior". I realise now that by this you mean the same as what I asked above, whatever it is. But the phrasing would be clearer with "we are able to directly evaluate the [properties conferred by the] prior". The fact is, you can evaluate the prior density for any weight setting fairly trivially!

Page 3, top: "under gradient flow to minimize a loss, the output distribution remains a GP". This is only true for squared loss, as far as I know.
review_title: Good work, but overstated claims 

Meta-review: This paper presents an empirical study focusing on Bayesian inference on NNGP - a Gaussian process where the kernel is defined by taking the width of a Bayesian neural network (BNN) to the infinity limit. The baselines include a finite width BNN with the same architecture, and a proposed GP-BNN hybrid (NNGP-LL) which is similar to GPDNN and deep kernel learning except that the last-layer GP has its kernel defined by the width-limit kernel. Experiments are performed on both regression and classification tasks, with a focus on OOD data. Results show that NNGP can obtain competitive results comparing to their BNN counterpart, and results on the proposed  NNGP-LL approach provides promising supports on the hybrid design as to combine the best from both GP and deep learning fields.

Although the proposed approach is a natural extension of the recent line of work on GP-BNN correspondence, reviewers agreed that the paper presented a good set of empirical studies, and the NNGP-LL approach, evaluated in section 5 with SOTA deep learning architectures, provides a promising direction of future for scalable uncertainty estimation. This is the main reason that leads to my decision on acceptance.

Concerns on section 3's results on under-performing CNN & NNGP results on CIFAR-10 has been raised, which hinders the significance of the results there (since they are way too far from expected CNN accuracy). The compromise for model architecture in order to enable NNGP posterior sampling is understandable, although this does raise questions about the robustness of posterior inference for NNGP in large architectures.
Decision: Accept (Poster)