Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The authors show that in the infinite width limit, deep GPs becomes a single-layer GP, generalizing the result that deep neural networks at initialization become a GP in the infinite width limit. The authors show that increasing depth of the GP makes the distribution of the final layer of deep GPs deviate from being Gaussian at finite width. The authors claim that this simplification implies that large widths are detrimental to neural network performance beyond some point.
main_review: ### Originality
The authors study the effect of the width and depth of deep GPs (a generalization of deep networks at initialization) and discuss the implications of the depth and width on the distribution of the output layer conditioned on the input features. This is an interesting generalization of the results that deep neural networks in the infinite limit become GPs at initialization.

### Quality
The quality of the theoretical analysis of the work is good. The experiments conducted on neural networks, and the authors' assertion that larger widths hurts neural network performance (because they make networks into GPs) has the following flaws:

1-0. In order to conduct a fair comparison of the performance of two neural networks with different width, one needs to conduct hyper-parameter search of basic parameters. At a minimum, it is standard to at least conduct a hyper-parameter search of the learning rate, which the authors do not do. This brings into question the validity of figure 4, because, for example, the performance of a width 5000 network might be worse than the performance of a width 500 network, because the fixed learning rate the authors used is more optimal for the width 500 network.

1-1.  For example, it is very possible that the drop-off of performance with respect to the width of the network for MNIST shown in figure 4 can be related to stability issues, as the maximum stable learning rate for wider networks tend to be lower. We can't know for sure, since the authors have not conducted a learning rate sweep for each width.

2-0. Besides the fact that the experiments conducted to support the author's assertion that "very large" widths hurt neural network performance has not been carefully conducted, even if one accepts the assertion that there exists an optimal width for performance, the reason the authors provide for it is not consistent with the existing literature.

2-1. If the authors' diagnosis for width hurting performance, i.e., that the neural network collapses to a GP in that limit, is correct, it must be the case that the performance of a neural-network GP (NNGP) cannot exceed that of a neural network for the MNIST tasks the authors explored.

2-2. This can be refuted by the experiments presented in work the authors cite, namely Lee et al. [49]. In table 1 and table 2 of their paper, the authors present the result of numerous experiments they conduct where they demonstrate examples where the NNGP performance exceeds that of a trained neural network. For example, the authors report an NNGP (obtained as the infinite width limit of a 3-layer network) that achieves 96.93% test accuracy on 5k MNIST, better than what the authors report in figure 4.

### Clarity
The theoretical analysis itself has been presented in a clear and consistent way. The connection between the theoretical results and the implication it has to the performance of neural networks has not been clearly presented, if presented at all. The authors try to imply that the fact that the infinite limit reduces the neural network at initialization to a GP (a well-known fact) is the reason that very wide networks should eventually under-perform networks with "optimal-width," but there is hardly an argument being made. In fact, Lee et al. [49] has systematically explored this problem with extensive experiments and have showed that the GP limit often out-performs trained neural networks, disproving the authors' claim.

### Significance
The paper contains some interesting analysis on deep GPs, but their claims connecting their theoretical results to the performance of actual neural networks are not consistent with experimental results in the literature.

---

### Summary of Author Correspondence

After discussion with the authors, the authors have agreed to make the following changes to their paper:
1. They have run additional experiments to show how widening the width of trained-NNs affect the NN performance, and agreed to provide the training parameters used.
2. They have agreed to clarify that their theoretical and experimental results on deep GPs cannot be directly be applied to trained-NNs, but is suggestive in nature.

My initial rating of the paper for a clear rejection (3) was based on a (perhaps misunderstood) assessment on my part that the paper is confusing the reader by implying that the Gaussianity of wide DGP marginals is what should be attributed to the performance loss of trained-NNs at very large widths, the evidence of which I found to be unconvincing. Given that the authors have agreed to clear up such confusion, I believe my initial rating to be too harsh now. At the same time, it has been clarified that the most interesting part of the paper to me, the consequence of the authors' work on trained-NNs, is indeed of a speculative nature. I thus have adjusted my rating to a marginal reject (5).
limitations_and_societal_impact: I do not have useful comments in this aspect.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary:      This paper provides a narrative of composite Gaussian process models. In specific the paper provides insights into the interplay between depth and width in the context of capacity and representative power. The paper further draws conclusions on the implications that this has for composite finite basis function models such as neural networks.

main_review: This is a very nice paper that I thoroughly enjoyed reading. The paper really tries to tell a story through related work and provides easy to follow mathematical derivations of the main concepts with plenty of intuition.

     My main question regarding the generalisation of the results from Deep GPs to neural networks. While it is true that Deep GPs subsumes neural networks and as is shown in [1] there exists an equivalence class in parametrisation between a sparse Deep GP and a neural network. However, the main difference comes in the difference between the marginal likelihood and the objective function used to train the neural network. It would be great if the authors could comment on this and what the implications of this is in interpreting the results in the paper for neural networks.
     
     While this paper wants to study the model in isolation from the inference mechanism I think the results implies something quite interesting with respect to sparse DGPs, the approximate model is actually more useful compared to the model it aims to approximate. The same type of behaviour can be seen with a VAE. I think this would be an interesting idea for the authors to discuss and what this actually means for the conclusions we should draw from inference and how we should design approximate inference schemes for Deep GPs.
     
     In summary I think this is a lovely written paper providing a nice narrative to many concepts that people have had an intuitive understanding of with respect to Deep GPs. Making these mathematically concrete and providing a narrative grounded in related work is important to move these methods forward with understanding beyond empirical results. The authors have done a good job of separating the material in the main paper from the detailed explanations in the appendix. For me it was crucial to have the verbose derivations in Appendix F,G to solidify my understanding from the paper. I would be happy to increase my score, however I feel (as is reflected by my confidence score) that I am less well versed in the related work that this paper builds upon so there might be things that I have missed.

     Even though this paper has *a lot* of references already, one paper that the authors might like is [2]. There are quite a few connections between these two works that you might find interesting for future work.
     
     1) Dutordoir, V., Hensman, J., Wilk, M. v. d., Ek, C. H., Ghahramani, Z., & Durrande, N., Deep neural networks as point estimates for deep gaussian processes, CoRR, (),  (2021). 
     2) Halverson, J., Maiti, A., & Stoner, K., Neural networks and quantum field theory, Machine Learning: Science and Technology, 2(3), 035002 (2021).  http://dx.doi.org/10.1088/2632-2153/abeca3
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper shows that in the limit of infinite width a Deep Gaussian Process (DGP) converges to an ordinary Gaussian Process (GP). Indeed, in the limit of infinite width the covariance function of each hidden GP layer becomes deterministic (as a consequence of the strong law of large numbers). This allows proving that the marginal prior covariance  of the DGP becomes gaussian (Theorem1). The  consequence of this theorem is that as the width increases DGP models lose their ability to adapt the hidden covariance functions to the input data (section 4).
This theoretical result is supported by numerical experiments (section 6.1).
The paper also addresses the convergence of a DGP to a GP with increasing depth and width (section 5).
Since it is known that deep neural networks (DNN) approach a GP as they are made wider and wider,  Theorem 1 is used to  claim that a very large width can be detrimental to the expressivity of a DNN (and not only a DGP). 


main_review: The paper is  well written and the proofs of the theorems/lemmas are, to my understanding, correct. The connections with the relevant literature are also discussed, with an important exception (see below). 

I have a concern related to the meaningfulness of the extension of the results to standard neural networks.  The test accuracy reported in Figure 4, rightmost panel, is significantly lower than the accuracy obtained with non-Bayesian approaches on CIFAR10. (see for example https://paperswithcode.com/sota/image-classification-on-cifar-10?tag_filter=3) This is for sure partially due to the lack of data augmentation (which is acceptable in a numerical experiment) but, in my opinion, also to the training protocol. If I understood correctly, the same weight decay is applied for every model. What would happen if one would tune this decay optimally? Maybe, as stated by the authors “.. by changing the weight decay parameter, may lessen the consequences of width that we observe” (line 322-323). Indeed, setting the weight decay at initialization amounts simply  to set the inverse variance of the gaussian prior on the network weights. It is well known empirically that smaller models require larger weight decay than wider models. It seems thus natural to choose a gaussian prior with small variance i.e. more restrictive for narrower networks than for wider ones. 

The fact that the author(s) have not optimized their training protocol might partially explain the (possible) discrepancy with other works: see for example  https://arxiv.org/abs/1901.01608 (which should be cited and discussed), ref 49, where it is explicitly shown that width improves accuracy *in practice*.

Indeed, the quality of an architecture is not only in its theoretical expressivity, but, even more, in its “trainability”. Even a large  width perceptron is, as has been well known since a long time, extremely “expressive”, but it cannot be trained.  The importance of trainability is in my opinion overlooked in an otherwise interesting and relevant paper.

In short, I think that either the detrimental effect of width is demonstrated using state-of-the-art training protocols, or the strong claims on the importance of Theorem 1 for standard DNNs should be weakened. 

 The code used for the experiments should be publicly released for the reproducibility of the results

limitations_and_societal_impact: In my opinion the limitations are not sufficiently discussed, in particular concerning  the connection of the results to the theory of DNNs.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper argues that deep Gaussian Processes with large widths and more than 2 layers have undesirable properties. In particular, the paper shows that firstly, deep GP prior in the GP limit is essentially shallow, and secondly, deep GP posterior then loses data adaptivity in the hidden layers. The paper further examines the role of depth and argues that the deep GP prior, if not equal to a GP, has heavy-tailed behaviors.

==================================
Post-rebuttal:

Thanks for the reply and clarification!

While I agree that very large width hurts in the context of deep GP as shown in the paper, I also agree with other reviewers that the message that very large width hurts generally is overly strong. The literature has a mixed bag of results concerning widths -- theoretically or empirically, so a general statement should be validated very carefully.

I'd like to keep my score.
main_review: The subject of the paper is timely, and the conclusions are interesting. In a way, it is not surprising that the deep GP prior becomes shallow in the GP limit; specifically Lemma 1 is expectable since intuitively a mixture of zero-mean Gaussians, if equal to a Gaussian, should comprise of same Gaussian components. (To be fair, Lemma 2 is more general than this.) What is interesting in the paper is the interpretation that in the hidden layers, a certain hierarchical structure is lost and hence so is data adaptivity in the posterior. This message is a strong and noteworthy one.

Yet this important message about the posterior is where I do not find it satisfying yet. In particular, the argument around (6) potentially ignores some important limit exchange issue, technically speaking. The conceptual problem I have in mind is the following. It is true that in the prior the kernel converges pointwise to a degenerate limit as the width increases. However this doesn’t exclude the possibility that in the process of computing the posterior (or in general, a process that involves fitting the training data), the movement of the weight of the first layer explodes with large width. In such scenario, it is unclear if in the posterior, the kernel can still be close to the degenerate limit (in an appropriate sense such that (6) holds).

Another issue — of less importance in my personal opinion — is that of the experimental results in Section 6.2. This section aims to argue against large widths in contexts beyond GP. However the way I read the results in Figure 4, they say that the performance does not necessarily increase monotonically with width. I feel even more mixed when contrasting this figure with other papers, such as [51]. The notion of “sufficient capacity” is not defined, nor is it illustrated numerically. I have a similar concern regarding Table 1, where the message that deeper is better is not clear.
limitations_and_societal_impact: Yes
review_title: null

Meta-review: This paper leverages an analysis of deep Gaussian processes to argue that an excess increase in the width of a neural network can degrade performance. The question of whether or not neural networks benefit from increased width is an important and unresolved question in the literature, and the reviewers and I agree that this paper provides an additional and important perspective on the topic that will be of interest to the NeurIPS community. While some reviewers found the experimental results unconvincing and the conclusions somewhat speculative, others found the framework to be illuminating and to provide new perspectives on this important problem. Overall, I do not expect this or any paper to fully and unambiguously resolve all questions relating to the benefits/drawbacks of large width networks, and I believe this paper provides novel and useful insights that shed light on this important problem. Therefore, I recommend acceptance.
Decision: Accept (Poster)