reviews: 
Comments: The paper proposes a novel way to learn Bayesian last layer (BLL) networks by deriving a sampling-free approach for optimizing normal, categorical, and generative classification likelihoods,
by relying on variational inference techniques. 
The approach is then evaluated on a series of regression and classification tasks against a range of baselines.


_____
_Edit: Given the improved presentation and evaluation, I increased my score._
Strengths: BLL networks are an interesting approach to solve the scalability problem Bayesian neural networks tend to suffer from. 
The paper introduces another variation to this family of approaches that is relatively straightforward, easy to understand, and implement.
The method is properly evaluated as the number of experimental setups is reasonably extensive both with respect to architectures and experimental tasks.
Weaknesses: Straight-forward contributions can be seen both as a strength and as a weakness depending on the situation. 
They are a strength if they are an easy solution to a complex problem that might not improve upon current approaches in all situations, but most. 
They are a weakness if they do not provide a clear theoretical benefit above current approaches and also come without clear performance improvements.
For me, the results point to the latter case as they are rather mixed despite some strong wordings of the authors in their claims. 

The abstract promises "improve[d] predictive accuracy, calibration, and out of distribution detection over baselines.", similarly in the contributions, and conclusion 
parts of the paper. Even more, the method not only improves, but it also performs "extremely strong" and "exceptionally strong".  
These are some exceptionally strong statements given the actual performance.  

Focusing on each of the experiments in turn. The first problem is the presentation, e.g., what does a bold number mean (see question below)?

_Regression Experiments._ Of 
the six data sets (see question below on this number) the proposal improves on two, slightly on one, equally on two (although better than neural net-based baselines), and worse than most of its baselines in the final one. Calling this "strong performance" is rather misleading. Two additional, though potentially minor, problems are that all of the baselines are simply cited from prior work (Watson et al., 2021). Given the wide performance variations between different train/test splits that can be observed for various UCI data sets the results are not entirely trustworthy. (Note that the reported error intervals are most likely standard errors, as is common on UCI, instead of standard deviations. But which they are is never specified.)
Secondly, the authors acknowledge in the appendix that there might be differences in the way the training data is normalized compared to the cited results. 
(Whether these problems strongly influence the results, or bias them in favor or against VBLL is unclear.)   

_Classification._
While "Extremely strong accuracy results" are mentioned, it just performs as well or worse than competitive baselines like Laplace or Ensembles. The same for ECE, NLL, OOD detection.  where "exceptionally strong performance" is claimed.

The method is somewhat simpler than baselines, but it lacks a convincing argument for why this should matter. As the authors advertise this simplicity, there should be additional results on practical runtime improvements compared to the baselines to provide some evidence for the claim that a reader should use this approach. 

### Minor
- Dropout is first claimed to have a "substantially higher computational cost" (Sec 1) and appears later as a "comparatively inexpensive" method (Sec 4).  Additional forward passes at test time are indeed rather inexpensive instead of a high computational cost.
- When submitting to ICLR please make sure that you follow the ICLR style guide. E.g., Table captions belong above tables, not below.

### Typos
- Sec 5.2 first par: "We refer to this model **as** G-VBLL-MAP..."
- (6): $y|\rho$
Recommendation: 6: marginally above the acceptance threshold
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 2 fair
Clarity: 3 good