Linking Neural Collapse and L2 Normalization with Improved Out-of-Distribution Detection in Deep Neural Networks
Authors: Jarrod Haas
William Yolland
Bernhard T Rabus
Venue: Accepted by TMLR ()

Abstract: We propose a simple modification to standard ResNet architectures--L2 normalization over feature space--that substantially improves out-of-distribution (OoD) performance on the previously proposed Deep Deterministic Uncertainty (DDU) benchmark. We show that this change also induces early Neural Collapse (NC), an effect linked to better OoD performance. Our method achieves comparable or superior OoD detection scores and classification accuracy in a small fraction of the training time of the benchmark. Additionally, it substantially improves worst case OoD performance over multiple, randomly initialized models. Though we do not suggest that NC is the sole mechanism or a comprehensive explanation for OoD behaviour in deep neural networks (DNN), we believe NC's simple mathematical and geometric structure can provide a framework for analysis of this complex phenomenon in future work.

Published in Transactions on Machine Learning Research (12/2022)
Linking Neural Collapse and L2 Normalization with Improved Out-of-Distribution Detection in Deep Neural Networks
Jarrod Haas
jhans@sfu..o
SA Rlab, Deportment of Engineering Science Simon Fruser University; Digitalist Group Canada
William Yolland
yollandu@gmail.comm com
MetaOptima
Bernhard Rabus
berahard t.rabs@ssf.fc ca
SARlab, Department of Engineering Science
Simon Praser University
Reviewed on OpenReview: https:
// openreview. net/ forum? id= f
Abstract
We proposc a simple modification standard ResNet architectures- 1, normalization over feature space that substantially improves out-of-distribution (OoD) performance on the pre- viously proposed Deep Deterministic Uncertainty (DDU) benchmark. We show that this change also induces carly Neural Collapse (NC), an effect linked to better OoD performance. Our method achieves comparable or superior OoD detection scores and classification accu- racy in a small fraction of the training time of the benchmark. Additionally, substantially improves worst case OoD performance over multiple, randomly initialized models. Though we do not suggest that NC the sole mechanism or a comprehensive explanation for OoD behaviour in deep neural networks (DNN), believe NC's simple mathematical and geo- metric structure can provide framework for analysis of this complex phenomenon in future work.
1 Introduction
It is well known that Deep Neural Networks (DNNs) lack robustness to distribution shift and may not reliably indicate failure when receiving out of distribution (OoD) inputs (Rabanser al. 2018 Chen al. 22200). Specifically, networks may give confident predictions in cases where inputs are completely irrelevant, e.g. image of a plane input. into a network trained to classify dogs or cats may produce high confidence scores for either dogs or cats. This inability for networks to 'know what they do not know" hinders the application of machine learning in engineering and other safety critical domains Henne et al. 2020).
A number of recent developments have attempted address this problem, the most widely used being Monte Carlo Dropout (MCD) and ensembles (Gal and Ghahramani 2016 Lakshminarayanan al. (017)) While supported by reasonable theoretical background, MCD lacks performance in some applications and requires multiple forward passes of the model after training (Haas and Rabus 2021 Ovadia et al. 2019). Ensembles can provide better accuracy than MCD, as well as better OoD detection under larger distribution shifts, but require a substantial increase in compute Ovadia et al. 2019
These limitations have spurred interest in deterministic and single forward pass methods. Notable amongst these Deep Deterministic Uncertainty (DDU) (Mukhoti et al.] 2021]. DDU is much simpler than many competing approaches (Liu et al. 2020 Van Amersfoort et al.] 2020 van Amersfoort et al.) (021), produces competitive results and has been proposed benchmark for uncertainty methods A limitation,

---

Published in Transactions on Machine Learning Research (12/2022)
5
5
0
0
-5
-5
-10 -5
0
5 10 100
-5
5
10
Figure 1: An illustration of the DDU method from Mukhoti et al. (2021) Left: In this hypothetical example with a two-dimensional feature space, DDU fits Gaussians over cach of three classes as the components of a GMM, ((y,)) Right: with standard decision boundaries (red), embeddings this space that are far (yellow points) from class centroids are labelled with high confidence (darker areas are higher confidence).
shown in our experiments, is that DDU requires long training times and produces models with inconsistent performance.
We demonstrate that DDU can be substantially improved via L2 normalization over feature space in standard ResNet architectures. Beyond offering performance gains in accuracy and OoD detection, L2 normalization induces neural collapse (NC) much earlier than standard training. NC was recently found to occur in many NN architectures when they are overtrained (Papyan et al.) 2020). This may provide a way render the complexity of deep neural networks more tractable, such that they can be analyzed through the relative geometric and mathematical simplicity of simplex Equiangular Tight Frames (simplex ETF) Mixxn al. 2022 Zhu et al.) 2021 Lu and Steinerberger 2020 Ji et al.j 2021). Although this simplex ETF is limited to the feature layer and decision classifier, these layers sumnarize a substantial amount of network functionality. While Papyan et al. demonstrate increased adversarial robustness under NC, to the best of our knowledge, we present the first study of the relationship between OoD detection and NC.
We summarize our contributions as follows:
L2 normalization over the feature space of deep learning models results in OoD detection and classi- fication performance that is competitive with or exceeds performance the DDU benchmark. Most notably, the worst case OoD detection performance across model seeds is substantially improved.
2) Models trained with L2 normalization over feature space produce the aforementioned performance benefits in 17% (ResNet18) to 29% (ResNet50) of the training time of the DDC benchmark. Our proposed L2 normalization does not add any significant training time versus models without it.
L2 normalization over feature space induces NC as much as five times faster than standard training. Controlling the rate of NC may be useful for analyzing DNN behaviour.
4) NC is linked with OoD detection under our proposed modification to the DDU method. We show evidence that fast. NC plays a role in achieving OoD detection performance with less training, and that training directly on NC has a substantially different effect on OoD performance than standard cross entropy (CE) training. This connection between simplex ETFs that naturally arise in DNNs and OoD performance permits an elegant analytical framework for further study of the underlying mechanisms that govern uncertainty and robustness in DNNs.
2

---

Published in Transactions on Machine Learning Research (12/2022)
AUROC CIFARIOO
Ancurucy
SVIN
Thgy ImageNet
CIFARIO Trst
ResNet18
No L2 350
No L2 60
L2 60
No L2 350
No L2
L3
No 350
No L2
No L22
No
1,2 00
0.877
0.518
0.924
0.601
0.878
0.881
0.9.0
0.89/5
0.922
0.8SI
0.49
0.063
0
0.885
0.815
0.886
0,009
0.872
0,911
0,941
0.912
0.29
Mean
0.151..088
0.9122 035
0.9381.010
0.8754.0088
0,824 014
0. 0.8912
0
0.8911.000
(009
0.841
0.90411.004
0.935 BOS
0,898 00S
0.9272.0001
ResNett0
No L2
100
100
350
No 100
350
No L2
L2
No
No L. 100
100
0.903
0.860
0.027
0.817
0.733
0.892
0.881
0,832
0.912
0.890
0.937
Max
0.987
0.080
0.955
0.501
0.900
0.896
0,509
0.003
0.918
0,946
0.027
0963
Moan
0.965-.1126 1126
0.9462.025 029
0,945 007
0.871:8008
0.817 1122
0,904:2.01
0 0.804E27221
0.8811
0.9164.002
0.9894.000
0.306 OBS
0.0411.002
Table 1: OoD detection and classification accuracy results for ResNet18 and ResNet50 models, 15 seeds per experiment, trained on CIFAR10, and SVHN, CIFAR100 and Tiny ImageNet test sets used as OoD data. For all models, we indicate whether L2 normalization over feature space was used (L2/No L2) and how many training epochs occurred 00//00//35), and compare against the DDU baseline (No L2 350). Note that the variability of AUROC scores substantially reduced under 1,2 normalization of feature space. With much less training, worst case OoD performance across model sceds improves substantially over the bascline, and mean performance improves or is competitive in all cases.
2 Background
2.1 Problem Definition
A standard classification model maps images to classes f x + y, where x E X, the set of images, and y E k), the set of & possible classes. The model is composed of a feature extractor which embeds images into feature space embeddings in Rd where d is the size of the feature vector, and a linear decision classifier that transforms z into vector of length k, commonly known as the logits. These logits are run through softmax function to produce probability over classes over which the argmar function identifies the class prediction y. We note that the set of images can be broken into X E Xin which are images drawn from the same distribution as the training data, and X E Xout. which are all other images.
To evaluate models, we merge ID and OoD images into a single test set. OoD performance is then a binary classification lask, where we measure how well OoD images can be separated from ID images using a score derived from our model. Our scoring rule is derived from a Gaussian Mixture Model (GMM) fit over = (as discussed below) rather than the softmax outputs used for classification. We retain softmax outputs we want to make sure that our ability to distinguish ID and OoD images does not hinder our ability to classify ID images.
2.2 Related Work
Until recently, most research loward uncertainty estimation in deep learning took a Bayesian approach. The high number of parameters in DL models renders posterior integration intractable, so approximations (typically variational inference) have been used (Gal and Ghahramani] 2016). Monte Carlo Dropout (MCD), which draws connection between test time dropout and variational inference, has emerged a popular method to estimate uncertainty (Gal and Ghahramani] 2016). Despite MCD's simplicity, scalability and applicability to nearly any model architecture, it is often outperformed by deterministic ensembles (Ovadia et al.) 2019). Lakshminarayanan et al. observed that a simple average over the predictions of multiple deterministic models with the same architecture but starting from unique parameter initialisations produces a competitive uncortainty estimate (Lakshminarayanan et al. 2017].
The extra compute required for running multiple model passes at test time and for training and testing ensem- bles has motivated recent research around single model, single pass uncertainty estimates. Van Amersfoort et al. enforce a bi-Lipschitz penalty on gradients during training to create distance-aware feature space in their Deep Uncertainty Quantification method (DUQ), which is then measured with a radial basis function (RBF) instead of a standard linear classifier (Van Amersfoort et al. 2020). While showing some promise, bi-lisschitz loss penalties can be unstable during training (Mukhoti et al. (021). In later work, van Amers- foort et al. (2021) replace the RBF apparatus with a deep kernel and a point Gaussian Process (GP), while maintaining distance-awareness in feature space by adding spectral normalization throughout the model. A
3

---

Published in Transactions on Machine Learning Research (12/2022)
similar approach by Liu et al. (200 uses a Random Fourier Feature approximation to construct the GP (called Spectral-normalizedd Neural Gaussian Process, or SNGP), but results are not as competitive. [Lee et al. (2018) propose fitting a wlasswwise conditional Gaussian with shared covariance matrix to multiple layers, but employ OoD and adversarial data to learn a weighted average of scores over layers. They also add noise to inpuls at test. time to enhance OoD separability. ODIN produces competitive results, but also requires OoD data. for hyperparameter tuning (Liang et al., 2017). Generalized ODIN relaxes this requirement (Hsu) et al. 2020), but both versions require an additional backward pass at test time to perturb inputs. Finally, Hendrycks and Gimpel (2016) proposed simply measuring maximum softmax scores of converged models benchmark for OoD detection. We believe that OoD detection has moved well past what out-of-the-box, uncalibrated softmax scores can accomplish, and that Deep Deterministic Uncertainty provides a better benchmark for the evaluation of single pass methods (Mukhoti et al., 2021
2.3 Deep Deterministic Uncertainty
Deep Deterministic Uncertainty (DDU) was proposed by Mukhoti et al. as a simple but very competitive benchmark to detect OoD samples (Mukhoti et al.) 2021). It requires only single pass through a single network, and empirically demonstrates that more complex mothods such as DUQ and SNGP are not nee- essary. To evaluate uncertainly, uses a class- wiss Gaussian Mixture Model (GMM) ((y, with mean and covariance parameters retrieved from the feature layer via one-time pass over the training set using a converged model. At test time, extracted features This log probability is used as the scoring method to assess separability of ID and OoD data.
evaluated under the GMM as g(2) E, (<11))g(v).
Mukhoti et al's approach is motivated by a simple idea: feature space must be "well regularized', i.e. images that are very different in input space are different in feature space ("sensitivity"), and images that are similar in input space are similar in feature space ("smoothness"). This can be interpreted as enforcing a bi-Lipschitz constraint over feature space:
(zd1 S d=(f0(zz) s Kedj(E1,
for feature extractor f with parameters 0, inputs T2, distance metrics d₁, dy over input space and feature
space respectively, and lower and upper bounds Ky. (sensitivity) and Ky (smoothness), ivspyctively (Mukholi
et al. 2021 Liu et al., 2020). Leaky ReLUs and strided average pooling of Ixl convolutions (to perform downsampling residual connections) can improve sensitivity, while spectral normalization can improve
smoothness and encourage bi-Lipschitz continuily in residual networks (Mukhoti el al. 2021 Smith et al.,
2021
Under well-rogllarzed feature space arising from converged model, OoD images should be mapped less often to ID feature space. OoD detection is then a matter of measuring the distance of an extracted feature f0(x) from dense, class-wise feature-space clusters that emerge during training. Additionally, fitting a GMM over these clusters prevents the network from assigning high confidence to points far away from the class clusters simply because they fall within a decision region (Figure D
Importantly, if feature extractor is too sensitive, ID feature vectors may not cluster tightly enough, and ID images would then get flagged as OoD due to their distance from modes in the GMM. If too smooth, the feature extractor won't pick up on differences between OoD and ID images, and may place OoD images too close to GMM modes.
Our ablation study results (Table 2 show that L2 normalization over feature space improves OoD detection when used in addition to the bi-T.ipshitzzenfrrinng regularization techniques proposed in the DDU benchmark. In Section 4 we show an empirical link between L2 normalization, Neural Collapse and OoD detection performance.
2.4 L2 Normalization of Feature Space and Neural Collapse
DDU works well under the assumption of well-regularized feature space. However, the bi-Lipschitz conti- nuity encouraged by the aforementioned methods affords no explicit structure over feature space. We suggest
4

---

Published in Transactions on Machine Learning Research (12/2022)
Intervention
AUROC
Accuracy
Spectral Norm
Leaky
GAP
L2-Norm
Min
Max
Mean
Min
Max
Mean
0.855
0.939
0.910
0.884
0.927
0.916
x
x
/
0.894
0.951
0.932
0.924
0.930
0.927
X
0.830
0.965
0.889
0.872
0.909
0.898
/
0.917
0.958
0.944
0.925
0.929
0.927
X
0.884
0.954
0.923
0.904
0.926
0.917
X
0.903
0.959
0.934
0.925
0.931
0.928
0.848
0.953
0.912
0.881
0.912
0.898
0.924
0.961
0.938
0.924
0.929
0.927
Table 2: ResNet18 ablation study of effects from the DDU benchmark along with our method, 15 seeds per experiment. Models trained on CIFARIO, with SVHN as OoD data. When combined with any other interventions, our method improves worst case and average AUROC scores. In many cases, minimum AUROC across all seeds is improved by several points.
that the simplex ETF structure arising from Neural Collapse improves scnsitivity and smoothness and is thereby linked with better OoD detection results.
Although feature space structured once model accuracy converges this being a simple consequence of the decision layer's need divide feature space in order to produce class predictions-the structure that emerges is limited and highly variable, even under bi-Lipschilz regularization techniques: class clusters have high non- uniform variance, and clusters are only separated to the degree that the decision layer can satisfy training demands for low cross-entropy loss (Figure B). As consequence, it is well known that data embedded far from class clusters can have high confidence scores (Hein et al. (018)). Figure 1. This is also evidenced by models with high accuracy and high-variance class clusters (Table 3
Given that we want to organize feature space in the manner most conducive to OoD detection, we suggest that following the principles of sensitivity and smoothness should be the aim: tightening clusters and placing them away from each other. Features in this proposed configuration would have low within-class variance (smoothness), and higher between-class variance (sensitivity). Intuitively, networks trained to adhere this structure should excel at mapping known inputs to within these light clusters, while unknown inputs should become difficult to map precisely and would tend to fall outside of the clusters and into the empty feature space around them.
Although there are many possible ways structure feature space, a reasonable starting point is a hy- persphere. Numerous works exist that seek to exploit the properties of hyperspheres in order make high-dimensional information-compariss
problems, like OoD detection, more tractable (Sablayrolles et al., 2018 Zhou et al. 2020 Zheng et al. 2022 Liu et al.] 2017). An interesting property of hypersphere RD is that if D is sufficiently larger than the number of points N that one wishes to embed on its surface, the maximum distance between all points is obtained when all point-wise distances are equal. In other words, constraining feature space to a hypersphere allows a structure over which class clusters could be maximally and equally distanced from one another, providing substantial unused space between clusters. We note that fully maximizing between-class distance probably not idcal, as this would rigidly enforce too much sensitivily i.e. we would like the feature extractor have some flexibility in placing more similar classes closer together. A balance must be struck between spreading classes out and respecting appropriate Lipschitz bounds.
Papyan et al. (2020) recently observed a phenomenon known as Neural Collapse (NC) which progresses as networks are trained, ultimately leading to feature space and decision space conforming to a structure known as simplex equiangular fight-frame (see Scction 3.2 for definition and measurement). One of the key properties of NC is variability collapse, wherein class clusters in embedding space collapse toward a single point in feature space. Additionally, feature space convergences to a simplex ETF, clusters positioned
5

---

Published in Transactions on Machine Learning Research (12/2022)
Intervention
NCI
NC2 EA
Means
NC2 EA
Class
NC2 EN
Means
NC2 EN Class
NC3
NC4
Speciral Norm
Leaky ReLU
NorNor
Mean
SD
Menn
SD
Mean
5D
Mean
5D
Mean
SD
Mecan
SD
Mean
No Leaky
No
1397
0.33
0.076
03007
03033
0.011
0.013
0.015
00002
NN
12
0.154
0.04
0.014
0.001
0.023
0.001
0.044
0.002
0.083
0.001
0.038
0.001
0.000
0.000
Leaky
2.668
0.329
0.100
0001
0.031
0.027
0.115
02017
0.112
02000
0.227
0:010
01005
00005
0.024
0.001
0.017
SN
No Lenky
No L2
122
0.163
2.199
0.055
0.412
0.015
0.08
0.002
0.019
0.000
0.024
0.013
0.001
0.108
0.047
0.002
00008
01285
0.085
00006
0.001
0.038
0.382
0.001
0.029
0.000
0000
0000
0.000
Lanky
No
2.00
0.051
1.020
0.124
0.005
0.005
0.024
0.023
0.001
0.049
SN. Leaky, No L2 360 Epuchs
0.305
0.164
0002
000S
00025
0.006
0.003
0.005
00225
0,022
0.04
0.001
00000
0000
Table 3: Noural Collapse measurements for ResNet18 ablation study and ResNet18 DDU bascline. 15 soeds for all experiments, ablation models trained for 60 epochs, DDU baseline trained for 350 epochs. Lower numbers indicate more NC. Most metrics are an order of magnitude lower with the L2 intervention. Models trained for 350 epochs slightly less but similar amounts of NC as L2 models trained for only 60 epochs.
at maximally equiangular distances from each other. In other words, DNNs gradually and automatically proceed toward the hypersphere structure we seek (Figure 2
The difficulty here is that NC fully converges only after the terminal phase of training, i.c. after cross- entropy loss goes to zero. This means that networks need to be substantially over-trained before feature space becomes structured in manner which we can fully exploit.
Since the Simplex ETF structure also results in class clusters existing on the surface of a hypersphere (a consequence of equinormality, see Section 3.2/, we hypothesize that it is possible to induce NC more quickly by simply constraining feature space to a hypersphere from the onset of training. T.2 normalization over feature space constrains feature vectors to a point on the surface of a hypersphere by restricting the magnitude of all feature vectors to be uniform:
Znorm
mar(( e)
(2)
where Rd is the space of real numbers over d dimensions and e is added in the event of the zero norm. We note that L2 normalization of the feature space immediately constrains features to satisfy the equinormality property of a Simplex ETF (see the second property of NC Section 3.2p from the onset of training. Since feature vector magnitude no longer a discriminatory factor for the classifier, feature vectors must be differentiated entirely by angular distance. For this reason, we view L2 normalization as a regularizer, limiting the set of possible feature spaces to only those defined on the surface of a hypersphere. This expedites the progression of equiangularity implemented in one line of code and does not require a sophisticated loss function, parameter tuning, nor any monitoring of NC measurements. In this way, L. normalization acts as a regularizer that constrains feature space to a hypersphere and expedites NC, and NC is linked with OoD dctection under the DDU method by explicitly and progressively structuring feature space as a simplex ETF.
well as the other aspects of NC (Table 3). Importantly, this can be easily
3 Methodology
3.1 Models and Loss Functions
For all experiments we used either ResNet18 ResNet50 models provided with the DDU benchmark code (Mukhoti et al., 2021). All were trained from fifteen independent seeds (training details can be found in Appendix A. All basclines used a standard cross entropy (CE) objective function during training. The NC intervention group described in Section 4.2 did not use a use a CE loss, but instead used a loss function containing the differentiable metrics described below in Section 3.2
[xc(f(x)) NC, NNeeaan ENelassiiiirr E.Amcans + FAddassifiir + NC
(3)

---

Published in Transactions on Machine Learning Research (12/2022)
Figure 2: Progression of Neural Collapse during training (left to right). Small blue spheres represent extracted features (classes are different shades of blue), blue ball-and-sticks are class-means, red ball-and-sticks are linear classifiers. Features collapse to low-variance class means and linear classifiers align with these. Note that the simplex ETF pictured is the 2D plane in 3D space, such that each arm is equidistant at 120 degrees. Image from Paayyn et al. 2020).
Note that the metric for NC4 is not used, as it requires an argmin function which is not differentiable. Although these metrics do not all have the same scale, they all proceed to zero, and we did not find it necessary to use any weighting scheme within the loss function.
3.2 Measuring Neural Collapse
NC has four properties:
NC1: Variability collapse: the within-class covariance of each class in feature space approaches zero.
NC2: Convergence Simplex Equiangular Tight Frame (Simplex ETF): the angles between each pair of class means are maximized and equal and the distances of each class mean from the global mean of classes are equal, i.e. class means placed at maximally equiangular locations on hypersphere.
NC3:
Convergence to self-duality; model decision regions and class means converge to symmetry where each class mean occupies the center of it's decision region, and all decision regions are equally sized.
NC4: Simplification to Nearest Class Center (NCC): the classifier assigns the highest probability for a given point in feature space to the nearest class mean.
Papyan el al. (2020) use seven different metrics (Eq. 3 to Eq. 8 to observe these properties. All are differentiable and used in the NC loss function for the experiment in Section 4, except for the NC4 metric (Eq. 9 which not differentiable.
The within-class variance, NC1, measured by comparing the within-class variance to the between-class variance,
NC1 Er E
where Tr is the trace operator, 1.]t indicates the Moore- Penro-e pscudoinverse, and C is the number of classes. Although Paapyan et al. (200)) could simply have used the trace of Σw to measure variability collapse, speculate they incorporated Σ/ (i.e. the between-class precision) because it becomes smaller as the class-means separate from each other around the hypersphere. Note that NC1 is not necessarily zero if cross-ent.ropy loss is zero: as shown in Figure class can have multiple areas where class confidence is maximized.
NC2 is indicated through four measurements. The equinormality of class means and classifier means is given by their coefficient of variation,
7

---

Published in Transactions on Machine Learning Research (12/2022)
stde(l the uG (2)
EN
(5)
avgalll Ue UG l12)
sldc(1] (1l w² 12)
E. E.Neasssi ficr avga(I) w (2)
(6)
where des ug in R² are the class means and global mean of classes, std and avg are standard deviation and average operators, and || ll is the L2-norm operator. The global class mcan is computed the mean of all class clusters, and class means themselves are each the mean of all feature vectors generated by the model for images of that particular class.
Maximum cquiangularity is measured via the Gram matrix of normalized class means (or classifier means), ue. The Gram matrix G =Il un 117- 4e || of class means (where |||| is the Frobenius norm, u. contains columnwise class means) contains, in its off-diagonal elements, the angles between each pair of class mean vectors. Maximum cosine distance between C' classes is -1/(C'- 1) (assumiin that the dimension of the embedding space is high enough to embed all vectors at equal angles from each other). We thus add this quantity to each element of the mutual coherence matrix, since all elements will then go to zero as we converge to equiangularity. Finally, we assign zero to the diagonal, as we do not need the distance of vectors with themselves, and we have
G + Coos - diog + Cmaz 1l
É E.Amaans
C * (C 1)
where G the Gram matrix of normalized class means, Cooe the matrix whose elements are set C1 - 1), and diag retrieves the diagonal of matrix. We take the L1-norm to map this quantity a single number, and then normalize by the number of pairwise combinations.
The self-duality of class means and classifiers, NC3, is measured the square of L2 normalized differences between classifier and class means,
NC, =1 WT - M
(8)
where W is the matrix of last-layer classifier weights, M is the matrix of class means, F denotes matrix with unit normalized means over columns, and || 1/ denotes the square of the T.2-norm
Finally, Nearest Class Center classification, NC4, is measured as the proportion of training set samples that are misclassified when a simple decision rule based on the distance of the feature space activations to the nearest class is used,
NC4 argmine ue ll2
(9)
where z is the feature space vector for given input.
4 Experiments
4.1 Faster and More Robust OoD Results
Our results in Table [ demonstrate that L2 normalization over feature space produces results that are competitive with or exceed those obtained using the DDU benchmark, and are obtained in less training time for ResNet18 and ResNet50 models. For ResNet18, our mean Area Under the Receiver-Operatee Curve (AUROC) scores exceed those of the bascline in only 60 epochs (17% of DDU bascline training time), with only .008 reduction in classification accuracy. For ResNet50, achieve higher mean accuracy than the
8

---

Published in Transactions on Machine Learning Research (12/2022)
Equiangularity of Class Means vs AUROC, Epochs 0-350
Equiangularity of Class Means vs AUROC, Epochs 150-350
0.9
0.94
0.8
0.92
0.7
AUROC
0.90
0.89
0.84
0.4
0.6 Equiangularity Means
0.09
0.08
Equiangularity Class 0.06
0.07
Figure 3: Equiangularity of class means (NC2) vs. AUROC scores during training for 15 ResNet18 seeds trained for 350 epochs (no L2 normalization). Each point is the score of individual model at particular epoch, every lenth epoch is captured. Colors are segmented by training epochs: blue 0-40, green 50-140, orange 150-190, and red 190-350. The Pearson R coefficient is -0.894 (NC is decreasing as AUROC increases). Right: only epochs 150-350 are shown (individual models tend to appear as clusters). Too much NC can have a slightly detrimental effect (less than 0.05 points AUROC on average) as we would expect, but there is very little risk in overtraining models, even without T,2 normalization.
the baseline in only 100 epochs (29% of DDU bascline training time), while mean AUROC is lower by only point.
Most notably, the lower bounds of OoD detection performance are significantly improved on both models (Table [] For Resnet18, improve the AUROC baseline by .047, and on ResNet50 improve by .024. These effects are even more pronounced when compared with the same nunber of training epochs in the absence of L2 normalization. Note that while AUROC and classification scores might improve with more training, our scope here is to emphasize that a substantial reduction in compute time is possible with this technique. We achieve substantially better worst case OoD performance with only a fraction of the training time stated in the DDU benchmark. Notably, the variability of AUROC performance across models trained from different initializations is also heavily decreased with L2.
An ablation study shows that mean accuracy, minimum accuracy, and AUROC are all improved across each of the pairwise 1.2 comparisons (Table 2 Furthermore, the worst performing models (min and mean AUROC) using L2 normalization still exceed the best performing models trained without L2 normalization across the same metrics.
We also assess how standard softmax scores perform when used as OoD decision scores. These results are detailed in Table in Appendix A.2. Softmax scores perform worse in all cases, with the exception of the ResNet50, No L2 350 epoch model.
Finally, note that large amount of NC results in a feature space that contains most information along h = numclasses dimensions, small subset of the total size of the feature space (c.g. B212 in ResNet18). This raises the question of whether a GMM could simply be fit over logit space after substantial NC, since logit space in R* We provide and discuss experimental results showing that GMMs are best fit to feature space Appendix A.3
4.2 Linking Neural Collapse with OoD Detection
Our intuition that L2 normalization over feature space would induce NC faster is empirically proven through- out our experiments. All aspects of NC are induced more rapidly, with nearly all measures showing greater NC in 60 epochs than the equivalent model counterparts trained for 350 epochs without L2 normalization (Table 3) We also note that in models without L2 normalization, NC is generally an order of magnitude less after 60 epochs than those with 1.2 normalization (Table 3).
9

---

Published in Transactions on Machine Learning Research (12/2022)
Mean Relative Change AUROC Control Group
Mean Relative Change AUROC Intervention
0.08
0.08
0.06
0.06
0.04
0.04
AUROC
0.02
0.02
0.00
0.00
-0.02
-0.02
Epoch
Epoch
Figure 4: NC loss intervention experiment. Mean relative change in AUROC when training from the 50th epoch of L2 ResNet18 over 15 unique model seeds for additional 20 epochs. Blue line is the mean over model seeds, blue shading is standard deviation. Teft: The control group improves with more training as expected, but the difference is small. Right: The intervention group mean AUROC score improves at a peak of 0.38 after two epochs, 4.75x more than the control group mean improvement of .008 over the same training period. The onset of NC has a clear and immediate impact on AUROC outcomes.
Figure shows the effect of training non-L2 normalized ResNet18s for 350 epochs on NC and AUROC scores. As expected, NC is strongly correlated with Ool) performance. We also note evidence that too much NC can slightly reduce performance from peak (Figure 3 Right).
4.2.1 Neural Collapse Training Intervention
To further investigate the connection between NC and OoD performance, we use random seeds of ResNet18 models trained for 350 epochs with L2 normalization. We observe during training that after 50 epochs, accuracy and AUROC scores both approach noisy plateaus. We use this observation to create two groups, control and an intervention. All models within these groups are initialized using the 50th epoch of their respective seed. Both groups then begin training at an order of magnitude lower loss to encourage conver- gence, and both groups step down the loss after 10 epochs to control for the possible effect of learning rate. The control group models continue training using the standard cross entropy loss. The intervention group is trained using the differentiable NC metrics listed in Equation
As shown in Figure 7 (Appendix A.5 models in the intervention group have substantially greater amounts of NC, expected. CE loss and classification accuracy remain relatively stable in the intervention models, owing to the fact that CE is excluded from the loss function. The intervention group is thus disentangled from confounding influence from the learning rate or the CE objective. However, in Figure 4 we see that AUROC is substantially different between the intervention and control groups.
Models perform better under NC and these improved scores arrive with less training: after only epochs. Intervention models improve AUROC an average of 0.038 points from intervention onset, 4.75x the average improvement of .008 from control models during this time. Even if allowed to train for the full 20 epochs, control models improve to maximum average of .077
4.2.2 Investigating Collapse
Figure 4 shows that too much training directly with NC loss hinders performance. This aligns with our intuition that too much NC can adversely effect the bi-Lipschitz constraint: extracted OoD features are not sufficiently differentiated from ID features and get collapsed into class clusters.
10

---

Published in Transactions on Machine Learning Research (12/2022)
Mean False Positives Before and After NC Loss Intervention
Mean Relative Change False Positives with Overtraining
8000
125
7000
100
75
6000
50
5000
25
4000
-25
-50
10 30203004550111 5253 58
50 60
70
80
100
110
120
130
140
Epoch
Epoch
Neural Collapse induced by NC loss leads to false positive images collapsing into class means quickly, but this effect is heavily mitigated when NC is induced under CE loss. We know OoD images are collapsed into class means because the variability of these means is decreasing as training continues and as false positives increase. The blue line is the mean over 13 model seeds, blue shading standard deviation. Left: Training on NC loss directly first decreases false positives (see also Figure 4 but too much NC starts collapse OoD and ID features together. Within only 20 epochs, this results over 500 more false positives. Right: A similar but far less pronounced trend is found
Figure 5:
overtrain our ResNet18 L2 normalized model from 60 to 150 epochs. In this case, false positives only increase by about 50: order of magnitude less OoD collapse over five times much training. L2 normalization thus allows models to be trained to convergence via standard loss and accuracy metrics without the need to monitor or tune NC.
To examine this, we measure the false positive detection rate before and during the NC intervention (Figure 5 As there are thousand ID test images with CIFAR10, false positives are defined as OoD images that have confidence scores the top ten thousand of all scores. As expected, the false positive rate drops steadily up to and shortly beyond intervention onset, but then begins to increase again. Since we know that variability is collapsing (Figure 7 as the false positive rate increases, we know that OoD images are getting higher scores due to their collapse into class means along with ID images.
To see if a similar effect occurs under NC from CE loss, we over-train our L2 normalized ResNet 18 model from epochs through to 150 epochs. Figure shows that the false positive rate remains stable for approximately epochs before increasing nearly monotonically for the remainder of training. However, effect is an order of magnitude smaller than when we use NC loss. Instead of an extra 500 (on average) false positives within 20 epochs, the L2 case only increases by average of 54 false positives after extra 90 epochs of training. When over-training our no I.2 baseline from 350 to 500 epochs, false positives increase by a maximum average of 20.
this case the
Using L2 normalization on feature space is quite stable, and peak or near-peak results can be obtained without much fine-tuning of training length. However, we do note that there trade off between clas- sification accuracy and OoD detection performance. When substantially overtraining to 350 epochs with L2 normalization, accuracy scores improve slightly over the no L2 cases, but OoD performance takes significant penalty (see Table 6 Appendix A..).. We thus note that our method is best used with short training schedules. Highest accuracy on validation can still be used to guide training, but shouldn't be used to determine optimal OoD performance.
wver-training directly with NC loss can have a large detrimental effect, despite it resulting in similar NC measurements to the counterpart CE loss cases. is not optimal to simply induce NC on a pre-convergod model via NC loss, although doing so can significantly boost AUROC scores. This suggests that CE loss, well it's potential interactions with spectral norm and leaky ReLUs plays important role in conditioning
11

---

Published in Transactions on Machine Learning Research (12/2022)
feature space with optimal bi-Lipschitz constraint. The benefit of this, however, is that our method is stable with respect to training-there is no need to monitor NC measurements in order to arrive at or near the peak of OoD performance or reap the benefits of much improved worst-cast performance.
Finally, we observe that while NC is directly correlated with improved OoD performance as explained above, per model amounts of NC are not directly correlated with peak AUROC. In other words, although optimal OoD detection happens within similar range of NC measurements, specific amounts of NC cannot be used to precisely predict peak AUROC scores for given model. Again, this harmonizes with our intuition that NC is only of several factors acting to condition feature space for OoD detection. These factors are all susceptible to the variability of training dynamics, well as situational dependencies such architectures and datasets.
5 Conclusion and Future Work
We propose a simple, one- one-line-of-code modification of the Deep Deterministic Uncertainty benchmark that provides superior OoD detection and classification accuracy results in a fraction of the training time. We also establish that L2 normalization induces NC faster than regular training, and that NC is linked to OoD detection performance under the DDU method. Although we do not suggest that NC is the sole explanation for OoD performance, we do expect that its simple structure can provide insight into the complex and poorly understood behaviour of uncertainty in deep neural networks. We believe that this connection is compelling area of future research into uncertainly and robustness in DNNs.
12

---

Published in Transactions on Machine Learning Research (12/2022)
References
Jiefeng Chen, Yixuan Li, Xi Wu, Yingyu Liang, and Somesh Jha. Robust out-of-distribution detection in neural networks. CoRR, abs/2003.09711, 2020. URL
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pages 1050 1059. PMLR, 2016.
Jarrod Haas and Bernhard Rabus. Uncertainty estimation for deep learning- based segmentation of roads in synthetic aperture radar imagery. Remote Sensing, 13(8), 2021. ISSN 2072-4292. doi: 10.3390//s13081447. URL htttps://ww..dtt wwt.:/ddpi.ccoo////mmmmmm cotm/2002-499/133////////1
Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem. CoRR, abs/ 1812.0570, 2018. URL http//xxvvvorr./..a....
05720
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distributiio examples in neural nelworks. CoRR, abs/1610. abs/1610.02136, 2016. URL
02136
Maximilian Henne, Adrian Schwaiger, Karsten Roscher, and Gereon Weiss. Benchmarking uncertainly estimation methods for deep learning with safety-related metrics. In SafeA1@ AAAI, pages 83 90, 2020.
Ven-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira, Generalized ODIN: detecting out-of-distributio image without learning from out-of-distributio data. CoRR, abs//2022... 20022.1202 2020. URL httthttts:///r/
11297
Wenlong Ji, Yiping Lu, Yiliang Zhang, Zhun Deng, and Weijie J Su. An unconstrained layer-peeled per- spective on neural collapse. arXiv preprint 1rx::22100002796, 2021.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncer- tainty estimation using deep ensembles. Advances in neural information processing systems, 30, 2017.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out- of-distribution samples and adversarial attacks. Advances in neural information processing systems, 31, 2018.
Shiyu Liang, Yixuan Li, and R. Srikant. Principled dctection of out-of-distribution examples in neural networks. CoRR, abs/1700 Abs/1706.02690, 2017. URL
02690
Jeremiah Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bodrax Weiss, and Balaji Lakshminarayanan. Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. Advances in Neural Information Processing Systems, 33:7498-75) 7512, 2020.
Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. Sphereface: Deep hypersphere embedding for face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 212-220, 2017.
Jianfeng Lu and Stefan Steinerberger. Neural collapse with cross-cntropy loss.
arXie preprint
arXiv:2012.08 084665, 2020.
Dustin G Mixon, Hans Parshall, and Jianzong Pi. Neural collapse with unconstrained features. Sampling Theory, Signal Processing, and Data Analysis, 20(2):1 13, 2022.
Jishnu Mukhoti, Andreas Kirsch, Joost. van Amersfoort, Philip H. S. Torr, and Yarin Gal. Deterministic neural networks with appropriate inductive biases capture epistemic and alcatoric uncertainty. CoRR, abs/2102.11582, 2021. URL
Yaniv Ovadia, Emily Fortig, Jie Ren, Zachary Nado, David Sculley, Scbastian Nowozin, Joshma Dillon, Balaji Lakshminarayanan, and Jasper Snock. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. Advances in neural information processing systems, 32, 2019.
13

---

Published in Transactions on Machine Learning Research (12/2022)
Vardan Papyan, XY Han, and David L Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences, 117(40):246522246) 2020.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- performance deep learning library. In Advances in Neurol Information Processing Systems 32, pages 8024-8035. Curran Associates, Inc., 2019. URL http::/papers: http:://pppprss.uuripp..... wwwurips.ccpppppppe pepptordlaarmmmpraiiie--e-te--/e aerprpeeeeeppe---appprrrreeeeeee
lwwrningg-ibraaaat pdf
Stephan Rabanser, Stephan Günnerann, and Zachary C. Lipton. Failing loudly: An empirical study of methods for detecting dataset shift, 2018. URL
Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, and Hervé Jégou. Spreading vectors for similarity search. arXiv preprint arXic:1806. 03198, 2018.
Lewis Smith, Joost van Amersfoort, Haiwen Huang, Stephen J. Roberts, and Yarin Gal. Can convolutional resnets approximately preserve imput distances? A frequency analysis perspective. CoRR, hatss///00022000 2021. URL
02469
Joost Van Amersfoort, Lewis Smith, Yee Whye Teh, and Yarin Gal. Uncertainty estimation using a single deep deterministic neural network. In International conference on machine learning, pages 9690-9700. PMI.R, 2020.
Joost van Amersfoort, Lewis Smith, Andrew Jesson, Oscar Key, and Yarin Gal. On feature collapse and deep kernel learning for single forward pass uncertainty. arXiv preprint 0rXiv:2102.114000 2021.
Jian Zheng, Jingyi Li, Cong Liu, Jianfeng Wang, Jiang Li, and Hongling Liu. Anomaly detection for high- dimensional space using deep hypersphere fused with probability approach. Complez & Intelligent Systems, pages 1-16, 2022.
Chang Zhou, Lai Man Po, and Weifeng Ou. Angular deep supervised vector quantization for image retricval. IEEE Transaclions on Neural Networks and Learning Systems, 2020.
Zhihui Zhu, Tianyu Ding, Jinxin Zhou, Xiao Li, Chong You, Jeremias Sulam, and Qing Qu geomet- ric analysis of neural collapse with unconstrained features. Advances in Neural Information Processing Systems, 14:28200298844 2021.
14

---

Published in Transactions on Machine Learning Research (12/2022)
A Appendix
A.1 Training Details
All models (except those explicitly noted in the ablation study) use spectral normalization, leaky ReLUs and Global Average Pooling (GAP), as these produce the strongest baselines. Each experiment was conducted with fifteen randomly initialized model parameter sets; no fixed seeds were used at any time for initialization. We set the batch size to 1024 for all training runs, except the NC intervention models, which were more stable when training with a batch size of 2048. All training was conducted on four NVIDIA V100 GPUs in PyTorch 1.10.1 Paszke et al. (2019).
Stochastic gradient descent (SGD) with an initial learning rate of 1e-! was used as the optimizer for all experiments. We used learning rate schedule that decreased by one order of magnitude at 150 and 250 epochs for the 350 epoch models, as per the DDU benchmark. We adjust the learning rate at 75 and 90 for the 100 epoch ResNet50 modcls, and at 40 and 50 for the 60 epoch ResNet18 models. Models were trained on the standard CIFAR-10 training data set with a validation size of 10% created with a fixed random seed.
A.2 Effect of L2 Normalization on Softmax Scores for OoD Detection
AUROC (CMMM Over Foature Spine]
SVHN
CIFAR100
Timy ImaggeNet
88
No
No L2
L22 80
No L22 550
12 350
550
Nu 1.2
1.2
Mia
0.577
0,879
0.545
0.924
0,561
0,570
0.796
0.878
0
0,885
0.898
Max
0.909
0.911
0.563
0.961
0.885
0,579
0.855
0LA86
0.909
03002
0877
0
Moon
0.055-00018
090-00011
0.91250035
0.804-0.000
0.870200000
0,87680000
0,804:00014
0.881:00012
0,8080000
0,80200004
0141140005
0. 0.90458004
ResNet50
No L2 350
L2 350
No L2 100
L3 100
No L2 850
L2 350
No L2 100
L2 100
No L2 350
L2 350
No L2 100
L2 100
Min
0,369
0,817
0.892
0,918
0.501
0.806
0.909
0.9158000
(a) The variability of AUROC scores is substantially reduced under L2 normalization of feature space. With much less training, worst case OoD performance across model seeds improves substantially over the baseline, and mean performance improves or is competitive in all cases.
AUROC (Softmax)
SWHN
CIFARIC0
Tiny ImageNet
ReaNet18
No 350
No L2
60
No 350
No L2
L2
350
No L2
L2
Min 0.893
0.791
0.851
0.816
0.841
0.862
0.842
0.857
0.869
Max
0.938
0.879
0.931
0.858
0.869
0.880
0.570
0.887
0.885
Mean
0.917
0.809
0.888+0.026
0.85240.004
0.000
0.872-0.005
0.860
0.670++0.010
0.878+00.05
ReaNet50
No L2 350
No L2 100
L2 100
No L2 350
No L2 100
L2 100
No L2 350
No L2 100
L2 100
Min
0.950
0.664
0.834
0.877
0.841
0.833
0.896
0.853
0.844
Max
0.975
0.948
0.952
0.882
0.877
0.888
D.903
0.906
0.902
Mean
0.96120.007
0,84340.07
073
0,910
0.879=0.002
0.800=0.010
0,839+0.016
0.899=0.002
0882200016
0,873=0.017
(b) Softmax performs worse cases versus GMMs on L2 normalized feature space with a singular exception: SVHN on ResNet 50.
Table 4: OoD detection results using (a) log probabilities from GMM fitted over feature space and (b) soft- max scores. ResNet18 and Res.et550 models were used, 15 seeds per experiment, trained on CIFAR1O, with SVHN, CIFAR100 and Tiny ImageNet test sets used as OoD data. For all models, we indicate whether L2 normalization over feature space was used (L2/No L,2) and how many training epochs occurred (60/100/350)) and compare against baseline (No L2 350). There is no clear pattern of behaviour when using softmax scores for OoD detection, but using GMMs provides superior results.
A.3 Fitting GMMs on Logit Space
A large amount of NC results in a feature space that contains most class information along k = numclasses dimensions, a small subset of the total size of the feature space (e.g. R512 in ResNet18). This raises the question of whether a GMM could simply be fit over logit space, since it is in R
Table shows the results of experiments with GMMs fit over logit space. This approach performs worse than GMMs fit over feature space in all cases. Intuitively, this makes sense: even under perfect NC, we
15

---

Published in Transactions on Machine Learning Research (12/2022)
AUROC
SVHN
CIFARTO0
Tiny ImageNet
ResNec18
350
No L2
60
No L2 350
60
L2 60
No L2 350
No L2
60
Min
0.867
0.782
0.767
0.848
0.739
Max
0.932
0.883
0.928
0.807
0.801
0.871
0.815
0,805
0.890
Mean
0.897=0.017
0.816#0.013
0.885=0.021
0.797=0.007
0.786
0.860=0.000
0. 0.798200014
0.780=0.011
0.879=20.008
ResNet50
No L2
No L2
L.
Min
0.908
0.682
0.838
0.827
0.761
0.826
0.813
0.770
0.816
Max
0.947
0.909
0.927
0.842
0.817
0.872
0.874
0.837
0.883
Mean
0.920:000011
0.831=0.001
0.899=0.023
0.831#0.0044
0.796
0.850=0.013
0.806+0.007
0.811=0.020
0.868=#0.012
Table 5: OoD detection results for ResNet18 and ResNet50 models using log probabilities taken from GMMs
fitted over logil. space instead of feature space (same experimental setup Table 4 This approach performs worse in all cases versus using GMMs L2 normalized feature space (see Table
would expect OoD inputs to increase the variability of class clusters in arbitrary dimensions of feature space. A Singular Value Decomposition (SVD) over feature space supports our intuitions. In Figure G we show the SVD all training embeddings for CIFARIO, along with the singular values for the test set and SVHN OoD test set projected onto the the same basis used for the training singular values. As we would expect, the first 10 singular values contain nearly all information. However, the latter 502 singular values contain significantly more information in the OoD case. This information is critical to identifying OoD examples in feature space and, due to dimensionality reduction, is severely reduced in logit space.
First 200 Singular Values Train, Test and OoD data
Tain
000
50
100
125
150
175
200
Singular Value
Figure The first 200 singular values for CIFARIO train and test sets, as well as the SVHN OoD set. As expected, singular value magnitudes fall off drastically after the first 10. Singular value magnitudes for OoD examples remain higher after 10, indicating greater deviation from the Simplex ETF structure. This information is exploited by the GMM to identify OoD examples, and is much less prevalent in the heavily reduced dimensionality of logit space.
16

---

Published in Transactions on Machine Learning Research (12/2022)
A.4 Overtraining with L2 Normalization
Table 6 shows the results of overtraining with L2 Normalization (L2 350). While there is not substantial penalty for overtraining by 10 to 100 epochs (Figure 5 Right), training for the full 350 epochs (as with the DDU bascline) starts to roduce OoD performance by few percentage points. We note that there is a tradeoff with accuracy, which does increase when overtraining to 350 epochs.
AUROC
SVHN
CIFARIIG
Tlay ImuggNNt
ResNet18
No
No
No
350
No
No L2
0M0--0011
080220005
0.838 0.000
0.870
0.87
0.88
0.904:0004
ResNet50
No L2 350
L2 360
No L2 100
12 100
No L2 350
La 350
No 100
12 100
No L2 350
L2 360
No L2 100
L2 100
Min
01MI5
0,000
0.51
U.714
0,892
01852
0.912
0.806
0:637+0022
0.504-0000
0.6
(a) OoD performance begins to decay substantially when models overtrained with L2 normalization, in line with our discussion in Section 4.2.2.
Accuracy
ResNot18
No L. 350
L,2 350
No 1.2 60
1.2 60
ResNet50
No 1.2 350
L2 350
No 1.2 100
1,2 100
Min
0.928
0.938
0.881
0.924
Min
0.330
0.947
0.856
0.937
Mean
0.935
0.938:00002
0.898200000
0.927:0.001
Mean
0.930-10.066
0.949100002
0,91620.008
0.941=20.002
(b) Accuracy increases slightly when substantially overtraining with L,2 normalization, but OoD performance drops. For ResNet50, higher accuracy is achieved in only 100 epochs compared with the baseline.
Table 6: OoD detection (a) and classification accuracy results (b) for ResNet18 and ResNet50 models, 15 seeds per experiment, trained on CIFARIO, with SVHN, CIFAR100 and Tiny ImageNet test sets used as OoD data. For all models, we indicate whether 1,2 normalization over feature space was used (T.2/No L22 and how many training epochs occurred (60/100/ (350), and compare against baseline (No L2 350).
17

---

Published in Transactions on Machine Learning Research (12/2022)
A.5
Neural Collapse Measurements for NC Loss Intervention
NC1: Variability Collapse
NC 2: EN Means
NC 2: EN Classifier
325
0.10
2.0
0.08
225
0.08
0.04
0.02
150
Classifier
Self-duality
0.18
0.16
011
0.14
010
0.12
0.09
0.10
0.08
0.08
0.18
0.0
0.06
0.16
0.06
0.04
55
60
85
55
60
70
NC Nearest Class Center
CE Loss
Training Accuracy
016
0.05
0.188
014
0.80
0.016
0.75
0.700
0.022
0.955
0.10
0.08
0.60
0.00
0.06
0.55
Figure 7: Mean (solid lines) ResNet18 NC measurements from the NC loss (sec Equation 3) experiment over 15 model seeds, along with CE loss and classification accuracy. Shading around the solid line shows standard deviation. Blue is the control group, orange is the intervention group. The intervention started at epoch 50, and proceeded for 20 epochs. Over the same training period, the intervention group has substantially more NC, while CE loss and training set classification accuracy are relatively unchanged. This indicates that NC and CE loss effects were controlled successfully.
18

---

Published in Transactions on Machine Learning Research (12/2022)
A.6 Additional Figures
Training models for low cross-entropy or high accuracy does not nocessarily structure embodding space in an optimal way for OoD detection. Left: MNIST training features in a 2-dimensional feature space, The only structure explicitly required by cross-entropy loss is that features are linearly separable. Right: CIFAR10 training features in 2-dimensional feature space. The network finds it more difficult to make these more complex features linearly separable, but even with spectral normalization, the structure is still arbitrary. In both cases, the within-class variance could be reduced substantially, preserving appropriate notions of sensitivity and smoothness while not affecting classification. These tighter class means could be separated to allow OoD features more inter-class distance to fall away from fitted GMM modes.
Figure 8: Building intuition about embedding spaces:
19


Reviews:
[{"review_id": "5k8QA4VnF1", "reviewer": "Reviewer_WGXv", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The authors of the paper conduct careful experiments to demonstrate the effect of L2 regularization on the neural collapse phenomenon. \n- The proposed method is simple yet seems to be effective. \n- The proposed method seems to be well-motivated. \n\nWeaknesses:\n- I find the description of the proposed L2 regularization to be very confusing. What is exactly the formula of the L2 regularization? Is Equation 9 of the paper the regularizer used? What is the hyper-parameter associated with this additional term? Also, why would L2 regularization over feature space \"constrain the feature vector to a point on the surface of a hypersphere\"? I feel like there is a lot of important but used explanation missing regarding the proposed method. \n- I still don't see a causal connection between the neural collapse and improved OoD performance. Why is NC helpful for the task of OoD detection? While the authors of the paper provide experimental evidence for this, it would be great if there is some additional explanation/derivation from a theoretical standpoint. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper tackles an important problem of out-of-distribution detection with neural networks. The authors of the paper build upon the previously proposed method of \"deep deterministic uncertainty\", and demonstrate that the use of $L_2$ regularization over the feature space can serve as a simple yet effective method to improve the method, both in terms of the out-of-distribution detection performance and in terms of the training time needed. The authors of the paper demonstrate that such improvement can be attributed to the fact that the L2 regularization induces early neural collapse of the model. Such neural collapse is experimentally demonstrated to be helpful for OoD detection. ", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "First of all, it would be great if the authors of the paper can address my concerns listed above about the method. In addition, I also find the structure of the method section to be a little bit confusing. It might be better to first mention the proposed L2 regularization first before talking about experimentation etc.. ", "broader_impact_concerns": "I don't see any concerns regarding this. ", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "yKwu0A1Mei", "reviewer": "Reviewer_jLid", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n1. The observation that L2 normalization has a positive effect on OoD performance is interesting and would be easily adopted.\n\n2. Correlation between NC and OoD performance is interesting but warrants further investigation.\n\n## Weaknesses\n\n1. No clear justification or explanations. For instance, there is no clear intuition/explanation provided for why L2 normalization yields NC. Also, it is not clear why NC would lead to better OoD improvements. I think the empirical observations would be strengthened if proper intuitions and/or explanations are provided. Specifically, NC --> OoD connection is problematic without any justifications as it is observed in the last para before sec. 5 that \"more NC is not always better\" in contrast to the rest of the paper.\n\n2. Writing lacks clarity. In many places, the writing is not clear and lacks clear explanations. \na) In Sec 2.1, the function definition should be a mapping from a set to another set, X \\in X_in doesn't make sense (should it be X\\subset X_in?), and Eq.1 doesn't make sense (should it be P(y|x \\in X_in \\cup X_out)?). Also please introduce all the symbols in all equations.\nb) In Sec. 2.2, what is a bi-lipschitz constraint? Please cite.\nc) NC1 would be zero if the cross-entropy is zero right? Please clarify this.\nd) what is the form mutual coherence matrix?\ne) For example, what is meant by \"global means of classes\" (u_G)?\nf) legend for fig. 3\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents an observation that L2 normalization of features improves OoD detection performance and/or enables the networks to reach better OoD performance in a short time. Additionally, they empirically show a correlation between neural collapse (NC) and OoD performance. Experiments are performed on cifar, svhn and tinyimagenet datasets with resnet architectures.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please address the writing comments and provide intuitions and/or justifications for 1) why L2 normalization might improve NC and OoD performance and 2) when better NC would lead to better OoD performance. ", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mLXXy4xdec", "reviewer": "Reviewer_11H5", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- Interesting problem\n\nWeaknesses:\n- Problem definition in section 2.1 is not very clear.  What does it mean in the sentence before Eq 1 \"For OOD detection, we can conceive of the model as learning a data generating distribution $p(y|x)$...\"?  Should it be $p(x, y)$? It also isn't clear to me how this formulation is useful for OOD detection, and Eq 1 is not referenced later in the paper.  It also isn't clear to me how this is related to the sentence that follows: \"Under this regime, we desire that the model learns a density over feature space...\"  I think it would also help to clarify the definition of OOD too, right now the text just says that $X_{in}$ contain inputs drawn from the same distribution as the training data and $X_{out}$.  I think it should be clear that the distributions that $X_{out}$ and $X_{in}$ are sampled from have different supports.\n- Missing some related works for OOD detection: ie)\n Hendrycks, Dan, and Kevin Gimpel. \"A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks.\" (2016).\nLiang, Shiyu, Yixuan Li, and R. Srikant. \"Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks.\" International Conference on Learning Representations. 2018.\n- Paper organization can be improved.  Section 2 is a very long background section and immediately jumps into Section 3 which starts off with experimental setup and mainly describes **how** the authors do certain things **but not why**.  It would be more helpful to shorten the background section and use it to motivate what the authors are trying to do and describe how the experiments answer those questions, and then transition into Section 3.  I also think that a large portion (specifically details on how models are trained) of Section 3.1 can be moved to the Appendix.  The main text should focus on describing which architectures, datasets, and baselines you use so that the readers can understand the experimental results and figures.\n- Not sure what is meant by \"evidence of a tightening lower bound\" in section 4.2, what lower bound?\n- Legend in figure 4 isn't particularly useful.  Overall figures 3 and 4 have a lot going on because 15 seeds are used, I'm not exactly sure what I should be looking at.  Maybe the authors can just plot the mean and variances?\n- Should include results for additional baselines for OOD: authors only compare to no L2 regularization", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors look at the problem of OOD detection and apply L2 regularization over the feature space and demonstrate some improvements in speedup and AUROC when combining with DDU.  The authors also investigate the relationship between neural collapse and OOD detection performance.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Critical adjustments:\n- I really think that major revisions should be made to the writing of the paper.  Outside of reading the introduction, I really had no idea what the authors contributions are throughout reading the paper.  There's also a lot of extraneous details about experimental setup and additional background info that is neither used as motivation nor used to explain the methodology.  The paper severely lacks a section that explains what the authors introduce (L2 regularization on the feature space) and motivate why the authors think this would improve performance.  Similarly there isn't anything motivating the study of neural collapse and connections with OOD performance, the authors just define neural collapse and move on to explain experimental setup.\n- More baselines for OOD, more datasets for NC experiments, can also look at metrics like AUPR\n- Some statements such as \"evidence of a tightening lower bound\" should be explained\n- the final conclusion of the study of NC and OOD performance ends with the conclusion that more NC sometimes helps OOD performance and that sometimes it doesn't but it isn't clear when this occurs.  The authors leave this as an open question, but given that part of the contributions is a study of the relationship between NC and OOD performance, I think there should be more work done to investigate this.\n\nMinor comments:\n- Could the authors format some of the citations to use parenthetical citations (\\citep{}) instead of in-text citations (\\citet{}) (specifically where I should not be reading the authors name as part of the sentence)?  That would greatly improve the legibility of the paper\n- Would also help to give timing results on how long training takes for the L2 regularized model vs DDU without L2 regularization", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper tackles an important topic (OoD), proposes a simple and promising looking method (L2 normalization), and establishes an interesting link with Neural Collapse (NC). We thank the authors for their engagement in the rebuttal process, and note that the paper was substantially improved during the process.

However, I think the current version requires some minor revisions before acceptance. In particular, while a link with NC is established, this relationship between OoD and NC is unclear, and whether this plays a major role in L2's improved OoD performance is unclear. I would like to request the following minor revisions before acceptance:

1) The title currently "Inducing Early Neural Collapse in Deep Neural Networks for Improved Out-of-Distribution Detection". Please soften the implication in the title that inducing early collapse is the main cause of improved OoD. The evidence shows the L2 improves OoD, and OoD and NC are related, but there is not sufficient support for this title.

2) Soften contribution (4). The paper can claim to uncover an interesting link between early NC and improved OoD detection. But I think the claim "NC encourages OoD detection under our proposed modification to the DDU method" is slightly too strong.

3) Please add an experiment that shows the performance of L2 with a large number of epochs (the same number as used in the baseline in Table 1). Even if the results are not good, this would help practitioners understand that this method is most beneficial in conjunction with short schedules.

The following is optional, and not required for acceptance, but I think would substantially strengthen the paper:

4) Add an experiment using L2, training on one dataset other than CIFAR10, ideally a larger dataset (e.g. ImageNet), where training cost is more likely to be a bottleneck, and training with a short schedule would be most beneficial in practice.

A couple of trivial changes needed:

* After the refactor, some concepts (e.g. NC2 on P5) are referenced before definition.
* There are still instances of "L2 regularization" instead of "L2 normalization" which caused confusion in the reviews.
* I saw one typo (standad->standard P2)
Decision: Accept (poster)