Post-Hoc Reversal: Are We Selecting Models Prematurely?
Authors: Rishabh Ranjan
Saurabh Garg
Mrigank Raman
Carlos Guestrin
Zachary Chase Lipton
Venue: NeurIPS 2024 poster (2024)
Keywords: temperature scaling
ensembling
stochastic weight averaging
noisy data
overfitting
double descent
early stopping
hyperparameter tuning
Primary area: evaluation
TL;DR: Post-hoc transforms like temperature scaling and ensembling can reverse performance trends, with important implications for selecting base models.

Abstract: Trained models are often composed with post-hoc transforms such as temperature scaling (TS), ensembling and stochastic weight averaging (SWA) to improve performance, robustness, uncertainty estimation, etc. However, such transforms are typically applied only after the base models have already been finalized by standard means. In this paper, we challenge this practice with an extensive empirical study. In particular, we demonstrate a phenomenon that we call post-hoc reversal, where performance trends are reversed after applying post-hoc transforms. This phenomenon is especially prominent in high-noise settings. For example, while base models overfit badly early in training, both ensembling and SWA favor base models trained for more epochs. Post-hoc reversal can also prevent the appearance of double descent and mitigate mismatches between test loss and test error seen in base models. Preliminary analyses suggest that these transforms induce reversal by suppressing the influence of mislabeled examples, exploiting differences in their learning dynamics from those of clean examples. Based on our findings, we propose post-hoc selection, a simple technique whereby post-hoc metrics inform model development decisions such as early stopping, checkpointing, and broader hyperparameter choices. Our experiments span real-world vision, language, tabular and graph datasets. On an LLM instruction tuning dataset, post-hoc selection results in >1.5x MMLU improvement compared to naive selection.

Post-Hoc Reversal: Are We Selecting Models Prematurely?
Rishabh Ranjan Saurabh Garg", Mrigank Raman', Carlos Guestrin 1.3 Zachary Lipton 'Stanford University, "Carnegie Mellon University, "Chan Zuckerberg Biohub (ranjanr, goetaansgggtetrtssstttttttoo edu, (sgarg2,mris mrigankr, waaggaak,,liipptee 211ptton) edu
Abstract
Trained models are often composed with post-hoc transforms such as temperature scaling (TS), ensembling and stochastic weight averaging (SWA) to improve performance, robustness, uncertainty estimation, etc. However, such transforms are typically applied only after the base models have already been finalized by standard means. In this paper, we challenge this practice with an extensive empirical study. In particular, where performance trends are reversed after applying post-hoc transforms. This phenomenon is especially prominent in high-noise settings. For example, while base models overfit badly early in training, both ensembling and SWA favor base models trained for of double descent and mitigate mismatches between test loss and test error seen in base models. Preliminary analyses suggest that these transforms induce reversal by suppressing the influence of mislabeled examples, exploiting differences in their learning dynamics from those of clean examples. Based on our findings, we propose post-hoc selection, a simple technique whereby post-hoc metrics inform model development decisions such as early stopping, checkpointing, and broader hyperparameter choices. Our experiments span real-world vision, language, tabular and graph datasets. an L.LM instruction tuning dataset, post-hoc selection results in >> 1.5x MMLU improvement compared to naive selection
demonstrate phenomenon that we call post-hoc reversal,
epochs. Post-hoc reversal also prevent the appearance
Introduction
Many widely used techniques in deep learning operate on trained models; we refer to these as post-hoc transforms. Examples include temperature scaling (TS) [19], stochastic weight averaging (SWA) (8
and ensembling [39]. These techniques have shown promise for improving predictive performance, robustness, uncertainty estimation, -uttof-distribution generalization, and few-shot performance 6 39 56,660 84). Typically, the pre-training and post-hoc stages are isolated. The workflow is: (I) pick model architecture, training recipe, hyperparameters, etc. optimize for individual model performance; (2) train one or more models; (3) pick best-performing checkpoints; (4) apply post-hoc transforms. We refer to this procedure as naive selection.
In this paper, we demonstrate interesting drawbacks of naive selection. In a large-scale empirical study, uncover post-hoc reversal a phenomenon whereby post-hoc transforms reverse performance trends between models (Fig. T}. We demonstrate post-hoc reversal with respect to training epochs, model sizes, and other hyperparameters like learning rate schedules. We further establish that post- hoe reversal is a robust phenomenon by experimenting on real-world datasets across domains and modalities, with diverse model classes and training setups.
38th Conference on Neural Information Processing Systems (NeurIPS 2024).
"Undertaken in part as a visiting researcher at Careegie Mellon University. *Code is available at https://github. http://tthu..coo/rssshhhbbb
com/rishabh wwwchhahabhrrajjaaa ranjan/ hoc reversal

---

2.5
b2
Base curve (no transform)
Test Loss (rross-Entropy)
Test Error (%)
44
b2
curve ESWA+Ens+T
43
gain from transform under current practices
b1
additional gain from methodology
P1
Naive selection (current practice)
1:5
Post-hoc selection (our methodology)
P2 36
P2
18
Epochs
Epochs
Figure An illustration of the phenomenon of post-hoc reversal on the FMoW dataset: base performance at epoch t2 is worse than at epoch 1 (b₂ >> bj). but post-hoc performance is better (p2 P1). The current practice of naive selection considers base metrics pick models at epoch f1- Our proposed technique of post-hoc selection instead uses post-hoc metrics to pick models at epoch ig. resulting > 2x improvement over naive selection in both test loss and error. SWA+Ens+T'S refers to the post-hoc transform obtained by composing SWA, ensemble (Ens) and temperature scaling (TS). Base curves show mean of runs, models from which constitute the ensembles. Individual runs are shown in lighter colors. See Fig. 5 for more detailed curves this dataset.
Post-hoc reversal is most prominent on noisy datasets (Fig. 2 Other phenomena exacerbated by noise include catastrophic overfitting double descent [55] and loss-error mismatch [19]. While these phenomena pose challenges model development, post-hoc reversal suggests a path alleviate them. Noise can arise not only from labeling errors, but also from inherent uncertainty in the prediction task, such as in next token prediction [60]. Indeed, severe performance degradation has limited multi-epoch training of large language models (LLMs) 8T. Here too, post-hoc reversal reveals promising path for sustained performance improvements over longer training.
The core intuition for post-hoc reversal is that models continue to learn generalizable patterns from clean examples, even when spurious patterns learnt from mislabeled examples worsen the overall performance. Post-hoc transforms exploit differences in the learning dynamics of clean and mislabeled examples [42] to reinforce the influence of the former, while suppressing that of the latter. When strong enough, this effect leads to reversal. We show evidence for these intuitions in $
Based on findings, we propose post-hoc selection-a simple technique whereby base models are selected based on post-transform performance. The technique is practical as the transforms of interest can be cheaply incorporated into the validation phase of the training loop. Post-hoc selection significantly improves the performance of the transformed models, with > 2 improvements over naive selection some cases (Fig. 2). terms of absolute performance, post-hoc selection leads to > 3-point reduction in test егrоr over naive selection on a satellite imaging dataset (Fig. [ The reduction is even higher (> points) when using -uttof-disttibuttin (OOD) val/test splits for the same dataset. On an LLM instruction tuning dataset, under our procedure a composed transform of SWA, ensemble and TS gives 1.5x MMLU improvement over a naive application of the same transform prematurely selected models.
Reduction in Test Loss (Cross-Emt Entropy)
C-10-N
C-100-N
Reduction in Test Error
C-10-N
IC-100-N
Naive selection
(Absolute %6)
Post-hoc selection
0.2
0.1
Clean (0)
Aggre
Noisy (40)
Aggre
Randl Worst Clean Noisy
(9)
(17)
(40)
(0)
(0)
(9)
(17) (40)
(0) (40)
Label Set (Approx. Noise %))
Label Set (Approx. Noise %)
Figure 2: comparison of naive and post-hoc selection on label sets from CEFAR-100100-- (abbr.
C-10/100-N) for the SWA+TS transform. On noisy label sets, post-hoc selection is often > 2x better.

---

2 Related Work
slew of empirical works 17 55 57 58 have revealed both challenges and opportunities for improving the understanding and practice of deep learning. Our work expands this list with novel phenomenon lying together noisy data learning and post-hoc transforms. Orthogonal to our work, a number of training-stage strategies for noisy data have been proposed (see (69 for survey).
TS belongs to a family of calibration techniques 22 proposed with the goal of producing well- calibrated probabilities. Ensembling is foundational technique in machine learning, with simple variants routinely used in deep learning [3] [3]. SWA (28) is the culmination a line of work [18] 25] seeking cheaply approximate ensembling. Despite their prevalence, thorough understanding of best practices for wielding these techniques is lacking, especially in the context of noisy data. Our work fills this gap. For more detailed discussion related work, see App. A
Preliminaries and Background
We describe our learning setup in $ 3..1 with emphasis on noisy data, a key focus of this work. In 3.2 introduce the post-hoc transtorms we study.
3.1 Learning on Noisy Data
Setup. We consider multi-class classification with classes, input E X' and label y y
(1...., C}. Training, validation and test sets are drawn i.i.d. from the data distribution D. A classifier f: Q x X - C outputs the logit vector
fx; 0), given parameter vector 0 θ
Predicted probability of class k: is f = k x] oz)),, where a is the softmax function.
Noise. Data D is said to be clean if PD [y/ x] is one-hot for all x, i.e., PD|y x] 1yy v" (x))
for some labeling function y" X + y. Then, for any example input (+) in the dataset, the observed label is (+))) y² (x()). When Pp x] is not one-hot, 'D is said to be noisy and the observed label is only a stochastic sample y( ~ Pply x x= x(1 from the underlying conditional distribution. Noise can arise due (1) -eneretrmmiiimm the prediction target (2) insufficient information in the input context, and (3) annotation errors. See App. B. for illustrated examples.
Metrics. A metric M: RCC y + Th compares the predicted logits with the observed la- bel (y = M[f = [[xf))- 0), y)] denotes the metric computed over D given f and 9. We use two metrics (1) classification error, or simply error, with a (z, y) 1 (arg max; 2ạ * y} and (2) cross-entropy loss, or simply loss, with (Mlos y) The exponentiated loss, also called perplexity, is common in language modeling, where it is computed on per-token basis. A standard result states that loss is minimized and only if the ground truth conditional probability is recovered 20) See App. B.T for additional background.
log (z),
3.2 Post-Hoc Transforms in Machine Learning
Definition 1 (Post-Hoc Transform) A post-hoc transform T maps classifier f: 1 X' + 1o another classifier T o f: @K X y. for some K.
Temperature Scaling (TS). TS 19 involves scaling the logits with a temperature T E IR obtained
by optimizing the cross-entropy loss the validation set, with model parameters fixed (Eqn. Temperature scaling preserves error as it does not affect the predicted class. We use the torchcal 63 implementation, which optimizes the temperature GPU with Newton's method [15].
(Trs f) 0) (x; with arg min Miloss
[ (=
(I)
Ensembling. this method, predictions from an ensemble of classifiers are combined. In deep learning, simply averaging the temperature-scaled logits is effective (Eqn. 2 0 O obtained from multiple training runs with the same architecture and dataset, with stochasticity from mini-batch sampling and random initialization, if applicable.
.a....,x, 0x are
3

---

K
1 -
(Tems 2 (x; ....... 0k)
fx; 0x), with T = arg min M
(2)
7k
Stochastic Weight Averaging (SWA). SWA 28 involves averaging weights ......, 0 from the same training run (Eqn. 3). BatchNorm statistics are recomputed after averaging, if required. We pick checkpoints at epoch boundaries. Unlike Izmailov al. 28 (warmup) or modify the learning rate schedule
do not skip the initial epochs
(TswA o f) (x; 0I.......... .0.......)))) 0k)= f
x; K
e
Compositions. TS, ensembling and SWA can be readily composed. particular, we consider SWA+TS and SWA+Ens+7S, for single- and multi-model settings respectively. We denote them with Ts+T = TTs TSwa and TS+ETT = TTs o TEms o TswA (explicit forms in App. B.2).
Post-Hoc Reversal: Formalization and Empirical Study
To use post-hoc transforms, one must first select models to apply them to. Current practice is to select the best-performing model independent of post-hoc transforms, rationalized by an implicit monotonic- ity assumption "better-perfore shall see, this assumption is often violated in practice. We call such violations post-hoc reversal. In $ 4.1 formalize post-hoc reversal and discuss ways to detect it. In 4.2 we empirically study various kinds of post-hoc reversal with special practical relevance.
models result in better performance after transformation". As we
4.1 Definitions
First, we give a general definition of post-hoe reversal (Def. ( If Def. holld with px 's which are optimal for the base metric My, then naive selection becomes suboptimal as it picks qı 's. but On's are better under the post-hoc metric MTof- Since the entire space of parameter tuples O can be large, we study post-hoc reversal restricted to indexed parameters (Def. 33. Indices can be, for example, training epochs
4.2.1) model sizes ($ 4.2.2 or hyperparameter configurations 4.2.3
Let post-hoc transform T map a classifier f: θ x X - y to T OK X y. T applied f exhibits post-hoc reversal for metric M if there exist (O,...., 0k) PI..... px) E O such that My(0n) 2 My((Pu) for all k 1,..., K but MT (o..... 0k) MTor(PP......
Definition (Post-hoc reversal)
Definition 3 (Index-wise post-hoc reversal) Let I be a set of indices and P: I - O map indices to parameter tuples. When Def. 2 holds with (0,..... 0x) = P(s), (p). ( xx) = P(t) for some 8, / I, we call it index-wise post-hoc reversal.
Diagnosis. To enable visual diagnosis of post-hoc reversal, we define hase and post-hoe curves (Def. 4 and relaxed notion of post-hoc reversal for them (Def. 5). Post-hoc reversal is characterized by non-monotonicity between the base and post-hoe curves, i.e., there exist regions where one improves while the other worsens. This happens, for instance, when one curve exhibits double descent but the other doesn't. Different optimal indices for the two curves is another indicator of post-hoc reversal.
Definition 4 (Base and post-hoc curves) The base and post-hoc curves M Mpoutt I + R are given by Maae (t) * HKmMpp (000 and (t)ss MMoo(O)........... 0x). where
(O....., 0x) P(I).
Definition 5 (Post-hoc reversal for curves) Base and post-hoc curves Muuse Moon I R ex- hibit post-hoc reversal when there exist 8, l. E I such that Mbuse (s) 2 Mbasee (t) but Moost (s) < MPos (t).
'Thus, variant of SWA frperparraeeter-fre..

---

Test Loss (pre-TS)
Test Loss (post-TS)
Test Error (%)
60
Individual
C-10-N N Worst, Base
Ensemble
C-10-N Worst, SWA
3.0
C-10-N Randl, Base
50
C-10-N Randl, SWA
2.5
C-10-N Clean, Base
40
C-10-N Clean, SWA
2.0
30
1.5
1.0
20
0.5
25
50
75
25
50
75
25
50
75
Epochs
Epochs
Epochs
Figure 3: Loss and error for CIFAR-10-N Clean (approx. 0% noise), Randl (approx. 17% noise) and Worst (approx. 40% noise). Except for ensemble curves, mean of runs is shown; individual runs are in lighter shades. Ensembles comprise models from these 8 runs. For example, observe post-hoc reversal for C-10-N Worst: (1) error plot: from epoch 5 to 50, solid red (base) curve worsens but solid orange (SWA) curve improves; (2) eгror plot: solid red (base) curve has double descent but dashed red (ensemble) curve does not; (3) loss plots: solid red (base) curve has a double descent pre-TS but not post-TS; (4) error plot: best error is at approx. epoch for solid red (base) curve but approx. epoch 60 for dashed orange (SWA ensemble) curve.
4.2 Experiments
4.2.1
Epoch-Wise Post-Hoc Reversal
When the indices in Def. are training epochs, we call epoch-wise post-hoc reversal. We use 0, to denote the model at the end of epoch t. For ensembles, a superscript j denotes the j-th training (out of runs). & E maps to parameters (()) E O (K for TS; N for ensemble; and for
SWA) as follows: Prs (4) = (0.): Peno (1) = (0) 0 Psw (0,, 02).
Experimental setup. We focus on the CIFAR-N dataset 174). CIFAR-1O-N uses the same images as CIFAR-10 but provides multiple numan-annotatte label sets, allowing the study of realistic noise patterns of varying levels in a controlled manner. Clean is the original label set; R,nd are sets human labels; Aggre combines Randl, 1,23 by majority vote; and Worst combines them by picking incorrect label, if possible. Similarly CIFAR- 100- has two label sets, Clean and Noisy, with the latter being human-labelen annealed learning rate. Additional details datasets and training setup are in App. C Fig, B shows test curves on CIFAR- 10-N Clean, Rand1 and Worst. Other label sets and CIFAR- 100NN are in App.
We train ResNet18 21 models for 100 epochs with cosine
For clarity, omit the SWA base curve M SWA(( (t) (M,(01) + ... + (0,))))))///... in the plots, and simply re-use the curve ba (t) M, compare with the post-hoe SWA curve. While deviating from Def. this better reflects the current practice of early stopping on the latest epoch's 's base metric.
Observations. First, we focus on the base curves: (1) Overfitting: noise increases, test curves go from single descent to double descent to a U-shaped curve with increased overfitting. (2) Double descent: Noise amplifies double descent, and the second descent worsens with increasing noise (as compared to the first). (3) Loss-error mismatch: Loss overfits more drastically than error, leading to mismatch with higher noise. Optimal models for loss and error can be different.
"Enemmblin models from possibly unequal epochs is covered in

---

CIFAR-10-N Worst
Exponential LR
Casine LR
Constant LR
3.0
(post-TS) (pre-1
2.8
2.5
2.4
2.0
2.0
1.6
Loss (post-TS)
1.8
1.4
1.5
1.3
60
48
Error (%)
Error (%)
44
50
40
45
36
25
50
20
40
20
40
20
40
ResNet Width
Epochs
Epochs
Epochs
Base
Individual
Base
Individual
SWA
Ensemble
SWA
Ensemble
Figure 4: C-10-N Worst test curves against model size. Best width for solid blue curves is 10 but for dashed orange curves, is ~ 50 for er- ror and 25 for post-TS loss.
Figure 5: FMoW test curves for 3 LR schedules. Note that the pre- TS loss is significantly higher than the post-TS loss. For example, observe post-hoc reversal w.r.t. cosine and constant LRs at epoch 50 between: (1) solid blue (base) and dashed blue (ensemble) error curves; (2) solid blue (base) and solid orange (SWA) post-T'S loss curves; (3) solid blue (base) curves for pre-TS, and post-TS loss.
Next, consider the general impact of post-hoc transforms: (4) Performance improvements: TS, SWA and ensemble always improve performace, both individually and in composition with larger gaps for noisy label sets. (5) Post-hoc reversal: Post-hoc reversal manifests non-mo con-monotonicity between the base and post-hoc curves, especially for noisy label sets. (6) SWA vs Ensemble: SWA recover much of the ensemble gain, but the optimal epoch often differs lot from the base curve. (7) Smoother curves: Base curves fluctuate wildly, but SWA and ensemble curves are smooth, making them more reliable for early stopping.
Finally, we discuss some benefits from post-hoc reversal: (8) Overfitting: All transforms reduce overfitting, often reverting performance degradation. (9) Double descent: SWA, ensemble and compositions flatten the double descent peak. TS, on the other hand, leads to a double descent for some cases where there was none before. (10) Loss-error mismatch: TS aligns the loss and error curves, enabling simultaneously good loss and error.
4.2.2 Model-Wise Post-Hoc Reversal
Here, indices represent model sizes. Models sizes are trained for T' epochs, large enough for convergence. Following (55), avoid early stopping. Notation-wise, we add subscript to to indicate model size 8. Parameters are indexed as follows: Pis(s) = (0r,s); Pens(s)
(......., NN..:: (s)a(s) (.).......,T.. sT
Experimental setup. We parameterize family of ResNet18s by scaling the number of filters in the convolutional layers. Specifically, we use [R:, 2k, 4k, 8k) filters for width k. The standard ResNet18 corresponds k 64. Otherwise the training setup is same before. Fig. 4 shows the curves. Concretely, the index set I = {2, 42,4.... 64) is the set of ResNet widths described above.
Observations. Post-hoc transforms improve performance (up to R 10 points for error) and mitigate double descent. Further, we see yet another way in which higher-capacity models are better: they give better results under post-hoc transforms when lower-capacity base models perform better.

---

% predictions filpped
Clean subset
80
Base model
subset Mislabeled subset
40
Mislabeled subset
Train error (%)
overfits
Base
20
SWA
40
0
SWA
20
40
60
80
100
20
Epochs
Figure Flipping of predicted class between consecutive epochs, for clean and mislabeled train subsets of C-10-N Worst. % of examples flipped is about twice as high for the mislabeled subset, suggesting an unstable influence on the decision boundary.
55
Test set Base
Test error
SWA
40
60
80
100
Temperatur
Epochs
Figure 6: Evolution of the fit/memorization of clean and mislabeled examples during training. for base and SWA models on C-10-N Worst. Train error drops earlier for the clean subset. the regime of post-hoe reversal (shaded), SWA further lowers the train error on the clean subset, while raising it on the mislabeled subset.
40
60
100
Epochs
Figure 8: Optimal temperature for TS C-10-N Worst increases with epochs, indicating increas- ing overconfidence of the neural network.
4.2.3 Hyperparameter-Wiss Wise Post-Hoc Reversal
In general, the index set I can contain any hyperparameter configurations. Here, we consider two hyperparamters: learning rate schedule and training epochs. To avoid repeating CIFAR-N epoch-wise curves, we experiment on a fresh dataset, FMoW.
Experimental setup. We experiment on learning rates (LRs) and training epochs, with index set I (const, exp, cos) (1,.... .., T}. Here, const, exp and cos refer to constant, exponentially decaying and cosine annealed LRs respectively, and is the total number of epochs. We train DenseNet121 [26) models the FMoW dataset [9] which constitutes 62-way classification of land use from satellite images. For more details, see App. C Fig. shows the curves.
LR-wise observations. We see some interesting instances of post-hoc reversal: (1) constant LR has the worst base performance but the best post-hoc performance; (2) under SWA and TS (composed), the curves continue improve at the later epochs for constant LR, but not for the decaying LR
Epoch-wise observations. Epoch-wise post-hoc reversal occurs for all LR schedules. SWA and ensembling convert the double descent into a strong single descent, with approx. 10-point improve- ment in error for the latter. For constant LR, this also changes the optimal epoch. SWA only recovers about half of the ensemble gain, and perhaps surprisingly, ensembling SWA models is not better than ensembling alone. Pre-TS loss curves show a strong mismatch with the errr curves, but TS enables simultaneously good loss and error with the last epoch models. Overall, these observations reinforce the trends gleaned from the CIFAR-N experiments.
Intuitions for Post-Hoc Reversal
In this section, we give hypotheses for post-hoc reversal, backed by experimental evidence.
Ensembling and SWA delay catastrophic overfitting, Models learn generalizable patterns from clean examples, and spurious patterns from mislabeled ones. The latter causes overfitting. When noise is low, the former dominates and overfitting is benign. Otherwise, overfitting is catastrophic. Ensembling and SWA improve fitting of clean examples, and reduce memorization of mislabeled ones. When this overturns the dominance of spurious patterns, observe reversal.
Fig. 6 validates this intuition for SWA on CIFAR-IO-N Worst. Fig. further suggests the underlying mechanism predictions the mislabeled train subset fluctuate much more during training, allow-
PPossibly due higher model variance with constant LR, beneficial for both ensembling and SWA.

---

Table 1: Naive vs post-hoc (ours) selection for SWA+TS and SWA+Ens+TS transforms. Better values are in bold. Except some clean cases, post-hoc selection is always better, often more than doubling the improvement over no transform. See Tabs. and
8
App. E for standard deviations.
Metric +
Test Loss
Test Error (%)
Transform +
None
SWA+TS
SWA+F
None
SWA+TS
SWA+ TS
Dataset
Naive
Ours
Naive
Ours
Naive
Ours
Naive
Ours
C-10-N
0.269
0.270
0.234
0.233
9.09
9.10
8.30
8.24
C.10.N Aggre
0.722
0.663
0.585
0.608
0.543
19.20
17.08
16.95
15.88
15.74
C-10-N Rand1
1.009
0.968
0.907
0,916
0.859
28.63
27.13
24.84
24.80
23.50
C-IN-N Word
1.511
1,483
1.443
1.437
1.399
46.84
46.12
44.14
44.30
42.88
C-100-N Clean
1.215
1.205
1.065
1.063
32.69
29.94
C-100-N Noisy
2,416
2.289
2.136
2.129
1.994
58.68
54.94
53.18
51.34
50.26
FMoW (ID)
1,583
1.627
1.554
1.494
1.305
43.20
42.69
39.92
37.95
34.93
FMoW (OOD) 1.831
1.840
1.788
1,700
1.571
49.32
49.70
46.75
46.74
41.56
ing SWA to easily revert their memorization. In App. G we extend this analysis to ensembling and solidify the intuition further by visualizing decision boundaries on a synthetic dataset. This explana- tion also applies to flattening of the double descent peak, which is manifestation of catastrophic overfitting.
TS mitigates loss-error mismatch. Once a neural net has fit a train example, the cross-entropy loss on it can be lowered by simply upscaling the weights of the linear output layer. This makes the model overconfident later in training, as shown [19]. For a mislabeled example, this leads to worse loss on similar test instances. The test error is not affected it is independent of the scale of the logits. In high-noise settings, test loss can worsen due to memorization of mislabeled examples, even as the test error improves from continued learning clean examples, leading to loss-error mismatch. TS fixes this by downscaling the logits. Indeed, one finds that the temperature (as obtained with a held-out set) increases with epochs (Fig.
Post-hoc reversal can occur against epochs, model sizes or other hyperparameters. Different variants of post-hoc reversal can be unified via effective model complexity (EMC), introduced in (55] to unify epoch- and model-wise double descent. EMC measures memorization capacity, which plays key role in post-hoc reversal. EMC increases with epochs and model size. Further, EMC increases with epochs more rapidly for constant LR than annealed LR, explaining our observations in $ 4.2.3
Post-Hoc Selection: Leveraging Post-Hoc Reversal in Practice
Our findings from motivvtt the principle of post-hoc selection, where model development decisions take post-hoc transforms into account. For concreteness, we discuss the choice of checkpoints from training runs under the SWA+TS and SWA+Ens+TS transforms. Checkpoint selection reduces the selection of the final epoch T, as SWA uses all checkpoints up to that epoch. M,ai denotes a metric of choice computed on the validation set.
SWA+TS. Naive selection picks epoch f arg miny My In contrast, post-hoc selection picks / arg miny M / ((e,)
SWA+Ens+TS. Here we have N different training runs to pick epochs for.
Naive selection picks T = arg miny My (0))) for each run independently. In contrast, post-hoc selection would ideally pick .i....,
., Tv = arg mint,...... TT / ((0)) (0 )Mm ) which jointly minimizes the
ensemble performance. This being computationally expensive, we instead minimize under the constraint 7; =
Results. Tab [ compares naive and post-hoc selection strategies for CIFAR-N and FMoW, Except for some clean label sets, post-hoe selection always better than naive selection, often with > 2x improvement from post-hoe selection as compared to naive selection. It remains effective with out-
Alternatively, one can select 7; arg miny Maral Turo (0) (Bf...., ..,...,,,,, 0%) as hybrid between post hoc selection
(within runs) and naive selection (across runs).
8

---

Perplexity
CLM Error (%)
MMLU (%)
5
33
48
Base
SWA+TS
4
46
SWA+ Ens+TS
Naive selection
29
Post-hoc selection
3
44
0
2
3
1
2
3
4
0
2
Epochs
Epochs
Epochs
Figure 9: Perplexity and causal language modeling (CLM) error on the Guanaco test set, and MMLU accuracy (higher is better) for instruction tuning I.L.MA---7B. Shading indicates post-hoc reversal. Base and SWA+TS curves are mean of 8 runs; SWA+Ens+TS ensembles models from these runs. Individual runs are not shown as they have high variance (see Tab. App.
of-distribution (OOD) val/test sets, as seen for FMoW (we use ID and OOD splits from WILDS 34 For some datasets, like C-100-N Noisy, post-hoc selection is only marginally better on test error. Often, in such cases, the error floor is already quite high (c.g., C-100-N Noisy has ~ 40% noise and R8 has 10% error clean C-100, a test error of 50% is already impressive), and test loss is a more appropriate metric.
Early stopping. We advocate monitoring post-hoc metrics for early stopping. Only running average needs to be updated for SWA, and TS involves a quick single-parameter optimization. Further, while the base curves can fluctuate wildly between consecutive runs, SWA+TS curves considerably smoother (see Figs. similarly monitor metrics for SWA+Ens+TS under parallel training runs.
and 10 making them more reliable for automated early stopping. One can
7 Experiments Across Domains and Modalities
In $ and 66 we introduced post-hoc reversal and selection with experiments the CIFAR-N and FMoW datasets. In this section, we supplement our experimental analysis with additional experiments across diverse domains and modalities to demonstrate the generality of our findings.
7.1 LLM Instruction Tuning
Language models are pre-trained or fine-tuned with a self-supervised objective of predicting the next token in a text corpus. There might be many acceptable tokens following given prefix, albeit with different probabilities. Thus next token prediction is noisy and one might reasonably expect to see post-hoc reversal. In this section, we test this hypothesis for the task of fine-tuning LLMs follow instructions (instruction tuning (2]). Instruction tuning datasets are naturally small (8 and amenable to multi-epoch training where catastrophic overfitting becomes an important concern. Recent works [53] 8T have argued for data repetitions for LLM pre-training as well, but such experiments are beyond the scope of this paper.
Experimental setup. We fine-tune LLaMA-2-7B 70 the Guanaco dataset 12 of chat comple- tions. We evaluate perplexity and causal language modeling (CLM) error on the test set, and also the MMLU accuracy [24] better contextualize model improvements. Fig. [ shows the curves. Tab. 7 in App. D gives exact numbers, and App. explores sub-epoch checkpointing. For TS, we use Shared temperature parameter scale the logits of all tokens and leave more involved strategies like long- long-horizon temperature scaling [66] to future work.
Observations. We observe post-hoc reversal between epochs 1 and 2 for perplexity and error, and between epochs and 3 for MMLU. Both SWA+TS and SWA+Ens+TS transforms show significant improvements, much of which only realized under post-hoe selection.
7.2 Other Text, Tabular and Graph Datasets
In this section, we further expand our experimental coverage to text, tabular and graph classification datasets from world applications.

---

Yelp
Income
Reddit-12k
Loss (post-TS) Loss (pre:TS)
2.56
0.60
3.85
0.84
0.38
1.37
0.98
0.44
1.58
Base
SWA
0.90
Individual
0.82
0.38
1.34
Ensemble
43.1
Eroo (%)
40.7
19.5
57.0
38.3
18.4
35.9
17.3
49.8
10
20
10
20
0
200
400
Epochs
Epochs
Epochs
Figure 10: Test curves for real-world noisy datasets. Note that the pre-TS loss significantly higher than the post-T'S loss. Examples of post-hoc reversal between the base curves given by the solid blue lines and the post-hoc curves given by the dashed orange lines (SWA ensemble): (1) optimal epoch is different for base and post-hoc curves for error and post-TS loss on all datasets; (2) for error Yelp, base curve shows double descent but post-hoc curve does not; (3) for error on Income, base curve overfits catastrophically approx. epoch 5 but post-hoc curve continues improving till approx. epoch 20; (4) for error on Reddit- 12k, base curve does not show double descent but post-hoc curve does.
Experimental setup. We consider the following tasks: (1) sentiment classification on the Yelp reviews dataset [5] (text) with a pre-trained transformer BERT [3]. (2) prediction tasks on census data from Folktables 14 (tabular) with MLPs and (3) community detection on the Reddit and Collab datasets X2 (graph) with graph neural networks (GNNs). Folktables has 5 prediction tasks: Income, PublicCoverage, Mobility, Employment and TravelTimee Reddit has 2 versions: Reddit-5k and 12k. For more details, see App. C Figure TO shows curves for Yelp, Income and Reddit- - 12k. Tab. 5 in App. D compares naive and post-hoc selection on all datasets.
Observations. Post-hoc reversal is recurring feature across datasets, transforms and metrics. The datasets show different patterns between the base and post-hoc curves, showing that post-hoc reversal can take variety of forms.
8 Conclusion
We empirically studied temperature scaling (TS), ensembling, stochastic weight averaging (SWA) and their compositions, and found that these transforms reverse model peformance trends (post-hoc reversal). Based on our findings, we presented the simple technique of post-hoe selection, and showed that it outperforms naive selection. We validated our findings and proposals over diverse settings.
Our work has broad implications for the field of deep learning. It shows that current practices surrounding the use of post-hoc transforms leave much room for improvement. This is especially true for noisy data, which is pervasive in real-world applications. Future directions include better strategies for checkpoint selection, developing theoretical understanding, investigating impacts scaling laws, and characterizing other instances of post-hoc reversal.
Summary of practical recommendations. We advocate for the use of TS, ensembling and SWA across deep learning applications. Further, such transforms should be tightly integrated into the model development pipeline, following the methodology outlined in the paper. In particular: (1) apply SWA+TS and SWA+Ens+TS transforms for better results in the single- and multi-model settings respectively: (2) track remperature-scall
loss overcome loss-error mismatch; (3) monitor post-hoc metrics to avoid premature early stopping; (4) make hyperparameter decisions informed by most-transform performance; (5) use post-hoc selection to pick model checkpoints.
10

---

Acknowledgements
ZL acknowledges Amazon AI, Salesforce Research, Facebook, UPMC, Abridge, the PwC Center, the Block Center, the Center for Machine Learning and Health, and the CMU Software Engineering Institute (SEI) via Department of Defense contract DA8702-15-D-0000 for their generous support of ACMI Lab's s research machine learning under distribution shift.
References
[1] Taiga Abe, E. Kelly Buchanan, Geoff Pleiss, and John P Cunningham. Pathologies of predictive diversity in deep ensembles. ArXiv, abs/2302.00704, 2023.
[2] Amr Alexandari, Anshul Kundaje, and Avanti Shrikumar. Maximum likelihood with bias- corrected calibration hard-to-beat at label shift adaptation. In International Conference on Machine Learning, pages 222 232. PMLR, 2020.
[3] Zeyuan Allen-Zhu and Yuanzhi Li. Towards understanding ensemble, knowledge distillation and self-distillation in deep learning. arXiv preprint arXiv:2012.09816 2020.
[4] Devansh Arpit, Huan Wang, Yingbo Zhou, and Caiming Xiong. Ensemble of averages: Improv- ing model selection and boosting performance in domain generalization. Advances in Neural Information Processing Systems, 35:8265-877 8277, 2022.
[5] Nabiha Asghar. Yelp dataset challenge: Review rating prediction. arXiv preprint
1r5iv:1605.05362 2016.
[6] Junbum Cha, Sanghyuk Chun, Kyungjae Lee, Han-Cheol Cho. Seunghyun Park, Yunsung Lee, and Sungrae Park. Swad: Domain generalization by seeking flat minima. Advances Neural Information Processing Systems, 34:22405 22418, 2021.
[7] John Chen, Qihan Wang, and Anastasios Kyrillidis. Mitigating deep double descent by con- catenating imputs. Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021.
[8] Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, al. Alpagasus: Training better alpaca with fewer data. arXiv preprint arXiv:2307.08700 2023.
[9] Gordon Christie, Neil Fendley, James Wilson, and Ryan Mukherjee. Functional map of the world. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6172-6180, 2018.
[10] Jeremy M. Cohen, Simran Kaur, Yuanzhi Li, J. Zico Kolter, and Ameet Talwalkar. Gradient descent neural networks typically occurs the edge of stability. ArXiv, abs/2103.00065, 2021.
[11] Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski. Jonathan Heek, Justin Gilmer, Andreas Peter Steiner, Mathilde Caron, Robert Geirhos, Ibrahim Alabdulmohsin, et al. Scaling vision transformers to 22 billion parameters. In International Conference on Machine Learning, pages 7480-7512. PMLR, 2023.
[12] Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. QLoRA: Efficient finetuning of quantized LLMs. In Thirry--evennt Conference on Neural Information Processing Systems, 2023.
[13] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
Ming-Wei
[14] Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt. Retiring adult: New datasets for fair machine learning. Advances in Neural Information Processing Systems, 34, 2021.
[15] Reuben Feinman. Pytorch-minimiiz: a library for numerical optimization with autograd, 2021. htthst::::/ ://

---

[16] Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan, Deep ensembles: A loss landscape perspective. arXiv preprint arXiv:1912.02757, 2019.
[17] Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv: Learning, 2018.
[18) Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry P Vetrov, and Andrew G Wilson. Loss surfaces, mode connectivity, and fast ensembling of dnns. Advances neural information processing systems, 31, 2018.
[19] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration modern neural networks. In International conference machine learning, pages 1321-1330. PMLR, 2017.
[20] Trevor J. Hastie, Robert Tibshirani, and Jerome H. Friedman. The elements of statistical learning. 2001.
[21] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[22] Tong He, Zhi Zhang, Hang Zhang, Zhongyue Zhang, Junyuan Xie, and Mu Li. Bag of tricks for image classification with convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 558-567, 2019.
[23] Reinhard Heckel and Fatih Yilmaz. Early stopping in deep networks: Double descent and how eliminate it. ArXiv, abs/2007. 2020.
[24] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint 20XY::00090033000 2020.
[25] Gao Huang. Yixuan Li, Geoff Pleiss, Zhuang Liu, John E Hopcroft, and Kilian Q Weinberger. Snapshot ensembles: Train 1, get m for free. arXiv preprint arXiv: 1rXiv:1704.0000 11.4100444909999 2017.
[26] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 4700-4708, 2017.
[27] Gabriel Ilharco, Mitchell Wortsman, Samir Yitzhak Gadre, Shuran Song, Hannaneh Hajishirzi, Simon Kornblith, Ali Farhadi, and Ludwig Schmidt. Patching open-vocabulary models by interpolating weights. Advances in Neural Information Processing Systems, 25:29262-229277, 2022.
[28] Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. Averaging weights leads to wider optima and better generalization. arXiv preprint 18Xiv=1803.05400 2018.
[29] Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. LIm-blender: Ensembling large language models with pairwise ranking and generative fusion. arXiv preprinl arXiv:2306.0256/1 2023.
[30] Lu Jiang, Di Huang, Mason Liu, and Weilong Yang. Beyond synthetic noise: Deep learning on controlled noisy labels. In International Conference on Machine Learning, 2019.
[31] Jared Kaplan, Sam McCandlish, T. J. Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeff Wu, and Dario Amodei. Scaling laws for neural language models. ArXiv, abs/2001.088 100361, 2020.
[32] Amr Khalifa, Michael C Mozer, Hanie Sedghi, Behnam Neyshabur, and Ibrahim Alabdulmohsin. Layer-stack temperature scaling, arXiv preprint arXiv:2211. 2022.
[33] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.0330 2016.
12

---

[34] Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, al. Wilds: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning, pages 5637-5664. PMLR, 2021.
[35] Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung. Sylvain Gelly, and Neil Houlsby. Big transfer (bit): General visual representation learning. In Computer Vision- ECCV 2020: 16th European Conference, Glasgow. UK, August 23-28, 2020, Proceedings, Part V 16, pages 491-507. Springer, 2020.
[36] Dan Kondratyuk, Mingxing Tan, Matthew Brown, and Boqing Gong. When ensembling smaller models is more efficient than single large models. arXiv preprint arXi::2005.005 2020.
[37] Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi-Rui Tam, Keith Stevens, Abdullah Barhoum, Nguyen Minh Duc, Oliver Stanley, Richárd Nagyfi, et al. Ope- nassistant conversations-demccatiiiiii large language model alignment. arXiv preprint arXiv:2304.07327, 2023.
[38] Ludmila I. Kuncheva and Christopher J. Whitaker. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning, 51:181-2001 2003.
[39] Balaji Lakshninarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Neural Information Processing Systems, 2016
[40] Kuang-Huel Lee, Xiaodong He, Lei Zhang, and Linjun Yang. Cleannet: Transfer learning for scalable image classifier training with label noise. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5447-5456, 2017.
[41] Weishi Li, Yong Peng, Miao Zhang, Liang Ding, Han Hu, and Li Shen. Deep model fusion: A survey. arXiv preprint (wX.:::339..6699 2023.
[42] Sheng Liu, Jonathan Niles-Weed, Narges Razavian, and Carlos learning regularization prevents memorization of noisy labels. Advances in neural information processing systems, 33:20331-20344 2020.
Granda. Early-
[43] Sheng Liu, Jonathan Niles-Weets Narges Razavian, and Carlos Feraadez-Granda. Early- learning regularization prevents memorization of noisy labels. ArXiv, abs/2007.00151, 2020.
[44] Sheng Liu, Zhihui Zhu, Qing Qu, and Chong You. Robust training under label noise by over-parameterizztion ArXiv, abs/2202.14026 2022.
[45] Sheng Liu, Zhihui Zhu, Qing Qu, and Chong You. Robust training under label noise by over- parameterization. In International Conference on Machine Learning, pages 14153-14172. PMLR, 2022.
[46] Shikun Liu, Linxi Fan, Edward Johns, Zhiding Yu, Chaowei Xiao, and Anima Anandkumar. Pris- mer: A vision- languaee model with an ensemble of experts. arXiv preprint arXiv:2303.02506, 2023.
[47] Raphael Gontijo Lopes, Yann Dauphin, and Ekin Dogus Cubuk. No one representation to rule them all: Overlapping features of training methods. ArXiv, abs/21 10. 12899, 2021.
[48] Keming Lu, Hongyi Yuan, Runji Lin, Junyang Lin, Zheng Yuan, Chang Zhou, and Jingren Zhou. Routing to the expert: Efficient soward-guided ensemble of large language models. arXiv preprint arXiv: 022311100692 2023.
[49] Xiaoding Lu, Adian Liusie, Vyas Raina, Yuwen Zhang, and William Beauchamp. Blend- ing is all you need: Cheaper, better alternative trillion-parameters Ilm. arXiv preprint a40iv:2401. 20t.2-221..9444 2024.
[50] Neil Rohit Mallinar, James B. Simon, Amirhesam Abedsoltan, Parthe Pandit, Mikhail Belkin, and Preetum Nakkiran. Benign, tempered, or catastrophic: taxonomy of overfitting. ArXiv, abs/2207. abs/2207.06569, 2022.
13

---

[51] Prem Melville and Raymond J. Mooney, Constructing diverse classifier ensembles using artificial training examples. In International Joint Conference on Artificial Intelligence. 2003.
[52] Christopher Morris, Nils M Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Marion Neumann. Tudataset: A collection of benchmark datasets for learning with graphs. arXiv preprint arXiv:2007.08663 2020.
[53] Niklas Muennighoff, Alexander M. Rush, Boaz Barak, Teven Le Scao, Aleksandra Piktus, Nouamane Tazi, Sampo Pyysalo, Thomas Wolf, and Colin Raffel. Scaling data-constrained language models. ArXiv, abs/2305.16264 2023.
[54] Preetum Nakkiran, Prayaag Venkat, Sham M. Kakade, and Tengyu Ma. Optimal regularization can mitigate double descent. ArXiv, abs/2003.0 018770 2020.
[55] Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang. Boaz Barak, and Ilya Sutskever. Deep double descent: Where bigger models and more data hurt. Journal of Statistical Mechanics: 2021.
Theory and Experiment, 2021(12):124000
[56] Yaniv Ovadia, Emily Fertig, Jie Jessie Ren, Zachary Nado, Sculley, Sebastian Nowozin, Joshua V. Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In Neural Information Processing Systems, 2019.
[57] Vardan Papyan, Xuemei Han, and David L. Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences of the United States of America, 117:24652 24663, 2020.
[58] Alethea Power, Yuri Burda, Harrison Edwards, Igor Babuschkin, and Vedant Misra. Grokking: Generalization beyond overfitting on small algorithmic datasets. ArXiv, abb/2201.021177 2022.
[59] Qu'en and Tartaglione. Can we avoid double descent in deep neural networks. arXiv preprint arXiv: 13255 2023.
[60] Alec Radford and Karthik Narasimhan. Improving language understanding by generative pre-training, 2018.
[61] Alexandre Rame, Matthieu Kirchmeyer, Thibaud Rahier, Alain Rakotomamoniy. Patrick Gal- linari, and Matthicu Cord. Diverse weight averaging for out-of-distribution generalization. Advances in Neural Information Processing Systems, 15:2111110886 2022.
[62] Alexandre Ramé, Nino Vieillard, Léonard Hussenot, Robert Dadashi, Geoffrey Cideron, Olivier Bachem, and Johan Ferret. Warm: On the benefits of weight averaged reward models. arXiv preprint arXiv:2401. 12187, 2024.
[63] Rishabh Ranjan. torchcal: post-hoc calibration on GPU, 2023. URL. htt.htttt///tthhui com/ rishabh-ranj an/torchcal
[64] Sunny Sanyal, Atula Tejaswi Neerkaje, Jean Kaddour, Abhishek Kumar, et al. Early weight averaging meets high learning rates for Ilm pre-training. In Workshop on Advancing Neural Network Training: Computational Efficiency. Scalability, and Resource Optimization (WANT@ NeurIPS 2023), 2023.
[65] Rylan Schaeffer, Mikail Khona, Zachary Robertson, Akhilan Boopathy, Kateryna Pistunova. Jason W. Rocks, Ila Rani Fiete. and Oluwasanmi Koyejo. Double descent demystified: Identi- fying, interpreting ablating the sources of a deep learning puzzle. ArXiv, 0bo/2303..4451.. 2023.
[66] Andy Shih, Dorsa Sadigh, and Stefano Ermon. Long horizon temperature scaling. In Interna- tional Conference on Machine Learning, pages 31422-31434. PMLR, 2023.
[67] Anand Siththaranjan, Cassidy Laidlaw, and Dylan Hadfield-Menell. Distributional prefer- ence learning: Understanding and accounting for hidden context in rlhf. arXiv preprint arXiv:2312.08358, 2023.
14

---

[68] Hwanjun Song, Minseok Kim, and Jae-Gil Lee. Selfie: Refurbishing unclean samples for robust deep learning. In International Conference on Machine Learning. 2019.
[69] Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Gil Lee. Learning from noisy labels with deep neural networks: A survey. IEEE Transactions on Neural Networks and Learning Systems, 2022.
[70] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.00 002888, 2023.
[71] Dongdong Wang, Boqing Gong, and Liqiang Wang.
On calibrating semantic segmentation models: Analyses and an algorithm. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23652-236602 2023.
[72] Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoe V. Le. Finetuned language models are zero-shot learners. ArXiv, abs/2109.016522 2021.
[73] Jiaheng Wei, Zhaowei Zhu, Hao Cheng, Tongliang Liu, Gang Niu, and Yang Liu, Learning with noisy labels revisited: A study using real-world human annotations. ArXiv, 10./2110012088 2021.
[74] Jiaheng Wei, Zhaowei Zhu, Hao Cheng, Tongliang Liu, Gang Niu, and Yang Liu. Learning with noisy labels revisited: A study using real-world human annotations. arXiv preprini arXiv:2/ /0. /2088, 2021.
[75] Ross Wightman, Hugo Touvron, and Hervé Jégou. Resnet strikes back: An improved training procedure in timm. arXiv preprint arXi::21 2021.
[76] Andrew G Wilson and Pavel Izmailov. Bayesian deep learning and probabilistic perspective of generalization. Advances neural information processing systems, 33:4697-4708, 2020.
[77] Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In International Conference on Machine Learning, pages 23965-23999 PMLR, 2022.
[78] Ruixuan Xiao, Yiwen Dong, Haobo Wang, Lei Feng, Runze Wu, Gang Chen, and Junbo Zhao. Promix: Combating label noise via maximizing clean sample utility. In Edith Elkind, editor, Proceedingy of the Thirty-Second International Joint Conference on Artificial Intelli- gence, IJCAI-23, pages 4442-4450, International Joint Conferences on Artificial Intelligence Organization, 2023. doi: 10. 10.24963//jcca 20234/44 Main Track.
[79] Tong Xiao, Tian Xia, Yi Yang. Chang Huang, and Xiaogang Wang. Learning from massive noisy labeled data for image classification. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2691-2699, 2015.
[80] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? arXiv preprint arXiv: 181000000. 2018
[81] Fuzhao Xue, Yao Fu, Wangchunshu Zhou, Zangwei Zheng, and Yang You. To repeat or not to repeat: Insights from scaling I1m under loken-crisis. ArXiv, abs/2305. 13230, 2023.
[82] Pinar Yanardag and SVN Vishwanathan. Deep graph kernels. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 1365-1374, 2015.
[83] Mert Yuksekgonul, Linjun Zhang, James Y. Zou, and Carlos Guestrin. Beyond confidence: Reliable models should also consider atypicality. ArXiv, abs/2305. 18262, 2023.
[84] Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few-shot performance of language models. International Conference on Machine Learning, pages 12697-12706. PMLR, 2021.
15

---

[85] Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. Lima: Less is more for alignment. arXiv preprint arXiv:2305. 2023.
16

---

A Expanded Related Work
Phenomena, Empirical works like double descent (55], grokking [58. scaling laws BTJ neural-
collapse [57], edge-of-stability [TO], poteey-tiicke--ypothee [17] have revealed both challenges and oppotunities for improving the understanding and practices of deep neural network training. Post-hoc reversal expands this list as novel phenomenon regarding learning dynamics under the lens of post-hoe transforms. It most intimately connected with double descent, offering a way to
mitigate it. Some works 7 23 54 59
65
76
show other mitigations, such as regularization and
data augmentation.
Temperature Scaling (TS). TS belongs to family of post-hoc calibration techniques 2 19 32, 66 833, with the unique property preserving classification error. Recently, calibration has been applied to large vision and language models [ [7] [4]. While loss-error mismatch has been reported the best of our knowledge, we are the first to report post-hoc reversal with TS.
before
Ensembling. Ensembling is a foundational technique in machine learning, encompassing bagging, boosting, etc. In deep learning, a uniform ensemble is most popular [ [9]. although recent work
on ensembling LLMs has explored more efficient routing-based ensembles [29 46 48 49]. Various
works have explored strategies to form optimal ensembles [36] 47 51 77, generally based on model
diversity [38]. but recently Abe et al. [M] have warned against this. In contrast, our recommendation
for forming ensembles relies directly on the validation performance of the ensemble, introducing no proxies, and still being computationally cheap.
Stochastic
Weight Averaging (SWA). SWA 28 is the culmination of a line of work [88 25 which seek to cheaply approximate ensembling. It has inspired numerous works which average 777 often in combination with ensembling. Recently, weight averaging has shown up in the LLM space [62 64]. While these works generally apply SWA with fixed training time determined independently, we present SWA in the role of early stopping and model selection. In practice, SWA has often been found to be unreliable and is often skipped from training recipes even when considered (35 75). Our work sheds some light on this, offering rather counter-intuitive choice of models to include in the weight average for best results.
weights in some form 4 6 27
41
61
Noise. Many training strategies have been introduced to deal with noisy data (see 169 for survey). However, the efficacy of simple post-hoc transforms has been left unexplored. Further, most of these works are motivated by labeling errors, which leaves some of the core practical considerations for dealing with general noisy data unaddressed. For instance, access to a clean validation set is assumed and test loss is overlooked as an important metrie 44 44
We also entirely avoid experiments synthetic noise, informed by recent work which questions the transferability of findings to realistic noise patterns (30 (73. Some recent datasets 30
70) make it possible study realistic noise along with known noise estimates. Noise due to insufficient information in the input context
40
68
73
(Fig. IT has also been studied under different settings, such as for RLHF (67].
Multi-epoch training of LLMs. Multi-epoch training of LLMs runs into severe catastrophic overfitting. Xue et al. [8]] examine the contributing factors and explore possible solutions. They find that regularization is not helpful, except for dropout. Muennighoff et al. [53] study scaling laws put forward post-hoc transforms
considering data repetitions. Complementarilyy solution with our post-hoc selection methodology. This is especially important for fine-tuning LLMs, e.g. in instruction tuning [72), where [85] and { advocate for fine-tuning with smaller amount of higher quality samples for more epochs.
effective
B Expanded Preliminaries and Background
B.1 Learning on Noisy Data
Figures 12 and 13 illustrate various sources of noise: aleatoric unertainty, epistemic uncertainty and annotation errors. Below we provide some background on Bayes-optimal classifier and use it to introduce the clean error metric and Bayes loss/error as measures of noise level.
See, for example, discussion at http://wwwwu.aaaoeo....////////a http:pprrrroooootttttisssss-aaas
858
17

---

write ballad on internet privacy and open-source is...
There
conputer, friendly old granmar...
Sorry,
indeed nisspell "was", here a...
There once was corputer, friendly ly old...
There once corputer, friendly and old,...
Verse Open source and privacy, Me need to protect our...
L- Laoks geod
you explain clicks endless...
Can you sodify it to carry an opposite nessege?
Change the chorus
chorus: Oh internet, let freedom be our guide...
Chorus: Oh internet, where information flovs...
(a) Gas Station
(b) Educational Institution
aowledge
Figure 11: Data can be noisy due insufficient information in the input context (epistemic uncer- tainty). Figures Tla and show satellite images from the FMoW dataset. The labels are correct, as corroborated by external map data. However, they cannot be determined with full certainty from the images alone.
Data can be noisy due non- determinism in the prediction target (aleatoric un- certainty). Figure shows message tree from the OpenAssistant Converstations (OASSTI) Dataset. chatbot can continue a conversation satisfacto- rily in many different ways, making next token prediction noisy.
Figure 12:
(a) L: Cat C: Frog
(b) L: Horse
(c) L: Airplane C: Ship
(d) L: Bee A: Sunfllowrr
(c) L: Dolphin A: Wooma
( L: Mountain
C: Deer
A: Cloud
Figure 13: Data can be noisy due annotation errors. Figures 13a 13b and 13c are mislabeled
images from CIFAR-10. 13d 13e and
13f are ambiguous images from CiFAR- 100 with multiple
correct labels among the given classes. (I. label in dataset, correct label, alternative label)
Bayes-optimal classifier. fp. given by fD(X) = log Pply = k x] minimizes both M and M and is called the Bayes-optimal classifier for D. The Bayes error MM [fo] and Bayes loss [fo] are measures of the noise level. y (x( = arg max; x is sometimes called the clean label. Using y" one may define the clean data distribution D with Pz|x| Pp x and Poly x] = 1y = y (x)). The clean error M is common metric label noise literature but not a focus our work as y" is typically inaccessible in more general noisy settings.
B.2 Hos Transforms in Machine Learning
The explicit forms of the composed transforms SWA+TS and SWA+Ens+TS (denoted as Ts+T and Ts+FaT) are given by Equations and respectively. averaged and the L resulting models are ensembled, followed by temperature scaling. temperature for weight-averaged models, and TEns is the temperature for the ensemble. As before, they obtained by optimizing the cross-entropy loss over the validation set, with model parameters fixed.
J For TS parameters ....... O' lxx are weight-
is the
K
K
1
(Tstrof) (x; 01,.... 0x) = 1
x;
0
with arg min M
e
K
K
1=1
Ki
(TsHErT o f) f)(x) (x: 1
oF ..... 0k
f
X:
0
(5)
TEas
K1
1=1
k=
18

---

Table Dataset Details.
Modality
Dataset
Train Size
Val Size
Test Size
Classes
Inpat Size
Units
CIFAR-10
40000
5000
5000
10
3 x 32 32
Vision
CIFAR-10O-N Coarse
TIA---00-N Fine
40000
40000
5000
5000
5000
5000
100
20
3 x 32
x
32 x
C X X
FMoW
76863
11483
11327
62
3 224
x 224
Text
Guanaco
88500
500
500
32000
X 4000
characters
Yelp
25000
5000
5000
2000
Income
156533
19566
19566
2
816
Public Coverage
1108-4
13855
13855
2
Tahular
Mobility
64265
8032
8032
2
101
features
Employment
303055
37881
37881
2
Travel Time
138008
17250
17250
2
615
Collab
4000
500
500
3
74.49, 2457.78
Graph
Reddit-5k
4001
499
499
5
508.52, 594.87
modes, edges (avg.)
Reddit- 12
95455
1192
1192
391.41, 456.888....8
Table 3: Training Details.
Dataset
Model
Pre-
Optimizer
LR
Weight Decay
Schedule
Epochs
Size
Batch
C-10//10-NN
ResNer18-D
22
Yes
SGD
0.1
Se-4
Cosine
100
500
FMoW
DenseNet121
Yes
Adam
1o-4
Constant
50
64
Guanaco
LLaMA-2-7B
70
Yes
Adam
2c-4
Constant
6
16
Yelp
BERT
13
AdamnW
Se-5
Lineur
Folktables
MLP
No
Adam
0.01
Exponcntial
50
256
Collab
GIN
No
Adam
0.01
Exponential
500
128
Reddit
GCN
33
No
Adam
0.01
Exponential
500
128
C Dataset and Training Details
Tabs. land] 3 summarize the datasets and training details for our experiments. They are described in detail below. We trained our models under these hyperparameters on 48 GB A6000 GPUs in single-GPU setup, except for LLaMA-2-1 fine-tuning on Guanaco, for which we used 80 GB A100 GPUs. Single model training completes in few hours for all datasets except FMoW and Guanaco, on which training took upto 12 hours. We experiment most extensively on the CIFAR-N datasets, where optimized script can train single model in 3-5 minutes
A6000 GPU.
CIFAR-N [74]. CIFAR-IO-T uses the same images as CIFAR-10 but provides multiple human- annotated label sets. Clean is the original label set; R,nd1,2.3 are sets of human labels; Aggre combines Randl .2,3 by majority vote; and Worst combines them by picking an incorrect label, possible. CIFAR-100 has 2 variants, fine-gra fine-grained one with 100 classes and a coarse-gre grainnd one with 20 classes, obtained by grouping the fine-graine aeee-gained classes. Correspondingly, there are CIFAR- -00-N Coarse and CIFAR-100-N Fine datasets. They have two label sets each: Clean and Noisy, with the latter being human-labeled. In the main paper, CIFAR- 100-N refers to the fine-grained version.
By cross-referencing with the original labels, it is possible to estimate the noise levels. These are shown in Table
CIFAR-N allows access to clean labels. In the literature, the validation and test sets for CIFAR-N typically use the clean labels 42
45 78). However, access to clean labels is luxury only available for label noise settings. Even there, obtaining clean labels is expensive, as it requires careful expert annotation. For other sources of noise might not even be feasible obtain clean labels. Hence, we restrict ourselves to using noisy (i.i.d. to train) validation and test sets. Since CIFAR-N only provides
Table 4: Noise levels for CIFAR-N (%), reproduced from 774
CIFAR-10-N
CIFAR-100.
Crarse
CIFAR-100-N Fine
Aggre
Clean
Noisy
Noisy
0.00
9.03
17.23
18.12
17.64
40.21
0.00
25,60
0.00
40.20
19

---

Table 5: Naive vs post-hoe (ours) selection for SWA+TS and SWA+Ens+TS transforms on some real-world datasets. Better values are in bold.
Metric +
Test Loss
Test Firor (%)
Transform
SWA+TS
SWA+En++TS
SWA+TS
SWA+Eis+TS
Dataset &
Naive
Ours
Naive
Ours
Naive
Ours
Naive
Ours
Yelp
0,908
0.890
0.854
0.841
0,824
39.41
38.02
37.33
36.18
36.14
Incoome
0.393
0.390
0.387
0.388
0.385
17.84
17.69
17.54
17.62
17.40
PublicCoverage
0.544
0.540
0.539
0.538
0.538
27.52
27.31
27.25
27.25
27.02
Mobility
0,474
0.472
0.471
0.471
0.468
21.43
21.38
21.42
21.17
21.24
Employment
0,380
0.379
0.378
0.378
0.377
17.94
17.77
17.80
17.72
17.83
TravelTime
0.593
0.591
35.77
35.35
35.23
Callab
0.492
0.475
0.460
0.439
0.404
20.65
21.58
20.27
20.40
18.80
Reddit-Sk
1.154
1.112
1.100
1.101
1.085
47.42
48.35
47.04
47.0
45.49
Reddit-12k
1.405
1,381
1.366
1.367
1.346
51.78
51.08
51.11
50.34
51.26
human labels for the original 50k CIFAR-10/100 train images, we split these into 40k/5k/5k images for train/val/test sets.
FMoW 9/.44). This is the version of the original FMoW dataset 9 as used in the WILDS benchmark [34]. For FMoW (ID) we use the in-distribution val and test sets, and for FMoW (OOD), we use the out-of-distribution val and test sets, where the val set is shifted with respect to the train set, and the test set is shifted with respect both the train and val sets. All splits are as provided by WILDS. The input is an RGB satellite image (rescaled to 224 x 224 pixels) and the label is one of 62 building or land use categories. The labels were obtained by combination of human annotation and cross rooss-effeence geographical information. The original dataset provides additional metadata about location, time, sun angles, physical sizes, etc. which is ignored in the WILDS dataset (and hence in ours). While the labels have low noise compared to the ground-truth, this dataset is noisy because of insufficient information. It hard to disambiguate the building or land use category with full certainty by looking at the satellite image alone. See Figure [T] Models and training setup are as used 9 except for the LR schedule, where we experiment with multiple alternatives.
This is a subset of the OASSTI dataset [37 containing only the highest-rated paths in the conversation tree. We follow the fine-tuning setup from 12] except that we use vanilla fine-tuning without any quantization or low-rank adapters.
Guanaco 12
Yelp (5). This is subset of the Yelp Dataset Challenge 2015 dataset with 25k reviews in the train set and 5k reviews each in the validation and test sets. The imput is review text and the label is one of 5 classes (1 to 5 stars). Assigning rating to review is intrinsically non-deterministic as different reviewers might have different thresholds for the star ratings. This introduces noise in the data.
Folktables [14]. Folktables consists of 5 classification tasks based on the US Census: Income, Employment, Health, TravelTime and PublieCoverage. The data is tabular. The available feature columns do not contain sufficient information to predict the targets with full certainty, even if the Census recorded the ground- ground-truth labels with high accuracy. This results in noise.
Collab and Reddit (52 82). These datasets are from TUDataset (52), and were originally introduced by Yanardag and Vishwanathan [2. Collab
scientific collaboration dataset. The input is an ego-network of researcher and the label is the field of the researcher (one of High Energy Physics, Condensed Matter Physics and Astro Physics). The Reddit-5k and Reddit- 12k datasets (originally called DEDDIT-MULTL, graph which corresponds to an online discussion thread from the social network site Reddit. Nodes correspond to users and there is edge if one user responded to another's comment. The task is to predict which subreddit a discussion graph belongs to. Reddit-5k is smaller with 5k examples and classes. Reddit- 12k is bigger with 12k examples and 11 classes.
and REDDIT-MULTL, 12K) are balanced datasets where the input is
20

---

Table 6: Detailed results for CIFAR-N datasets. Base denotes no transform and Final denotes the SWA+Ens+TSS transform. Gain shows performance improvement. A shows change from naive selection to post-hoc selection. Since Base and Gain columns involve 8 individual runs, we report mean Esid. dev. of the metric. C-10-N, C-100-N-C and C-100-N-F are shorthands for CO-AR-1O-N, (IFAR-100-N Coarse and CIFAR- 100- Fine respectively.
Metric
Tost Loss
Test Error (%)
Dataser
Select
Ppochs
Base
Final
Gain
Epochs
Base
Final
Gain
C-10-N Clean
Naive
90+9
0,435=a.012
0.234
0..022
92+s
9.75+0.24
8.30
1.45+0.24
Post-hoe
100
0,433
0.233
0.200
96
9.822a.27
8.24
1.58
10
1.001go..00 ood
0.001
4gs
0.07 10
0.066
C-10-N
Naive
10g3
0.722=a.01
0.608
0. 0.114:20.018
94
19.20 39
15.88
3.33
Post-hoc
53
0.977
0.543
0.434+0.0c0
58
22.2 +0.02
15.74
6.47+0.62
Aggre
A
t 13ga
0.255gg.027
0.065
36gs
1
3.0000077
0.14
C-10-N
RandI
Post hoc
Naive
31
8+2
1.189=a.011
1.009+0.com oos
0.859
0.916
0.330
0.093+0.cos
67
22+32
31.58-20.51
28. 6.7.7
23.50
24.80
8.08
3.83+0.67
A
23++2
1 0.181 =0.018
0.057
44
1 2.95+0..5
4 1.30
Naive
10
1.040 oos
0.931
0.108820.oo
14g6
29.9010.4
25.44
4.47
C-10-N Rand2
Post-hoe
30
1.189=0.00
0.888
0.301+0.037
74
31.15+0.38
24.12
0.02=0.38
t g
0.150 oas
0.043
60
1 11224.0.50 56
1.32
Naive
9g2
1.005
0.910
0.09510.011
24
28.96 28.96110.s
24.86
4.10.20.s
C-10-N
Rand3
Post hoc
32
1.179ga os7
0.866
0.3155..027
38
32.399...68 68
23.44
8.95
A
23
0.174=00081
0.046
114
1 3.4331..0
1.1222
C-10-N Worst
Naive
8+2
1.511+a.00
1.437
0.073+a.cos
10+a
46.8440.56
44.30
2.54+0.66
Post-hoe
25
1,6-13
1.399
0.24540.011
21
19.6740.74
42.88
6.79.0.74
t 1729
1 0.133ga..oe
0.039
142a
t 2.83310.84
1.42
C-100-N-C
Naive
33
+0.14
0.669
0.342+0.c14
91+4
23.12+0.40
19.36
3.76
Post-hoe
100
1,040=a..0z
0.606
0,435
72
21.3920.43
19.52
4.87:20.43
Clean
t 67235
1 0.029 2.02999..020
0.063
19
1 1.27710.26
0.16
Naive
8ga
1.431
1.234
0.198 ooe
40
41.4240.45
34.42
7.00
Noisy
Post-hoc
32
1.744=0.040
1.150
0.594
38
45.45+00.98
33.54
+0.08
24
1 0.3.13=a..0
0.084
2
t 4.03
0.888
C-100:N:F
Naive
93*5
1.508=a.0z
1.065
0.443
88+5
33.83++0.37
29.90
3.93
Clean
Post-hoc
75
1.567 =0.07
1.063
0.504
95
33.86+0.58
29.94
3.92
A
18+5
1 0.059=a.014
0.002
7+5
0.03++0.31
0.04
C-100-N-F
Naive
7=2
2,116=a.oo 022
2.129
0.287
91
08
51.34
7.34=0.49
Noisy
Post-hoc
27
3.015+a.00 0099
1.994
1.021+0.070
32
63.53+0..6
50.26
13.27
1 20
1 0.598 077
4 0.135
4 59
1 4.85
4 1.08
D Post-Hoc Selection Results for Remaining Datasets
T compares naive and post-hoe selection for datasets not covered in the main paper. Post-hoc selection is mostly better than naive selection, although with varying margins. Post-hoc selection is sometimes worse, but only marginally
E
Detailed Results
Tables 6 and 8 proviid detailed results for CIFAR-N, LLM instruction tuning, and other datasets respectively.
F Optimal Checkpointing for Small Number of Epochs
Throughout the main paper, we use checkpoint interval of epoch. For small-epoch settings, such as LLM pre-training or fine-tuning, it might be better to checkpoint more frequently, at fractional epochs. In this section, investigate the impact of checkpoint interval on the best MMLU score, and the epoch at which it is achieved, for the LLM instruction tuning setup of $ 7.1
'This may be attributed to (1) picking the same epoch for all runs in post hoc selection, and (2) generalization error between validation and test sets for the selected epoch.
21

---

Table Detailed results for LLM instruction tuning. Better values are in bold. Since Base and Gain columns involve 8 individual runs, we report mean 2std. dev. of the metric.
Transform
Nome
SWA+TS
SWA+Ens+
Metric 4
Naive
Ours
Naive
Ours
Perplexity
3.756
3,471
3.461
3,245
3.142
Emor MMLU
32.84
30.81
29.68
30.16
28.93
46.64
46.78
17.03
47.23
17.54
Table 8: Detailed results for other datasets. See Table 6 caption for description.
Objective
Test
(%)
Dalusel 4
Select 4
Epochts
Base
Final
Gain
Egoochs
Base
Final
Gain
Naive
220
1.58320.014
1.194
0.089 -0.011
15219
13. 23.20
37.95
5. 5.2422.48
FMoW (ID)
Post-hac
50
2.83120.053 2.003
1.305
2226_0.000 25.053
43.1810.55
34.93
8.24
A
48
t 1.248
4. 0.189
33
0.02
4.3.02
Naive
2+0
1.831-0.018
1.700
0.131-0.015
3-11
49.32+0.as
46.74
2.58+0.38
FMoW (OOD)
Post-hoc
50
3. 3.399
1,571
1.828 1.828
50
50.0850.38
11.56
8.52.0.38
48 n
20.044
0.129
47
0.755g..0 on
4 5.19
Naive
2+1
0.908+0.008
0.841
0.067-6.008
9+
40.41
36.18
3.23+0.70
Yolp
Post-hoc
3
0.990=0.044
0,824
0.106=0.064
3
10.28+1.29
36.14
1.1441.29
1 1
t 0.082 20.010
J 0.017
1. 62s
0.87
4 0.001
Naive
5
0.393
0.388
0.005_o.00:
722
17.8420.15
17.62
0.2220.15
Income
Post-hac
11
0.421=0.007
0.385
0.036=0.007
19.21+0.14
17.40
1.8120.14
A
1 6
t 0.0288...0m
J. 0.003
12
1.3720:22
0.222
Public Coverage
Naive
10+2
0.544=0.001
0.538
0.006=0.001
12-a
27.52+0.24
27.25
0.28
Post-hac
18
0.554_0.002
0.538
0.016go.002
22
27.9620.21
27.02
0.94
A
8+4
1 0.010-0.o02
0.000
104a
0.444..25
0.22
222
0.474
0.003
1326
21.43
21.17
0.26
Mobility
Post-hoe
14
0.476-0.008
0,468
0.008-0.008
11
21.40+0.17
21.24
0.16+0.17
1 822
1 0.002200003
0.003
4 222
0.03
1 0.07
Nuive
8
0.380_o.000
0.378
0,003
14
17.9410.08
17.72
0.2220.08
Employment
hac
0.383_0.001
0.377
0.006_0.001
18.27
17.83
0.43
A
7+1
0.003
J 0,000
16-4
0.33+..18
1 0.11
Travel Time
Naive
6+2
0..07=0.002
0.596
0.001=0.0m2
35.77
35.44
0.32-20.44
Post-hoc
15
0.626_0.003
0.591
0.035_0.003
16
36. 46.4010.20
35.23
1.17
1 8
00020_0.o03
7
0.64g0 42
4 0.21
Naive
0.492=0.044
0.439
0.053-6.064
75-22
20.65+
20.40
0.25+11.0
Collab
Post-hoe
163
1.075
0.401
0.6712o.122
152
20.95 228
18.60
2.15
111
1 0.583
J. 0.033
77
0.30 31
4 1.60
Naive
15
1.154_0.022
1.101
0.053
13
47. 47.42
47.09
0.33
Reddit-Sk
Post-hoc
44
1.448-0.058
1.085
0.362-6.068
30.75+1.88
45.49
5.20+1.88
A
29
0.29120.059
0.015
32:28
3.3332.05
4 1,60
16+3
1.405
1.367
0.038-0.01.
17-4
51.78+1.05
50.34
1.45++1.0
Reddit 12k
Post hoc
41
1.585 023
1.346
0.239 0.239_0.028
64
55.85
51.26
4.59
25
1 0.100000002
J. 0.021
47
4.07+1.59
1 0.92
Figs. 14a and 14b show the results. We find that a checkpointing interval of 0.7 epochs gives the best results, with higher and lower intervals performing slightly worse. This makes sense- higher intervals include too few checkpoints for SWA, lower ones include too many weaker checkpoints from earlier in training.
Also, we find that the optimal epoch is shifted further at smaller checkpointing intervals (by about epochs when the checkpointing interval is 0.1 epochs), showing that post-hoc reversal is even more important in this setting. This is likely because with more checkpoints being averaged, even more overfitted checkpoints can be accomodated while still increasing the overall performance.
G Visualizing Post-Hoe Reversal on a Synthetic Dataset
Here, we replicate post-hoe reversal on a synthetic dataset with input features, with the aim of visualizing learnt decision surfaces to solidify our intuitions.
22

---

Guanaco
L1aMA-2-78
3.5
18
MMLU
47.0
46.5
2.0
14
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
085
1.0
Checkpoint interval (epochs)
Checkpoint interval (epochs)
12
SWA+TS
SWA+TS --r Ers
Epochs
(a) Best epoch vs checkpoint freq.
(b) MMLU vs checkpoint freq.
200
400
500
500
Figure 14: Best MMLU and epoch at which it is achieved vs checkpoint- ing interval, for the LLM instruction tuning setup of $ 7.1 Checkpointing every 0.7 epochs gives the best results. Best epoch is shifted further smaller checkpointing intervals, i.e. post-hoe reversal is more prominent this setting.
The syn- thetic dataset setup in G exhibits post-hoc reversal between epochs 440 and 1000.
Figure 15:
Model 1
Model
Ensemble
Epoch 440
Epoch 1000
Figure 16: Decision surfaces of 2 models and the ensemble (of 16 models) on a synthetic 2D dataset of spirals, at epochs 440 and 1000, between which post-hoc reversal occurs (Fig. 15)
We train 4-layer MLPs with 512 ReLU units per hidden layer on a 2-class spirals dataset of 1000 training examples, with 20% of the labels flipped at random. We train 16 MLPs and track the mean test eтrоr across epochs, as well as the test error of the ensemble (Fig. [5
As per 3 [6] ensembling and SWA help when the data has a "multi-view" structure, or equivalently, the loss landscape has multiple modes. This is hard to achieve for 2D datasets, instead we simulate the effect by training each MI.P on a random 50% subsample the training data.
Fig. 16 shows decision surfaces epochs 440 and 1000 for 2 MLPs and the ensemble. Decision boundaries are spiky around noisy examples and smoother around clean ones. While the generalizable parts of the spiral are retained in the ensemble, the effects of noisy examples are diminished. Between epochs 440 and 1000, individual models spike around noisy examples more prominently than they learn new parts of the spiral, but the ensemble surface is relatively unchanged, except for small improvements to learning the spiral.
This reinforces our intuitions from $ that that mislabeled examples have a more unstable influence on the decision boundary, and post-hoc transforms exploit this to reduce their impact, while amplifying generalizable patterns learnt from clean examples.
23

---

Table 9: Naive vs post-hoc (ours) selection for CIFAR-N trained with cross-entropy (CE) loss. Better values are in bold.
Metrie
Text Loss
Test Emor (8)
Transform +
SWA+TS
SWA+Em+IS
Nome
SWA+TS
SWA+Ems-TS
Dataset
Naive
Ours
Naive
Ours
Naive
Ours
Naive
Ours
C-10-N Clean
0.435
0.269
0.270
0.234
0.233
9.75
9.09
9.10
8.30
8.24
C-10.N Aggre
0.585
19.20
17.08
15.74
C- C-10-N Rand
1.009
0.968
0.907
0.916
0.859
28.63
27.13
24.84
24.80
23.50
C-10-N Rand2
1.040
0.983
0.935
0,931
0.888
29.91
27.60
25.69
25.44
24.12
10-N Rand3
1.005
0.963
0.911
0.910
0.864
28.96
26.91
25.09
24.86
23.44
10-N Worst
1.511
1,483
1.443
1.437
1.399
46.84
46.12
44.14
44,30
42.88
Clean
L011
0.786
0.686
0.669
0.606
23.12
21.30
21.38
19.36
19.52
Noisy
1.431
1,330
1.235
1.234
1.150
41,42
38.08
35.87
34.42
33.54
C-100-N Clean
1,508
1.215
1.205
1.065
1.063
33.83
32.67
32,69
29.90
29.94
C-100-N Noisy
2.289
2.136
2.129
58.68
54.94
50.26
Table 10: Naive vs post-hoc (ours) selection for CIFAR-N trained with SOP loss. Better values are in bold.
Metric +
Tost
Tost Error (%)
Transform -
None
SWA+TS
TWs+F
None
SWA-TS
TS
Datasct
Naive
Ours
Naive
Ours
Naive
Ours
Naive
Ours
C-10.N Clean
0.425
0.270
0.269
0,236
0.235
9.65
8.82
8.81
7.96
8.00
C-10-N Aggre
0.728
0.693
0.573
0.634
0.541
18.03
16.55
16,56
15.58
15.56
C-10-N
1.025
0.980
0.925
0.851
26.91
24.50
23.20
23.14
C-10-N Rand2
1.045
1.015
0.920
0.957
0.883
27.39
25,34
25.25
24.12
24.16
C-10-N Rand3
1.016
0.975
0.889
0,921
0.851
26.66
24.23
24.23
23.02
22.96
C-IO-N Werst
1.514
1.492
1.451
1.447
1.413
46.78
46.26
44.29
44.50
42.78
Clean
1.018
0.742
0.686
0.623
0.608
23.07
21.47
19.18
19.78
Noisy
1.427
1.347
1.229
1.247
1.145
41.39
38.01
35.85
34.32
33.94
C-100.N N Cleun
1.513
1.213
1.203
1.063
1.061
33.79
32.66
32,68
29.46
29.56
C-100-N Noisy
2.415
2.268
2.137
2.118
1.997
58.34
54.76
53.48
51.06
50.54
H
Noise-Aware Training
While our experiments in the main paper use the standard cross-entropy (CE) loss, here we consider two leading training objectives from the label noise literature: (1) SOP 45 and (2) ELR 42 Tables 9 and 11 an compare naive and post-hoc selection strategies for CIFAR-N datasets under CE. SOP and ELR losses respectively. Here again we find that post-hoe selection is superior to naive selection in general. We also note that the differences between CE, SOP and ELR are minimal. This is likely because we use i.i.d. (and therefore noisy) validation and test sets, unlike the original papers which use clean validation and test sets.
Table 11: Naive vs post-hoc (ours) selection for CIFAR-N trained with ELR loss. Better values are in bold.
Metric +
Tost
Test Error (%)
Transform +
None
SWA+TS
SWA+Ens+TS
None
SWA-TS
TWA+Ens+
Dataset 4
Naive
Ours
Naive
Ours
Naive
Ours
Naive
Ours
C-10-N Clean
0.421
0.271
0.269
0.233
0.232
9.53
8.92
9.01
7.98
7.92
C-10-N Aggre
0.730
0.659
0.584
0.606
0.541
19.02
16.86
16.80
15.34
15.52
C-10-N Rand1
1.019
0.975
0.911
0.921
0.861
29.42
26.68
24.86
24.30
23.56
C-10-N Rand2
1.42
0.994
0.939
0.941
0.893
29.79
27.98
25.74
26.12
24.50
10.N Rand3
0.964
0.913
0.866
24.84
24.52
23.32
1O.N Woest
1.508
1.492
1.443
1.444
1.397
46.94
46.27
44.03
44.64
42.48
Clean
1.030
0.760
0.686
0.644
0.605
23.07
21.27
21.40
19.28
19.24
Noisy
1.415
1.317
1.236
1.228
1.152
41.55
38.40
35.74
34.72
33.60
C-10O-N Cleun
1.518
1.223
1.210
1.070
1.068
34.05
32.92
32.97
29.68
29.66
C-100-N Noisy
2.432
2.287
2.140
2.130
1.997
58.85
54.84
53.24
50.86
50.50
24

---

I Limitations
We find post-hoe reversal be an important phenomenon when the base curve exhibits performance degradation due to overfitting. However, under some scenarios, the base curve shows a monotonic improvement performance with additional training (or increasing model size). Examples include: (1) the data has low noise, (2) the training is heavily regularized, and (3) there is an abundance of data, so that single data point not repeated enough to cause overfitting. such cases, post-hoc selection outcomes are similar to naive selection. Since our suggested approach only ensembles models trained for the same number of epochs during post-hoe selection, does not subsume the naive selection search space, leading to marginally worse performance sometimes, although this can be easily overcome in practice.
25

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Please see App. !
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
26

---

Justification: [NA]
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
Code reproduce experiments is available at hattps://anonynnn 4open science//rpost-hoc-- reversal Comprehensive dataset and training details are pro- vided in App.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
27

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Please code (including instructions to download data and reproduce the main results) at atlttpss:/amonyyoooo .oop// 4open. sciiecc/rr/pott-oc-reeeeeeeee
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://mips.cc.
aablic//uuddes/Coeeeubbiis caapssionoolyyyy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ wwtps/cccppplgguuuiieeeeae
Ciaeee//CSSSuSbisSS..S.. www.ssssiicgcyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please see App. C
particular, see Tab. 3 for training details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper reperreport error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification; Please see Tabs. 6 and for detailed results with standard deviations. We don't 't report error bars for ensembles as it is computationally prohibitive to train many independent ensembles.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
28

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please see App. C for training details, including compute resources used.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: htteee://eeuuuiee www./peu.p.../. wwttrrps.cc/pullie cc/public/Bthiccchieeelllll
Answer: [Yes]
Justification: The research conducted in the paper conforms, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the broader impact of our work in Sec.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
29

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All datasets and models used in this paper under permissive licenses allowing for research use. We have credited their authors by exhaustively citing sources.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
30

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssit
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
[NA]
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
31

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
32


Reviews:
[{"review_id": "s3rQzTU04J", "reviewer": "Reviewer_vEtm", "summary": "The paper discusses the phenomenon of *post-hoc reversal*, where the performance trend is reversed after applying post hoc transforms, namely, temperature scaling (TS), stochastic weight averaging (SWA), and ensembling (Ens).\n\nThe paper conducts an empirical study to observe the phenomenon across different epochs, model sizes, and hyperparameters.\n\nThey propose a *post-hoc selection* strategy where the optimal epoch/model size/hyperparameters are selected considering the performance after applying post-hoc transforms.\n\nThe paper focuses on the noisy data setting and shows experimental gains in that setting for some datasets. \n\nExperiments have been conducted for the FMoW dataset, CIFAR-10-N, and CIFAR-100-N, as well as some additional experiments on text, tabular, and graph datasets.", "questions": "**Questions**\n1. Why is post-hoc reversal prominent in the noisy data setting? Can the authors provide intuition or reasoning (other than empirical observations)?\n2. Can the authors provide intuitions to (at least) some of the observations? For example, why do SWA and Ens handle double descent, but TS causes it?\n3. In Figure 7, what is the SWA ensemble? It has not been discussed in Section 5.\n4. For the datasets Yelp, Income, and Reddit-12K, the post-hoc reversal is observed in Figure 7, but the post-hoc selection strategy either shows worse performance or marginally better. Does this indicate that the post-hoc selection strategy is ineffective in these settings? Can the authors provide any reasoning/explanations?\n\n**Suggestions**\n* The paper title should include \"noisy data setting\", as the paper focuses on the noisy data setting.\n* Include intuitions/reasonings/explanations (wherever possible) for the observations in the experimental results.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper demonstrates the phenomenon of post-hoc reversal.\n2. They conduct experiments on diverse domains.\n3. Performance gains are achieved with the proposed strategy in some of the noisy data settings.", "weaknesses": "1. The paper proposes *post-hoc selection* strategy, simply selecting the optimal epoch (or model size, etc.) after applying already existing (and widely used) post-hoc transforms (SWA, En, TS). They do not propose any *novel* mechanism to tackle *post-hoc reversal*.\n2. It is a findings paper; the study is empirical and lacks theoretical insights. Given the empirical nature of the study, the performance gains in some datasets are marginal (e.g., C-100-N Noisy test error, Table 1).\n3. For the datasets in Table 5, the post-hoc selection strategy does not consistently provide performance gains, i.e., it performs worse in some cases.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer vEtm,\n\nThank you for your review, including suggestions to improve the paper.\nWe address your concerns below:\n\n> They do not propose any _novel_ mechanism to tackle _post-hoc reversal_.\n\nFirst, we would like to clarify that\npost-hoc reversal (PHR) is not a problem.\nOn the contrary,\nPHR usually provides\nan opportunity to improve performance.\nBy post-hoc selection (PHS), we show that\na simple change to existing methodology\nis sufficient to reap benefits.\nWe leave smarter checkpoint selection to future work,\nas it is beyond the scope of this work,\nwhose primary aim is to demonstrate and characterize PHR.\n\n\n> Given the empirical nature of the study, the performance gains in some datasets are marginal (e.g., C-100-N Noisy test error, Table 1).\n\n> For the datasets in Table 5, the post-hoc selection strategy does not consistently provide performance gains, i.e., it performs worse in some cases.\n\nWhen solely considering the test error metric, this is indeed true.\nPlease note that test error is capped by the Bayes error and the model's strength.\nFor example, C-100-N Noisy has \\~40% noise and ResNet18 gets \\~10% test error with clean data.\nGiven this lower bound of \\~50%, the achieved error of 50.26% could still be considered impressive.\n\nIn fact, test error is not the best metric for datasets with high Bayes error.\nHence our equal focus on the test loss metric, which evaluates predicted probabilities.\nWe humbly point out that across _all_ our datasets and transforms, PHS outperforms naive selection, and in most cases quite significantly so.\n\nEven for test error, PHS is worse in only 5 of our 21 datasets, and except in one of these (Reddit-12k) it's less than 0.2 pts worse. Given that PHS is simple and cheap,\nwe stand by our recommendation to employ it in practice.\n\n\n> Why is post-hoc reversal prominent in the noisy data setting? Can the authors provide intuition or reasoning (other than empirical observations)?\n\nPlease see our global rebuttal\nwhere we provide detailed intuitions\nfor PHR and highlight it's important connection to noise.\nOur explanations cover\nepoch-, model-, and hyperparamter-wise PHR;\nTS, Ens and SWA transforms;\nand imporant consequences such as to\nloss-error mismatch, catastrophic overfitting and double descent.\n\nWe further validate our proposed intuitions\nwith experimental analysis on CIFAR-10N,\nand visualization of the learnt decision surfaces\non a synthetic dataset.\n\n\n> Can the authors provide intuitions to (at least) some of the observations? For example, why do SWA and Ens handle double descent, but TS causes it?\n\nWe again refer you to our global rebuttal for\ncomprehensive explanations of various observations.\nHere we restate our explanation for the particular observation mentioned,\nnamely, why SWA and Ens handle double descent but TS causes it.\n\nIn all our experiments,\nwe do not observe double descent in the base test loss curves.\nOverfitting occurs too drastically in the test loss\nfor the second descent to occur.\nThis is because once a noisy training point is fit,\nthe model can lower the loss simply by upscaling the logits.\nThis leads to overconfident predictions\nand poor generalization loss around noisy training points.\nHowever, scaling the logits does not affect error.\nIndeed test error overfits less,\nsometimes even exhibiting a second descent,\nwhere the continued learning overpowers the overfitting.\nBy rescaling the logits, temperature scaling\ndoes not so much cause double descent\nas it removes the loss-error mismatch.\nIf the test error curve has a double descent,\nthe post-TS test loss curve does too,\nsimply because the post-TS loss tracks the error more closely.\n\nAs mentioned earlier, we don't observe double descent in loss curves,\nso also no instances where TS removes double descent,\nas TS is unable to affect the test error metric.\n\nFor intuitions on why Ens/SWA can suppress double descent,\nplease refer to [Intuition 3] in the global rebuttal.\n\n\n> In Figure 7, what is the SWA ensemble? It has not been discussed in Section 5.\n\nSWA ensemble refers to the ensemble of SWA models,\ni.e., first apply SWA to checkpoints from the same training run,\nthen ensemble the SWA models across different runs.\nThank you for pointing this out.\nWe will clarify in the manuscript.\n\n\n> For the datasets Yelp, Income, and Reddit-12K, the post-hoc reversal is observed in Figure 7, but the post-hoc selection strategy either shows worse performance or marginally better. Does this indicate that the post-hoc selection strategy is ineffective in these settings? Can the authors provide any reasoning/explanations?\n\nThis is a fair observation.\nLooking carefully at Figure 7,\none finds that while PHR occurs quite prominently,\nthe optima of the post-hoc curves are only marginally better than\nthe optima of the corresponding base curves.\nAnother source of error is that while PHR curves\nare all drawn for the test set,\nfor the PHS results,\nthe epoch is selected based on the val set,\nand the reported numbers are evaluated on the test set.\n\nWe believe that this does not diminish the usefulness\nof PHS, because it is never substantially worse\nthan naive selection.\n\nFurther, depending on the setting, one might improve results\nby ensembling more models.\nWe ensemble 8 models throughout the paper for uniformity,\nbut for small tabular datasets like Income,\nit is feasible to ensemble many more models.\n\nA final factor that may be playing a role here,\nis that post-hoc transforms under any selection strategy\ncannot be expected to give too much improvement,\nif the base models already achieves close to\nthe Bayes error for the dataset,\nbut this is hard to evaluate beforehand.\n\n\nPlease let us know if any of your concerns are still unaddressed.\n\nThanks,\n\nAuthors", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7WttHAPCjK", "reviewer": "Reviewer_NSoG", "summary": "The authors investigate applying three post hoc transforms, namely temperature scaling (TS), ensembling, and stochastic weight averaging (SWA), to trained models after training, separately and jointly. They oppose their approach to the \"naive approach\" and provide empirical observations of a phenomenon they refer to as \"post hoc reversal.\"  This phenomenon corresponds to the performance trends of models that change after these transformations. In particular, they focus on the observation in noisy settings of the following observations.\n\n1- Epoch-wise post hoc reversal. The authors show that post hoc transforms reduce overfitting, double descent, and loss-error mismatch. In particular, SWA and ensembling reduce overfitting and flatten double descent peaks.\n\n2- Model-wise post hoc reversal. The model's size influences the improvement in the performance of the post hoc transforms, i.e., larger models tend to perform better under post hoc transforms.\n\n3- Hyperparameter-Wise Post Hoc Reversal. It appears that constant learning rates improve performance better than decaying learning ones and the optimal number of epoch shifts. \n\nThe experiments are broad, and metrics show good non-trivial improvement based on the method.", "questions": "If the author can provide a good answer to the question above that would address my concerns.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1 - The paper provides an extensive empirical study across various domains (vision, language, tabular, graph), aiming to demonstrate the generality of their approach, i.e., post hoc transforms. \n\n2- The authors use a variety of performance metrics (error rates, loss, MMLU accuracy) to validate the effectiveness of post hoc transforms, leading to a robust evaluation.\n\n3- The authors identify a potentially impactful post hoc reversal phenomenon that could challenge commonly adopted practices.\n\n4- The approach seems to lead to consistent performance gains across different datasets and settings. The focus is on noisy environments, which can be particularly relevant for less curated datasets, which might be what the field needs. \n\n5- The authors suggest that their approach could lead to guiding principles for model development, including early stopping and checkpoint selection.", "weaknesses": "The main weakness of the approach, which is recurrent in most current empirical LLM observations, is that understanding the source of the improvement needs to be clarified. \n\nIn fact, it is not clear why the benefits of post hoc transforms are more pronounced in high-noise settings.\n\nMaking assumptions to verify potentially synthetic and controlled datasets could be helpful. As of one, this corresponds to an observation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer NSoG,\n\nThank you for your review.\nWe provide a detailed reponse\nregarding the intuitions and explanations for post-hoc reversal (PHR)\nin the global rebuttal, along with experimental analyses on CIFAR-10N\nand a synthetic dataset to back our claims.\nBelow we highlight how the global rebuttal answers your questions.\n\n> the source of the improvement needs to be clarified.\n\nEven when performance degrades due to overfitting,\nthe model continues to learn generalizable patterns\nfrom the clean training points,\nbut this is outweighed by the spurious patterns\nlearnt from the noisy training points.\nUnder post-hoc transforms,\nthe generalizable patterns reinforce each other,\nwhereas the spurious patterns cancel out.\nThis is responsible for the performance improvement\nseen under PHR.\n\nWe expand on this intuition in the [Intuition 3] section of the global rebuttal,\nand further demonstrate it for CIFAR-10N under Experiment 2,\nand for a synthetic dataset under Experiment 3.\n\n\n> it is not clear why the benefits of post hoc transforms are more pronounced in high-noise settings.\n\nIn high-noise settings, noisy training points have\na greater adverse effect on the learnt model.\nPost-hoc transforms subdue this to a large extent.\nThis produces a more pronounced benefit.\nIn contrast, the base models themselves perform well in the low-noise setting,\nleaving less room for improvement by post-hoc transforms.\n\nWe give a more detailed explanation in the global rebuttal,\nseparately discussing the mechanisms by which temperature scaling\nand ensembling/SWA operate in the presence of noise\n([Intuition 2] and [Intuition 3] sections respectively),\nthereby elucidating their increased efficacy with higher noise levels.\n\n\n> Making assumptions to verify potentially synthetic and controlled datasets could be helpful.\n\nThank you for suggesting this.\nIn the global rebuttal,\nwe replicate post-hoc reversal\non a synthetic controlled-noise dataset with 2 input features,\nand visualize the learnt decision surfaces to\nverify our proposed intuitions\n(Experiment 3).\nWe plan to further improve this analysis\nand incorporate it in our paper,\nalong with the intuitions/explanations above.\n\nPlease let us know if any of your concerns are still unaddressed.\n\nThanks,\n\nAuthors", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oEXpLEGww3", "reviewer": "Reviewer_GiJ5", "summary": "This paper shows empirical evidence that common approaches to model selection can be improved upon by less greedy alternatives. In particular, authors highlighted what they referred to as post-hoc reversal, when post-training transformations reverse the trends observed in independent training runs. For instance, in certain overfitting situations, transformations such as temperature scaling, weight averaging, and ensembling, reversed growing test error trends and closed the generalization gap to a greater extent than common model selection approaches. This observation then yielded practical recommendations of incorporating post-training transforms into the model selection pipeline, which resulted in improved performance across a number of settings, as well as on smoother error curves that render model selection less noisy. Authors attributed post-hoc reversal to the high variance involved in the training of neural networks, which can be smoothed out by some of the studied transformations. The paper lays out clear practical recommendations as a result of the reported empirical observations, which I believe would be highly relevant to the community.", "questions": "- Would it be possible to replicate a subset of the results reported in section 4 and 5 for a larger scale dataset (e.g., ImageNet)?\n\n- Does post-hoc reversal still happen in situations where the test set is somehow shifted relative to the validation set used for post-hoc selection?\n\n- How does one leverage post-hoc selection in a non-multi-epoch training situation?", "limitations": "Please refer to the \"Weaknesses\" section.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well written and easy to follow, and tackles a highly relevant problem in practice.\n\n- The evaluation is broad and covers a number of model classes and sizes, and data modalities.\n\n- Conclusions lay out very clear and easy to implement recommendations to improve model selection.", "weaknesses": "- This requires no action from the authors but just for the record, I have mixed feelings about the presentation choices regarding the use of notation. The text is notation heavy, which makes it precise, but at the cost of readability. The discussion seems simple enough to enable the use of almost plain text only. But I reiterate that I consider this to be mostly matter of writing style, and I expect no effort from authors addressing this comment.\n\n- The bulk of the empirical assessment focuses on a somewhat small scale setting involving variations of the Cifar-10 dataset and relatively small neural networks. While this is obviously due to the high cost involved in replicating these experiments in larger scale settings, it limits the strength of the presented empirical evidence. However, section 6 adds results for other settings including fine-tuning of very large models, which address this concern to an extent.\n\n- The experiments focus on multi-epoch training settings, and it's a bit unclear how those results transfer to now common pre-training situations where very few epochs are used. While one can replace epoch-end checkpoints by checkpoints obtained every k steps, it's unclear how the choice of k affects results, for instance.\n\n- One component that seems to have been left out of the analysis is robustness. Does post-hoc reversal still happens in situations where the test set is somehow shifted relative to the validation set used for post-hoc selection? It might be the case that there's not so pronounced of a gap between post-hoc and greedy/naive selection in such a scenario.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer GiJ5,\n\nThank you for your review\nand thoughtful suggestions about writing.\nWe will take them into account\nwhile revising the manuscript.\nWe address your concerns regarding experiments below.\n\n> Would it be possible to replicate a subset of the results reported in section 4 and 5 for a larger scale dataset (e.g., ImageNet)?\n\nDue to limited time in the rebuttal period,\nwe are unable to report results on ImageNet\nat this time.\nHowever, we appreciate this suggestion\nand will look into incorporating larger scale datasets\nin the final version of this work.\n\n> Does post-hoc reversal still happen in situations where the test set is somehow shifted relative to the validation set used for post-hoc selection?\n\nYes.\nWe report an experimental analysis\non the FMoW dataset below.\n\n## Experiment 1:\nWe used the distribution-shifted\nval and test sets provided by WILDS [1].\nHere, the val set is shifted w.r.t. the train set\nand the test set is shifted w.r.t. both the train and val sets.\nThe rest of the setup is same\nas for Table 1 in the paper.\nFor convenient comparison,\nwe also reproduce the\nin-distribution FMoW numbers\nfrom the paper.\nThe in-distribution FMoW dataset is denoted with ID,\nwhile the distribution-shifted (out-of-distribution)\nversion is denoted with OOD.\nThe 4 tables below\nshow the results for\nthe test loss and test error metrics\nand for the SWA+TS and SWA+Ens+TS transforms.\n_Base_ column has numbers without any post-hoc transform,\nwhile _Naive_ and _Ours_ represent\nthe application of post-hoc transforms\nwith naive and post-hoc selection strategies respectively.\n\n\n__Test Loss, SWA+TS:__\n|     | Base  | Naive | Ours      | Diff  |\n|-----|-------|-------|-----------|-------|\n| ID  | 1.583 | 1.627 | **1.554** | 0.073 |\n| OOD | 1.831 | 1.840 | **1.788** | 0.052 |\n\n__Test Loss, SWA+Ens+TS:__\n|     | Base  | Naive | Ours      | Diff  |\n|-----|-------|-------|-----------|-------|\n| ID  | 1.583 | 1.494 | **1.305** | 0.189 |\n| OOD | 1.831 | 1.700 | **1.571** | 0.129 |\n\n__Test Error (%), SWA+TS:__\n|     | Base  | Naive | Ours      | Diff  |\n|-----|-------|-------|-----------|-------|\n| ID  | 43.20 | 42.69 | **39.92** | 2.77  |\n| OOD | 49.32 | 49.70 | **46.75** | 2.95  |\n\n__Test Error (%), SWA+Ens+TS:__\n|     | Base  | Naive | Ours      | Diff  |\n|-----|-------|-------|-----------|-------|\n| ID  | 43.20 | 37.95 | **34.93** | 3.02  |\n| OOD | 49.32 | 46.74 | **41.56** | 5.18  |\n\n\nWe observe that post-hoc selection is about as effective\nin the OOD case as in the ID case.\nInterestingly, the improvement\nfor OOD as compared to ID\nis slightly lower for test loss\nbut higher for test error.\n\nThank you for suggesting this experiment.\nWe will add it to the paper.\n\n\n> The experiments focus on multi-epoch training settings, and it's a bit unclear how those results transfer to now common pre-training situations where very few epochs are used. While one can replace epoch-end checkpoints by checkpoints obtained every k steps, it's unclear how the choice of k affects results, for instance.\n\nThank you for suggesting this analysis;\nwe believe it would be a valuable addition to the paper.\nWe have conducted this experiment\non our LLM instruction tuning setup from Section 6.1,\nwhere best results are obtained within 3-4 epochs.\n\n## Experiment 2:\nWe vary the checkpointing interval\nas a fraction of 1 epoch,\nand record the best test accuracy,\nas well as the epoch at which it is obtained\nfor both SWA+TS and SWA+Ens+TS.\n\nFigs. 4 (a) and (b) in the PDF attached to the global rebuttal\nshows the results.\nWe find that a checkpointing interval of 0.7 epochs gives the best results,\nwith higher and lower intervals performing slightly worse.\nThis makes sense:\nhigher intervals include too few checkpoints for SWA,\nlower ones include too many weaker checkpoints from earlier in training.\n\nAlso, we find that the optimal epoch is shifted\nfurther at smaller checkpointing intervals\n(by about 2 epochs when the checkpointing interval is 0.1 epochs),\nshowing that post-hoc reversal is even more important\nin this setting.\nThis is likely because with more checkpoints being averaged,\neven more overfitted checkpoints can be accomodated\nwhile still increasing the overall performance.\n\n\nPlease let us know if any of your concerns are still unaddressed.\n\nThanks,\n\nAuthors\n\n[1] WILDS: A Benchmark of in-the-Wild Distribution Shifts", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
# Overview

We thank all the reviewers for their feedback
and helpful suggestions on improving the work.
We have added extensive explanations and intuitions,
backed by numerous additional experiments and analyses.
We summarize them below:
1. Epoch-, model-, and hyperparameter-wise post-hoc reversal (PHR)
can be viewed under the common lens of effective model complexity (EMC),
allowing us to focus on intuitions for epoch-wise reversal (Intuition 1).
2. Loss-error mismatch results from increasingly overconfident predictions
blowing up the test loss in the presence of noise, but not test error.
Temperature scaling fixes this by downscaling the logits,
resulting in PHR (Intuition 2).
3. Neural networks learn noisy points differently
than clean ones.
To wit,
predictions for noisy points fluctuate more during training
(see Expt. 1 on CIFAR-10N),
indicating unstable decision boundaries around them
(see Expt. 3 on synthetic dataset).
4. Building on the above,
ensembling and SWA act differently
on patterns learnt from clean and noisy points
(see Expt. 2 on CIFAR-10N),
reinforcing the former and
suppressing the latter
(see Expt. 3 on synthetic dataset).
Post-hoc reversal is observed when after transform
continued learning from clean points outshines
overfitting from noisy points (Intuition 3).
5. Post-hoc selection is effective even under distribution shift
(See Expt. 1 in response to reviewer GiJ5).
6. Post-hoc reversal is even more relevant for
checkpointing at fractional epochs in very-low-epoch settings.
(See Expt. 2 in response to reviewer GiJ5).

# [Intuition 1] Effective Model Complexity (EMC)

Epoch-, model-, and hyperparam-wise PHR can be unified via EMC,
introduced in [1] to unify epoch-, and model-wise double descent.
EMC measures memorization capacity,
and is important to us because
memorization of noisy point plays a key role in PHR.
EMC increases with epochs and model size,
and different hyperparams can impact it in different ways.
For example, EMC increases with epochs more rapidly for constant LR
than annealed LR, explaining our observations in Section 4.2.3.


# [Intuition 2] Temperature Scaling (TS) and Loss-Error Mismatch

Once a neural net has fit a training point,
the cross-entropy loss on it
can be lowered simply by upscaling the weights of the linear output layer.
This makes the model overconfident later in training,
as shown in [2].
For a noisy training point,
this leads to worse loss on similar test points.
The test error is not affected
as it depends only on the argmax of the class probabilities.
In high-noise settings,
test loss can worsen due to fitting noisy training points,
even as the test error improves from continued learning on clean points,
leading to loss-error mismatch.
TS fixes this by downscaling the logits.
Indeed, one finds that the temperature
(as obtained with a held-out set)
increases with epochs.
(see Fig. S1 in the Supplementary of [2]).


# [Intuition 3] Ens/SWA and Delayed Catastrophic Overfitting

We focus on test error as post-TS loss behaves similarly
and the intuitions transfer.
Flattening double descent is a special case
of delayed catastrophic overfitting,
as applied to the ascent to the peak.

From clean training points,
models learn generalizable patterns
and from noisy points, spurious ones which cause overfitting.
When noise is low,
the former dominates
and overfitting is benign.
Otherwise, overfitting is catastrophic.

The core intuition for Ens/SWA delaying catastrophic overfitting
is that generalizable patterns across checkpoints get reinforced,
while the spurious patterns cancel out.
Further intuition is that this is enabled by
the decision boundary being more "unstable" around noisy points
as compared to clean ones.
The experiments below substantiate these claims.


## Experiment 1
In Fig. 1, we see that
across epochs, the prediction flips
for a much higher fraction of the noisy points
than of the clean ones,
indicating higher instability.
The dataset here is CIFAR-10N Worst (~40% noise),
and training setup is same as in the paper.

## Experiment 2
Here, we measure the average predicted probability
for the clean and noisy subsets of CIFAR-10N Worst,
as proxy for the extent of memorization.
In Fig. 2 (a) and (b),
we find that SWA lowers the memorization
of clean points only a bit (\~0.1 probability),
but of noisy points by a lot (\~0.5 probability),
clearly establishing the differential effect.

## Experiment 3
Here, we replicate PHR on a synthetic dataset
with 2 input features,
with the aim of visualizing learnt decision surfaces
to solidify our intuitions.

We train 4-layer MLPs with 512 ReLU units per hidden layer on a 2-class spirals dataset of 1000 training points, with 20% of the labels flipped at random.
We train 16 MLPs and track the mean test error across epochs, as well as the test error of the ensemble (Fig. 3 (b)).

As per [3,4] Ens/SWA helps when the data has a "multi-view" structure,
or equivalently, the loss landscape has multiple modes.
This is hard to achieve for 2D dataset,
so instead we simulate the effect by training each MLP on a random 50% subsample
of the training data.

Fig. 3 (a) shows decision surfaces at epochs 440 and 1000 for 2 MLPs
and the ensemble.
Decision boundaries are spiky around noisy points
and smoother around clean ones.
While the generalizable parts of the spiral are retained in the ensemble,
the effects of noisy points are diminished.
Between epochs 440 and 1000,
individual models spike around noisy points more prominently
than they learn new parts of the spiral,
but the ensemble surface is relatively unchanged,
except for small improvements to learning the spiral.

We will further polish and incorporate the above in the paper.

# References

[1] Deep Double Descent: Where Bigger Models and More Data Hurt

[2] On Calibration of Modern Neural Networks

[3] Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning

[4] Deep Ensembles: A Loss Landscape Perspective

Meta-review: **Summary**

This is an empirical study that shows that performance trends observed during training might reverse after post-hoc transforms like temperature scaling or stochastic weight averaging. This leads to the practical recommendation of *post-hoc selection*, i.e. to choose intermediate checkpoints based on their contribution to post-hoc performance rather than individual-run performance. Results are surprising, e.g. sometimes this method favors overfitting on individual runs.

**Recommendation**

This work is interesting for the research community and has some practical implications. I am happy to support its acceptance.
Decision: Accept (poster)