On the Reproducibility of Neural Network Predictions
Authors: Srinadh Bhojanapalli
Kimberly Jenney Wilber
Andreas Veit
Ankit Singh Rawat
Seungyeon Kim
Aditya Krishna Menon
Sanjiv Kumar
Venue:  (2021)
Keywords: reproducibility
churn
confidence

Abstract: Standard training techniques for neural networks involve multiple sources of randomness, e.g., initialization, mini-batch ordering and in some cases data augmentation. Given that neural networks are heavily over-parameterized in practice, such randomness can cause {\em churn} -- disagreements between predictions of the two models independently trained by the same algorithm, contributing to the `reproducibility challenges' in modern machine learning. In this paper, we study this problem of churn, identify factors that cause it, and propose two simple means of mitigating it. We first demonstrate that churn is indeed an issue, even for standard image classification tasks (CIFAR and ImageNet), and study the role of the different sources of training randomness that cause churn. By analyzing the relationship between churn and prediction confidences, we pursue an approach with two components for churn reduction. First, we propose using \emph{minimum entropy regularizers} to increase prediction confidences. Second, we present a  novel variant of co-distillation approach~\citep{anil2018large} to increase model agreement and reduce churn. We present empirical results showing the effectiveness of both techniques in reducing churn while improving the accuracy of the underlying model.

Under review as a conference paper at ICLR 2021
ON THE REPRODUCIBILITY OF NEURAL NETWORK PREDICTIONS
Anonymous authors Paper under double-blind review
ABSTRACT
Standard training techniques for neural networks involve multiple sources of ran- domness, e.g., initialization, mini-batch ordering and in some cases data augmenta- tion. Given that neural networks heavily over-pat wver-parameterized in practice, such randomness can cause churn - disagreements between predictions of the two mod- els independently trained by the same algorithm, contributing to the 'reproducibility challenges' in modern machine learning. In this paper, we study this problem churn, identify factors that cause and propose two simple means of mitigating it. We first demonstrate that churn is indeed an issue, even for standard image classi- fication tasks (CIFAR and ImageNet), and study the role of the different sources of training randomness that cause churn. By analyzing the relationship between churn and prediction confidences, we pursue an approach with two components for churn reduction. First, we propose using minimum entropy regularizers to increase prediction confidences. Second, we present a novel variant of co-distillation approach (Anil et al., 2018) increase model agreement and reduce churn. We present empirical results showing the effectiveness of both techniques reducing churn while improving the accuracy of the underlying model.
INTRODUCTION
Deep neural networks (DNNs) have seen remarkable success in range of complex tasks, and significant effort has been spent on further improving their predictive accuracy. However, an equally important desidcratum of any machine learning system is stability or reproducibility in its predictions. In practice, machine learning models continuously (re)-trained as new data arrives, or to incorporate architectural and algorithmic changes. A model that changes its predictions on a significant fraction of examples after each update is undesirable, even if each model instantiation attains high accuracy.
Reproducibility of predictions is challenge even if the architecture and training data fixed across different training runs, which is the focus this paper. Unfortunately, two key ingredients that help deep networks attain high accuracy over-paramettrrizatonn, and the randomization of their training algorithms - pose significant challenges to their reproducibility. The former refers to the fact that NNs typically have many solutions that minimize the training objective (Neyshabur et al. 2015) Zhang et al. (017). The latter refers to the fact that standard training of NNs involves several sources of randomness, e.g., initialization, mini-batch ordering, -enerrterminism in training platforms and in some cases data augmentation. Put together, these imply that NN training can find vastly different solutions in each run even when training data the same, leading to a reproducibility challenge.
The prediction disagreement between two models is referred to as churn (Cormier et al. 2016 Concretely, given two models, churn is the fraction test examples where the predictions of the two models disagree. Clearly, churn is zero if both models have perfect accuracy an unattainable goal for most the practical settings of interest. Similarly, one can mitigate churn by eliminating all sources of randomness in the underlying training setup. However, for random initialization and the order of data, inherent non-determinism in the current computation platforms is hard to avoid (see $2.3 Moreover it is desirable to have stable models with predictions
if one controls the seed used
Madani et al. 2004 referred to this as disagreement and used it estimate for generalization error and
model selection

---

Under review as a conference paper at ICLR 2021
unaffected by such factors in training. Thus, it is critical to quantify churn, and develop methods that reduce it.
In this paper, we study the problem of churn in NNs for the classification setting. We demonstrate the presence of churn, and investigate the role of different training factors causing it. Interestingly experiments show that churn is not avoidable on the computing platforms commonly used in machine learning, further highlighting the necessity of developing techniques to mitigate churn. We then analyze the relation between churn and predicted class probabilities. Based on this, we develop a novel regularized co-distillation approach for reducing churn.
Our key contributions summarized below:
(i) Besides the disagreement in the final predictions of models, we propose alternative soft metrics to measure churn. We demonstrate the existence of churn on standard image classification tasks (CIFAR-I0, CIFAR- 100, ImageNet, SVHN and iNaturalist) and identify the components of learning algorithms that contribute the observed churn. Furthermore, we analyze the relationship between churn and model prediction confidences (cf. $2).
(ii) Motivated from our analysis, we propose a regularized co-distillation approach to reduce churn that both improves prediction confidences and reduces prediction variance (cf. $ Our approach consists of two components: a) minimum entropy regularizers that improve prediction confidences (cf. I3.I). and b) a new variant co-distillation (Anil et al. 2018) to reduce prediction variance across runs. Specifically, disagreement, with a linear warmup and joint updates across multiple models (cf. $3.2).
use a symmetric KL divergence based loss to reduce model
(iii) We empirically demonstrate the effectiveness of the proposed approach in reducing churn and (sometimes) increasing accuracy. We present ablation studies over its two components show their complementary nature in reducing churn (cf.
1.1 RELATED WORK
Reproducibility in machine learning. There is broad field studying the problem of reproducible
research (Buckheit & Donoho 1995 Gentleman & Lang 2007 Sonnenburg et al. 2007 Kovacevic
2007 Mesirov] [0110] Pengl [011] McNutt 2014] Braun Ong. 2014; Rule et al.] 2018) which
identifies best practices to facilitate the reproducibility of scientific results. Henderson et al. (2018)
analysed reproducibility of methods in reinforcement learning, showing that performance certain methods is sensitive to the random seed used in the training. While the performance of NNs on image classification tasks is fairly stable (Table 2). we focus on analyzing and improving the reproducibility of individual predictions. Thus, chur can be seen specific technical component of this reproducibility challenge.
Cormier al. (2016) defined the disagreement between predictions of two models churn. They proposed an MCMC approach to train an initial stable model so that it has a small churn with its future version, say model B. Here, future versions are based on slightly modified training data with possibly additional features. In Goh et al. p): Cotter et al.] (2019). constrained optimization is utilized reduce churn across different model versions. In contrast, we interested in capturing the contribution of factors other than training data modification that cause churn.
More recently. Madhyastha & Jain 2019) study instability in the interpretation mechanisms and average performance for deep NNs due to change in random seed, and propose a stochastic weight averaging (Izmailov et al. 2018) approach to promote robust interpretations. In contrast, we are interested in robustness of individual predictions.
Ensembling and online distillation. Ensemble methods (Dietterich) 2000
Lakshminarayanan
et al. 2017) that combine the predictions from multiple (diverse) models naturally reduce the churn by averaging out the randomness in the training procedure of the individual models. However, such methods incur large memory footprint and high computational cost during the inference time. Distillation (Hinton et al. 2015; Bucilua et al. (2066) aims to train a single model from the ensemble to alleviate these costs. Even though distilled model aims to recover the accuracy of the underlying ensemble, it is unclear the distilled model also leads to churn reduction. Furthermore, distillation is two-stage process, involving first training ensemble and then distilling it into single model.
To avoid this two-stage training process, multiple recent works Anil et al. 2018): Zhang et al. (2018); Lan al. (2018); [Sng & Chai (2018); Guo et al. (002)) have focused online distillation, where
2

---

Under review as a conference paper at ICLR 2021
multiple identical similar models (with different initialization) are trained while regularizing the distance between their prediction probabilities. At the end of the training, any of the participating models can be used for inference. Notably, Anil al. (2018), while referring to this approach as co-distillation, also empirically pointed out its utility for churn reduction on the Criteo Ad churn
datase 2 In contrast, we develop deeper understanding of co-distillation framework reduction mechanism by providing theoretical justification behind its ability to reduce churn. We experimentally show that using a symmetric KL divergence objective instead of the cross entropy loss for co- co-distillation (Anil et al. 2018) leads to lower churn and better accuracy, even improving over the expensive ensembling-o distiilaatiiiiioinn approach.
Entropy regularizer. Minimum entropy regularization was earlier explored in the context of semi-supervised learning (Grandvalet Bengio, 2005). Such techniques have also been used to combat label noise (Reed et al. 2015). In contrast, we utilize minimum entropy regularization in fully supervised settings for distinct purpose of reducing churn and experimentally show its effectiveness.
1.2 NOTATION
Multi-class classification. We consider a multi-class classification setting, where given an instance E X. the goal is to classify it as a member of one of K classes, indexed by the y _ [K]. Let W be the set of parameters that define the underlying classification models. In particular, for w E W, the associated classification model f(:: w): X Ag maps the instance r E X in the K K-dimensional simplex AK c RK Given f (æ; w), is classified as element of class Vrww such that
Dzw = arg max jey [a; w)--
This gives the misclassification error l0 (y. B2mw)) 1 Ay), where y is the true label for æ. Let Fx be the joint distribution over the instance and label pairs. We learn a classification model by minimizing the risk for some valid surrogate loss l of the misclassification error fo1: L(w) & [x f(;; w. In practice, since have only finite samples S (X y)", we minimize the corresponding empirical risk.
L( S) & [S] Eeenees (v. f (z;
2 CHURN: MEASUREMENT AND ANALYSIS
In this section we define churn, and demonstrate its existence on CIFAR and ImageNet datasets. We also propose and measure alternative soft metrics to quantify churn, that mitigate the discontinuity of churn. Subsequently, we examine the influence of different factors in the learning algorithm on churn. Finally, we present relation between churn and prediction confidences of the model.
We begin by defining churn as the expected disagreement between the predictions of two models (Cormier et al.) 2016).
Definition (Chum between two models). Let W1, w1 W define classification models ff; X - AK, respectively. Then, the churn between the two models is
Churn(w1, w2) Ex| (Yr.,y A Px # Yxwuz].
(3)
where
arg maxjey f(z; w1); and Y 22wg arg max f(;;; w2);-
Note that if the models have perfect test accuracy, then their predictions always agree with the true label, which corresponds to zero churn. In practice, however, this is rarely the case. The following rather straightforward result shows that churn is upper bounded by the sum of the test error of the models. See Appendix B for the proof. We note that a similar result was shown in Theorem 1 of Madani et al. (2004).
Lemma 1. Let P PERW A Px Yx;en and Per,, A P Yx be the misclassifica- tion error for the models un and W2, respectively. Then, Churn( (orr,nWw,,a W2) & Perr, + Prr
3

---

Under review as a conference paper at ICLR 2021
Remove angreestation
Mdentical Inttialicatin
Ascunac Churn;%)) SChorry (76)
+:
=
93.88:
0.24
$7,05
0.12
3.96 0:26
0.18
0.10
0.21
0.16
0:39
0.09
0.18
0.17
033
0.34
0.14
14:11 0.17
Table 1: Ablation study of churn across 5
CIFAR-10 with ResNet-55. Holding the initialization constant across models always decreases churn, but using identical mini-batch ordering and completely removing augmentation can increase churn with decrease in accuracy.
Despite the worst-case bound in Lemma] [I imperfect accuracy does not preclude the absence of churn. As the best-case scenario, two imperfect models can agree on the predictions for each example (whether correct or incorrect), causing the churn be zero. For example, multiple runs of deterministic learning algorithm produce models with zero churn, independent of their accuracy.
This shows that, in general, one cannot infer churn from test accuracy, and understanding churn of an algorithm requires independent exploration.
2.1 DEMONSTRATION OF CHURN
In principle, there are multiple sources randomness standard training procedures that make NNs susceptible churn. We now verify this hypothesis by showing that, in practice, these sources indeed result in non-trivial churn for NÑs on standard image classification datasets.
Table repoorts churn over CIFAR-10. 10, CIFAR-100 and ImageNet, with ResNets (He al. 2016a b as the underlying model architecture. As per the standard practice in the literature, we employ SGD with momentum and step-wiss wise learning rate decay train ResNets. Note that the models obtained in different runs indeed disagree in their predictions substantially. To measure the disagreement comes mainly from misclassified examples, we measure churn on two slices of examples, correctly classified (Churn correct) and misclassified (Churn incorrect). We notice churn even among the examples that are correctly classified, though we see a relatively higher churn among the misclassified examples.
This raises two natural questions: (i) do the prediction probabilities differ significantly, or is the churn observed in aablemminnly artifact of the argmax operation in (I], when faced with small variation in prediction probabilities across models?, and (ii) what causes such high churn across runs? We address these questions the following sections.
2.2 SURROGATE CHURN
Chum in Table could potentially be manifestation of applying the arg max operation in (.. despite the prediction probabilities being close. To study this, we consider the following soft metric to measure churn, which takes the models' entire prediction probability mass function into account.
Definition (Surrogate churn between two models). Let f(s vo)), w2): X - A be two models defined by W1, w2 W, respectively. Then, for a IR the surrogate churn between the models is
1 2
Ex II(
f(X:
f(X: w2)
SChurnawuy w2)
(4)
max f(X; w1)
max f(X: (x)
As :0, this reduces the standard Churn definition B In Table 2 we measure SChurn, for a 1, which shows that even the distance between prediction probabilities is significant across runs. Thus, the churn observed in Table 2 is not merely caused by the discontinuity of the arg max, but indeed highlights the instability of model predictions caused by randomness in training.
2.3 WHAT CAUSES CHURN?
We now investigate the role played by randomness in initialization, mini-batch ordering, data augmen- tation, and non-determinism in the computation platform in causing churn. Even though these aspects are sources randomness in training, they are not necessarily sources of churn. We experiment by holding some of these factors constant and measure the churn across 5 different runs. We report results in Table where the first column gives the baseline churn with no factor held constant across
4

---

Under review as a conference paper at ICLR 2021
Model initialization is source of chum. Simply initializing weights from identical seeds (odd columns in [ can decrease churn under most settings. Other sources of randomness include mini-batch ordering and dataset augmentation. To hold the former constant, ensure that every model iterates over the dataset in the same order; to hold the latter constant, we remove all augmentation during training. These two aspects contribute randomness between training runs, but fixing them does not decrease churn; rather, they appear to have regularizing affect on our hardware platform.
Finally, there is churn resulting from an unavoidable source during training, the non-determinism in computing platforms used for training machine learning models, e.g., GPU/TPU (Morin & Willetts PPTochh 2019 Nvidial (020). The experiments in Table [ were run on TPU. Even when all other aspects of training are held constant (rightmost column), model weights diverge within 100 steps (across runs) and the final churn is significant. We verified that this is the sole source of non-determinism: models trained 10,000 steps on CPUs under these settings had identical weights.
These experiments underscore the importance of developing and incorporating churn reduction strategies in training. Even with extreme measures to eliminate all sources of randomness, we continue to observe churn due to unavoidable hardware non-determinism.
2.4 REDUCING CHURN: RELATION TO PREDICTION PROBABILITIES
We now focus on the main objective of this paper churn reduction NNs. Many factors that have been shown to cause churn in $ 2.3 crucial for the good performance of NNs and thus cannot be controlled or climinated without causing performance degradation. Morcover, controlling certain factors such hardware non-determinism is extremely challenging, especially in the large scale distributed computing setup.
Towards reducing churn, we first develop an understanding of the relationship between the distribution of prediction probabilities of a model and churn. Intuitively, encouraging either larger prediction confidence smaller variance across multiple training runs should result in reduced churn.
To formalize this intuition, let us consider the prediction confidence realized by the classification model [(; w) on the instance X:
Nag [a; w) Becw arg max (x;w))
where Vane is the model prediction in [ Note that Yew denotes the difference between the probabilities of the most likely and the second most likely classes under the distribution f(z; w). and is not the same as the standard multi-class margin (Koltchinski et 2001): captures how confident the model [(; w) is about its prediction Uapes without taking the true label into account.
The following result relates the prediction confidence to churn. See Appendix B for the proof.
Lemma 2. Let Taamn and Yawwrg be the prediction confidences realized X by the classification models f(; w1) and w2), respectively. Then,
Churn(wi, w2) Px((xx * &x.wz) &
[(X; w2)) min(7x00 (X }]
Here, DLJ (f: w1), f(;; w2)) Ejey [f(x; w1), [z; W2), measures the L1 distance. Lemmal establishes that, for given instance æ, the churn between models f(; w,) and f(; w2) becomes less likely as their confidences Taroory and Tapue increase. Similarly, the churn becomes smaller when the difference between their prediction probabilities, Dr. (f(æ;; wi), [a;; w2)), decreases.
REGULARIZED CO-DISTILLATION FOR CHURN REDUCTION
In this section we present approach for chur reduction. Motivated by Lemma 2 approach with two complementary techniques for churn reduction: 1) We first propose entropy based regularizers that encourage solutions e W with large prediction confidence Naww for each instance x. 2) We next employ co-distillation approach with novel design choices that simultaneously trains two models while minimizing the distance between their prediction probabilities.
we consider an
Note that the entropy regularizers themselves cannot actively enforce alignment between the prediction probabilities across multiple runs the resulting objective does not couple multiple models together.
5

---

Under review as a conference paper at ICLR 2021
On the other hand, the co-distillation in itself does not affect the prediction confidence of the underlying models (as verified in Figure Thus, our combined approach promotes both large model prediction confidence and small variance in prediction probabilities across multiple runs.
3.1 MINIMUM ENTROPY REGULARIZERS
Aiming increase the prediction confidences of the trained model (Tr,w we propose novel training objectives that employ one of two possible regularizers based on: (1) entropy of the model prediction probabilities; and (2) negative symmetric KL divergence between the model prediction probabilities and the uniform distribution. Both regularizers encourage the prediction probabilities to be concentrated on small number of classes, and increase the associated prediction confidence.
Entropy regularizer. Recall that the standard training minimizes the risk [(w).. Instead, for a [o, 1] and {zz,, befm]. we propose to minimize the following regularized objective to increase prediction confidence of the mode??
Lentropy (w;tropp(w;30 8) (1 (1-a) a) L(w; 8) + 1 E H (f(ar,))) 11
(6)
where H (f(z;w)) -je(k] f(z; w), log f(;; w)y denotes the entropy of predictions f(; w).
Symmetric KL divergence regularizer. Instead of encouraging the low entropy for the prediction probabilities, we can alternatively maximize the distance of the prediction probability mass function from the uniform distribution we utilize the symmetric KI. divergence the distance measure. Let Unif e Ax be the uniform distribution. Thus, given n samples S ((x; yi) ve(n).
regularizer enhance the prediction confidence. In particular,
we propose to minimize
(w; S) (1 a) L(w; a Σ NE[N
(f Unif),
where (),, Unif) =
+ (T,(Unil]/((x:)))
As discussed in the beginning of the section, the intuition behind utilizing these regularizers is to encourage spiky prediction probability mass functions, which lead to higher prediction confidence. The following result supports this intuition in the binary classification setting. See Appendix B for the proof. As for multi-class classification, we empirically verify that the proposed regularizers indeed lead to increased prediction confidences (cf. Figure (
Theorem 3. Let f(: w) and f(= be two binary classification models. For a given x E we have H(f(z;))) & (=:(ww;))) or SKL(J(;; > SKL(J (2;; w')), then T/x: 2 "sxqu"-
Note that the effect of these regularizers is different from increasing the temperature in softmax while computing (a; w)) Similar to max entropy regularizers (Pereyra et al. 2017), they are independent of the label, a crucial difference that allows them to reduce churn even among misclassified examples.
3.2 CO-DISTILLATION
As the second measure for churn reduction, we now focus on designing training objectives that mini- mize the variance among the prediction probabilities for the same instance across multiple runs of the learning algorithm. Towards this, we consider novel variants of co-distillation approach (Anil et al., 2018). In particular, employ co-distillation by using symmetric KI. divergence (Co-distillski)) with linear warm-up. to penalize the prediction distance between the models instead of the popular step-wise cross entropy loss (Co-distillce) used in Anil al. (2018) (see CC. We first motivate the proposed objective for reducing churn using Lemma
Recall from Lemma 2 that, for given instance x, chur across two models f(: w1) and f(: w2) decreases as the distance between their prediction probabilities (f w1). f(:; wz)) becomes smaller. Motivated by this, given training samples {(x;; [e]]e one can simultaneously train two models corresponding to W1, E W by minimizing the following objective and keeping either
We restrict ourselves to the convex combination of the original risk and the regularizer terms as this ensures that the scale of the proposed objective is the same as that of the original risk. This allows us to experiment with the same learning and other hyperparameters for both.

---

Under review as a conference paper at ICLR 2021
Training
Accuracy
Charu(%)
SChurmg (56)
correct
Chura incorrect
cost
Boseline
Ix
93.97=0.11
5.72::.118
5.1120.115
262:+0.12
53.82:1144
CIFAR-
Frempyy (tis peper)
Ix
94.05-0 18
5 5.99+0021
6.17+0.177
2,55+0
53,53+111
ResNet
SKL (this pupert
Ix
93.35 0.13
5.1910.18
6.5110.16
2.5610.15
52.611.9
2-Enoemble
2015
4.11
491100111
47:25:1:555
Co-distller: 2012
2x
56220112
Chdiatillgkt paper)
2x
466+00111
2
+Entropy (ikss paper)
9463-0.15 2x
4,21::0115
4.61=0.14
1.94:0.13
44.29=119
Boselin:
1x
73.26=0222
26.77+0.26
57.19
11422+026
68.77=00.72
CIFAR: 100
Enempy (his
1x
7324 0.29
26.5510.26
34.58:10.29
11:29
08.32:10.66
ResNet-56
SKL (Cai paper)
25.220.36
28,81 10.35
01.1620.4
65.42:0055
disill. Dinoon 2015
3x
18.6610.26
Codistill Aai
19.55+0.27
9.98
48,09=2.44
Co-disillgırr Uha paper
26
0.16
17.7020.33
24.9
203
45.98
+Fatnopy (this paper)
26
76.51-0.26
17.09+0.1
24.06-0..28
7,2+00.26
49.4+1111
Rouelie:
1x
76+0.11
15
35.01+0.244
5.87+0.11
44.77+0.44
ImageNet
0.11
14921000
$2.9310.188
5.75 1..11
44.5610.33
ResNet-x2-50
SKL (chis paperi
153210011
2-Ensemble disil. et 2015
129240.14
34.22002
492-0.11
Chdiistil Ani
76-12-088
1162+021
32,67+14
Codisillggt 10n papec)
24
76674-0066
11450011
20.6600.20
5.11
1.,41=1.115
+Entropy (this paperi
28
0.:56-0022
0,33
22.92
4,30 0.33
32,89 1.15
Baseline
18
60.97 0.26
71.33:10.11
9.65:003
$4.881005
NNaturaiss ResNet
Enmmpy
02174_0.15
25222
66.111047
9.23:10:4
53.82:00335
distille::: Aail et 2010)
28
26.9210.22
Codistilogs (0in pape))
24
61.50-0228
21.73+0.26
68.1)+1117
7,48-0334
44.53-0.599
Biseline
16
87.16-0043
14.3=04
17,52+0.48
5,9400333
70, 10.30:1.24
SVIIN
Enempy (this porers
18
$7.24000
14.1210442
16.2210.52
6.141.0477
19.31:1
I LaNe15
Codistilog
Asil 2018
2x
88.91003
9.43+00.3
10.44+0.36
3.84+00377
$4.00-1.8
distillatt.
papo
2%
89.61_0.25
8.49
12.210035
3,59100.4
51.11:144
Table 2: Estimate of Churn (ef. (3) and SChurn (cf. 4 on the test sets. For each setting, we report the mean and standard deviation over 10 independent runs, with random initialization, mini-batches and data-augmentation. We report the values corresponding to the smallest churn for each method (see in $ for the exact parameters). We boldface the best results in each column. First we notice that both the proposed methods effective at reducing churn and Schurn, with Co-distillsKL showing significant reduction in churn. Additionally, these methods also improve the accuracy. Finally combing the entropy regularizer with Co-distillgkL offers the best way reduce churn, and Co-distillce approaches. Note that for co-distillation
improving over the ensembling-distillae we measure churn of single model (e.g. f(; wi) in eq (9) across independent training runs.
of the two models as the final solution.
3 LCa-ditillaa (W1, W2: S) A L( + L(W2;8) "1
ii] DLi (f (x) f(ggw))) (8)
From Pinsker's inequality, Dr. wi), ((; < /2 KL[J(a [ w2)). Thus, one can alternatielly utilize the following objective.
Lco-distllssa (w1, W2; S) A L) + L(w2;S) +
n Σ SKL(s(a,,)) ie[n]]
where (fI, wi (2:; w2)) (I; wi (1) w2)) + (L( (2) (2)]] denotes the symmetric KL-divergence between the prediction probabilities of the two models. In what follows, we work with the objective in (9) as we observed this to more effective in our experiments, leading to both smaller churn and higher model accuracy.
In addition the co-distillation objective, we introduce two other changes the training procedure: joint updates and linear rampup of the co-distillation loss. We discuss these differences in $
Regularized co-distillation. In this paper, we also explore regularized co- co-distillation, where we utilize the minimum entropy regularizes (cf. $ (.J) in our co-distillation framework. We note that the best results for the combined approach are achieved when we use a linear warmup for the regularizer coefficient as well. Note that combining the Co-distillgkı, objective with an entropy regularizer is not the same as using the cross entropy loss, due to the use the different weights for each method, and rampup of the regularizer coefficients. This distinction is important in reducing churn (Table 2
EXPERIMENTS
We conduct experiments on 5 different datasets, CIFAR-10. CIFAR-100, ImageNet, SVHN and iNaturalist 2018. We use LeNet5 for experiments on SVHN, ResNet-56 for experiments on CIFAR- 10 and CIFAR-100, and Res5et-v2-50 for experiments ImageNet and iNaturalist. We use the
7

---

Under review as a conference paper at ICLR 2021
Bada
Bate
Bata
77.5
17
0.9
Co-distilaation (SKL)
16
Schurai (deahee))
77.0
15
0.7
Recurecy
70.3
14
88 12
13
0.6
12
0.5
Entropy Regularizrr
26
0.0
0.1
0.
60
04
05
75.5
0.0
0.1
0.7
5.3
0.6
0.5
10
0.0
0.1
0.2
0.3
0.4
0.5
Alpha
Alpha
Alpha
Figure 1: ImageNet ablation study: We plot the effect of the entropy regularizer (top1o). for varying a, and Co-distillskI. for varying 8. on the prediction entropy, accuracy and churn. These plots shows the complementary nature of these methods in reducing churn. While entropy regularizer reduces churn by reducing the prediction entropy, Co-distillskL reduces churn by improving the agreement between two models, hence increasing the entropy.
0.00
0.10
0.9
Codiistilaatonn (SKLI
24.5
Co-dibillttoon
0.8
Confidence
0.7
Entropy Regularizer
0.5
6.1
0.5
Alpha
Churn
Figure 2: ImageNet: L of the entropy regularizer and our proposed variant of co-distillation, Co-distillskr. on the prediction confidence. Right comparison with Co-distiller. Anil et al. 2018).
We plot the trade-oft off between accuracy and churn for Co-distillcg and Co-distillsku by varying B.
The plot clearly shows that the proposed variant achieves a better tradeoff, compared Co-distillcg-
Datased
Method
Accurncy
Chura(%)
Woight decay =
Weight decay = 0.0001
Weight decay =
Weight decay 0.0001
Raseline
91 02
93.97:00.11
8.4+0.21
5.724-0.18
CIFAR-10
Entropy (this piper)
91 1..66 0.26
$4.05:0.118
8.45::0.25
5.59::0.31
papar)
22
93.75-20.33
7.84++0.24
5.79-20.18
Co Co-distillss (this paper)
92.1110.11
94.63 0.115
6.74
4.29 0.11
Baseline
68.6+:0.31
73.26:20.22
02.23-+:045
16.77=0.26
CIFAR-100
Entropy (this pager)
68.41 33
73.24:50.29
32 42.02::::22
26.55=0.26
SKL (this
69.02.1.0.33
73.35
31 2.:10.3333
25.62:00.6
Co-disilllger. i paper)
72.29::0.42
76.56:10..0
23.14:10.33
Table 3:
Weight decay ablation studies. Similar to Table provide results accuracy and churn for baseline and the proposed approaches, with and without weight decay. We notice that the proposed approaches improve churn both with and without weight decay.
same hyperparameters for all the experiments a dataset. We use the Cross entropy loss and the Momentum optimizer. For complete details we refer to the Appendix A
Top; regularizer. For problems with a large number of outputs, e.g. ImageNet, is required to penalize all the predictions to reduce churn. Recall that prediction confidence is only a function of the top two prediction probabilities. Hence, on ImageNet, we consider the top,, variants of the proposed regularizers, and penalize the entropy/SKL only on the top & predictions, with ie = 10.
Accuracy and churn. In Table 2 we present the accuracy and churn of models trained with the minimum entropy regularizers, co-distillation and their combination. For each dataset we report the values for the best a and f3. We notice that our proposed methods indeed reduce the churn and Schurn. While both minimum entropy regularization and co-distillation are consistently effective in reducing Schurn, their effect on churn varies, potentially due to the discontinuous nature of the churn. Also note that the proposed methods reduce churn both among the correctly and incorrectly classified examples. Our co-distillation proposal, Co-distillskL consistently showed a significant reduction in churn, and has better performance than Co- Co-distillge (Anil et al. (018)) that is based on the cross cross-entropy loss, showing the importance of choosing the right objective for reducing churn. We present additional comparison in Figure showing churn and Top-1 error ImageNet, computed for
8

---

Under review as a conference paper at ICLR 2021
Method
Training cost
CIEAR-10 ECR (x1c2)
CIFAR-I00 ECE (x1c22)
ImageNet- ECE (x le2)
15.2340032
3.11-0.12
Entreyy (tis poper)
Ix
4.19=0.18
17.43=0026
6.17#009
SKL (ths paper)
5.1120.12
21882:0.16
6,19:10.12
2-Ensemble
et
13.85-20.23
Co-disillerr (Ani al. 2015)
2x
4.13=0.12
15.86=00.13
11.97=0007
Co-disillasst (hhs
0.18
11.39
2.48::0.09
+Entropy (this paper)
3.94
11.55:10.23
3.89:10.29
Table 4: Expected Calibration Error. We compute the Expected Calibration Error (ECE) (Guo itt al., 2017) to evaluate the effect on calibration of logits by the churn reduction methods considered this paper, for different datasets. We report ECE for the predictions of the models used report accuracy and churn in 2 We note that while the minimum entropy regularizers, predictably. increase the calibration error, our combined approach with co-distillation results calibration error competitive with 2-ensemble distillation method.
different values of / (cf. )9). Finally, we achieve an even further reduction in churn by the combined approach of entropy regularized co-distillation. Interestingly, the considered methods also improve the accuracy, showing their utility beyond churn reduction.
Ensembling. We also compare with the onsembling-distillaatt approach (Hinton al. 2015 in Table 2, where we use 2 teacher ensemble for distillation. We show that the proposed methods consistently outperform mnsembiing-distilaaiin
approach, despite having a lower training cost.
Ablation. We next report results of our ablation study on the entropy (toP1o) regularizer coefficient a, and the Co-distillskL coefficient the prediction entropy shows different behavior between these methods. While entropy regularizers improve churn by reducing the entropy, co-distillation reduces churn by reducing the prediction distance between two models, resulting in an increase in entropy. This complementary nature explains advantage in combining these two methods (cf. T)ble
Figure While both methods improve accuracy and churn,
We next present ablation studies on weight decay regularization in Table 3 showing that the proposed approaches improve churn on models trained both with and without weight decay.
Fixed initialization. Earlier in Table we showed that fixing the initialization of five runs of the baseline model lowers churn from 5.81 0.08 to 5.56 + 0.12 for CIFAR-IO on ResNet-56. However, our proposed Co-distillskL model is more effective, reducing churn to 4.29 + 0.14 (see Table 2). Using fixed initialization on this model does not significantly affect churn (4.24 + 0.07).
DISCUSSION
Connection to label smoothing and calibration. Max entropy regularizers, such as label smooth- ing, often used to reduce prediction confidence (Pereyra et al. 2017 Müller et al., 2019) and provide better prediction calibration. The minimum entropy regularizers studied in this paper have the opposite effect and increase the prediction confidence, whereas co-distillation reduces the prediction confidences. To study this more concretely we measure the effect of churn reduction methods on the calibration.
We compute Expected Calibration Error (ECE) (Guo et al. 2017) for methods considered in this paper and report the results in Table We notice that the minimum entropy regularizers, predictably, in- crease the calibration error. whereas the propose co-distillation approach (Co-distillskL) significantly reduces the calibration error. We notice that our joint approach is competitive with the ensemble distillation approach on CIFAR datasets but incurs higher error on ImageNet. Developing approaches that jointly optimize for churn and calibration is interesting direction of future work.
9

---

Under review as a conference paper at ICLR 2021
REFERENCES
Rohan Anil, Gabriel Pereyra, Alexandre Passos, Robert Ormandi, George E. Dahl, and Geoffrey E. Hinton, Large scale distributed neural network training through online distillation. International Conference Learning Representations, 2018.
Mikio L. Braun and Cheng Soon Ong. Open science in machine learning. In Victoria Stodden, Friedrich Leisch, and Roger D. Peng (eds.), Implementing reproducible research. CRC Press/Taylor and Francis, Boca Raton, FL. 2014. ISBN 978 1466561595
Cristian Bucilua, Rich Caruana, and Alexandru Niculescu-Mizil Model compression. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 535-541, 2006.
Jonathan B. Buckheit and David L. Donoho. Wavelab and reproducible research. In Wavelets and Statistics, pp. 55 81. Springer New York, New York, NY, 1995. ISBN 978-1-466 1-4612-2544-7.
Quentin Cormier, Mahdi Milani Fard, Kevin Canini, and Maya Gupta. Launch and iterate: Reducing prediction churn. Advances in Neural Information Processing Systems 29, pp. 3179-3187. 2016.
Andrew Cotter, Heinrich Jiang, Maya Gupta, Serena Wang. Taman Narayan, Seungil You, and Karthik Sridharan. Optimization with non-differentiable constraints with applications to fairness, recall, churn, and other goals. Journal of Machine Learning Research, 20(172):1-33 2019.
Thomas G. Dietterich. Ensemble methods machine learning. In Multiple Classifier Systems, pp. 1-15, Berlin, Heidelberg, 2000. Springer Berlin Heidelberg. ISBN 978- 778-3-540-45014-6.
Robert Gentleman and Dunean Temple Lang. Statistical analyses and reproducible research. Journal of Computational and Graphical Statistics, 16(1):1-23, 2007.
Gabriel Goh, Andrew Cotter, Maya Gupta, and Michael P Friedlander. Satisfying real-world goals with dataset constraints. In Advances in Neural Information Processing Systems 29, pp. 2415-2423. 2016.
Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimization. In Lawrence K. Saul, Yair Weiss, and Léon Bottou (eds.), Advances in Neural Information Processing Systems 17, pp- 529 236. MIT Press, Cambridge, MA, 2005.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International Conference on Machine Learning, pp. 1321-1330, 2017.
Qiushan Guo, Xinjiang Wang. Yichao Wu, Zhipeng Yu, Ding Liang, Xiaolin Hu, and Ping Luo. Online knowledge distillation via collaborative learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016a.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European conference on computer vision, pp. 635.645.25 Springer, 2016b.
Peter Henderson, Riashat Islam, Philip Bachman, Joelle Pineau, Doina Precup, and David Meger. Deep reinforcement learning that matters. In Thirty-Second AAAI Conference Artificial Intelli- gence, 2018
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. arXiv preprint arXiv:1503.0253 2015.
Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. Av- eraging weights leads to wider optima and better generalization. arxiv preprint trXiv:1803.05400 2018.
10

---

Under review as a conference paper at ICLR 2021
Vladimir Koltchinskii, Dmitriy Panchenko, and Fernando Lozano. Some bounds the general- ization error of combined classifiers. In Advances in Neural Information Processing Systems 13, pp. 245-251, 2001.
Kovacevic, How to encourage and publish reproducible research. In 2007 IEEE International Conference on Acoustics, Speech and Signal Processing- ICASSP '07, volume 4, pp. IV-1273-IV- 1276, 2007.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances Neurul Information Processing Systems 30, pp. 6402-6413, Curran Associates, Inc., 2017.
Xu Lan, Xiatian Zhu, and Shaogang Gong. Knowledge distillation by on-the-fly native ensemble. In Advances in Neural Information Processing Systems 31, pp. 7517-7527. 2018.
Omid Madani, David Pennock, and Gary Flake. Co-validation: Using model disagreement on unlabeled data to validate classification algorithms. Advances neural information processing systems, 17:873-880, 2004.
Pranava Swaroop Madhyastha and Rishabh Jain. On model stability function of random seed. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pp- 929 9399,99,,220 2019.
Marcia McNutt. Reproducibility. Science, 343(6168):229 229, 2014. ISSN 0036-8075.
Jill P. Mesirov. Accessible reproducible research. Science, 327(5964):415 2010. ISSN 0036-8075.
Miguel Morin and Matthew Willetts. Non-determinism in tensorflow resnets. arXiv preprint
arXiv:2001.11396, 2020.
Rafael Müller, Simon Kornblith, and Geoffrey Hinton. When does label smoothing help? In Advances in Neural Information Processing Systems, pp. 14996-4705 2019.
Behnam Neyshabur, Ryota Tomioka, and Nathan Srebro. In search of the real inductive bias: On the role of implicit regularization deep learning. In International Conference on Learning Representations workshop track, 2015.
Nvidia. Tensorflow determinism, 2020.
URL htcps://githtbb.cmm///DDDD
tensorf fensorfllow-determminn sm
Roger D. Peng. Reproducible research in computational science. Science, 334(6060); 2011. ISSN 0036-8075.
Gabriel Pereyra, George Tucker, Jan Chorowski, Łukasz Kaiser, and Geoffrey Hinton. Regularizing neural networks by penalizing confident output distributions. arXiv preprint arx.v:1701.005588 2017.
Py PyToc.. Reproducibility, 2019. URL http::///pttorcc..oogg//////s///
randomness html
Scott E. Reed, Honglak Lee, Dragomir Anguelov, Christian Szegedy, Dumitru Erhan, and Andrew Rabinovich, Training deep neural networks on noisy labels with bootstrapping. In Yoshua Bengio and Yann LeCun (eds.). 3rd International Conference on Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Workshop Track Proceedings, 2015.
Adam Rule, Amanda Birmingham, Cristal Zuniga, Ilkay Altintas, Shih-Cheng Huang, Rob Knight, Niema Moshiri, Mai H. Nguyen, Sara Brin Rosenthal, Fernando Pérez, and Peter W. Rose. Ten simple rules for reproducible research in jupyter notebooks. arXiv preprint arXiv:1810.08055. 2018.
Guocong Song and Wei Chai. Collaborative learning for deep neural networks. In Advances in Neural Information Processing Systems 31, pp. 1832-1841. 2018
11

---

Under review as a conference paper at ICLR 2021
Sören Sonnenburg, Mikio L. Braun, Cheng Soon Ong, Samy Bengio, Leon Bottou, Geoffrey Holmes, Yann LeCun, Klaus-Robert Müller, Fernando Pereira, Carl Edward Rasmussen, Gunnar Rätsch, Bernhard Schölkopf, Alexander Smola, Pascal Vincent. Jason Weston, and Robert Williamson. The need for open source software in machine learning. J. Mach. Learn. Res., 8:2443-2466 December 2007.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understand- ing deep learning requires rethinking generalization. In International Conference on Learning Representations, 2017.
Ying Zhang, Tao Xiang, Timothy M Hospedales, and Huchuan Lu. Deep mutual learning. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 4320-4328. 2018.
12

---

Under review as a conference paper at ICLR 2021
Dataset
Method
a
B
Temperature
Accuracy
Churn
Baseline
93.97+0.11
5.72+0.18
CIFAR-I0
Entropy
0.15
94.05+0.118
5.59+0.21
ResNet-56
SKL
0.05
93.75.0.11
5.79
2-Ensemble distillation
3.0
94.62++0.07
4,47+0.13
Co-distill
0.01
94.25-+0.15
5.14
Co
0.01
94.63-10.15
0.29
+Entropy
0.1
0.01
9..63
4.21
Baseline
73.26
26.77:
CIFAR-100 ResNet-56
Entropy
0.25 0.35
73.24
26.55
SKL
03.
25.62
2-Ensemble distillation
4.0
76.27
18.86+0.226
Co-distille'
0.04
73.11
19.55
Co-distillgRı
0.04
76,58 0.16
17.70+0.33
+Fatropy
0.2
0.04
76,53
17.09
Baseline
76+
15.22+00116
ImageNet
Entropy
0.2 0.05
76.39
04.92
ResNet-v2-50
SKL
75.98
15.32+0.111
2-Ensemble distillation
1.0
7.08
12.92+0.144
Co-distill
0.02
1..12
11.62
Co-distillgKL
0.06
0.07
11.45
+Fmtropy
0.2
0.06
7..02
11.05
Baseline
00.97:10.26
0..9
INaturalist
Entropy
0.4
62.74:0.15
25.84
ResNet-v2-50
Co-distiller
0.02
6111+
26.89+0.22
Co-distillsKL
0.05
6..59
21.7331..26
Baseline
87.16-0.43
14.3+004
SVHN LeNet5
Entropy
0.2
8..24
14.12::0.42
Co-distilley
0.02
88.93 0.39
9.42+00.3
Co-distillsKL
0.02
0.225
8.49
Table 5: In this table we list the hyper-parameter values corresponding to the settings for the results Table For each method we experiment with a range of paree--prameters, and 10 independent runs, described in the Section A and report the results for the setting with the best performance.
A EXPERIMENTAL SETUP
Architecture: For our experiments we use the standard image classification datasets CIFAR and ImageNet. For our experiments with CIFAR, we use ResNet-32 and ResNet-56 architectures, and for ImageNet we use ResNet-vv-50. These architectures have the following configuration in terms of (Mayyrrs Aiiterr stride), for each ResNet block:
ResNet-32- [(5, 16, 1), (5, 32, 2), (5, 64, 2)]
ResNet-56- [(9, 16, 1), (9, 32, 2), (9, 64, 2)]
ResNect-v2-00- [(3, 64, 1), (4, 128, 2), (6, 256, 2), (3, 512,
where 'stride' refers the stride of the first convolution filter within each block. For ResNet-v2-50 the final layer of each block has 4 Pitter filters. We use L2 weight decay of strength le-4 for all experiments.
Learning rate and Batch size: For our experiments with CIFAR, we use SGD optimizer with 0.9 Nesterov momentum. We use a linear learning rate warmup for first epochs, with peak learning rate of 1.0. We use stepwise decay schedule, that decays learning rate by a factor of 10, at epoch numbers 200, 300 and 400. We train the models for a total of 450 epochs. We use a batch size of 1024.
For the ImageNet experiments, we use SGD optimizer with a 0.9 momentum. We use a linear learning rate warmup for first 5 epochs, with peak learning rate of 0.8. We again use stepwise decay schedule, that decays learning rate by a factor of 10, at epoch numbers 30, and 80. We train the models for total of 90 epochs. We batch size of 1024.
13

---

Under review as a conference paper at ICLR 2021
Data augmentation: We use data augmentation with random cropping and flipping.
Parameter ranges: For our experiments with the min entropy regularizers, we use [0.05, 0.1, 0.11, 0.2, 0.25, 0.35, 0.4, 0.5]. and report the corresponding to best
churn in Table 2 For our experiments with the co-distillation approach, we use 3 [0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08], and report the 3 corresponding to best churn in Table
For our experiments entropy regularized co-distillation, coefficient. We use the same range for as described above, and use only the best & value from the earlier experiments.
linear warmup of the regularizer
Finally, we run our experiments using TPUv3. Experiments on CIFAR datasets finish in an hour, experiments ImageNet take around 6-8 hours.
Hyper parameters for T Finally we list in The best hyperparameters used to obtain the results in Table ?
For reference we also include the accuracy and the churn again for all methods.
B PROOFS
Proof of Lemma 1 Recall from Definition I that
Churn(w1, w22 Px(Yx.e, Yx,wal = Px, Px,er Yx,wz]
Px,([1Xx.x Y, 9x,uz * Y) U (8x,is * Y, Px.u, * 8xw=
Pxxrl =Y, Yx # +|+1|\Yxuu # Y, Px,us # Yx,ea]
(4)) P P\Yx,u> # +|+P|YYx.u # #Y]
Perr,w Pe
(10)
where (1) and (it) follow from the union bound and the fact that P[A] & P[B] whenever S B. respectively.
Proof of Lemma 2 Note that, for any j # Daww..
f(x; W2) W2)
- f(x; w2)s
fxx; W2)
f(x; wi)ss.., + f(x; W1) f(x; wi)s + +f(x; - f(x; w2)
1f(rr
jey
Dr.1 (J (Ir; (2; w2)).
(11)
Similarly, for any * Dr,w2, we can establish that
f(z; w1) W1) ja.w,
f(x; W1); Ya,wz DLA (f(2; w1), f(; w2)).
(12)
Note that experiencing churn between the two models on is equivalent to (geww # (n,wz)
{jj Un,ws f(;;; W2) fx; U # Úz,wz (1::W1) < <f(=;11);)
@ (fr; f(; Ta (Di ((r > N (13)
where (i) follows from 1II and 12 Now, 13
implies that
Px(
\x Fx (f f(X, min(7x.. wx
(14)
Proof of Theorem 3
Let the prediction for given x be f(x;; w). W.I.O.G.. let > 1 - p. The
prediction confidence is then
Yar,u0 p
2p 1.
(15)
14

---

Under review as a conference paper at ICLR 2021
Entropy case. Now write entropy in terms of the prediction confidence (ef. (13
H([(ww)) w)) - plog(p) - (1 11-27 log(1
(1 7a,w)
7x.w
Yeww)
Tar
x log 2
log
2
(16)
Now the gradient of g(-) is V Te I (Ta,w) log
which is less than for Neow [0, 1]. Hence, the function g(7r,w)) is a decreasing function for inputs in range [0, 1J. Hence, (()(ww))))) g(Zz,w) implies Ya.w! Te,w.
1-
1+
Symmetric-KL. divergence case. SKL(f(I; w), Unif) (f) + KL(Unif))](;;;r - (f(z; - 1/K) log (K f(z; w);)
Recall that
1
[(aww) log (f(; - K
log (((xw)))
For binary classification this reduces to,
1 log(p(1 - p)) Hp))
SKL(f(z; w), Unif) -2 2
By using 15) and 16
we can rewrite this function in terms of YYr, as:
(KI( w), Unif) log(p(1 -p) H(p)
1 ( 1 7z,
1
7s,
log
g(
loog
((),,)))
2
2
2
Now notice that both the above terms are an increasing function of 7z... as log is an increasing function, and w(),,w) is decreasing function. Hence SKL(f(x; w)), Unif) SKL(f(#; w Unif) implies Twrw 2 Yazw".
C DESIGN CHOICES IN THE PROPOSED VARIANT OF CO-DISTILLATION
Here, we discuss two important design choices that are crucial for successful utilization of co- distillation framework for churn reduction while also improving model accuracy.
Joint vs. independent updates. Anil et al. (2018) consider co-distillation with independent updates for the participating models. In particular, with two participating models, this corresponds to independently solving the following two sub-problems during the t-th step.
w arg wew L[wr;S) + B- Σ [E[n]]
(L (f
(17a)
wh = arg min L(w2;S) 3.
1 Σ [s(n]]
KI. (I,((2;; T)lff ([ w2)).
(17b)
where w and w/ corresponds to earlier checkpoints of two models being used to compute the model disagreement loss component at the other model. Note that since w ---7 and w are not being optimized in (17a) and (1b), respectively. Thus, for each model, these objectives cquivalent to regularizing its empirical loss via cross-entropy terms that aims to align its prediction probabilities with that of the other model. In particular, the optimization steps in (17) equivalent to
= arg win L( (ww S)
Σ ke[n]
H(f(z w (Inww))
(18a)
n
w = arg
min L(w2; S) + ke[n]
(18b)
15

---

Under review as a conference paper at ICLR 2021
where H (f;;; w1), f(æ; w2)) = -, f(; w1), log fa; w2), denotes the eross-entropy between the probability mass functions f(x; w1) and [x; w2)-
In our experiments, we verify that this independent updates approach leads to worse results as compared to joinlly training the two models using the objective in (9).
In general, imple- menting co-distillation approach by employing the model disagreement based loss component (f.g.SKL(f(x; w1), f(z; w2)) in our proposal) from the beginning leads to sub-optimal test accu- racy the models are very poor classifiers the initialization. As a remedy, [Anil al. (2018) proposed step-wise ramp iff to. with to burn-in steps. Besides step-wise ramp up, we experimented with linear ramp-up. i.e, 3r = minfc- t, } and observed that linear ramp up leads to better churn reduction. We believe that, for churn reduction, it's important to ensure that the prediction probabilities the two models start aligning before the model diverges too far during the training process.
Deferred model disagreement loss component in co-distillation.
of such loss component, i.c., time-dependent B(t) such that (()
D COMBINED APPROACH: CO-DISTILLATION WITH ENTROPY REGULARIZER
In this paper, have considered different approaches to reduce churn, involving minimum entropy regularizers and co-distillation framework, respectively. This raises an important question if these two approaches are redundant. As shown in Figures 2 and 3 this is the case. In fact, these two approaches are complementary in nature and combining them together leads to better results in term of both churn and accuracy.
Note that the minimum entropy regularizers themselves cannot actively enforce alignment between the prediction probabilities across multiple runs the resulting objective does not couple multiple models together. On the other hand, as verified in Figure 2 the co-distillation framework in itself does not affect the prediction confidence the underlying models. Thus, both these approaches can be combined together to obtain the following training objective which promotes both large model prediction confidence and small variance in prediction probabílities across multiple runs.
I combined Co-disillgga, (w1, W2;; S) L.Co (W1, W12; () W2: + a ERRe(w1, S), (19)
where
CReg(wi, W2;S) = - 71
ie[mn] Hf( wi)) (f(xi, W2))
EReg(w1, S) - 1 n [E[n]
(SKL(f(z; Unif) + SKL(f(#; w2), Unit))
E ADDITIONAL EXPERIMENTAN RESULTS
In this section present additional experimental results.
Ablation on CIFAR- 100. Next we present our ablation studics, similar to results Figure I for -IFAR-100. In Figure we plot the mean prediction entropy, accuracy and churn, for the proposed SKL regularizer (7) and the co-distillation approach (9). for varying a and B, respectively. Similar Figure [ the plots show the effectiveness of the proposed approaches in reducing churn, while improving accuracy.
E. ON HAARDWAE NONDETERMINISM. GPU BASELINE ABLATION RESULTS
We now repeat the experiments in Sec. 2.3 on GPUs to investigate the unavoidable nondeterminism introduced by different hardware platforms.
We report results in Table 6 Compared to Table [ there are two main differences. First, these experiments are GPU with smaller ResNet-32 models. Second, these results always data
16

---

Under review as a conference paper at ICLR 2021
Beta
Beta
0.00
0.02
0.04
0.06
0.08
0.10
0.00
0.02
0.04
0.06
0.08
0.10
0.00
0.02
0.04
0.06
0.08
0.10
26
40
Co-diisiilation (SKL)
76
26
24
36
75
22
32
20
28
0.2
73
24
Entropy Regularizer
0.0
20
0.0
0.1
0.2
0.3
0.0 0.1
0.0 Alpha
Alpha
Alpha
Figure 3: CIFAR-100 ablation study: Similar to Figure] plot the effect of the SKL regularizer (cf. (7., for varying a, and our proposed variant co-distillation (Co-distillsku) for varying 8, on the prediction entropy, accuracy, and churn. These plots shows the complementary nature these methods in reducing churn. While the regularizer reduces churn, by reducing the prediction entropy, Co-distillskL reduces churn by improving the agreement between two models, hence increasing the entropy.
Identical initialization
Ident. input data
Accuracy
91.67 + 0.22
91.61 0.16
91.53+ 0.13
91.86- 0.05
Churn (%)
8.44 0.27
8.25 0.11
2.73 0.07
1.33 0.10
SChurn (%)
8.71 + 0.26
8.69 - 0.12
2.98 - 0.08
1.47 - 0.05
Table 6: Ablation study of churn across runs on CIFAR-I0 with a ResNet-32 on GPU. Holding the initialization constant across models decreases churn significantly, and using identical input data (keeping minibatch ordering and augmentation constant) lowers churn further. Unlike Table [ these experiments were performed on GPU rather than TPU. Under this setting, using fixed initialization does achieve lower churn at a much more expensive computation cost.
augmentation, but control for the randomness differently by ensuring that all models within each run perform the same augmentations during training; we call these "identical input data" ablations because they control for both minibatch ordering and data augmentation. Again, the the first column gives the baseline churn with no factor held constant across runs, and the last column has every possible source churn except hardware nondeterminism held constant across runs.
Model initialization is significant source of chum. Simply initializing weights from identical seeds (columns 1-2 in Table I can significantly decrease churn, no matter what other aspects of training held constant. Holding input data constant across runs does reduce randomness and further reduces churn, but is less significant compared to constant initialization.
The rightmost column eliminates all possible sources of churn except for unavoidable determinism in the computing platform used for training machine learning models, e.g., GPU/TPU (Morin & Willetts 2020 PyTorch] 2019 Nvidial 2020 The experiments in Table were run on GPU, and even when all other aspects of training are held constant (rightmost column), model weights diverge within 100 steps (across runs) and the final churn is significant. We verified that this is the sole source of non-determinism models trained to 10,000 steps on CPUs under these settings had identical weights. Comparing these results the rightmost column of Table seems TPU platforms currently introduce a higher baseline amount of churn than GPUs. However, shown in this work, our Co-distillskL method can alleviate this somewhat.
E.2 ENSEMBLE
In Table provide the results for 2-ensemble models and compare with distillation and co- distillation approaches. We notice that the proposed approach with entropy regularizer and co- distillation achieve similar better churn, with lower inference costs. However the the ensemble models achieve better accuracy on certain datasets, and could be an alternative in settings where higher inference costs acceptable.
17

---

Under review as a conference paper at ICLR 2021
Training
Accurccy
Chara(5)
SChuray
641=0.15
RecNew-6
Codiailic
Costiilllg popet
2.8020355
44.20:::19
CIFAR-100
2-Eeembl:
20232:039
ReaNc-566
2-eeabble diall. Hitos 2015
26
18.6600266
254322233
7.222.223
54.3-0.86
Codivillen
15
19.5510227
25.08:0227
48,99
Codiuillakd,
76.5
17.2010:33
24.9920.11
9.03-0.3
45.9822.00
76.53
17.09.03
7.200.26
35.01
ImageSet
Ensemble
07
11.13100133
27.71 20.15
4210011
34.6320443
ResNes
2 Encemble distl. Hinos 2015
75
12.92:0.14
34.02202
49220.11
36,00:20.99
disilice
76-122005
11002:0:21
1434
491:20.36
32,672:
Co-dinillact,
11:45:00115
26.06-13.00
5.1.20.36
32,41::1.15
+tomopy
76.56-00022
11.15-00.66
22.92.1100
4.32.00.32
32.89--1.15
Table 7: Similar to Table 2 we provide results on accuracy and churn for 2-ensemble and proposed approaches. We notice that proposed approaches are either competitive improve churn over 2-ensemble method, while keeping the inference costs low.
F Loss LANDSCAPE: ENTROPY REGULARIZATION VS. TEMPERATURE SCALING
Figure visualises the entropy regularised loss (6) in binary case. We do so both in terms of the underlying loss operating on probabilities (i.e., log-loss), and the loss operating on logits with an implicit transformation via the sigmoid (i.e., the logistic loss). Here, the regularised log-loss is (p), for binary entropy H Similarly, the logistic loss is é: R R,, where ((f) = (1 - a) log (
é: [0, R. where E(p) (1 Q) log
f) + a Hain (o())) for sigmoid (.). The effect of increasing a is to dampen the loss for nigh-confidence predictions e.g., for the logistic loss, either very strongly positive or negative predictions incur low loss. This encourages
model to make confident predictions.
Figure 5 by contrast, illustrates the effect of changing the temperature 7 in the softmax. Temperature scaling is a common trick used to decrease classifier confidence. This also dampens the loss for high-confidence predictions. However, with strongly negative predictions, still incurs high loss compared to strongly positive predictions. This is in contrast to the more aggressive dampening of the loss achieved by the entropy regularised loss.
0.0
0.25
0.75
0.75
1.0
0.5
Figure 4: Visualisation of entropy regularised loss (eq. 6 in binary case. On the left panel the regularised log-loss (1 a) -olo, + H (p), which accepts probability in [0, as input. On the right panel is the logistic loss (1 - ax) log a (f) + a Hom (a)(()()) which accepts a score in IR as input and passes it through sigmoid a(-) to get probability estimate.
18

---

Under review as a conference paper at ICLR 2021
8D
1.25
2.0
10
10
Figure 5: Visualisation of temperature scaling on binary logistic lau. We plot (f/T for varying temperature parameter T.
19


Reviews:
[{"review_id": "tc_jTy4F7oN", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper investigates two methods to reduce churn in neural network classification prediction. Churn is when two networks trained on the same data produce outputs that disagree, due to randomness in the training process. The authors identify several sources of randomness, from underlying hardware differences to parameter initialization and more. The authors propose two ways to mitigate churn. One is to use entropy minimization to favor more confident predictions. The second is to use co-distillation, a form of online ensemble learning. The authors show that both together do a good job of reducing churn on three data sets.\n\nThe paper does a decent job of making an important point about churn, investigating its prevalence, and proposing a solution. The approach is sound and promising. For a narrow result like this, specific to one metric of neural networks, I would like much more empirical validation that the authors provide. Only three data sets and three baselines does not seem like enough, given that the experiments provide the main take-home message of the paper.\n\nI'd like a deeper discussion about why churn is bad. Can the authors give a concrete example where churn will make a machine learning system more undesirable? For example, imagine a facial recognition system. What if new data or new training lead to a new model that is just as accurate but makes mistakes on different people than before. In what application is that inherently bad? Can you formulate the problem with churn more formally? In the current paper, it's mostly assumed to be undesirable. To an extent, I agree, but I'd like to understand more clearly why it is undesirable. I think the comparison to reproducible scientific experiments is a little loose. A machine learning algorithm is not a scientific experiment. I don't think the authors need to cite quite so many papers about the much more general problem of reproducibility in science.\n\nThis finding is interesting and instructive: \"churn observed in Table 2 is not merely caused by the discontinuity of the arg max\".\n\nThis finding is fascinating: \"Even with extreme measures to eliminate all sources of randomness, we continue to observe churn due to unavoidable hardware non-determinism.\"\n\nI have a question about the minimum entropy procedure. Doesn't it depend on the confidence scores being accurate? For example, if some scores were overconfident, the minimum entropy procedure would tend to select those predictions and reduce accuracy. Imagine that confidence scores are normally distributed: some confidence scores are accurate but some are under- or over-confident. Minimum entropy will tend to pick the over-confident ones even though the confidence is in error. Is this a real danger and do the authors observe this at all with their technique?\n\nThis is more a question for Cormier et al. than for the current authors, but why use the word \"churn\" instead of \"disagreement\"? Is there a difference? From what I can tell, churn and disagreement are the same thing, and churn has a different English meaning. Disagreement seems like the better term for this.\n\nThe following paper explored the use of disagreement as a model selection tool and I think may have also proven Lemma 1:\n\nhttps://papers.nips.cc/paper/2603-co-validation-using-model-disagreement-on-unlabeled-data-to-validate-classification-algorithms\n\nMinor comments and typos:\n\nWhy is Table 2 on page 7 when it is referred to on page 3? Why is Table 2 referred to before Table 1?\n\n\"linear warmup and join updates\"\nDid you mean?:\nlinear warmup and joint updates\n\n\"any of the participating model can be used for inference\"\nDid you mean?:\nany of the participating models can be used for inference\n\nworst cast bound\nworst-case bound\n\nruns.We\nruns. We\n\nIntuitively,encouraging\nIntuitively, encouraging", "extra_scores": {}, "extra_text": {}}, {"review_id": "Sh942wQpsy9", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes methods to address churn in deep neural networks for classification, defined as the extent of disagreements in predictions of two models trained on the same data with the same algorithm. In addition to an existing measure of churn that is based on exact match of predicted classes, the paper introduces a soft measure of churn that measures disagreement by comparing the two models' class probability distributions. The paper proposes three regularization terms that can be added to the primary loss function used during training to reduce churn: two single-model regularization terms (based on cross entropy and KL divergence respectively) that encourage the model to output a more uneven probability distribution for an example, and a divergence-based term that is used by training two models simultaneously and that imposes a KL-divergence-based penalty that encourages the two models to output probability distributions that are as similar as possible. Experiments with ResNet architectures on CIFAR-10/100 and ImageNet indicate that the proposed approaches and their combination indeed reduce churn and do so to a larger extent than the two-model divergence-based approach applied in conjunction with cross-entropy that was proposed in work by Anil et al. in 2018 (although the improvement seems quite minor on ImageNet).\n\nMy primary concern is that the paper does not evaluate the effect of the regularization terms on calibration of the probability estimates. This issues is briefly discussed in Section 5. In practice, it seems vastly more important to have well-calibrated probability estimates than no churn. The single-model regularization terms proposed in the paper seem to encourage the model to become overconfident. This will reduce churn but is clearly not desirable in most practical applications.\n\nAnother concern is that the hyperparameters \\alpha and \\beta appear to be tuned by maximizing performance on the *test* data. This inflates the performance estimates on this data obtained for the proposed methods compared to the baseline, which does not have these hyperparameters to play with.\n\nI would also like to see results for churn when standard forms of regularization are included, such as L_2 or L_1 regularization. As these should also increase the stability of the learning process, it stands to reason that they will also reduce churn (particularly when the hyperparameters are tuned on the test data).\n\nThe paper appears to claim that the ordering of the mini batches (over multiple runs!) has an effect on accuracy. This does not appear to make sense (assuming only random orders are considered).\n\nThe paper eliminates churn due to data augmentation by removing data augmentation. Instead, the random number generator used for augmentation should be initialized in a deterministic manner.\n\nGiven that the best-performing model is based on training two networks, the paper should also include results for two-member ensembles that are obtained by averaging the probability distributions across the two networks. If memory consumption and inference time are not critical, and this two-member ensemble turns out to reduce churn substantially, it would be a useful solution.\n\nSome small issues:\n\n\"disagreements between predictions of the two models independently trained by the same algorithm\" --- on the same training data???\n\nTable 2 is discussed before Table 1.\n\nWhy is SChurn_1 denoted a percentage in Table 1?", "extra_scores": {}, "extra_text": {}}, {"review_id": "4HOnX-eOiUn", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work studies the ‘churn’ (disagreement between predictions of two replicates) caused by different sources of variation in the training procedure and proposes solutions to reduce it. One solution is to use minimum entropy regularization to increase prediction confidences and the second solution is to force model agreement via co-distillation.\n\n===============================\n\nPros:\n1. The paper is well written and clear.\n2. Studies dissect many components, e.g. churn caused different sources of variation, ablation study of the proposed co-distillation+entropy. \n3. Results are compared to reasonable baselines. \n\nCons:\n1. I find the problem statement unconvincing. How much retraining from scratch affects generalization? Beside variability on the test set accuracy, is there any evidence that the fluctuations observed are representative of variations of the true risk (on all data distribution)? Or is it only noise due to the small size of the test set?\n2. Modification of two independent variables (dependent and independent variables in planning of experimental procedures) in the experiments of Table 1 likely make the results misleading. (More on this below) \n\n===============================\n\nReasons for score:\n\nI would vote for a weak reject. The breadth of the empirical analysis is sufficient but subtle details (as further explained below) make the results misleading for Table 1.\n\n===============================\n\nAdditional observations\n\nModification of two independent variables (dependent and independent variables in planning of experimental procedures) in the experiments of Table 1 make the results potentially misleading. The churn depends to some degree on the level of accuracy, and data augmentation significantly affects accuracy as well. Indeed, removing data augmentation leads to a drastic drop of 4% accuracy. In the same way, but less drastic, the random data order from one epoch to another affects accuracy. When modifying data augmentation or data order, it is not possible to determine whether the change of churn is strictly due to data augmentation/data order or to accuracy drop. One way of avoiding this confounding effect would be to conduct hyperparameter optimization in a way to enforce a given level of accuracy (ex: 88%). We could then compare with augmentation at 88% accuracy vs no augmentation at 88% accuracy. If for instance a sub-optimal learning rate with data augmentation yielding 88% accuracy leads to the same level of churn than a good learning rate with no data augmentation yielding 88% accuracy, then we could conclude the effect on churn is mainly due to accuracy itself. I would nevertheless assume data augmentation to reduce churn indeed as it can be seen as increasing the dataset size, which reduces the level of noise. Therefore, to measure the effect of accuracy alone on churn I would also run two experiments where I optimize the learning rate to find accuracies of 84% and 88% (both without data augmentation) to see the relation between accuracy and churn on equal dataset size.\n\nOn the same topic, the authors should avoid removing altogether data augmentation when they want to remove its effect of variation. They should rather seed it. I understand it is more effort as I recently went through the process, but it is possible. This way they would study the effect of varying data augmentation vs fixing it across replicates without losing its regularization effect. The same applies for data order. It is possible to seed data order without losing the randomization from one epoch to another. \n\nIn section 2.3, the authors say that ‘Even when all other aspects of training are held constant (rightmost column), model weights diverge within 100 steps (across runs) and the final churn is significant.’ I am not familiar with TensorFlow, but I know ResNet implementations based on the cudnn backend require the deterministic operators for perfect reproducibility. ResNet is trainable in a deterministic way using PyTorch for instance. It turns out I have also studied these sources of noise and the residual variance resulting from this noise (numerical noise due to operation order on GPU) is significantly smaller than the one caused by data sampling, weighs init, data ordering or data augmentation. The results last column on table 1, where churn is only caused by numerical noise, suggests that the large increase in churn when removing data augmentation is mainly due to the decrease in accuracy. When only numerical noise is present there is smaller variance in results so I would expect churn to be smaller. The fact that it increases in Table 1 suggests to me that we are indeed observing a confounding effect where the main cause is the drop of accuracy rather that the removal of data augmentation.\n\n\nBold results in table 2 are misleading. Many of them are not significantly different yet only one result per column is in bold. All top results that are not significantly different should be in bold. \n\nThe co-distillitation + entropy procedure proposed in this paper introduces 2 new hyperparameters. The optimization of these hyper-parameters can lead to misleading results if hyperparemeters of baselines are not optimized accordingly with similar budgets. The experimental section should report these optimization procedures so that we can assert the reliability of the results. Also, were the alpha and beta optimized to provide better accuracy of lower churn in Table 2?\n\n===============================\n\nTypos, minor comments, questions\n\nIntro in section 2 should restate that the definition builds open the work of Cormier et al 2016.\n\nIn the co-distillation process, how do you choose which model to retain to compute the churn? As I understand it the churn is computed on 2 models that are trained independently with other co-distillation ‘siblings’. Do you pick randomly within the co-distillation pairs?\n\nPage 1, first paragraph: a a novel -> a novel\nPage 5, Intuitively,encouraging -> Intuitively, encouraging\n\n===============================\n\nPost-Rebuttal\n\nI thank the authors for the detailed answer. In light of the response of the authors and the other reviews, I still recommend rejecting the paper, with a rating of 5.\n\nSome comments based on the rebuttal:\n\nI find the data in table 6 to support my point on confounding variables. The churn with data augmentation fixed on GPU is systematically higher than with random data augmentation on TPU when model initialization is random. Note that the main differences here beside the fact that data augmentation is fixed or random, is that the accuracy is lower by 1.5%-2.5%. We see again an increase in churn related to a decrease in accuracy. Just as when the data augmentation was removed altogether. The authors say that accuracy change itself isn't predictive of churn because we could make a training perfectly reproducible with lower accuracy thus leading to 0 churn, but the same argument would hold for removing a fixed data augmentation from a perfectly reproducible training. When not fixing the whole training process, two different interventions leading to equivalent accuracy decrease could lead to equivalent churn. This for instance would be a direct consequence of a binomial modelisation of the model performance as a function of test set size and model average accuracy (and by the way which models fairly well test accuracy variation for the datasets-architectures in this paper). The lower the accuracy, the higher the variance.\n\n> Table 2. We boldfaced the results in table 2 with the best mean performance, which we believe is a standard practice.\n\nIt is unfortunately common practice indeed, but it is a bad practice. Results that are so close that random fluctuations could explain the difference should not be considered as different.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Fitting a neural net is a stochastic process, with many sources of stochasticity, including initialization, batch presentation, data augmentation, non-deterministic low-level operations and the non-associativity of rounding errors in multi-threads systems such as GPUs and TPUs. In this paper, the authors aim to alleviate this randomness by incorporating specific regularizers during learning or by using co-distillation.

As the reviewers pointed out, the paper is quite clearly written, but the motivation for this work is not clear. The example of system updates does not correspond to the current study that targets the internal variability of the learning process. Reproducibility is an important issue, but in a statistical context, why would it be relevant to assess reproducibility by the individual decisions made by a single estimate? The usual way of assessing learning algorithms is to look at (a summary of) the distribution of performance for a given learning problem characterized by a data distribution, not to look at individual decisions made by a particular estimate. Furthermore, this study ignores the randomness due to the selection of hyper-parameters. Why would the partial reproducibility studied here, for a fixed choice of hyper-parameters, be of particular interest? As is, either the work is ill-defined and incomplete, or it lacks a clear rationale, and I thus recommend rejection. 

I would also like to point a reproducibility issue in the proposed experimental study. The exact meaning of the variability measures reported in the tables is not given, but I assume that it is the standard deviation of the different runs (for example, the 5 replicates in Table 1). These figures are not directly related to the variability of each setup, as they ignore the variability due to the random selections during the ablation study (for example, as I understand it, the last result of Table 1 was obtained for a single arbitrary initialization and a single arbitrary batch order). 

Decision: Reject