Leveraging Simple Model Predictions for Enhancing its Performance
Authors: Amit Dhurandhar
Karthikeyan Shanmugam
Ronny Luss
Venue:  (2020)
Keywords: simple models
interpretability
resource constraints

Abstract: There has been recent interest in improving performance of simple models for multiple reasons such as interpretability, robust learning from small data, deployment in memory constrained settings as well as environmental considerations. In this paper, we propose a novel method SRatio that can utilize information from high performing complex models (viz. deep neural networks, boosted trees, random forests) to reweight a training dataset for a potentially low performing simple model such as a decision tree or a shallow network enhancing its performance. Our method also leverages the per sample hardness estimate of the simple model which is not the case with the prior works which primarily consider the complex model's confidences/predictions and is thus conceptually novel. Moreover, we generalize and formalize the concept of attaching probes to intermediate layers of a neural network, which was one of the main ideas in previous work \citep{profweight}, to other commonly used classifiers and incorporate this into our method. The benefit of these contributions is witnessed in the experiments where on 6 UCI datasets and CIFAR-10 we outperform competitors in a majority (16 out of 27) of the cases and tie for best performance in the remaining cases. In fact, in a couple of cases, we even approach the complex model's performance. We also conduct further experiments to validate assertions and intuitively understand why our method works. Theoretically, we motivate our approach by showing that the weighted loss minimized by simple models using our weighting upper bounds the loss of the complex model.

Under review as a conference paper at ICLR 2020
LEVERAGING SIMPLE MODEL PREDICTIONS FOR ENHANCING ITS PERFORMANCE
Anonymous authors Paper under double-blind review
ABSTRACT
There has been recent interest in improving performance of simple models for mul- tiple reasons such as interpretability, robust learning from small data, deployment in memory constrained settings as well as environmental considerations. In this paper, propose novel method SRatio that can utilize information from high performing complex models (viz. deep neural networks, boosted trees, random forests) to reweight a training dataset for a potentially low performing simple model such as a decision tree or a shallow network enhancing its performance. Our method also leverages the per sample hardness estimate of the simple model which is not the case with the prior works which primarily consider the complex model's confidences/prediittio
and is thus conceptually novel. Moreover, we generalize and formalize the concept of attaching probes to intermediate layers of neural network to other commonly used classifiers and incorporate this into our method. The benefit of these contributions is witnessed in the experiments where on 6 UCI datasets and CIFAR- 10 we outperform competitors in majority (16 out of 27) of the cases and tie for best performance in the remaining cases. In fact, in a couple of cases, we even approach the complex model's performance. We also conduct further experiments to validate assertions and intuitively understand why our method works. Theoretically, we motivate our approach by showing that the weighted loss minimized by simple models using our weighting upper bounds the loss of the complex model.
INTRODUCTION
Simple models such as decision trees or rule lists or shallow neural networks still find use in multiple settings where a) (global) interpretability is needed, b) small data sizes are available, or c) memory/compptttional constraints are prevalent (Dhurandhar et al. 20188). In such settings compact or understandable models are often preferred over high performing complex models, where the combination of a human with interpretable model can have better on-field performance than simply using the best performing black box model (Varshney et al. 2018). For example, a manufacturing engineer with an interpretable model may be able obtain precise knowledge of how out-of-spec product was produced and can potentially go back to fix the process as opposed to having little-to-no knowledge of how the decision was reached. Posthoc local explainability methods (Ribeiro al.) 2016, Bach et al. 20155 Dhurandhar et al. 2018a) can help delve into the local behavior of black box models, however, besides the explanations being only local, there is no guarantee that they are in fact true (Rudin 2018). There is also a growing concern of the carbon footprint left behind in training complex deep models (Strubell al. 2019), which for some popular architectures is more than that left behind by a car over its entire lifetime.
In this paper, we propose a method, SRatio, which reweights the training set improve simple models given access to highly accurate complex model such as a deep neural network, boosted trees, or some other predictive model. Given the applications we are interested in, such as interpretability or deployment of models in resource limited settings, assume the complexity of the simple models to be predetermined or fixed (viz. decision tree of height < 5). We cannot grow arbitrary size ensembles such in boosting or bagging (Freund & Schapire 1997). Our method applies potentially to any complex-sing simple model combination which is not the case for some state-of-the-a methods this
space such as Knowledge Distillation (Geoffrey Hinton! 2015) Profweight (Dhurandhar al.

---

Under review as a conference paper at ICLR 2020
2018b), where the complex model is assumed to be deep neural network. In addition, we generalize and formalize the concept of probes presented in (Dhurandhar et al.) 2018b) and provide examples of what they would correspond to for classifiers other than neural networks. Our method also uses the priori low performing simple model's 's confidences to enhance its performance. We believe this to be conceptually novel compared to existing methods which seem only leverage the complex model (viz. its rrediction/ccofidenees) The benefit is seen in experiments where we outperform other competitors in majority of the cases and are tied with one or more methods for best performance in the remaining cases. In fact, in couple of cases we even approach the complex model's performance, i.e. a single tree made be as accurate as 100 boosted trees. Moreover, we motivate our approach by contrasting it with covariate shift and show that our weighting scheme where we now minimize the weighted loss of the simple model is equivalent to minimizing an upper bound on the loss of the complex model.
RELATED WORK
Knowledge Distillation (Geoffrey Hinton 2015 Tan et al. 2017 Lopez-Paz et al., 2016) is one of the most popular approaches for building "simpler" neural networks. It typically involves minimizing the cross-entropy loss of a simpler network based on calibrated confidences (Guo et al., 2017) of a more complex network. The simpler networks are usually not that simple in that they are typically of the same (or similar) depth but thinned down (Romero et al., 2015). This is generally insufficient to meet tight resource constraints (Reagen et al. 2016). Moreover, the thinning down was shown for convolutional neural networks but it is unclear how one would do the same for modern architectures such ResNets. The weighting of training inputs approach the other hand can be more easily applied to different architectures. to models optimizing losses other than cross-entropy (viz. hinge loss, squared loss) with some interpretation of which inputs are more (or less) important. Some other strategies to improve simple models (Buciluă et al. 2006 Ba Caurana 2013 Bastani et al. 2017 are also conceptually similar to Distillation, where the actual outputs are replaced by predictions from the complex model.
also has another advantage in that it can be readily applied
In Dehghami et al. (2017). authors use soft targets and their uncertainty estimates inform student model on a larger dataset with more noisy labels. Uncertainty estimates are obtained from Gaussian Process Regression done on dataset that has less noisy labels. In (Furlanello al. (2018), authors train a student neural network that is identically parameterized to the original one by fitting to soft scores rescaled by temperature. In our problem, the complexity of the student is very different from that of the teacher and we do compare with distillation-like schemes. Frosst & Hinton (2017) define new class of decision trees called soft decision trees to enable to fit soft targets (classic distillation) of neural network. Our methods use existing training algorithms for well-known simple models. Ren et al. (2018) advocate reweighting samples as a way to make deep learning robust by tuning the weights a validation set through gradient descent. Our problem is about using knowledge from pre-trained complex model to improve simple model through weighting samples.
The most relevant work to our current endeavor is Pro/Weight (Dhurandhar al. (0188), where they too weight the training inputs. Their method however, requires the complex model to be neural network and thus does not apply to settings where we have a different complex model. Moreover, their method, like Distillation, takes into account only the complex model's 's assessment of an example's s difficulty.
Curriculum learning (CL) (Bengio et al. 2009) and boosting (Freund & Schapire 1997) are two other approaches which rely on weighting samples, however, their motivation and setup are significantly different. In both CL. and boosting the complexity of the improved learner can increase as they do not have to respect constraints such as interpretability (Dhurandhar et al. 2017 Montavon et al., 2017) or limited memory/power (Reagen al.) 2016 Chen et al. 2016). In CL. typically, there is no automatic gradation of example difficulty during training. In boosting, the examples are graded with respect to previous weak learner and not an independent accurate complex model. Also, as we later show, our method does not necessarily up- up-weight hard examples but rather uses a measure that takes into account hardness assessed by both the complex and simple models.
2

---

Under review as a conference paper at ICLR 2020
3 METHODOLOGY
In this section, we first provide theoretical and intuitive justification for our approach. This is followed by description of our method for improving simple models using both the complex and simple model's predictions. The key novelty lies in the use of the simple model's prediction, which also makes the theory non-trivial yet practical. Rather than use the complex model's prediction, we generalize concept from (Dhurandhar et (2118b)))) where they attached probe functions each layer of a neural network, obtained predictions using only the first & layers (k being varied up to the total number of layers), and used the mean of the probe predictions rather than the output of only the last layer. They empirically showed the extra information from previous layers to improve upon only using the final layer's output. Our generalization, which call graded classifiers and formally define below, extracts progressive information from other models (beyond neural networks). The graded classifiers provide better performance than using only the output of complex model, illustrated by our various experiments in the subsequent section.
3.1 THEORETICAL MOTIVATION
Our approach in section 3.3 can be motivated by contrasting it with the covariate shift (Agarwal et al., 2011) setting.
X Y is the input-output space and p(x, y) and q(., y) are the source and target distributions in the covariate shift setting, then it is assumed that p(y[z)= q(y (r) but p(x) * (z)). One of the standard solutions for such settings is importance sampling where the source data is sampled proportional to
s(ær] in order to mimic as closely possible the target distribution. p(xr
In case, the dataset is the same but the classifiers (i.e. complex and simple) are different. We can think of this as a setting where p(x) = q(zr) as both the models learn from the same data, however, p A q(ylar) (z) where p(ylæ:) and (y(zz) correspond to the outputs of complex and simple classifiers, respectively. Given that we want the simple model to approach the complex models performance, a natural analog to the importance weights used in covariate shift is weight samples by p(ylz) which is the essence of our method as described below in section 3.3 ((1/=))
Now, let us formally show that the expected cross-entropy loss of a model is no greater than the reweighted version with an additional positive slack term. This implies that training the simple model with this reweighting is a valid and sound procedure of the loss we want to optimize.
Lemma 3.1. Let pe po(y(z)) x) be the softmax scores on specific model from simple model space O. Let l ② be the set of simple model parameters that is obtained from a given learning algorithm for the simple model on training dataset. Let Pe(y) x)) be pre-trained complex classifier whose loss smaller than 6* on the training distribution. Let Z 1 be scalar clip level for the ratio Poly)ay (r)/ pur (vlz)) Then we have:
Pa(ylla) (r) Pa- (z)lar) :))) log
E log (y|z:) <
max
1, min
po (wll))
Pa- (y/zx) }))
E
log
min
log(
Proof.
1
[))]] E log min
Pe- (y|xr) }
Pe(ylar)
log(oooo (b]=))
E log po(y)
min
x)
Po- (3)(z) [z)
1
(o(() Pa- (3)x2)
E log
max
min
p)(y)
E
log
min
Po- (3/(
Pe(()
E
max
1. 1,nin
Po- (2/2²)
log (o(ul
E
log
min
log(())
Po- (y|xr)
3

---

Under review as a conference paper at ICLR 2020
(a) The inequality log(wx) I w log(z) + 1og((3) holds for all 2 w 2 1, where 3 2 is any arbitrary clip level.
We observe that re-weighing every sample by max(1, min( Pe(pla) Pa- (vla) B)) and re- optimizing using the simple model training algorithm is a sound way to optimize the cross-entropy loss of the simple model on the training data set. The reason we believe that optimizing the upper bound could be better is because many simple models such as decision trees are trained using a simple greedy approach. Therefore, reweighting samples based an accurate complex model could induce the appropriate bias leading to better solutions. Moreover, in equation 2 the second inequality the main place where there is slack between the upper bound and the quantity we are interested in bounding. This incquality exhibits a smaller slack w = Pe (=(yz) is not much smaller than 1 with high probability. This is typical when P. comes from a more complex model that is more accurate than that of 0
Remark 1:
pe- (s/(z)
Remark 2: The upper bound used for the last inequality in the proof leads to a quantification of the bias introduced by weighting for particular dataset. Note that in practice, we determine the optimal 33 via cross- validdaiii..........
3.2 INTUITIVE JUSTIFICATION
Intuitively, assuming w 2 1 implies that the complex model finds an input easier (i.e. higher score or confidence) to classify in the correct class than does the simple model. Although in practice this may not always be the case, it is not unreasonable to believe that this would occur for most inputs, especially if the complex model is highly accurate.
The motivation for our approach conceptually does not contradict (Dhurandhar et al. 2018b). where hard samples for complex model are weighted low. These would still be potentially weighted low as the numerator would be small. However, the main difference would occur in the weighting of the easy examples for the complex model, which rather than being uniformly weighted high, would now be weighted based on the assessment of the simple model. This, we believe, is important information as stressing inputs that are already extremely easy for the simple model to classify will possibly not lead to the best generalization. It is probably more important to stress inputs that somewhat hard for the simple model but easier for the complex model, as that is likely to be the critical point of information transfer. Even though easier inputs for the complex model are likely to get higher weights, ranking these based on the simple model's assessment is important and not captured in previous approaches. Hence, although our idea may appear be simple, we believe is significant jump conceptually in that it also takes into account the simple model's behavior to improve itself.
Our method described in the next section is generalization of this idea and the motivation presented in the previous section. If the confidences of the complex model are representative of difficulty then we could leverage them alone. However, many times as seen in previous work (Dhurandhar et al.. 2018b), they may not be representative and hence using confidences of lower layers or simpler forms of the complex classifier can be very helpful.
3.3 METHOD
We now present our approach SRatio in algorithm [. which uses the ideas presented the previous sections and generalizes the method in (Dhurandhar et al. (018b) to be applicable to complex classifiers other than neural networks.
In previous works (Akshayvarun Subramanya 2017 Dhurandhar et al. 2018b). it was seen that sometimes highly accurate models such as deep neural networks may not be good density estimators and hence may not provide accurate quantification of the relative difficulty of input. To obtain a better quantification, the idea of attaching probes (viz. linear classifiers) to intermediate layers of deep neural network and then averaging the confidences was proposed. This, seen in the previous work. led to significantly better results over the -fate-of-thh-ar art. Similarly, we generalize our method where rather than taking just the output confidences of the complex model as the numerator,
4

---

Under review as a conference paper at ICLR 2020
Algorithm 1 Our proposed method SRatio.
Input: (graded) classifiers Ç1+ .... Sn. learning algorithm for simple model Ls. dataset Ds of cardinality N, performance gap parameter r and maximum allowed ratio parameter /3.
1) Train simple model on Ds. S < Ls Iw) and compute its (average) prediction error es. (Obtain initial simple model where each input is given unit weight.)
2) Compute (average) prediction errors é1, ... for the graded classifiers and store the ones that
are at least y more accurate than the simple model i... I {i E [l....,n) Es 6, 2 7
3) Compute weights for all inputs as follows: w(x) Eicr fc(x) where m the cardinality of
mS(z)
set I and ((z) is the prediction probability/score for the true class of the simple model.
4) Set w(x) + 0, if w(x) B. {Ignore extremely hard examples for the simple model.)
5) Retrain the simple model on the dataset Ds with the corresponding learned weights w, Sw f
Lg(Ds, w)
6) Return S.
we take an average of the confidences over a gradation of outputs produced by taking appropriate simplifications of the complex model. We formalize this notion of graded outputs as follows:
Definition (6-graded) Let X x Y denote the input-output space and p(x, y) the joint distribution over this space. Let S1, S2. .... Cn denote classifiers that output the prediction probabilities for a given input x E X for the most probable (or true) class y Y determined by p(y (r:). We then say that classifiers Çi, E2. Sn are 5-graded for some 5 E (0, 1] and (measurable) set Z C X Vf Z, (xx) S (2(x))
(I)),, where Jrez p(x) 8.
Loosely speaking, the above definition says that a sequence of classifiers is ô-graded if a classifier in the sequence is least accurate as the ones preceding it for inputs whose probability measure is at least 5. Thus, sequence would be l-graded if the above inequalities were true for the entire input space (i.e. Z X). Below are some examples of how one could produce d-graded classifiers for different models in practice.
Deep Neural Networks: The notion of attaching probes, which are essentially linear classi- fiers (viz. a [W + b)) trained on intermediate layers of deep neural network (Dhurandhar et al.
2018b Alain & Bengio 2016) could be seen as way of creating 5-graded cl classifiers, where lower layer probes are likely to be less accurate than those above them for most of the samples. Thus the idea of probes simplifications of the complex model, as used in previous works, are captured by our definition.
Boosted Trees: One natural way here could to consider the ordering produced by boosting algorithms that grow the tree ensemble and use all trees up to certain point. For example, if we have an ensemble of 10 trees, then Çi could be the first tree, Ç2 could be the first two trees and so where Ç10 is the entire ensemble.
Random Forests: Here one could order trees based on performance and then do a similar grouping as above where Ç1 could be the least accurate tree, then (2 could be the ensemble of (1 and the second most inaccurate tree and so on. Of course, for this and boosted trees one could take bigger steps and add more trees to produce the next Ç so that there is a measurable jump in performance from one graded classifier to the next.
Other Models: For non- ensembeeee models such as generalized linear models one too could form graded classifiers by taking different order Taylor approximations the functions, or by setting the least important coefficients successively to zero by doing function decompositions based on binary, ternary and higher order interactions (Molnar et al. 2009), or using feature selection and starting with model containing the most important feature(s).
Given this, we see in algorithm I that take input graded classifiers and the learning algorithm for the simple model. Trivially, the graded classifiers can just be the entire complex classifier where only consider its output confidences. We now take ratio of the average confidence of the graded classifiers that are at least more accurate than the simple model by y > 0 and the simple model's s confidence. If this ratio is too large (i.e. >> (3) we set the weight zero and otherwise the ratio is the
5

---

Under review as a conference paper at ICLR 2020
Table 1:
Below the averaged % errors with 95% confidence intervals for the different methods on six real datasets. Boosted Trees and Random Forest (100 trees) are the complex models (CM), while a single decision tree and linear SVM are the simple models (SM). Best simple model results are indicated in bold. indicates the simple model has approached the complex models performance.
Complex Model
CM Error
Simple Model
SM
Distill-proxy Error (SM)
ConfWeight Error (SM)
SRafio
Dataset
Error
Error (SM)
Tree
10.95
10.95
11.42
8.57
Boosted Trees
8.10
004
10.4
-0.8
10.5
+0.4
SVM
12.38
11.90
11.90
10.47
=0.6
+0.6
=0.6
+0.5
Ionosphere
Tree
10.95
10.95
11.42
10.42
Random Forest
6.19
-04
10.4
=0.4
:0.11
10.4
SVM
12.38
12.38
12.38
11.42
-0.6
+00.6
-0.6
+0.3
Tree
15.62
15.62
15.62
15.62
Boosted Trees
4.68
00.8
=1.0
+0.5
+0.4
SVM
1.56
1.56
1.56
1.56
-0 0.4
+0.4
0.4
+0.4
Ovarian Cancer
Tree
15.62
15.62
14.06
14.04
Random Forest
6.25
0..8
10.8
0011
10.1
10.8
SVM
1.56
1.56
1.56
1.56
=0.4
+0.4
=0.4
+0.4
Tree
23.88
22.77
23.33
22.77
Boosted Trees
15.55
-0.7
10.1
-0.3
10.2
+0.6
SVM
17.22
16.67
17.22
16.77
=0.2
+0.3
=0.2
+0.2
Heart Disease
Tree
23.88
23.88
25.55
22.77
Random Forest
15.88
00.7
+00.7
00.5
+0..3
+0.6
SVM
17.22
17.22
16.67
16.67
-0.2
+0..2
0.3
+0.2
Tree
25.43
25.06
25.10
25.06
Boosted
12.96
-0.2
+0.1
-0.1
+0.1
+0.1
SVM
14.70
15.33
14.70
13.72
-0.2
+0.00
-0.2
+0.2
Waveform
Tree
25.43
25.43
25.43
25.06
Random Forest
10.90 10.1
-0.2
10.2
-0.2
10.1
SVM
14.70
14.33
14.30
12.72
-0.2
+0.0
=0.2
+0.5
Tree
7.93
7.93
7.86
7.15
Boosted Trees
6.32
-0.2
10.1
-0.2
10.1
+0.0
SVM
14.56
15.85
13.92
13.92
Human Activity
=0.1
+0.1
-0.1
+0.2
Recognition
Tree
7.93
7.23
7.21
6.67
Random Forest
2.34
-02
+.0.1
=0.1
+0.0
+0.0
SVM
14.56
13.92
14.21
13.92
-0.1
+0.11
+:0.1
Tree
4.49
6.11
4.45
4.06
Boosted Trees
4.06
-0.1
+00.1
-0.1
+0.1
+0.1
SVM
6.11
6.29
6.41
5.48
-0.1
+0.11
+0.1
Musk
Tree
4.49
4.49
4.47
3.89
Random Forest
2.45
-0.1
+0.1
-0.1
10.1
10.1
SVM
6.11
6.16
5.96
5.53
=0.1
+0.1
=0.1
+0.1
weight for that input. Note that setting large weights zero reduces the variance of the simple model because prevents dependence on select few examples. Moreover, large weights mostly indicate that the input is extremely hard for the simple model classify correctly and expending effort and ignoring other examples will most likely be detrimental to performance. Best values for both parameters can be found empirically using standard validation procedures.

---

Under review as a conference paper at ICLR 2020
4 EXPERIMENTS
In this section, we empirically validate our approach as compared with other state-of-the-art methods used to improve simple models. We experiment 6 real datasets from UCI (Dheeru & Karra Taniski dou, 2 Ionosphere, Ovarian Cancer (OC), Heart Disease (HD), Waveform, Human Activity Recognition (HAR), and Musk. Data characteristics are given in the appendix.
4.1 UCI DATASETS SETUP
We experiment with two complex models, namely, boosted trees and random forests, each of size 100. For each of the complex models we see how the different methods perform in enhancing two simple models: single CART decision tree and linear SVM classifier. Since ProfWeight is not directly applicable in this setting, we compare with its special case ConfWeight which weighs examples based on the confidence score of the complex model. We also compare with two models that serves as and Disti 11 11rooxyy 2 since distillation the
a proxy to Distillation, namely Di st stsss-llpppp 11pprxyy mainly designed for cross-entropy loss with soft targets. For D st stllooy 1, hard targets predicted by the complex models (boosted trees or random forests) as labels for the simple models. For Distill setiiy-p 2, we use regression versions of trees and SVM for the simple models to fit the soft probabilities of the complex models. For multiclass problems, we train separate regressor for fitting a soft score for each class and choose the class with the largest soft score. This version performed worse and numbers are relegated to the supplement. We only report numbers for Dist proxx 30% test. Results for all methods are averaged over 10 random splits and reported in Table S with 95% confidence intervals.
the main paper. Datasets are randomly split into 70% train and
For our method, graded classifiers based the complex models are formed as described before in steps of 10 trees. We have 10 graded classifiers (10 x 10 = 100 trees) for both boosted trees and random forests. The trees in the random forest ordered based on increasing performance. Optimal values for y and 3 are found using 10-fold vrossvaalidation
meggoved points Highwal weihted Rareiiiiig poirs
17-
ST-T SY-M
siosa
3.5%
N Train
3.5%
16
Math lancsphare
0%
Muxi
HAR
Waveforn
HD
Mtunk
HAR
Weneffom
0C
Figure 1: Above (left) see the of training set points assigned weight by SRatio at optimal 3 values for each complex (BT, RF) and simple model (Tree, SVM) combination on the 6 UCI datasets. We see that 1% of the training set has weights in all cases. Above (right) we analyze why intuitively our reweighting seems to work by considering the % of nearest neighbors that have zero weight, high weight, and in-between weight. Results are averaged over all complex-simphl model combinations. Both plots represent averaged values over 10 random train/test splits with 95% confidence intervals.
4.2 CIFAR-10 10 SETUP
The setup we follow here very similar to previous works (Dhurandhar et al. 2018b). The complex model is an 18 unit ResNet with 15 residual (Res) blocks/units. We consider simple model that consists of 3 Res units, 5 Res units and 7 Res units. Each unit consists of two x 3 convolutional layers with either 64, 128, or 256 filters (the exact architecture is given in the appendix). A 3 x convolutional layer with 16 filters serves an input to the first ResNet block, while an average pooling layer followed by fully connected layer with 10 logits takes as input the output of the final ResNet block for each of the models.
Tensorflow 1.5.0 used for CIFAR-10 experiments
7

---

Under review as a conference paper at ICLR 2020
Table 2: Below we observe the averaged accuracies (%) of simple models SM-3 (3 Res units), SM-5
(5 Res units) and SM-7 (7 Res units) trained with various weighting methods and distillation. The complex model achieved 84.5% accuracy, Statistically significant best results are indicated in bold.
SM-3
SM-5
SM-7
Standard
73.15 (# 0.7)
75.78 (=0.5)
78.76 (+0.35)
ConfWeight
76.27 (#0.48)
78.54 (H0036)
81.46 (E:0.50)
Distillation
65.84 (+0060)
70.09 (+-0119)
73.4 (+F0.64
ProfWeight
76.56 (+0.51)
79.25 (+0.36)
81.34 (#=0,49)
SRatio
77.23 (#0.14)
80.14 (#0.22)
81.89 (#:028)
We form 18 graded classifiers by training probes which are linear classifiers with softmax activations attached flattened intermediate representations corresponding to the 18 units of ResNet (15 Res units 3 others). As done in prior studies, we split the 50000 training samples from the CIFAR-10 dataset into two training sets of 30000 and 20000 samples, which are used to train the complex and simple models, respectively. 500 samples from the CIFAR-10 10 test set are used for validation and hyperparameter tuning (details in appendix). The remaining 9500 are used to report accuracies of all the models. Distillation (Geoffrry Hinton| 2015) employs cross-entropy loss with soft targets to train the simple model. The soft targets are the softmax outputs of the complex model's last layer rescaled by temperature t 0.5 which was selected based on cross-validation. For ProfWeight, we report results for the area under the curve (AUC) version as it had the best performance in a majority of the cases in the prior study. Details of 3 and y values that experimented with to obtain the results in Table 2 are in the appendix.
4.3 OBSERVATIONS
In the experiments on the 6 UCI datasets depicted in Table 8 we observe that we are consistently the best performer, either tying or superseding other competitors. Given the 24 experiments based on dataset, complex model, and simple model combinations (6 x 2 x 2 = 24), we are the outright best performer in 14 of those cases, while being tied with one or more other methods for best performance in the remaining 10 cases. In fact, in 2 cases where we are outright best performers, dataset=lonosphere, complex model =boosted trees, simple model= = Tree and dataset=Musk, complex model =boosted trees, simple model = Tree, our method enhances the simple model's performance to match (statistically) that of the complex model. We believe this improvement to be significant. In fact, the Musk dataset, we observe that the simple tree model enhanced using method, where the complex model is a random forest, supersedes the performance of the other complex model. On the Ovarian Cancer dataset, linear SVM actually seems perform best, even better than the complex models. A reason for this may be that the dataset is high dimensional with few examples. Due to this, it also seems difficult for any of the methods to boost the simple model's performance.
figure[leet) We now offer an intuition as to why our weighting works. First, we see in figure Ieett) that our assertion of only a very small fraction of the training set being assigned 0 weights based on parameter 3, which upper bounds the weights, is true (< 1% is assigned weight 0). For Ovarian Cancer (OC), SVM was better than the complex models and hence no points had weight O.
In figure [ riight), we see the intuitive justification for the learned weights. Given 10 nearest neighbors (NN) of a data point, let v/ and va denote the number of those NNs that belong to its class (i.e. same class), and most frequent different class respectively. Then the Y-axis depicts P._.. x 100. This metric gives a feel for how difficult it is likely to be to correctly classify point. We thus see that most of the 10 NNs for the O weight points lie in different class and so likely are almost impossible for a simple model to classify correctly. The highest weighted points (i.e. top 5 percentile) have nearest neighbors from the same class almost 50% of the time and are close to the most frequent (different) class. This showcases why the O weight points are so difficult for the simple models to classify, while the highest weighted points seem to outline an important decision boundary. With some effort simple model should be able to classify them correctly and so focusing on them is important. The remaining points (on average) seem to be relatively easy for both complex and simple models.
vatua
8

---

Under review as a conference paper at ICLR 2020
On the CIFAR-10 dataset, the simple model has 3 Res units and Res units. For Res units, we tie with ProfWeight and ConfWeight. Given the motivation of resource limited settings where memory constraints can be stringent (Reagen et al.] 2016) Chen et al. 2016), SM-3 and SM-5 are anyway the more reasonable options. In general, we from these experiments that the simple model's predictions be highly informative in improving its own performance.
that our method outperforms other of-h-of-the-art methods where
5
DISCUSSION
Our approach and results outline an interesting strategy, where even in cases that one might want a simple model, it might be beneficial build an accurate complex model first and use enhance the desired simple model. Such is exactly the situation for the manufacturing engineer described in the introduction that has experience with simple interpretable models that provide him with knowledge that complex model with better performance cannot offer.
Although our method may appear to be simplistic, we believe it to be a conceptual jump. Our method takes into account the difficulty of sample not just based on the complex model, but also the simple model which a priori not obvious and hence possibly ignored by previous methods that may or may not be weighting-base based. Moreover, we have empirically shown that our method either outperforms or matches the best solutions across a wide array of datasets for different complex model (viz. boosted trees, random forests and ResNets) and simple model (viz. single decision trees, linear SVM and small ResNets) combinations. In fact, in a couple cases, a single tree approached the performance of 100 boosted trees using our method. In addition, we also formalized and generalized the idea behind probes presented in previous work (Dhurandhar et al. (018b) to classifiers beyond deep neural networks and gave examples of practical instantiations. In the future, we would like to uncover more such methods and study their theoretical underpinnings.
REFERENCES
D. Agarwal, L. Li, and A. J. Smola. Linear-Time Estimators for Propensity Scores. In 14 Intl. Conference on Artificial Intelligence and Statistics (AISTATS), pp. 93- 100, 2011.
R. V.Venaatsh Babu Akshayvarun Subramanya, Suraj Srinivas. Confidence estimation in deep neural networks via density modelling. arXiv preprint arXiv:1707.07013. 2017.
Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01444, 2016.
Lei Jimmy Ba and Rich Caurana. Do deep nets really need to be deep? CoRR, abs/1312.6184, 2013. 6184
URL httt:::/aaxivvoorg//o
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Tlaus-Robers Müller, and Wojeiech Samek. On pixel-wise explanations for non-linear classifier decisions by layer- wise relevance propagation. PloS one, 10(7)=e0130) 30140, 2015.
Osbert Bastani, Carolyn Kim, and Hamsa Bastani. Interpreting blackbox models via model extraction. arXiv preprint
2017.
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th Annual International Conference on Machine Learning, 2009,
Cristian Buciluă, Rich Caruana, and Alexandru Niculescu-Mizii, Model compression. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2006.
Yu-Hsin Chen, Joel Emer, and Vivienne Sze. Eyeriss: A spatial architecture for energy-efficient dataflow for convolutional neural networks. In 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture, 2016.
Mostafa Dehghani, Arash Mehrjou, Stephan Gouws, Jaap Kamps, and Bernhard Schölkopf. Fidelity- weighted learning. arXiv preprint arXiv: 17179977991099.11 2017.

---

Under review as a conference paper at ICLR 2020
Dua Dheeru and Efi Karra Taniskidou. UCI machine learning repository, 2017. URL http: //archive ics. uci edu/ml
Amit Dhurandhar, Vijay Iyengar, Ronny Luss, and Karthikeyan Shanmugam. Tip: Typifying the interpretability of procedures. arXiv preprint arXiv: 1706. 02052, 2017.
Amit Dhurandhar, Pin- Yu Chen, Ronny Luss, Chun-Chen Tu, Paishun Ting, Karthikeyan Shanmugam, and Payel Das. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. In Advances in Neural Information Processing Systems 31. 201 8a.
Amit Dhurandhar, Karthikeyan Shanmugam, Ronny Luss, and Peder Olsen. Improving simple models with confidence profiles. Advances of Neural Inf. Processing Systems (NeurIPS), 2018b.
Yoav Freund and Robert E. Schapire. Decision-theoretie generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1):119-139 19-139, 1997.
Nicholas Frosst and Geoffrey Hinton. Distilling neural network into soft decision tree. arXiv
preprint arXiv: /7/ 1.09744, 2017.
Tommaso Furlanello, Zachary C Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar. Born again neural networks. arXiv preprint 1rXiv:1805.04770, 2018.
Jeff Dean Geoffrey Hinton, Oriol Vinyals. Distilling the knowledge in neural network. In hutps://arzisorg/ab///588.0. 50000005510 2015.
Chuan Guo, Geoff Pleiss, Sun, and Kilian Weinberger. On calibration of modern neural networks. Indl. Conference on Machine Learning (ICML), 2017.
David Lopez-Paz, Leon Bottou, Bernhard Schölkopf, and Vladimir Vapnik. Unifying distillation and privileged information. International Conference on Learning Representations (ICLR 2016), 2016. URL/tttp://leon wwt.ou. htt.:///.....rpppppp////ppeeeeee
-211
Christoph Molnar, Giuseppe Casalicchio, and Bernd Bischl. Quantifying interpretability of arbitrary machine learning models through functional decomposition. CoRR, 2019. URL https://
arxiv. abs/ /1opss//994.. 1109/994..... 03867
Grégoire Montavon, Wojciech Samek, and Klaus-Robert Müller. Methods for interpreting and understanding deep neural networks. Digital Signal Processing, 2017.
Brandon Reagen, Paul Whatmough, Robert Adolf, Saketh Rama, Hyunkwang Lee, Sae Kyu Lee, Jose Miguel, Hernandez-Lobato, Gu-Yeon Wei, and David Brooks. Minerva: Enabling low-power, highly-accurate deep neural network accelerators. IEEE Explore, 2016.
Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. Learning to reweight examples for robust deep learning. arXiv preprint arXiv: 180. 803,09050, 2018
Marco Ribeiro. Sameer Singh, and Carlos Guestrin. "why should trust you?" explaining the predictions of any classifier. In ACM SIGKDD Inil. Conference on Knowledge Discovery and Data Mining, 2016.
Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. arXiv preprint arXiv: 2015.
Cynthia Rudin. Please stop explaining black box models for high stakes decisions. NIPS Workshop on Critiquing and Correcting Trends in Machine Learning. 2018.
Emma Strubell, Ananya Ganesh, and Andrew McCallum. Energy and policy considerations for deep learning in nlp. CORR, 2019. UtRLpattt:://axxiii org/ abs/ 02243
Sarah Tan, Rich Caruana, Giles Hooker, and Yin Lou. Auditing black-box models using transparent model distillation with side information. CoRR, 2017. URL
1710.06169
Kush R. Varshney, Prashant Khanduri, Pranay Sharma, Shan Zhang, and Pramod K. Varshney. Why interpretability in machine learning? an answer using distributed detection and data fusion theory. ICML. Workshop Human Interpretability in Machine Learning (WHI), 2018.
10

---

Under review as a conference paper at ICLR 2020
A
EXPERIMENTAL DETAILS
Table 3: Dataset characteristics, where denotes dataset size and is the dimensionality.
Dataset
N
d
# of Classes
lonosphere
351
34
Ovarian Cancer
216
4000
Heart Disease
303
13
Waveform
5000
40
Human Activity
10299
561
Musk
65988
166
CIFAR-10
60000
32 x 32
10
Units
Description
Init-conv
3 3 conv, 16
3 3 conv, 64
Resunit:1-0
3 x 3 conv, 64
3 X conv, 64
(Resunit:1-x)x 4
4
3 x conv,
3 x 3 conv, 128
(Resunit:2-0)
3 x conv, 128
3 x 3 conv, 128
(Resunit:2-x)x
x 4
x 3 conv, 128
3 x conv, 256
(Resunit:3-0)
3 conv, 256
x conv, 256
(Resunit:3-x)x
4
3 x 3 conv, 256
Average Pool
Fully Connected 10 logits
Table 4: 18 unit Complex Model with 15 ResNet units.
Table 5: Residual Network Model used as the complex model for CIFAR-10 experiments in Section 4.2
Simple Model IDs
Additional Resunits
Rel. Size
SM-3
None
~ 1/5
SM-5
(Resunit:1xx)>
2 1/3
(Resunit:2-x)x1
SM-7
(Resunit:1-x)
(Resunit:2-x)x4
1/2
(Resunit:-xxxx1
Table 6: Additional Resnet units in the Simple Models apart from the commonly shared ones. The last column shows the approximate size of the simple models relative the complex neural network model in the previous table.
A.1 ADDITIONAL TRAINING DETAILS
CIFAR-10 Experiments
Complex Model Training: We trained with an 6-2 weight decay rate of 0.0002, sgd optimizer with
Nesterov momentum (whose parameter is set to 0.9). 600 epochs and batch size 128. Learning rates according to the following schedule: 0.1 till 40k training steps, 0.01 between 40k-60k:
11

---

Under review as a conference paper at ICLR 2020
Probes
8
Training Set 2
0.298
0.439
0.4955
0.53855
0.5515
0.5632
0.597
0.6173
0.6418
Probes I0
11
12
13
14
15
16
17
18
Training Set 2
0.66104
0.6788
0.70855
0.7614
0.7963
0.82015
0.8259
0.84214
0.845
Table 7: Probes at various units and their accuracies on the training set 2 for the CIFAR-I0 experiment. This is used the ProfWeight algorithm choose the unit above which confidence scores needs to be averaged.
training steps, 0.001 between 60k: the standard schedule followed in the code by the Tensorflow authors We keep the learning rate schedule invariant across all results.
80k training steps and 0.0001 for 80k: training steps. This is
Simple Models Training:
Standard: We train a simple model as on the training set
2. ConfWeight: We weight each sample training set 2 by the confidence score of the last layer of the complex model on the true label. As mentioned before, this is a special case of our method, ProfWeight.
3. Distilled-temp-l:: We train the simple model using cross-entropy loss with soft targets. Soft targets are obtained from the softmax ouputs of the last layer of the complex model (or equivalently the last linear probe) rescaled by temperature
distillation of Geoffrey Hin- ton (2015). By using cross validation, we pick two temperatures that are competitive on the validation set (t 0.5 and 40.5) in terms of validation accuracy for the simple models. We cross-validated over temperatures from the set (0.5, 3, 10.5, 20.5, 30.5, 40.5, 50).
4. ProfWeight (>= f): Implementation of our ProfWeight algorithm where the weight of every sample in training set is set the averaged probe confidence scores of the true label of the probes corresponding to units above the ε-th unit. We set f = 13, 14 and 15. The rationale is that unweighted test scores of all the simple models in Table are all below the probe precision of layer 16 on training set 2 but always above the probe precision at layer 12. The unweighted (i.e. Standard model) test accuracies from Table 2 can checked against the accuracies of different probes on training set 2 given in Table 5 in the supplementary material.
5. SRatio: We average confidence scores from é 13, 14 and 15 done above for ProfWeight and divide by the simple models confidence, In each case, we optimize over B which is increased in steps of 0.5 from 1.5 to 10.
A.2 EXPERIMENTAL RESULTS FOR DISTISL PROXX We provide results for the second variant of Distillation Di Disti Distill-prox in Table g
"Code is taken from:
12

---

Under review as a conference paper at ICLR 2020
Table 8: Below we see the averaged % enrors with 95% confidence intervals for Distil (regression versions of trees and SVM for the simple models that fit soft probabilities from the complex models) on the six real datasets. The results reported using Distill-proxy are in the main paper and are superior to these. Boosted Trees and Random Forest (100 trees) are the complex models (CM), while single decision tree and linear SVM are the simple models (SM).
Complex CM
Simple
SM
Distill-proxy 2
Dataset
Model
Error
Model
Error
Error (SM)
Tree
10.95
10.95
Boosted Trees
8.10
+0.4
+0.4
10.4
SVM
12.38
12.17
10.6
10.3
Ionosphere
Tree
10.95
10.95
Random Forest
6.19
=0.4
#0.4
=0.4
SVM
12.38
12.38
#0.6
+0.6
Tree
15.62
15.62
Boosted Trees
4.68
=0.8
+0.8
=0.4
SVM
1.56
=0.4
+0.4
Ovarian Cancer
Tree
15.62
15.62
Random Forest
6.25
+0.8
+0.8
+0.88
SVM
1.56
1.56
10.4
10.4
Tree
23.88
23.69
Boosted Trees
15.55 10.6
+0.7
+0.2
SVM
17.22
17.01
10.2
10.1
Heart Discase
Tree
23.88
23.88
Random Forest
15.88
=0.7
+0.7
#0.6
SVM
17.22
17.22
=0.2
#0.2
Tree
25.43
25.26
Boosted Trees
12.96 =0.1
=0.2
#0.1
SVM
14.70
15.39
=022
#0.1
Waveform
Tree
25.43
25.43
Random Forest
10.90 +0.1
+0.2
+0.2
SVM
14.70
14.54
10.2
10.0
Tree
7.93
7.93
Boosted Trees
6.32
+0.2
+0.1
10.0
SVM
14.56
16.04
Human Activity
10.1
10.2
Recognition
Tree
7.93
7.45
Random Forest
2.34
=0.2
+0.1
+0.0
SVM
14.56
14.23
=0.1
+0.3
Tree
Boosted Trees
4.06
=0.1
+0.1
=0.1
SVM
6.11
6.34
+0.1
+0.1
Musk
Tree
4.49
4.49
Random Forest
2.45
+0.1
+0.1
+0.1
SVM
6.11
6.19
10.1
10.2
13


Reviews:
[{"review_id": "HkxMc1a65r", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a method for transferring knowledge from a complex, high-performing model to a small/lower-performing one. The approach is based on reweighting the train examples according to the ratio of confidences of the complex and simple models, and then retraining the simple model on the reweighed dataset. \n\nPros:\n* Some interesting ideas motivating the approach\n\nCons:\n* An important error in the main result and various other issues (imprecise statements, lack of details) raise doubts on the theoretical contribution of the paper.\n* Limited novelty. Contribution boils down to importance reweighting via a ratio of probabilities.\n* Limited awareness of related work.\n* Experimental framework lacks meaningful baselines and important details .\n\nBased on the points above, I have to recommend rejection of this paper to ICLR. \n\nDetailed Comments:\n* The claim that per-sample hardness reweighting is novel to this work is a bit of a stretch. Most reweighting-based methods for distillation use some form of this idea, though how each method estimates “hardness” varies, ranging from raw confidence scores of the teacher model (e.g., classic distillation), to scaled versions of it, to a ratio of this and the student’s score (this work).\n* More generally, there’s quite a few very relevant related works which are not cited here (see list below for a non-exhaustive list).\n* I believe the inequality used to prove Lemma 3.2 is not true in general. It’s easy to come up with counterexamples for which w>=1 and x>1 yet log(wx) <= w log(x) does not hold. For example, for w=2, any x in (1,2) will not satisfy this. In general, this inequality is only true if x >= w^{1/(w-1)}. I would be prepared to pass this off as a minor error, were it not for the fact that the w and x in the proof of Lemma 3.2 could quite conceivably be outside this range, i.e., rendering the inequality (and therefore the bound) invalid.\n* There are various other technical aspects of the paper that are either non-rigorous or lack details. For example, Lemma 3.1 requires more information. For which y does w>= hold? Is it just for the true class y*? The proof is equally obscure, and seems to use the same problematic inequality of Lemma 3.2 (for the cross-entropy loss). \n* The statement of Lemma 3.2 is confusing. The last sentence should read loss smaller than *that* of theta*. Also, I don’t see the optimality of theta* being used anywhere. The statement could very well be phrased in terms of any two models, one of which has a lower loss than the other. \n* Assuming the complex “teacher” classifiers are very accurate (which is assumed several times throughout the paper), re-weighting training examples by max(1, p_c / p_theta) does not provide too much information beyond the ground truth labels. This is a core problem in knowledge distillation, already identified as early as Hinton et al (2015), and which is often alleviated by some form of temperature annealing or by operating on pre-softmax logits rather than final probabilities. That doesn’t seem to be the case here, so I’m quite puzzled by the “intuitive justification” provided in this work.\n* The discussion in the first paragraph of pg 4 is a well known dichotomy (namely, up-weighting easy or hard examples) in the distillation/boosting/instance-reweighting literature. This discussion would really benefit from awareness of previous work discussing this duality (e.g., (Bengio et al 2009; Ren et al 2019)\n* Where and how is the ordering of the “graded” classifiers used? In step (3) of Algo 1 all \\zeta’s seem to be weighted equally, so I’m missing why the classifiers need to be delta-graded.\n* I find it quite surprising that in almost half of the datasets in Table 1, distillation yields the exact same performance as just training the Simple Model, and sometimes even degrades the performance. Section 4.1 points out that this is actually “an equivalent model to Distillation”, but no further explanation is given, so it is hard to conjecture what might be the problem. Futhermore, as the authors point out in the introduction, Distillation alla Hinton 2015 usually assumes both teacher and student models are soft predictors (e.g., neural nets), but the models used in the UCI experiments trees and SVMs, so I wonder how exactly they did this. \n* For problems involving distillation into simple interpretable models (like trees), such as the UCI experiments, there are many reasonably strong baselines from previous work that could have been compared against (e.g. Frosst and Hinton 2017), which could have provided a more meaningful evaluation.\n\n\n[1] Learning to Reweight Examples for Robust Deep Learning, Ren et al. ICML 2018\n[2] Distilling a Neural Network Into a Soft Decision Tree, Frosst and Hinton, 2017\n[3] Born-Again Neural Networks, Furlanello et al., ICML 2018\n[4] Fidelity-Weighted Learning, Deghani et al., ICLR 2018", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "HklTsbvv5B", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper is about re-weighting training sets in such a way that simple and interpretable models like trees or small networks can mimic the performance of potentially very complex learning architectures. The main difference to some existing approaches of this kind is the universal applicability to many complex-simple model combinations (and not only layered networks). Technically, this paper is an extension and formalization of some ideas proposed in (Dhurandhar 2018). The main methodological contribution is the formal justification of the proposed weighting scheme for samples which utilizes the ratio of conditional probabilities p_complex(y|x)/p_simple(y|x) in the Lemma 3.1 and 3.2. The practical algorithm described in the paper is based on the concept of delta-graded classifiers which basically defines a nested set of classifiers of increasing accuracy -- such as, for instance, simple classifiers trained on intermediate layers of a deep network. The second important input argument is the learning algorithm for the simple model, which is trained on the re-weighted samples. The motivation for this algorithmic procedure is somewhat \"hand-waving\", but rather intuitive. Experiments for different benchmark datasets and different complex-simple model combinations nicely demonstrate that this method is indeed quite useful in practice. In summary, I think that this work addresses a highly relevant problem and provides a relatively simple method that might be very useful in many applications. A potential weakness could be the gap between the formal treatment of the cross-entropy loss in Lemma 3.2. and the rather \"ad hoc\" practical algorithm that crucially depends on two tuning parameters. Nevertheless, in my opinion, this work provides some interesting ideas that have the potential to trigger future research in this area.        ", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "HklCMyvFcS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a simple yet effective trick to reweigh the samples fed into a simple model to improve the performance. The new weights are functions of both the simple model and another complex (and strong) model. Some theoretical and intuitive explanations are provided to support the main claim of the paper. \nThe main weakness of the paper is its presentation and this is apparent from the abstract of the paper and afterwards. Referring to a prior work (Dhuranhar, 2018b) in the abstract which might be known to the reader is not a good idea. The connection of lemma 3.1 to the rest of the paper is not clear. Similarly, it's not readily clear how lemma 3.2 leads to algorithm 1. The introduction of the graded classifiers was very sudden. It's difficult to pinpoint how we replaced the complex model with the graded classifiers. To me, the graded classifiers were and incremental work and make the main point of paper (section 3.1 and 3.2) could have been emphasized better. \nWhat would be the effect of changing the number of graded classifiers? Some theoretical explanations or empirical results could be beneficial. \nThe reweighting scheme looks very related to importance sampling ratios in off policy evaluation in reinforcement learning and counterfactual analysis. Connecting the dots could be interesting. ", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "H1eBe2V19H", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a means of improving the predictions of a \"simple\" (low-capacity) model. The idea is to take the predictions of a \"complex\" (high-capacity) model, and weight the loss in the simple model based on the ratio of complex to simple models' predictions. Intuitively, this seeks to focus on instances which the complex model can fit, but the unweighted simple model cannot. Experiments show the proposed method to have some benefits over existing approaches.\n\nThe application of importance-weighting to the \"model distillation\" problem is interesting, and the paper gives a reasonable intuition for why this approach might work. One general comment is that in contrasting their approach to a number of existing approaches, they note that several of them are typically employed with fairly complex simple models (e.g., neural networks). This may be true, but it was not clear that any of them require this to be the case. Surely they can also be used with simple models as ones you consider in the paper? In this case, I would've liked more elucidation as to why the proposed method can be expected to offer superior performance.\n\nThe theoretical justification of the approach is provided by means of Lemmas 3.1 and 3.2, for which I have some comments:\n- Lemma 3.1: the notation here is a bit imprecise. In general, a loss for example (x, y) takes in a true label y and predicted score z(x). You refer to the loss of a probabilistic prediction p(y | x), but do not refer to the actual label y itself. From the proof, it is implicit that you are considering y to be binary, and the use of a margin loss. This is ok, but for the hinge loss one doesn't use a probability estimate p(y | x) as input to the loss, but rather, a real-valued score.\n\nI think the result itself could be proven by noting that if φ is non-increasing and q < p, then φ(p)/φ(q) <= 1 < p/q.\n\n- Lemma 3.2: the result is interesting, but it seems that for practical purposes you are only using the first term, since it is the only quantity that depends on θ. Since max(1, .) >= 1 and -log pθ(y | x) >= 0, it seems one could trivially bound the LHS by the first term since -log pθ(y | x) <= max(1, .) * -log pθ(y | x)? Would this not suffice for the purposes of justifying your method? It should also be noted here how the subsequent requirement that the weights be capped (so as to prevent outliers) fits into the analysis.\n\nIn describing the method itself, the authors introduce a notion of δ-graded subsets. I found this to be a bit difficult to parse, and it was not clear why this notion was needed. It does not seem to feature in the description of Algorithm 1, nor the subsequent discussion. On the other hand, I felt that the meaning and need for parameters β and γ ought to have been discussed more prominently upfront.\n\nThe experiments show favourable performance of the proposed method over baselines, including the distillation approach of Hinton. The datasets are mostly small-scale, but this is in keeping with the goal of the paper, viz. addressing scenarios where simple models may be desired. Per earlier comments, I did not have a deep sense of what additional information the proposed method exploits so as to improve performance. I gather that the weighting including the predictions of the simple model is one difference; it might have been nice to give a sense of what fraction of points this amplifies or suppresses, compared to just using the predictions of the complex model.\n\nThere is a nice illustration in Fig 1 (right) as to the class-labels of the training samples assigned various weights. This shows that points with low weight tend to have low agreement with their neighbours' labels. One question is how using a nearest neighbour probability estimate itself (i.e., using this as the \"complex\" model) would fare.\n\nMinor comments:\n- the title is a bit confusing. It is not clear what \"its\" refers to. You are leveraging the predictions of a complex model to enhance those of a simple model?\n- the text has a number of long sentences that could benefit from rewriting or splitting.\n- proof of Lemma 3.2, use \\cdot not *.\n- proof of Lemma 3.2, θ* should use superscript.\n- Fig 1, the caption is overlong. Most of this should be in the text.\n- Fig 1, use crisper fonts for the text.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors propose a sample reweighting scheme that helps to learn a simple model with similar performance as a more complex one. The authors contained critical errors in their original submission and the paper seems to lack in terms of originality and novelty of the proposed method.
Decision: Reject