Towards Disentangling Information Paths with Coded ResNeXt
Anonymous Author(s)
AffiliationAddress
Abstract
The conventional, widely used treatment of deep learning models as black boxesprovides limited or no insights into the mechanisms that guide the neural networkdecisions.
Significant research effort has been dedicated to building interpretablemodels to address this issue.
Most efforts either focus on the high-level featuresassociated with the last layers, or attempt to interpret the output of a single layer.
In this paper, we take a novel approach to enhance the transparency of the functionof the whole network.
We propose a neural network architecture for classificationin which the information that is relevant to each class flows through specific paths.
These paths are designed in advance before training leveraging coding theory.
Moreover, the paths do not depend on the semantic similarities between classes.
A key property is that each path can be used as an autonomous single-purposemodel.
This enables to obtain, without any additional training and for any class,a lightweight binary classifier that has at least 60% fewer parameters than theoriginal network.
Furthermore, our coding theory based approach allows the neuralnetwork to make early predictions at intermediate layers during inference, withoutrequiring its full evaluation.
Remarkably, the proposed architecture provides allthe aforementioned properties while significantly improving the overall accuracy.
We demonstrate these properties on a slightly modified ResNeXt model tested on
CIFAR-10/100 and ImageNet-1k.
Introduction
Most successful deep learning architectures for image classification consist of a certain building blockapplied sequentially several times: one block follows another until a linear operation finally outputsthe model prediction.
In deep convolutional neural networks (CNNs), the block consists of multipleconvolutional operations [33, 34] applied sequentially.
Nonetheless, there are numerous proposalsplacing the convolutional layers in parallel, forming multi-branch designs.
For instance, inceptionmodels [65, 66] use blocks with multiple branches, each applying some convolutional operationson the block’s input and finally concatenating at the end the output of all branches.
The multibranch design framework can also accommodate skip connections [19], as initially done in ResNeXtnetworks [75], and later refined using squeeze-excitation in [23], or a split-attention mechanism in[79].
A first question we ask in this work is: what is the purpose of multi-branch architectures?
Initially, in AlexNet [30] two branches were employed to allow the distribution of the model acrosstwo GPUs, which at that time had limited memory.
Nowadays, multi-branch architectures arecommonly used for distributing the parameters of a block into branches such that each one appliesa separate transformation to the input.
However, rare are the cases where each branch is shown tocontribute in a different way.
One example is SKNet [36], in which each branch is associated withdifferent receptive field size and zooming in or out of an input image activates the appropriate branch.
Nonetheless, the value of multi-branch architectures is mostly justified by achieving a higher accuracy of the whole network.
Multi-branched blocks are also used for network architecture search, wherethe block/cell architecture is optimized selecting the number of branches, the operation that eachperforms, and how they are combined [38, 51, 53, 81, 83].
Nevertheless, the focus therein was onimproving accuracy.
In this work, we investigate how to ensure that each branch provably contributes in a different wayin a multi-branch architecture.
We propose a novel way to organize in a class-wise manner thetransformations carried out by the branches.
Leveraging concepts from coding theory, we designthe assignment of each branch to a specific set of classes without having to rely on the semantics ofeach class. Specifically, for each block in the network, a binary “codeword” of length equal to thenumber of branches of the block is matched to each class, indicating which branches will activate forthat class.
That way, keeping only the network’s branches assigned to a class, it is possible to forma unique path for that class, which traverses the whole network and through which the informationrelated to that class flows.
To showcase the unique features and the advantages of our idea, we usethe state-of-the-art multi-branch architecture ResNeXt [75] to which we add an architectural tweak.
Our main contributions can be summarized as follows:
• We modify the ResNeXt architecture so that it functions in a more transparent way by forcing the information related to each class to flow through well-defined network paths.
• As a proof of concept, we show that in order to form those paths, it is not necessary to rely on thesemantics of each class and the similarity between classes.
• Without any additional training, we can obtain a single-purpose model per class operating as a binary classifier that has 60% fewer parameters as compared to the complete network.•
We demonstrate that the intermediate layers can be used both for making early predictions and for providing a confidence level for correctness of the network’s final prediction.
• The proposed Coded ResNeXt significantly improves ResNeXt accuracy across all tested datasets.
Related Work
Numerous attempts have tried to understand how complex deep neural networks actually work.
Forinstance, activation maximization is used for finding the input that increases the activation of a neuron[11, 49, 47, 76].
Saliency maps [60, 59, 61, 63, 50, 54] find the pixels that have the largest influenceon the model prediction.
The pitfall of such approaches is that they cannot really explain how thenetwork functions and they mainly serve as post hoc visualization methods [55].
There are manyinterpretable by design proposals [70, 80, 6, 45, 28]; however, most of them concentrate on enhancinginterpretability only in the last layers of the neural network. In [70], the final linear layer is replacedwith a differentiable decision tree, and in [80] a loss is used to make each filter of the very high-levelconvolutional layer represent a specific object part. In [6], the model’s output is compared with learntprototypes, whereas in [28] it represents concepts on which humans can intervene.
Another direction for enhancing interpretability is through disentanglement.
Although there is notyet a generally accepted definition, disentanglement aims at separating out the main factors thatare present in the data distribution [4, 21, 40].
Still, the efforts concentrate on disentangling thefactors at a single vector/tensor (the latent representation), which is either the input or the output ofa network.
This is common across many proposed architectures such as (variational) autoencoders[22, 5, 7, 27, 12], generative adversarial networks [8, 42, 26], normalizing flows [13, 58], or evenarchitectures that aim to decompose the content to the style representation of an image [16, 14].
Incontrast to prior art, our work approaches disentanglement as “the way information travels through thenetwork” [56] and looks at the neural network architectures for classification in a more general way.
Specifically, our goal is to control the paths through which information flows throughout the networkby assigning each part of the network to specific subset of classes.
We only allow a small fraction( 6% ) of the parameters at the beginning of the network to be shared across all classes.
Related to ourwork is [72], in which they interpret a deep neural network by identifying such information paths;one key difference is that they use a post hoc method so the paths are identified and not designed.
Finally, there is a stream of work proposing to dynamically control the path that a sample followsthrough the network [35, 9, 2, 15, 37, 71, 68, 74, 39]. An extra network, named gated network orcontroller, is usually trained to choose which parts (blocks or filters) to be pruned/omitted.
Duringinference this brings memory and speedup gains since the samples do not pass through the entirenetwork.
Despite being related to our work, there are some key differences.
First, we do not use any
[Figure]
extra network that has to be trained to learn good paths. Second, in our work, the paths are defined inadvance, prior to training (and thus not learnt).
Most importantly, these works provide no guaranteethat two samples with same semantics (for example of the same class) follow the same path, so it isnot possible to extract any of those paths and use them as lightweight single purposed models.
Coded ResNeXt3.1 The block
The typical ResNeXt block [75] is depicted in Fig. 1(a).
It takes input x ∈ R C × H × W
( C is thenumber of input channels and H, W are the height and width of the input planes, respectively)and outputs y of the same dimensions.
It consists of N paths/branches ( N is called cardinality in[75]).
Each branch, which is called sub-neural network (subNN) here, performs transformation
T n , n ∈ { 1 , · · · N } which is aggregated together with the input x , giving the block’s output y :
[Equation]
3.1.1 Energy Normalization
For the Coded ResNeXt block depicted in Fig. 1(b), the sole architectural change we introduce is the
Energy Normalization applied just before aggregating the transformed inputs T n ( x ) ∈ R C × H × W .
For convenience, let t n :=
T n ( x ) .
If ( t n ) c,h,w ∈ R is the element of t n in position ( c, h, w ) , then wedefine function E as:
[Equation]
which gives the mean energy of the output signal of the n -th subNN.
Energy Normalization simplydivides the outputs of all branches by a scalar value equal to the square root of the total mean energy:¯ t n = t n (cid:112)
E avg with E avg =
N
N (cid:88) i =
E ( t i ) , ∀ n ∈ { 1 , · · · , N } .
Given that E ( ax ) = a 2 E ( x ) for scalar a ∈ R ≥ 0 , it is easy to see that this step normalizes the totalenergy, since after it the sum of the energy of all subNNs becomes (cid:80) Nn =1 E ( ¯ t n ) = N .
3.1.2 Coding Loss
We present here our first algorithmic addition.
After the Energy Normalization, we compute a novelloss function, coined coding loss L code .
Consider a classification problem of K classes.
Let l be theindex of the position of a ResNeXt block within the network.
As seen in Fig.
1(c), for that block, weassign to each class a binary codeword w l,k , k ∈ { 1 , · · · , K } of length N , indicating which subNNswe want to activate for that class.
If the n -th subNN operates for class k , then the n -th digit of w l,k is( w l,k ) n = 1 , and ( w l,k ) n = 0 otherwise.
To ensure that each class receives the same number N act,lof operating subNNs, all K codewords are designed with exactly N act,l ones.
We define the ratio
[Equation]
(4) which measures how much each class uses the block’s total computational resources.
We term themapping of the classes to codewords, as in Fig.
1(c), the coding scheme of the block.
Given an input of class k , the coding loss forces the mean energies of the subNNs inactive for class kto zero and those of the active subNNs to positive values.
The coding loss for the l -th block is
[Equation]
Note that after the Energy Normalization, the total subNNs mean energy is (cid:80) Nn =1 E ( ¯ t n ) =
N , whilethe codeword has N act,l = r l N ones, hence we multiply E (¯ t l ) by r l .
We remark that the choice ofsetting the exponent to 4 is carefully made.
For example, setting it to 2 , the accuracy for CIFAR-10drops from 94 .
4% to 93 .
1% , which further drops to 87 .
1% if the absolute value is used.
The exponentof 2 is much more demanding than the one of 4 on matching precisely the output energies to the rulesof the coding scheme.
The exponent of 2 seems to considerably restrict the flexibility of the functionof the subNNs, thus degrading the overall performance.
This trend is exacerbated with using theabsolute value.
3.1.3 DropSubNNs
The second algorithmic addition is a type of dropout [62], similar to techniques such as SpatialDropout[67], StochasticDepth [24], and DropPath [32].
Seeing each subNN as one more complicated neuron,we apply dropout to it, so its output is zeroed with a fixed probability p drop .
This method is coined as
DropSubNNs .
Our aim is to reduce the “co-adaptation” effect [62] on the subNN level, according towhich subNNs collaborate in groups instead of trying to independently produce useful features.
Inour implementation, we apply the same random mask to all blocks that have the same coding scheme.
3.2 The network
The complete network is constructed as a sequence of blocks.
The Energy Normalization, L code,l ,and dropSubNNs are applied only to blocks whose subNNs we want to specialize in some subsets ofclasses.
Thus, for blocks with r l = N/N = 1 , we use the conventional ResNeXt block as in Fig. 1(a).
In that sense, the ResNeXt model is a Coded ResNeXt model where all blocks have r l = N/N .
3.2.1 Coding Scheme Construction
We construct one coding scheme per ratio r l so that a coding scheme is uniquely characterized bythe ratio r l and any two blocks l, l ′ with r l = r l ′ have exactly the same coding scheme.
A generalrule we follow is that the deeper in the network a block is (i.e., the larger l is), the smaller the r lbeing assigned.
The first blocks have r l = N/N so that their subNNs produce low-level features,potentially useful for recognizing any of the classes.
Deeper blocks have smaller r l so that theirsubNNs specialize on a subset of classes.
This rule is not only intuitive but works better in practice.
[Table] instance changing
The primary goal of this work is to make a proof of concept that it is possible to specialize subNNs tosubsets of classes defined before training, even in the case when the classes within those subsets maynot be semantically related . Although, for a given r l , the coding scheme is designed in an agnosticway with respect to the nature of the classes, to ensure good performance we still have to design itsmartly.
For example, as stated in Section 3.1.2, we want to ensure that each class receives the samenumber of dedicated parameters and therefore of operating subNNs. This leads to the first rule:
A.
The number of “1”s must be equal to N act,l = r l N with N being the codeword length.
Moreover, we want to avoid under- or over-utilizing any subNN, in the sense of assigning too few ortoo many classes for it to process.
As a result, the second rule is:
B. Seeing the coding scheme as a binary table, as in Fig.
1(c), the sum of each column shouldbe approximately the same.
Finally, we aim at making the set of subNNs dedicated to work for a class to be as different as possiblefrom sets assigned to the rest of the classes.
This translates to:
C. The Hamming distance of any pair of codewords should be as high as possible.
Given r l and N , many coding schemes that follow the above rules may exist.
We have tested that anyof those schemes give experimentally the same results.
However, finding any of those schemes is achallenging NP-Hard problem.
We present in Appendix A our algorithm for generating the codingschemes used here.
3.2.2 Architecture and Total Loss
We succinctly describe a Coded ResNeXt block as [ C out , d, r l ] , with C out being the number ofchannels the block outputs and d being the bottleneck width as in ResNeXt [75].
A conventional
ResNeXt block is expressed as [ C out , d, N/N ] .
Table 1 presents the networks trained for CIFAR-10(C10), CIFAR-100 (C100) [29], and ImageNet 2012 [57] classification datasets.
In CIFAR-10/100we chose N to be small yet sufficiently high to enable reducing r l to less than 0 .
25 and still obtaininga coding scheme with minimum Hamming distance not less than 4.
For ImageNet we used the defaultvalues of ResNeXt-50.
Remarkably, even though the number of classes increases exponentially acrossdatasets ( K ∈ { 10 , 100 , 1000 } ), a strong coding scheme can be found to efficiently share the subNNsbetween classes, so that (a) random pairs of classes are assigned to very different subsets of subNNs;and (b) only a linear increase of the number of subNNs ( N ∈ { 10 , 20 , 32 } ) is needed.
Let L class be the conventional cross entropy loss and B code the set of indices l pointing to the blockswith ratio r l < 1 .
The total loss used in order to train the network is
[Equation]
with µ being a constant balancing the two losses.
Experiments
In this section, we present various experimental results to assess the performance of the proposed
Coded ResNeXt.
First, we show that our algorithm achieves subNN specialization.
To demonstratethis we show that when the subNNs specialized on the class of interest are removed, the performancedegrades, whereas it remains the same or even improves when the subNNs removed are not specializedfor that class.
To further prove the specialization, given a class, we keep only the subNNs assigned tothat class.
That way, we retrieve a lightweight single-purpose binary classifier accurately decidingwhether the input sample belongs or not to the class.
Finally, we show that it is possible to get goodpredictions from intermediate blocks without evaluating the whole network.
Those predictions canalso be used to provide confidence on whether the final network’s prediction is correct.
4.1 Setup and Validation Accuracy
( µ, p drop )
CodedResNeXt
ResNeXt
CIFAR- (6, [Table] [Table]
[Table]
CIFAR- (6, [Table] [Table] [Table]
ImageNet (
[Table]
[Table]
In order to make a fair comparison with
ResNeXt, on ImageNet [57] we followthe training process proposed by timm library [1].
The epochs are 250 (first 5 aswarmup [17] and last 10 cooling down),the batch size is 1536 , and the learningrate 0 . 6 .
RandAugment [10] of 2 layersand magnitude 7 (varied with a standarddeviation of 0 . 5 ) is used and also randomerasing augmentation [82] with probability 0 .
4 and 3 recounts.
We diverge fromthe timm’s proposed process only on the resolution of the input training images.
We reduce theresolution to 160 (instead of 224 ) because on the TPU-v2 of Google COLAB, which is the platformused for all our experiments, the training would require 1 month.
Nonetheless, timm reports 79 .
77%accuracy for the ResNeXt-50, which is clearly smaller than the one of Coded ResNeXt-50 eventhough it is trained with reduced resolution.
In Appendix B we describe the training process for CIFAR and give additional details.
In Table 2 we compare the accuracy of Coded ResNeXt against the corresponding ResNeXt (i.e.,when setting all ratios r l equal to 1).
We observe a clear improvement of the accuracy across alldatasets.
Surprisingly, forcing the subNNs to specialize to specific set of classes yields significantgains even if the assignment of classes to subNNs is done in a way agnostic to the semantics of theclasses.
Table 2 presents the default values used for the introduced hyperparameters ( µ, p drop ) and theachieved validation accuracy.
In Appendix D we perform an ablation study on those hyperparameters.
4.2 Specialization
A key idea of our work is to specialize each subNN to specific subset of classes; hence the firstexperiment is designed to test whether our proposed architecture actually succeeds in achievingspecialization.
Assuming a subNN is assigned to actively work for some class, then if this subNNhelps indeed on the classification process of images belonging to that class, removing this activesubNN should negatively impact this process.
On the other hand, if that subNN is not assigned tothat class, then it should remain inactive during the process so removing it should have no impact(degradation) on the performance.
For the first experiment we pick a block l from which we randomly remove subNNs 2 in two ways.
Given the class of the input image sampled from the validation set, the first way randomly removesk ≤ N act,l subNNs from the set of active for that class subNNs.
The second way randomly removesk ≤ N − N act,l subNNs from the (complementary) set of inactive subNNs for that class.
For illustration,in Fig.
2a we pick the last block of stage c3 in the architecture for CIFAR (see Table 1) and the secondof stage c5 for that for ImageNet.
Figures with respect to other blocks are presented in Appendix F. [Figure]
In Fig.
2a, we observe the same behavior across all datasets, which confirms that the more activesubNNs are removed, the more the performance degrades.
Interestingly, when removing inactiveones, the accuracy tends to increase.
Our interpretation is that even though the inactive subNNsare trained to output zero signal, this is never perfectly achieved in practice and their output alwaysinterferes with that of the active subNNs.
Thus, taking out the interferers could improve the accuracy.
Note that this higher accuracy of the neural network is not actually achievable since to remove asubNN we need to know a priori the class of the input so as to know the set of (in)active subNNsfor that class.
Finally, we remark that even if all active subNNs are removed from one block, theperformance does not necessarily plummet.
We believe that the reason behind this is that, in that case,information can still pass from the previous block to the next one through the skip connection.
4.3 Binary Classifier
Having confirmed that the subNNs specialize on their assigned subset of classes, we proceed withtesting this property to the extreme.
For that, instead of randomly removing few subNNs from oneblock, given a class k , we remove from all blocks all subNNs not assigned to class k .
The rationalebehind this is to check whether by keeping only the subNNs specialized on one class we can obtain abinary classifier (BC) capable of recognizing that class among the others.
tench) of each dataset (CIFAR-10, CIFAR-100, ImageNet, respectively).
In Fig.
3 we pick the first class of CIFAR-10/100 and ImageNet (“airplane”, “apple”, and “tench”,respectively) and remove all inactive subNNs for that class.
We also remove from the final linear layereverything except its first row of parameters so as to keep only the first logit corresponding to the firstclass.
That way, we retrieve a sub-model whose output is one dimensional.
Figure 3 depicts withblue the output distribution when inputting samples of the validation set belonging to the first classof the dataset (i.e., in-distribution positives), and with red when the samples belong to some otherclass of the dataset (i.e., in-distribution negatives).
Evidently, the extracted sub-models do operate as BCs giving high output when fed with samples of the class for which they are specialized.
To furthershowcase the specialization, for the sub-models trained on CIFAR-10 (resp.
CIFAR-100) we inputsamples that belong to the validation set of CIFAR-100 (resp.
CIFAR-10).
Those are considered outof-distribution predictions, since the sub-model has never been trained on such samples.
Nevertheless,as Fig.
3 shows, the extracted BC model continues to perform well.
Therefore, with a single training of a large multi-purpose neural network, we can straightforwardlyextract multiple single-purpose models that are considerably lighter ( 38% , 27% , and 35% of theinitial parameters for CIFAR-10, CIFAR-100, and ImageNet architectures, respectively) .
Given athreshold distinguishing between positive and negative predictions, each of those models becomes a
BC.
We set that threshold to the value maximizing the F1-score of the BC when fed with samplesfrom the training dataset.
In Figs.
2b and 2c 3 the performance of the BCs (on the validation setand the out-of-distribution set) is depicted in precision-recall plots.
Notably, for CIFAR-10, theworst performance is obtained by the BC for “cats” when fed with CIFAR-100’s out-of-distributionsamples.
This seems reasonable, since we request from the classifier to distinguish cats from classeslike leopard, lion, and tiger, but without having “seen” any sample of them during training.
It is known that in ResNet, complete blocks can be removed without severely degrading the accuracy[69].
Hence, it is reasonable to ask whether the conventional ResNeXt is also robust to the removalof subNNs and whether good BCs can be extracted from it without the need for our proposedmodifications.
Interestingly, this is not the case and the answer is negative.
In CIFAR-10 for instance,the extracted BCs from the Coded ResNeXt give on average precision 93% and recall 77% (indistribution).
Attempting to extract likewise BCs from a ResNeXt leads to precision 13% and recall56% .
Finally, given a class, the complete ResNeXt architecture can be seen as a BC by consideringits output to be only the corresponding logit.
Comparing such BCs to the extracted BCs seems unfairsince the extracted BCs not only have 2 .
5 times fewer parameters but also have never been trained asindependent models.
Nonetheless, it may serve as a baseline.
For CIFAR-10 this baseline gives BCswith average precision 79% and recall 94% .
Additional details and plots are provided in Appendix F.
4.3.1 Why ResNeXt?
In this subsection, we provide insights on why the subNNs achieve specialization and we highlightwhy ResNeXt serves as the appropriate architecture upon which to build our idea.
The objective ofour work is to construct networks in which the per-class information is forced to flow through specificpaths (determined here by the coding schemes).
To achieve this, we employ (i) an operation (energynormalization) that limits how many subNNs can be activated; and (ii) a loss function forcing whichones should be activated.
Intuitively, those operations should suffice for constraining the informationto flow through the active subNNs.
A natural question that arises is how accurate this is.
Let usassume that it is accurate.
Then, keeping those operations unaltered and changing only the way thesubNNs’ outputs are passed to the subsequent blocks should not impact the flow of information.
However, if instead of aggregating them by summation , they are concatenated , the performanceof the extracted BCs becomes poor (precision < 20% ).
It seems that the concatenation inhibitsthe “information” to pass only through the designated paths, since the performance degrades wheninactive subNNs (which in theory should not participate in those paths) are removed.
Let us see why.
When concatenating the outputs of the l -th block, the information about which are the inactivesubNNs is preserved, thus the l +1 -th block may depend its function on which subNNs of the l -thblock provide zero output.
This allows information to “leak” from the inactive subNNs.
On thecontrary, the information that some subNNs provide zero output is lost when adding them to thefinal output of the block.
For that reason, the ResNeXt architecture (which aggregates the outputs bysummation) is very well suited for developing our idea of controlling the information paths CIFARCIFAR- ImageNet
[Figure]
[Table] 4.4 Early Decoding Coded ResNeXt improves accuracy over ResNeXt while enabling the extraction of multiple lightersingle-purpose models with a single training and providing transparency on how information flowsthroughout the network.
Here we show that leveraging coding theory to design when and whichsubNNs should activate allows exploiting Coded ResNeXt in other ways.
Given block l with r l < 1 , i.e., l ∈ B code , the coding scheme maps each class k ∈ { 1 , · · · , K }in a one-to-one fashion to a codeword w l,k and then the training pushes the energies v l ∈ R N ≥
ofthe block’s subNNs output to match that codeword.
This allows for each l ∈ B code to measure thevector v l ∈ R N ≥ 0 , find the codeword w l,k , k ∈ { 1 , · · · , K } having the minimum distance to v l , andconsequently predict the class of the sample.
As a result, each block l ∈ B code becomes an earlydecoder predicting label: [Equation] with || · || 2 being the L2 norm.
This early decoding is enabled and reinforced by the rule C used tocreate the coding schemes (see Section 3.2.1).
That rule forces the codewords to be as far apart fromeach other as possible.
The farther apart the codewords are, the more subNNs should incorrectly beactive or inactive for the early decoder to predict a wrong label.
In Fig.
4 we depict the accuracy ofevery block l ∈ B code when functioning as an early decoder.
Interestingly, as a sample passes fromone block to the next one, the probability of being correctly decoded increases.
In Appendix D weshow that as the coefficient µ of L code,l increases, the early decoders improve, but past a certain pointthis comes at the expense of the overall accuracy.
Finally, we illustrate another possible utility of the early decoders.
In Table 3 we measure the accuracyof the network’s final prediction given how many early decoders also provide the same prediction. It is obvious that the more decoders agree with the final prediction, the higher the probability to becorrect. Therefore, early decoders provide a confidence estimation on the correctness of the network’sprediction. Specifically, they can be a source of extra features used to improve state-of-the-artconfidence calibration methods [18].
Due to space limitation, this is further analyzed in Appendix C.
Conclusion
In this paper, we proposed a network architecture in which the information related to each class flowsthrough distinctive and clearly defined paths.
We depart from the ResNeXt architecture and applyfew yet crucial modifications that attain both higher accuracy and have several additional attractiveproperties.
First, we specialize each part of the model on a specific subset of classes, which –given acertain class– enables to obtain a binary classifier for that class by keeping only the relevant parts ofthe network.
Second, it allows obtaining early predictions without the need for entirely evaluatingthe network.
Third, if fully evaluated, a confidence level can be produced on the correctness ofthe final prediction.
While in this work we achieved specialization without having to rely on thesemantic similarities between classes, we conjecture that further gains (higher accuracy, lighter binaryclassifiers, smaller network, etc.) can be obtained by exploiting those similarities.
Checklist
For all authors...
(a Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] The datasets are publicly available and the code is included in the supplementary material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes]
(c) Did you report error bars (e
g., with respect to the random seed after running experi-
ments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
[-K do 21: score ← S CORE ( C ) 22: if score = 
-]
[-(3 , 4) for CIFAR-
-]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]