Class Interference of Deep Networks
Authors: Dongcui Diao
Hengshuai Yao
Bei Jiang
Venue: Submitted to ICLR 2023 (2023)
Keywords: Minima sharpness
generalization
loss contour
visualization

Abstract: Recognizing and telling  similar objects apart is even hard for human beings. In this paper, we show that there is a phenomenon of class interference with all deep neural networks. Class interference represents the learning difficulty in data and it constitutes the largest percentage of generalization errors by deep networks. To understand class interference, we propose cross-class tests, class ego directions and interference models. We show how to use these definitions to study minima flatness and class interference of a trained model. We also show how to detect class interference during training through label dancing pattern and class dancing notes. 

Under review as a conference paper at ICLR 2023
CLASS INTERFERENCE OF DEEP NEURAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
Recognizing and telling similar objects apart is even hard for human beings. In this paper, we show that there is a phenomenon of class interference with all deep neural networks. Class interference represents the learning difficulty in data and it constitutes the largest percentage of generalization errors by deep networks. To understand class interference, we propose cross-clas, tests, class ego directions and interference models. We show how to use these definitions to study minima flatness and class interference of a trained model. We also show how to detect class interference during training through label dancing pattern and class dancing notes.
INTRODUCTION
Deep neural networks are very successful for classification (LeCun et al. 2015 Goodfellow al. 2016) and sequential decision making (Mnih et al.) 2015 Silver et al. 2016). However, there lacks good understanding of why they work well and where is the bottleneck. For example, it is well known that larger learning rates and smaller batch sizes can train models that generalize better. Keskar et al. (2016) found that large batch sizes lead models that look sharp around the min- ima. According to Hochreiter & Schmidhuber (1997), flat minima generalize better because of the minimum-dessrrition-leeee length principle: low-complexity networks generalize well in practice.
However, some works have different opinions about this matter (Kawaguchi et al. 2017: Dinh et al., 2017 Li et al. 2018). Dinh al.] (2017) showed that sharp minima can also generalize well and a flat minimum can always be constructed from sharp one by exploiting inherent geometric sym- metry for ReLUU based deep nets. Li al. (2018) presented an experiment in which small batch minimizer is considerably sharper but it still generalizes better than large batch minimizer by turn- ing on weight decay. Large batch training with good generalization also exists in literature (De et al.] 2017 Goyal al. 2017). By adjusting the number of iterations, |Hoffer et al. (2017) showed there is no generalization gap between small batch and large batch training.
These works greatly helped understand the generalization of deep networks better. However, it still remains largely mythical. In this paper, we show there is an important phenomenon of deep neural networks, in which certain classes pose a great challenge for classifiers tell them apart at test time, causing class interference.
Popular methods of understanding the generalization of deep neural networks based on minima flatness, usually by visualizing the loss using the interpolation between two models [Goodfellow
al.
2015: Keskar al. 2016, Im et al. 2016, Jastrzebski et al.] 2017 Draxler et al. 2018] Li
et al. 2018 Lucas et al. 2021; V laar & Frankle, 2022; Doknic & Möller 2022). Just plotting the
losses during training is not enough to understand generalization. Linearly interpolating between
the initial model and the final trained model provides more information the minima.
basic finding in this regard is the monotonic property: as the interpolation approaches the final model, loss decreases monotonically (Goodfellow et al.) (015)). Lucas et al. (202T) gave a deeper study of the monotonic property on the sufficient conditions as well as counter-examples where does not hold. Vlaar & Frankle (2022) showed that certain hidden layers initial model, and the shape of the linear path is not indicative of the generalization performance of the final model. (Li et al.; 2018) explored visualizing using two random directions and showed that it is important to normalize the filter. However, taking random directions produces stochastic loss contours. It is problematic when compare models. We take deterministic approach and
more sensitive to the

---

Under review as a conference paper at ICLR 2023
study the loss function in the space of class ego directions, following which parameter update can minimize the training loss for individual classes.
The contributions of this paper are as follows.
Using metric called CCTM that evaluates class interference on test set, we show that class interference the major source of generalization error for deep network classifiers. We show that class interference has symmetry pattern. In particular, deep models have similar amount of trouble telling "class A objects are not class B", and "B objects are not not A"
To understand class interference, we introduce the definitions of class ego directions and interference models.
In the class ego spaces, small learning rates can lead to extremely sharp minima, while learning rate annealing leads minima that are located at large lowlands, in terrains that are much bigger than the flat minima previously discovered for big learning rates.
The loss shapes in class ego spaces are indicative of interference. Classes that share similar loss shapes in other class ego spaces likely to interfere.
We show that class interference can also be observed in training. In particular, it can be detected from a special pattern called label dancing, which can be further understood better by plotting the dancing notes during training. Dancing notes show interesting interference between classes. For example, surprise is that we found FROG interferes CAT for good reasons in the CIFAR-10 data set.
CLASS INTERFERENCE
2.1
GENERALIZATION TESTS AND THE CLASS INTERFERENCE PHENOMENON
Let eι and 02 be class labels. We use the following cross-class test of generalization, which is the percentage of c2 predictions for the e₁ objects in the test set:
# predicting as C2 CCTM()),0)) 02) = #total C1 objects
Note this test being an accuracy or error metric depends on whether the two classes are the same or not. Calculating the measure for all pairs of classes over the test set gives matrix. We refer to this measure the CCT matrix, and simply the CCTM for short. CCTM extends the confusion matrix in literature by probability measure, which can be viewed combination of the true positive rates and false positive rates in a matrix format This extension facilitates a visualization of the generalization performance as heat map.
Figure shows the CCTM for VGG19 (Simonyan & Zisserman, 2015) and ResNet18 (He et al, 2015) the CIFAR-10 10 (Krizhevsky al.) 2009) test set with heat map. Models were trained with SGD (see Section for the training details). From the map, we can see that the most significant generalization errors from CAT and DOG for both models. This difficulty is not specific to models. It represents class similarity and learning difficulty in data. For example, in Table [ the accuracies in the columns of CAT and DOG are significantly lower than the other columns for all the four deep models. It is also observable that class interference has symmetry pattern: If classifier has trouble in recognizing that e₁ objects are not class cz will also have hard time ruling out class for 62 objects. This can be observed from CAT and DOG in the plotted CCTM.
We call generalization difficulties of deep neural networks between classes like CAT and DOG the class interference. If CCTM(c).C) is large, we say that class 02 interferes C1 or class C₁ has interference from 02. Class interference happens when classes are just similar. In this case, cats and dogs are hard to recognize for humans well, especially when the resolution of images is low. Examining only the test error would not reveal the class interference phenomenon because is an overall measure of all classes. The classes have much varied difference in their test accuracies. For example, in VGG19, the recall accuracy of CAT, i.e., CCTM(CAT,CAM) is only about 84.5%
See https://eeet wiki ptt.
ficity for example.
2

---

Under review as a conference paper at ICLR 2023
CCTM: VGG19
CCTM: ResNet18
plane
plane
0.06
0.06
bird
cat
0.04
0.04
dog
0.03
dog
frog
0.02
horse
horse
ship
0.01
truck
truc
0.00
0.00
plane car bird cat deer dog frog horse ship truck
plane car bird cat deer dog frog horse ship truck
class interference between CAT and DOG for both models. A darker color means a higher interference from the column class the row class. The diagonal entries are not plotted for clearer looking heat map. ResNet18 8 has less interference than VGG19 between CAT and DOG, indicated by: (1) there are fewer darker cells the CAT row; and (2) the DOG-CAT cell is brighter (red vs. dark grey).
Figure 1: CCTM for VGGI9 and 8 CIFAR-10 lest There is
plane
car
bird
cat
deer
dog
frog
horse
ship
truck
VGG19
96.0
96.6
91.9
84.5
94.7
89.0
96.0
96.4
95.2
96.3
ResNet18
96.9
97.9
94.2
86.5
97.0
92.6
96.9
96.3
96.5
97.3
GoogleNet
96.4
96.7
93.8
86.9
97.3
92.0
97.2
95.5
96.5
96.6
DLA
95.4
97.8
92.6
88.7
96.6
90.8
96.1
97.3
97.0
96.4
Table 1: Class recall accuracy (the diagonal entries of the CCTM) CIFAR-10 test set. Classifying CAT and DOG is the bottleneck for all the models. Figure 1 shows that this due to all the models suffer from the interference between CAT and DOG.
and DOG recall is about 89.0%. For the other classes the recall accuracy is much higher, e.g., CAR 96.6%. As shown in Table ResNet18 (He et al.) 2015), GoogleNet (Szegedy al. [014) and DLA (Yu et al., (017) have less class interference than VGG19 especially for CAT and DOG. For example, for ResNet18, (CAT, CAT) 86.5% and CCTM (OGG 92.6%.
2.2 DEFINITIONS
Let w² be a trained neural network model, e.g., VGG19 or ResNet18. We use the following defini- tions.
Definition 1 (Interference Model Set). Let De be the samples of class in data set. Define the gradient of class as the average gradiens that is calculated on this set:
fi (w") def 1
Σ fx,)
De
(X.Y)ED.
Accordingly, there are a set of class gradient directions for the model, (V (Vel |c 1,2,..., C}, where C the number of classes.
An ego model of class с is generated by using scalar in the class gradient direction:
w = - ((n)
The set, MM = {w. (c) = 1,..., is called the ego model set.
........,,, mc), is the ego model set of class c. The set union, M Mo
This definition based on that each w is in the direction of minimizing the loss for predicting class Note that (e))) is sample of "ego-centric" update, which minimizes the loss for class

---

Under review as a conference paper at ICLR 2023
only. It therefore could cause increase in the prediction emrors for the other classes. We refer to the gradient of class с as the ego direction of the class. Measuring the loss on the interference models thus tells the interference between classes.
Definition 2 (Interference Space). The model space (1o0)1e2)(01 E O 22) is called the interference model space of class 0₁ and Cg2 where an interference model is defined by
((r =w" (00r))=-'-'-'V" (1
Define Flar.s) ([(w(( (O,,,, 02) E U O2). which is the set of interference losses between the two classes. The 3D space, θ Q2 F(),,e),, is the loss interference space, or simply, the interference space (of class and class C2 for model w*).
Proposition 1. Any interference model is a convex combination of the ego models of the two classes.
Proof. Let w ( and w ((0)) be the ego model of class e₁ and 02, respectively. According to their definition,
Amoe) + + (1-A))uf = -w=- - Xas V/(c) + (1 \)w - -(1 (1->)aa V ((())) A)wg
- - (Na, + X)ay ((())(
= w((c)))))
where setting 0 = λoa and 02 = Alay finishes the proof.
MINIMA: FLAT OR SHARP?
Our first experiment is to understand minima sharpness of learning rate using class ego directions. We will visualize in the interference space, O O₂ x F(er,e)), We use this loss: mistake rate for the z-axis, which is the percentage of classification mistakes the training set give a loss measure in the same range across different plots. We visualize the loss of the models on the training versus Q O2, which is uniform grid over [-a,a] [-o, a). with 19 points in each direction. This gives 361 interference models between a given class pair. We use the ego directions of CAT-DOG (the most interfering class pair), TRUCK-CAR (with significant level of interference), and HORSE-SHIP (with little interference). These plots measure how sensitive the training loss changes with respect to the directions that focus on optimizing specially for individual classes and the linear combinations of these directions. The center of each plot corresponds to the origin, (8, 0, 0g 0), at which trained VGG19 or ResNet located.
We study the models of VGG19 and ResNet18 trained with the following optimizer setups:
big-Ir. This optimizer uses big learning rate, 0.01. The momentum and weight decay are the same as the small-Ir optimizer. Figure shows for VGG19 (top row) and ResNet18 (bottom row).
small-Ir. This SGD optimizer uses small learning rate 0.0001. It also has a momentum (rate 0.9) and a weight decay (rate 0.0005).
anneal-Ir. Similar the above optimizers, but with
bigger (initial) learning rate. A big constant learning rate 0.1 leads to oscillatory training loss and poor models. We thus decay with an initial value of 0.1 using Cosine rule (Loshchilov & Hutter 2016). This is the optimizer setup used to train the models in Section 2.1
The input images are transformed with RandomCrop and RandomHorizontalPlin and normalization. The batch size is 128. The Cross Entropy loss used. Each model trained with 200 epochs. The test accuracies for the models are shown in the following table.
ViG-bi VGG-small-Ir 84.99%
VGG-anneal-Ir 93.87%
ResNet-big-Ir ResNet-small-Jr
ResNet-anneal-Ir 95.15%
88.76%
86.88%
91.31
4

---

Under review as a conference paper at ICLR 2023
100 -rnnin aCCHERC
100
traiiing ANCUrSS
100
30
50
60
VGG-19
62
40
40
-big-Ir
20
20
100
0.10
0.10
dresttinn
tgo Seeeiin
-005
0.00
3.00
0.00
0.5
0.10
-0.30
-0.55
cat ago direction
0.00
0.05
direction
-0.10
-0.10
-0.05
ship direction
0.00
0.05
-0.10
100
100
10
60
60
ResNet18 -big-Ir
60
100 training
60
100
20
20
0.30 dog ago dieetion
- 0.00
cn ege srecion
0.00 0.05
-0.10 de Skeddm
0.00
0.05
0.05
310
-0.05
ago directon
0.00
0.05
0.10
0.00 truck discttin
0.005
6,05
-0.10
slip
0.05
0.05
Figure 2: Loss visualized in the class ego directions for models optimized with big-Ir. Model: VGG19 (top row); ResNet18 (bottom row). Each point stands for the loss of an interference model w())))))
This confirms that big learning rates generalize better than small ones discovered by the com- munity. Interestingly, the anneal learning rate leads to models that generalize even much better, for which there has been no explanation the best of our knowledge.
Let's first take look at VGG19 trained with big-Ir, whose interference spaces are shown at the top row of Figure 2 The loss exhibits strong sharpness in the CAT-DOG ego visualization. From the minimum (the trained VGG19 at the center), a small step of optimizing the CAT predictions easily deteriorates the loss, in particular the red flat plateau corresponds set down to merely 10%. The loss change is extremely sensitive in the CAT ego direction. It similarly sensitive in all directions except near the DOG ego direction, which looks still very sensitive. According to Proposition[] any interference model in this space is convex combination of CAT ego model and a DOG ego model. This plot thus shows that the CAT ego is very influential even the weight of the DOG ego is large.
accuracy the training
The visualizations in the CAR-TRUCK and HORSE-SHIE ego spaces show that the loss changes much less sensitively than for CAT and DOG when we update the model for the purpose of improv- ing or even sacrificing the prediction accuracy of the four classes. However, close to the directions of TRUCK ego plus negative CAR ego, and negative TRUCK ego plus CAR ego, the loss also changes abruptly. If we cut the loss surface 135 degrees in the z-y axis, we end up getting a minimum that looks sharp. On the other hand, random likely renders less sharp minimum. The case of HORSE-SHIP is similar. Thus whether the minimum looks flat or sharp is dependent on how the loss contour is cut. Some care needs to be taken when we discuss minima sharpness, especially the space in which the loss is plotted. Most previous discussions on minima sharpness are based on the difference between initial model and trained model, directions. Both methods have randomization effects and yet they get descent loss contours. While it is amazing, the reason why random cuts render reflective loss contours is unclear. Our guess is that most directions renders sharpness and sampling random one is likely fine. However, when we compare the levels of sharpness between models, random cuts may not be accurate.
flat look of the
random
Figure 2 bottom row shows for ResNet18 optimized with the optimizer. The loss change near the minimum is also extremely sensitive in the CAT-DOG ego space. Interestingly, for ResNet18, the loss in the DOG ego direction is more sensitive than in the CAT direction. This seems "transposed" effect of VGG19, because the influence the DOG ego is stronger on the loss now. For both VGG19 and ResNet18, the loss visualized in the CAT-DOG ego space has clear narrow valley structure the minimum. This kind of loss functions are known to be very challenging for gradient descent, e.g., see the Rosenbrock function also known the Banana function (Rosenbrock) 1960). In the
5

---

Under review as a conference paper at ICLR 2023
100
Tomnn AccUro
100
scomery
80
80
100
60
40
VGG-19
20
201
100
40
-small-Ir
100
0,10
0.10
-0.013.a.005 000 0,005
ego dructin
0.030
- 005ge
0.000
de
-0.100 10.000
muck ago direction
0.000
0,005
0.010
0010
-0.00
0.000
0.000
Pa 0.010
d..03
0.000
0d05
horee ego direction
0.000
00.05
0.020
100
300 cenntrr acCUTNY
100
100 900
ResNet18 -small-ir
20
100
20
0010
b 0.00
c.05
-9,007
-0.00
-0.010 0070.000
0.000
direction
0,010
-A.12.005 A.12a.00%..00 0,000 0,005
0,00s
Monci -3.10 ege diectio
ego dioction amo - 0101
egu atrention
0.01 00000
005
Figure 3: Loss visualized the class ego directions for models optimized with small-Ir. ResNet18 is spiky (extremely sharp) minimum according to the visualization in the ego spaces, even though the area of the illustration 1/100 of Figure 2 (big-Ir).
100
commnn wearnn 601
100
80
100
VGG-19
anneal-Ir
40
300
20
-1.0
dog ago
0.0
1.0
-2000 8 call direction
-1.0
track direction
.05
0.0
10
-2.0
-0.5
car ago directen
9.
1.0
Top ago dinctto
-10
69
10
NOTEA Orenzon
100
training accuraty
80
120
ResNet18 anneal-Ir
60
86
moltre Socuing
40
60
170
-1.0 -0.5
-10 1 200 cst ego direction
muck ago diection
0g 0o 0. or ego directen
-1.0 Mapp ago direcioon
0.
20
1.0 barso drection
6.5
dog ago direction
1.0
1.0
-1.0
Figure 4: Loss visualized in the class ego directions for models optimized with anneal-Ir. In this illustration area 100 times of Figure (big-Ir), both minima located large flat lowland. The ResNet18 8 has extremely large flat area.
CAR-TRUCK space, the loss of R is much less curvy up than that of VGG19. In particular, for VGG19 sensitive in both the ego directions, while for ResNer18, only near the direction about 135 degrees (z-y axis) it is sensitive. For VGG19, the SHIP direction has lots of sensitivity. For ResNet18, the HORSE direction instead is more sensitive.
Our results show the minima being flat or sharp is dependent what spaces the loss is illustrated. We think better way of discussing generalization is the area of flatness around the minima in crit- ical directions. Our plots in different class ego spaces show that minimum in certain visualization spaces (e.g., ResNet18 in the CAR-TRUCK ego space), while at the same time it can look very sharp in other spaces (e.g., R8 8 in the CAT-DOG space).
be flat minimum

---

Under review as a conference paper at ICLR 2023
CAT
DOG
HORSE
CAR
TRUCK
CAT-D0G- space
FROG
DEER
BIRD
PLANE
SHIP
CAT-DOG-
CAT
DOG
HORSE
CAR
TRUCK
CAR-TRUCK:
FROG
DEER
BIRD
PLANE
SHIP
space
Figure 5: Class-wine wise losses. This shows class prediction losses are more sensitive in the CAR- TRUCK space than in the CAT-DOG space. In particular, the CAR ego direction is very sensitive, which means all the other class prediction losses are greatly influenced by the movement in mini- mizing maximizing (gradient ascent) the CAR loss. Model: VGGI9. Optimizer: anneal-Ir.
Figure shows the small learning rate. This time ResNet18 8 is an extremely sharp minimum in all the three ego spaces. In a small area around the minimum in ego spaces, the loss changes dramatically. Beyond that small area, the loss is invariantly high (plateau). VGG19, instead, has change of loss in a small area although in the CAT ego direction the loss changes abruptly too (which forms cliff). This shows when the learning rate small, the loss contour can be near non-smooth and sharp minima do not necessarily generalize worse (comparing to VGG19). This confirms the findings by Dinh et al. (2017) and (Li et al. 2018) that there exist models that are sharp minima and yet they still generalize well. In particular, ResNet18 8 has a better generalization than VGG19, 86.88% versus 84.99% in this case. Our results show that flat minima generalize better when the learning rate is well tuned (not too small). However, when the learning rate is small, the minima can be sharp and they can generalize even better than less sharp ones.
smooth
Finally, Figure shows for the models optimized with learning rate annealing. These two models have superior generalization, with 93.87% for VGG19 and 95. for ResNet18. The visualization in the ego spaces show that the area of flatness is very large, especially ResNet18. Comparing to fixed big learning rate, the models trained by annealing have a much higher level of in-sensitiveness parameter changes in the class ego directions. Presumably, the big initial learning rate helps establish a larger flat area. This level of flatness has not been observed before, especially in previous experiments of learning rates. We may thus refer minima located in large flat terrain the lowland minima.
4 ANALYZING CLASS INTERFERENCE
4.1 INTERFERENCE FROM ONE CLASS TO THE OTHERS
We also would like understand the interference from class to the others for trained model. Figure Slohows the interference of CAT, DOG, CAR and TRUCK to all the classes. First let's look at the CAT loss in the CAT-DOG space (first plot). It shows CAT loss increases in the cat ego direction,

---

Under review as a conference paper at ICLR 2023
recall rate--cat
recall rate-ding
55 of cats predicted PLAME
cats predicted
-U of cals pa predicted as DEER
5% cats predicted DOG
cats
FROG
cats predicted
5% cats predicted as SHIP
5% of cats predicted TRUCK
106
1:0
opochs
Figure 6: The CAT-DOG dance. Each red arrow is a moment that DOG interferes CAT, with a local high DOG recall rate (cyan dashed line) and drop of the CAT recall rate (black line), and local peak of mis-classified cats as DOG (red cross). Each blue arrow is a sample moment that both recall rates drop, in response high mis-classification rate of cats third class, e.g., BIRD (black rectangle). Model: VGG19. Optimizer: anneal-lr.
i.e, the gradient ascent direction, which is intuitive. It also shows the CAT loss increases most when we minimize the DOG loss. This another verification that DOG interferes CAT. Interestingly. following the joint direction of gradient ascent directions to maximize the CAT loss and the DOG loss doesn't 't increase the CAT loss much. In the case of CAR loss in the CAR-TRUUK space, the situation is little different. In particular, CAR loss increases significantly whether gradient descent or ascent direction of TRUCK as long as we move in the ascent direction of CAR. TRUCK loss is
follow the
complicated. The loss increases in the joint direction of ascent directions of CAR and TRUCK losses. In addition, TRUCK loss also increases if we follow the the descent direction of CAR. This means minimizing the CAR loss has the effect of increasing the TRUCK loss. This is also a sign that CAR and TRUCK interferes. For the other classes, their prediction losses respond more sensitively to the ego directions of CAR-TRUCK than those of CAT-DOG.
In the CAT-DOG space, CAR, TRUCK, PLANE, and SHIP all increase their losses in one same corner. HORSE and DEER losses both increase as we get closer the corner where DOG loss increases; in addition, the increase of HORSE is more than DEER in this process.
In the CAR-TRUCK space, CAT, FROG, DOG, and DEER losses have very similar shapes. This suggests these losses increase in roughly the same directions in the CAR-TRUCK space. HORSE's loss shape is also similar these four classes, but the similarity is less. CAR and PLANE losses have very similar shapes. TRUCK and SHIP losses have similar wing-like structure too. CAR, PLANE, TRUCK and SHIP have similar loss shapes on the left side of the plots shown. These observations suggest that loss shapes in class ego spaces indicative of interference. Classes that share similar loss shapes in other class ego spaces are likely to interfere. This going to be discussed further in the next experiment.
4.2 CLASS INTERFERENCE TRAINING
The above experiments are for a trained model. We were wondering whether class interference can be observed in training. To study this, we plot the per-class training accuracy which is the recall rate for each class. Figure shows for CAT and DOG. The two recall rates are both highly oscillatory, especially in the beginning stage of training. Importantly, there are many moments that one rate being high while the other being low at the same time, which we call label dance or CAT- DOG dance for this particular case. This dancing pattern is strong indicator that CAT and DOG interfere. To further confirm this, we plot in the same figure the row of the CCTM for the training set that correspond CAT. i..., CCTM(CAT c). for each non-CAT class c. during the same training process. As the caption of the figure shows, a rise in the DOG recall rate is often caused by a high interference of DOG to CAT. After some (about 1 118) epochs, DOG interference dominates CAT
8

---

Under review as a conference paper at ICLR 2023
Cless that duminates intter serrfeeence
CAT(3] DOG(S) notes
CARI1) TRUCKI9) notes
175
The dancing notes for CAT, DOG, CAR and TRUCK. A data point of the CAT notes curve is the most often false prediction at that epoch. Model: VGG19. Optimizer: anneal-Ir.
Figure 7:
predictions errors and eventually weeds out following the other classes. In this phase of training circled in the figure), the recall rates of CAT and DOG highly symmetric to each other (horizontally), further indicating that DOG interference is the major source of error in predicting cats and vice versa.
Figure 7 plots the "argmax" operation of the CCTM for the rows corresponding to four classes at each epoch, excluding the diagonal part. The plot looks similar to music notes. So we term this plot "dancing notes". For DOG notes, there are many pink markers at the line y = 3, which is the class label corresponding to CAT. The stretched markers laying continuously a clear sign of CAT interference to DOG. In the CAT notes, continual red crosses also persist at y 5, which is the class label of DOG, showing interference persists longer than the other way. For better presentation of the results, we plot y 2 if no class interferes more than 0.1%.
DOG to CAT. It also shows that CAT interference to DOG
The notes of CAR (class label 1) and TRUCK (class label 9) show similar duration of interference, and it appears the interference from CAT to TRUCK seems to have a close strength to the other way around. It is also interesting observe that both CAR and TRUCK have interference from class labels = 0 and = 8, which correspond to PLANE and SHIP. This is intuitive because these are all human made metallic crafts. It appears that the interference from PLANE TRUCK is more often than to CAR, probably because trucks are bigger in size than cars.
CAT has interference from BIRD (2) given their similar fluffy looks. Surprisingly, FROG (6) also interferes CAT pretty often. We checked the CIFAR-10 images visually and it is probably because the images mostly close looks of the objects; in this case cats have two pointy ears which are easily confused with frogs who have their eyes positioned atop. Besides CAT, DOG has interference from HORSE (7) and DEER (4) because they are all four-legged. It is interesting to observe that CAT, on the other hand, almost does not have interference from HORSE, with only two three moments interference out of 200 epochs. This means HORSE is very helpful to differentiate be- tween CAT and DOG, which is the largest source of generalization error DOG also has a little interference from BIRD (2) similar to CAT does.
discussed in Section
2.1
CONCLUSION
This paper illustrates a phenomenon called class interference of deep neural networks. We show it is the bottleneck of classification, which represents learning difficulty in data. The proposed cross-class generalization tests, class ego directions, interference models and the study of class-wise losses in class ego directions provide a tool set for studying the generalization of trained deep neural networks. The study of label dancing via the dancing notes provides a method of detecting class interference during training. With the provided tools in these two dimensions, hope this paper is useful to understand the generalization of deep nets, improve existing models and training methods, and understand the data better well as the learning difficulty of recognition.

---

Under review as a conference paper at ICLR 2023
CCTM: GoogleNet
CCTM: DLA
car
0.05
car
0.05
bird
bird
0.04
0.04
deer
deer
dog
trog
frog
0.02
ship
0.01
ship
0.01
truck
truck
bird cat deer frog horse ship truck
bird deer frog horse truck
Figure 8: CCTM heatmaps for GoogleNet and DLA on CIFAR-1O set. CAT and DOG still have the largest generalization errors among all the class pairs. GoogleNet wins DLA in the DOG class. However, loses the CAT class. See Table [ for the recall rates of individual classes.
APPENDIX
The CCTM heatmaps of GoogleNet and DLA are shown in Figure The interference between CAT and DOG is similarly high (see Figure D for VGG19 and ResNet18).
Let us examine the CAR row in details this time. The CAR-TRUCK cell stands out. For GoogleNet, the color is a distinet Orange while other cells have very light colors. By looking at the colorbar, Orange color is about 0.025, which is 2.5%. In Table I GoogleNet's CAR recall is 96.7%. If add them together, 96.7% 2.5% = 99.2%. (For the TRUCK-CAN cell, it shows symmetry.) This means the majority of the generalization errors happen for predicting According to our experiments, CAR and TRUCK also interfere and we investigate their individual losses and illustrate their interference in training. Relevant discussions Section 4.1 (Figure and Section 4.2 (Figure 7)
TRUCK and vice
For DLA, the color of CAR-TRUCK is much brighter than GoogleNet, which Orange. Note the colorbar range of the two nets is the same and the colors across the two plots are comparable. This color about 0.018 according the colorbar. In Table D DLA's recall for CAR is 97.8%. We have 97.8% + 1.8% 99.6%. This shows most of DLA's mistakes for cars happen for predicting them as TRUCK, similar to GoogleNet. We can also observe here that the CAR-TRUCK mistake of DLA (1.8%) is better than that of GoogleNet (2.5%). This leads to overall better classification cars for DLA (97.8%) than GoogleNet (96.7%).
Yellow than
Reviewer question: interfeernce is something bad for classification (as you say "interference is the bottleneck"), then why PLANE/BIRD with lower interference (brighter colors) than CAR/TRUCK have better numbers Table e.g. for GoogleNet? PLANE/BIRD: 96.4/93.8 vs CAR/TRUCK 96.7/96.6?
We take a look at the raw CCTM data for GoogleNet, in particular the rows for PLANE, CAR, BIRD, and TRUCK. The data shown in Table 2 So this confirms the recall rates Table are correct, in particular, PLANE/BIRD: 96.4% vs. CAR/TRUCK 96.7%/96. 76.7%%%966%
Regarding why CAR/TRUCK have better recall rates than PLANE/BIRD, we can look at the CAR row first: All the numbers are low (0.00X or 0) except for the TRUCK column. This means GoogleNet rarely mistakes cars for classes other than TRUCK. Class interference is mainly for comparing the column classes for each row class. For the CAR row, TRUCK intereferes it lot, much more than the other classes.
This section henefits from the discussions with reviewer of ICLR 2023. It was added due to his suggestions.
10

---

Under review as a conference paper at ICLR 2023
Class
PLANE
CAR
BIRD
CAT
DEER
DOG
FROG
HORSE
SHIP
TRUCK
PLANE
0.964
0.001
0.012
0.001
0,002
0.002
0.001
0.011
0.006
CAR
0.004
0.967
0.
0.001
0.001
0.
0.
0.001
0.005
0.021
BIRD
0.01
0.
0.938
0.01
0.023
0.009
0.009
O.
0.001
O.
TRUCK
0.004
0.021
0.001
0.003
0.
O.
0.005
0.966
Table 2: Sampled rows of GoogleNet for Figure
For the PLANE row, (PLANE, BIRD) and (PLANE.SHIP) are both over 0.01. It makes sense because they both have sky background the data set. (PLANE, TRUCK) is also a bit high, 0.006, because both are metallic.
That is, for the PLANE row, there are three other classes that interfere it, while for CAR/TRUCK, there is only one class that interferes it (let's say we use interference threshold 0.005. Class B interferes class A if the cell (A, B) is bigger than 0.5%). Although the errors of (CAR, TRUCK) and (TRUCK, CAR) are high, the model does make much other mistake for predicting cars and trucks. Thus their recall rates are high.
For the BIRD row, there are five classes with high interference too. Thus for PLANE and BIRD, there is also significant interference from other classes besides the most interfering class (the most dark color cell in a row). This leads lower recall rates for PLANE/BIRD than for CAR/TRUCK.
This question and discussion shows that "many interfering classes for a class" is also bad for the class addition to a single, strong "most interfering class". In short, "interference is the bottleneck" means the certain classes have strong interference from one or multiple other classes.
REFERENCES
Soham De, Abhay Yadav, David Jacobs, and Tom Goldstein. Automated Inference with Adaptive Batches. In Aarti Singh and Jerry Zhu (eds.), Proceedings of the 20th International Confer- ence on Artificial Intelligence and Statistics, volume 54 of Proceedings of Machine Learning Research. pp. 1504-1513 PMLR, 20-22 Apr 2017. URL http:::/ppooceeeiiiiicc .ml.. mmrr. press/v54/ee720 /el
Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. Sharp minima can generalize for 2017. URL http://aaxxvv..oo /tttp::/axxxv..oggab///////
deep nets. CoRR,
Aleksandar Doknic and Torsten Möller. Funnscope: Visual microscope for explorn the loss landscape of fully connected neural networks, 2022. URL https::/arxiv.org
abs / 2204.22422 04511
Felix Draxler, Kambis Veschgini, Manfred Salmhofer, and Fred Hamprecht. Essentially no barriers in neural network energy landscape. In International conference on machine learning, pp. 1309- 1318. PMLR. 2018.
Ian Goodfellow, Oriol Vinyals, and Andrew Saxe. Qualitatively characterizing neural network op- timization problems. In International Conference on Learning Representations, 2015. URL
http:
Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning, volume 1. MIT Press, 2016.
Priya Goyal, Piotr Dollár, Ross B. Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. Accurate, large minibatch SGD: training im- agenet in 1 hour. CoRR, abs/1706.00677,, 2017. URL attp:///axxvv.org/abba.. 02677
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition, 2015.
03385
Sepp Hochreiter and Jürgen Schmidhuber. Flat minima. Neural computation, 9(1):1-42, 1997.

---

Under review as a conference paper at ICLR 2023
Elad Hoffer, Itay Hubara, and Daniel Soudry. Train longer. generalize better: closing the general- ization gap in large batch training of neural networks. 2017. doi: 1048550/ARXX..7755.8744. URL
abs/ 705. 08741
Daniel Jiwoong Im, Michael Tao, and Kristin Branson. An empirical analysis of the optimization of deep network loss surfaces, 2016. URL https: aaaii. org/ abs /ales/1666 1612. 04010
Stanislaw Jastrzebski, Zachary Kenton, Devansh Arpit, Nicolas Ballas, Asja Fischer, Yoshua Ben- gio, and Amos J. Storkey. Three factors influencing minima in SGD. CoRR, abs/1711 1o0//771 2017.
Kenji Kawaguchi, Leslie Pack Kaelbling, and Yoshua Bengio. Generalization in deep learning. arXiv preprint arXiv: 2017.
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal. Mikhail Smelyanskiy, and Ping Tak Pe- Tang. On large-batch training for deep learning: Generalization gap and sharp minima. CoRR, abs/1609.04883 04436, 2016. URL http:t::// arxiv. org/ abs /bb9/669, 04836
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436 444, 2015.
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss land-
scape of neural nets, 2018.
Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts, 2016. URL https: org/ hwtpg//////////o///////////aaee/ /1608 03983
James Lucas, Juhan Bac, Michael R. Zhang, Stanislav Fort, Richard Zemel, and Roger Grosse.
Analyzing monotonic linear interpolation neural network loss landscapes, 2021. URL https:
//axxiv.orq//abs/2211
11044
Volodymyr Mnih, Koray Kavukcuoglu, David Silver. Andrei A Rusu, Joel Veness, Mare G Belle- mare, Alex Graves, Martin Riedmiller, Andreas Fidjeland, Georg Ostrovski. et al. Human-level control through deep reinforcement learning, nature, 5 1177540):529-533 2015.
HoHo Rosenbrock. An automatic method for finding the greatest or least value of a function. The computer journal, 375- 1960.
David Silver, Aja Huang, Chris Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche,
Julian Schrittweeser, loannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):4844889 2016.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. International Conference on Learning Representations, 2015.
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Du- mitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions, 2014.
Tiffany Vlaar and Jonathan Frankle. What can linear interpolation of neural network loss landscapes tell us? ICML,, 2022. UR..nnttpss//axxxi..rr/////////2
6004
Fisher Yu, Dequan Wang, Evan Shelhamer, and Trevor Darrell. Deep layer aggregation, 2017. URL https: /tts:///rxvv... org/20//1707..5883
12


Reviews:
[{"review_id": "lIP0euZlRH", "reviewer": "Reviewer_6BAT", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose the notion of class interference represents the learning difficulty in data. They propose using a CCTM metric for understanding class interference whereas a larger interference metric indicates a sharper minimum and worse generalization. They also give an explanation for why annealed learning rate results in better generalization. The authors perform analysis on multiple architectures on CIFAR 10 to validate their claim", "summary_of_the_review": "Overall, I believe the paper is not complete in terms of writing and experiments. Although the observations are interesting, they are not supported by enough empirical and theoretical analysis. I've changed the score to strong reject as it seems that the authors are not open to improvements and I believe this paper is quite below ICLR standard. ", "strength_and_weaknesses": "Strength: \n1. The visualization validates the claims.\n\nWeakness: \n1. In section 1, I'm confused about how the interpolation monotonicity is related to class interference. The motivation is not quite clear. Besides, it seems to be related to the form in Proposition 1, yet this proposition is not used or explained in this paper.\n2. CCTM seems to be an extension for the confusion matrix when the number of classes is over 2.  Therefore, I'm concerned about the novelty of this metric. \n3. CCTM can't explain the difference between the failed generalization of an overfit model and an underfit model, which by intuition has very different loss landscapes. For example, in an underfit model, the landscape can be smooth yet CCTM would still be quite high.\n4. The notion of smoothness is only observed from visualization and lacks any metrics. Computing the eigenvalue of Hessian would be helpful for estimating the correlation between CCTM and smoothness.\n5. As shown in Table1, the recall accuracy is similar for VGG, ResNet, DLA and GoogleNet and may be the reason why they observe the same pattern in CCM. I believe it would be more convincing if the author can try a wider range of models with different complexity, including ViT.\n6. Only CIFAR-10 is used in the analysis and may be an exception. Hope more multi-label classification datasets could be analyzed, e.g. MNIST, and ImageNet.\n\nQuestions: \n1. Could you please provide some intuition why higher CCTM results in sharper minimum, especially when the loss space is not in the two classes involved in CCTM calculation ( as shown in Figure 4) ? \n2. Is class interference possibly an inherent problem of a dataset? As stated in the paper, cats are often confused with dogs because of low resolution. It seems that training techniques like anneal-lr can alleviate this problem, but is there a lower-bound on class interference? ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "iwJv5OAGoNX", "reviewer": "Reviewer_3tyq", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies deep neural network through the lens of \"class interference\". Class interference corresponds to difficulty for a pair of classes, where one class and another class are hard to distinguish for the neural network. More specifically the cross-class test of generalization matrix (CCTM) is used to measure the class interference between classes. Class interference becomes severe when classes are conceptually similar, as given by the example of Cats vs. Dogs. The paper further defines the ego model of each class, which is an gradient-updated model based on the average gradient of samples for only that class. Based on this idea, the paper defines an interference space, which is used to study trained deep neural networks and during training of deep neural networks.", "summary_of_the_review": "Although the paper works on an interesting problem with new and existing tools, overall, due to the reasons I wrote in the previous sections, I am feeling that in its current form, it is below the acceptance threshold. I hope the authors can incorporate the comments from the reviewers to make the paper better.", "strength_and_weaknesses": "Strengths:\n- Studying the minima flatness/sharpness and learning difficulty in data are important topics, and class interference provides us with insights about the datasets. We can have a better understanding of CIFAR-10 using the tools provided in the paper.\n\nWeaknesses:\n- Section 2 explains how the difficulty of data based on CCTM is not specific to models, but it seems to be model dependent, since the results depend on the model used (although we can see similar trends between the different models).\n- One of the main contribution is the class interference measure based on the proposed CCTM, but CCTM may not be novel, since it seems to be equivalent to the confusion matrix (or the normalized version of it). Confusion matrix is already used heavily to study the class-wise performance of deep neural network classifiers, especially in the industry.\n- The discussions about \"dance\" seems interesting, but I'm not sure if I understood the phenomenon correctly. It would be helpful if there are more discussions on why if the training recall of one class rises, the other class's training recall will decrease. Furthermore, it is hard to visually confirm if this is happening, so would be nice to see the correlation value between these two time-series data.\n- Currently the paper only studies the CIFAR-10 dataset, but it would be interesting to see if the same results, e.g., symmetry pattern, arise for other datasets as well.\n- From the perspective of learning difficulty in data, there are many papers recently working on this, such as: \"Deep Learning Through the Lens of Example Difficulty\" (NeurIPS 2021), \"Estimating Example Difficulty Using Variance of Gradients\" (CVPR 2022), \"Understanding Dataset Difficulty with V-Usable Information\" (ICML 2022). There are also papers such as \"Evaluating State-of-the-Art Classification Models Against Bayes Optimality\" (NeurIPS 2021) that study class-wise difficulty (Appendix B.1 shows how CAT and DOG are the most difficult classes for CIFAR-10, which is consistent with the experimental results in the paper under review). It would be interesting to see discussions about related work.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "_sq-_inRM5", "reviewer": "Reviewer_aWAb", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper investigates 'class interference', i.e. how does update on one class affect the others. The work looks into the flatness of the minima for the converged models of two architectures (VGG and ResNet) trained with three different hyper parameter settings (smaller lr, larger lr and annealed lr). Finally, the authors look at how class interference evolves throughout training.", "summary_of_the_review": "Given my comments above, I believe that the paper is not ready for a publication at ICLR. Apart from my criticisms above, I will put suggestions on how to improve the paper below:\n\n* I think it would be extremely helpful to make a synthetic task with two overlapping gaussians to illustrate the authors' point. The gaussians are super easy to plot and very easy to vary the mean/std to get the desired effect. \n* I know this is hard to do, but having a set of actionable insights will be extremely helpful for the researchers or practitioners. For instance, what should we do about the observed phenomenon? \n* How, do you think, the irreducible error is related to class interference?", "strength_and_weaknesses": "### Strengths\n\n* The paper shows some interesting phenomena, e.g. the fact that lr annealing leads to flat minima for both of the architectures. \n* I liked the introduction doing the overview of the related work and providing the context for the study.\n\n### Weaknesses\n\nClass interference is an intuitive phenomenon. However, whether this intuition aligns with optimisation challenges is unclear. It is also unclear what to do with this: yes, updating on one class only can make the other class deteriorate. What do we do with this? Should we even do something with this? If yes, why?\n\nApart from this, I don't think the paper demonstrates what it claimed to have demonstrated. For instance, looking at the conclusion:\n\n* We show it [the class interference] is the bottleneck of classification <- I don't think the paper actually showed that\n* ...we hope this paper is useful to understand the generalisation of deep nets <- it is unclear what exactly we understood from the fact that the classes interfere and that an updating one class leads to deterioration on the other.\n* ...useful to ... improve existing models and training methods <- this would be an extremely useful application, but it is unclear how to use the proposed metrics to do so. \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject