Original: C LASS I NTERFERENCE OF D EEP N EURAL N ETWORKS
Revised: C LASS I NTERFERENCE OF D EEP N EURAL N ETWORKS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Recognizing and telling similar objects apart is even hard for human beings.
Revised: Recognizing and telling similar objects apart is even hard for human beings.

Original: In this paper, we show that there is a phenomenon of class interference with all deep neural networks.
Revised: In this paper, we show that there is a phenomenon of class interference with all deep neural networks.

Original: Class interference represents the learning difficulty in data and it constitutes the largest percentage of generalization errors by deep networks.
Revised: Class interference represents the learning difficulty in data and it constitutes the largest percentage of generalization errors by deep networks.

Original: To understand class interference, we propose cross-class tests, class ego directions and interference models.
Revised: To understand class interference, we propose cross-class tests, class ego directions and interference models.

Original: We show how to use these definitions to study minima flatness and class interference of a trained model.
Revised: We show how to use these definitions to study minima flatness and class interference of a trained model.

Original: We also show how to detect class interference during training through label dancing pattern and class dancing notes.
Revised: We also show how to detect class interference during training through label dancing pattern and class dancing notes.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deep neural networks are very successful for classification (LeCun et al., 2015; Goodfellow et al., 2016) and sequential decision making (Mnih et al., 2015; Silver et al., 2016).
Revised: Deep neural networks are very successful for classification (LeCun et al., 2015; Goodfellow et al., 2016) and sequential decision making (Mnih et al., 2015; Silver et al., 2016).

Original: However, there lacks a good understanding of why they work well and where is the bottleneck.
Revised: However, there lacks a good understanding of why they work well and where is the bottleneck.

Original: For example, it is well known that larger learning rates and smaller batch sizes can train models that generalize better.
Revised: For example, it is well known that larger learning rates and smaller batch sizes can train models that generalize better.

Original: Keskar et al.
Revised: Keskar et al.

Original: (2016) found that large batch sizes lead to models that look sharp around the minima.
Revised: (2016) found that large batch sizes lead to models that look sharp around the minima.

Original: According to Hochreiter & Schmidhuber (1997), flat minima generalize better because of the minimum-description-length principle: low-complexity networks generalize well in practice.
Revised: According to Hochreiter & Schmidhuber (1997), flat minima generalize better because of the minimum-description-length principle: low-complexity networks generalize well in practice.

Original: However, some works have different opinions about this matter (Kawaguchi et al., 2017;
Revised: However, some works have different opinions about this matter (Kawaguchi et al., 2017;

Original: Dinh et al., 2017; Li et al., 2018).
Revised: Dinh et al., 2017; Li et al., 2018).

Original: Dinh et al.
Revised: Dinh et al.

Original: (2017) showed that sharp minima can also generalize well and a flat minimum can always be constructed from a sharp one by exploiting inherent geometric symmetry for ReLU based deep nets.
Revised: (2017) showed that sharp minima can also generalize well and a flat minimum can always be constructed from a sharp one by exploiting inherent geometric symmetry for ReLU based deep nets.

Original: Li et al.
Revised: Li et al.

Original: (2018) presented an experiment in which small batch minimizer is considerably sharper but it still generalizes better than large batch minimizer by turning on weight decay.
Revised: (2018) presented an experiment in which small batch minimizer is considerably sharper but it still generalizes better than large batch minimizer by turning on weight decay.

Original: Large batch training with good generalization also exists in literature (De et al., 2017; Goyal et al., 2017).
Revised: Large batch training with good generalization also exists in literature (De et al., 2017; Goyal et al., 2017).

Original: By adjusting the number of iterations, Hoffer et al.
Revised: By adjusting the number of iterations, Hoffer et al.

Original: (2017) showed there is no generalization gap between small batch and large batch training.
Revised: (2017) showed there is no generalization gap between small batch and large batch training.

Original: These works greatly helped understand the generalization of deep networks better.
Revised: These works greatly helped understand the generalization of deep networks better.

Original: However, it still remains largely mythical.
Revised: However, it still remains largely mythical.

Original: In this paper, we show there is an important phenomenon of deep neural networks, in which certain classes pose a great challenge for classifiers to tell them apart at test time, causing class interference .
Revised: In this paper, we show there is an important phenomenon of deep neural networks, in which certain classes pose a great challenge for classifiers to tell them apart at test time, causing class interference .

Original: Popular methods of understanding the generalization of deep neural networks are based on minima flatness, usually by visualizing the loss using the interpolation between two models (Goodfellow et al., 2015; Keskar et al., 2016; Im et al., 2016;
Revised: Popular methods of understanding the generalization of deep neural networks are based on minima flatness, usually by visualizing the loss using the interpolation between two models (Goodfellow et al., 2015; Keskar et al., 2016; Im et al., 2016;

Original: Jastrzebski et al., 2017; Draxler et al., 2018; Li et al., 2018; Lucas et al., 2021; Vlaar & Frankle, 2022; Doknic & M¨oller, 2022).
Revised: Jastrzebski et al., 2017; Draxler et al., 2018; Li et al., 2018; Lucas et al., 2021; Vlaar & Frankle, 2022; Doknic & M¨oller, 2022).

Original: Just plotting the losses during training is not enough to understand generalization.
Revised: Just plotting the losses during training is not enough to understand generalization.

Original: Linearly interpolating between the initial model and the final trained model provides more information on the minima.
Revised: Linearly interpolating between the initial model and the final trained model provides more information on the minima.

Original: A basic finding in this regard is the monotonic property: as the interpolation approaches the final model, loss decreases monotonically (Goodfellow et al., 2015).
Revised: A basic finding in this regard is the monotonic property: as the interpolation approaches the final model, loss decreases monotonically (Goodfellow et al., 2015).

Original: Lucas et al.
Revised: Lucas et al.

Original: (2021) gave a deeper study of the monotonic property on the sufficient conditions as well as counter-examples where it does not hold.
Revised: (2021) gave a deeper study of the monotonic property on the sufficient conditions as well as counter-examples where it does not hold.

Original: Vlaar & Frankle (2022) showed that certain hidden layers are more sensitive to the initial model, and the shape of the linear path is not indicative of the generalization performance of the final model.
Revised: Vlaar & Frankle (2022) showed that certain hidden layers are more sensitive to the initial model, and the shape of the linear path is not indicative of the generalization performance of the final model.

Original: (Li et al., 2018) explored visualizing using two random directions and showed that it is important to normalize the filter.
Revised: (Li et al., 2018) explored visualizing using two random directions and showed that it is important to normalize the filter.

Revised: However, taking random directions produces stochastic loss contours.

Revised: It is problematic when we compare models.

Original: We take a different approach and study the loss function in the space of class ego directions , following which parameter update can minimize the training loss for individual classes.
Revised: We take a deterministic approach and study the loss function in the space of class ego directions , following which parameter update can minimize the training loss for individual classes.

Original: The contributions of this paper are as follows.
Revised: The contributions of this paper are as follows.

Original: • Using a metric called CCTM that evaluates class interference on a test set, we show that class interference is the major source of generalization error for deep network classifiers.
Revised: • Using a metric called CCTM that evaluates class interference on a test set, we show that class interference is the major source of generalization error for deep network classifiers.

Original: We show that class interference has a symmetry pattern .
Revised: We show that class interference has a symmetry pattern .

Original: In particular, deep models have a similar amount of trouble in telling “ class A objects are not class B ”, and “ B objects are not A ”.
Revised: In particular, deep models have a similar amount of trouble in telling “ class A objects are not class B ”, and “ B objects are not A ”.

Original: • To understand class interference, we introduce the definitions of class ego directions and interference models.
Revised: • To understand class interference, we introduce the definitions of class ego directions and interference models.

Original: • In the class ego spaces, small learning rates can lead to extremely sharp minima, while learning rate annealing lead to minima that are located at large lowlands, in terrains that are much bigger than the flat minima previously discovered for big learning rates.
Revised: • In the class ego spaces, small learning rates can lead to extremely sharp minima, while learning rate annealing leads to minima that are located at large lowlands, in terrains that are much bigger than the flat minima previously discovered for big learning rates.

Original: • The loss shapes in class ego spaces are indicative of interference.
Revised: • The loss shapes in class ego spaces are indicative of interference.

Original: Classes that share similar loss shapes in other class ego spaces are likely to interfere.
Revised: Classes that share similar loss shapes in other class ego spaces are likely to interfere.

Original: • We show that class interference can also be observed in training.
Revised: • We show that class interference can also be observed in training.

Original: In particular, it can be detected from a special pattern called label dancing , which can be further understood better by plotting the dancing notes during training.
Revised: In particular, it can be detected from a special pattern called label dancing , which can be further understood better by plotting the dancing notes during training.

Original: Dancing notes show interesting interference between classes.
Revised: Dancing notes show interesting interference between classes.

Original: For example, a surprise is that we found FROG interferes CAT for good reasons in the CIFAR-10 data set.
Revised: For example, a surprise is that we found FROG interferes CAT for good reasons in the CIFAR-10 data set.

Original:  C LASS I NTERFERENCE 2. G ENERALIZATION T ESTS AND T HE C LASS I NTERFERENCE P HENOMENON
Revised:  C LASS I NTERFERENCE 2. G ENERALIZATION T ESTS AND T HE C LASS I NTERFERENCE P HENOMENON

Original: Let c 1 and c 2 be class labels.
Revised: Let c 1 and c 2 be class labels.

Original: We use the following cross-class test of generalization , which is the percentage of c 2 predictions for the c 1 objects in the test set:
Revised: We use the following cross-class test of generalization , which is the percentage of c 2 predictions for the c 1 objects in the test set:

Original: [Equation]
Revised: [Equation]

Original: Note this test being an accuracy or error metric depends on whether the two classes are the same or not.
Revised: Note this test being an accuracy or error metric depends on whether the two classes are the same or not.

Original: Calculating the measure for all pairs of classes over the test set gives a matrix.
Revised: Calculating the measure for all pairs of classes over the test set gives a matrix.

Original: We refer to this measure the CCT matrix, and simply the CCTM for short.
Revised: We refer to this measure the CCT matrix, and simply the CCTM for short.

Revised: CCTM extends the confusion matrix in literature by a probability measure, which can be viewed as a combination of the true positive rates and false positive rates in a matrix format 1 .

Revised: This extension facilitates a visualization of the generalization performance as a heat map.

Original: Figure 1 shows the CCTM for VGG19 (Simonyan & Zisserman, 2015) and ResNet18 (He et al., 2015) on the CIFAR-10 (Krizhevsky et al., 2009)
Revised: Figure 1 shows the CCTM for VGG19 (Simonyan & Zisserman, 2015) and ResNet18 (He et al., 2015) on the CIFAR-10 (Krizhevsky et al., 2009)

Original: test set with a heat map.
Revised: test set with a heat map.

Original: Models were trained with
Revised: Models were trained with

Original: SGD (see Section 3 for the training details).
Revised: SGD (see Section 3 for the training details).

Original: From the map, we can see that the most significant generalization errors are from CAT and DOG for both models.
Revised: From the map, we can see that the most significant generalization errors are from CAT and DOG for both models.

Original: This difficulty is not specific to models.
Revised: This difficulty is not specific to models.

Original: It represents class similarity and learning difficulty in data .
Revised: It represents class similarity and learning difficulty in data .

Original: For example, in Table 1, the accuracies in the columns of CAT and DOG are significantly lower than the other columns for all the four deep models.
Revised: For example, in Table 1, the accuracies in the columns of CAT and DOG are significantly lower than the other columns for all the four deep models.

Original: It is also observable that class interference has a symmetry pattern : If a classifier has trouble in recognizing that c 1 objects are not class c 2 , it will also have a hard time in ruling out class c 1 for c 2 objects.
Revised: It is also observable that class interference has a symmetry pattern : If a classifier has trouble in recognizing that c 1 objects are not class c 2 , it will also have a hard time in ruling out class c 1 for c 2 objects.

Original: This can be observed from CAT and DOG in the plotted CCTM.
Revised: This can be observed from CAT and DOG in the plotted CCTM.

Original: We call generalization difficulties of deep neural networks between classes like CAT and DOG the class interference .
Revised: We call generalization difficulties of deep neural networks between classes like CAT and DOG the class interference .

Original: If CCTM ( c 1 , c 2 ) is large, we say that class c 2 interferes c 1 , or class c 1 has interference from c 2 .
Revised: If CCTM ( c 1 , c 2 ) is large, we say that class c 2 interferes c 1 , or class c 1 has interference from c 2 .

Original: Class interference happens when classes are just similar.
Revised: Class interference happens when classes are just similar.

Original: In this case, cats and dogs are hard to recognize for humans as well, especially when the resolution of images is low.
Revised: In this case, cats and dogs are hard to recognize for humans as well, especially when the resolution of images is low.

Original: Examining only the test error would not reveal the class interference phenomenon because it is an overall measure of all classes.
Revised: Examining only the test error would not reveal the class interference phenomenon because it is an overall measure of all classes.

Original: The classes have a much varied difference in their test accuracies.
Revised: The classes have a much varied difference in their test accuracies.

Original: For example, in VGG19, the recall accuracy of CAT, i.e., CCTM ( CAT, CAT ) , is only about 86.7% and DOG recall is about 90.0%.
Revised: For example, in VGG19, the recall accuracy of CAT, i.e., CCTM ( CAT, CAT ) , is only about 84.5% [Table] is and DOG recall is about 89.0%.

Original: For the other classes the recall accuracy is much higher, e.g., SHIP is 96.5%.
Revised: For the other classes the recall accuracy is much higher, e [Table] g., CAR is 96.6%.

Original: As shown in Table 1, ResNet18 (He et al., 2015), GoogleNet (Szegedy et al., 2014) and
Revised: As shown in Table 1, ResNet18 (He et al., 2015), GoogleNet (Szegedy et al., 2014) and

Original: DLA (Yu et al., 2017) have less class interference than VGG19 especially for CAT and DOG.
Revised: DLA (Yu et al., 2017) have less class interference than VGG19 especially for CAT and DOG.

Original: For example, for ResNet18, CCTM ( CAT, CAT ) = 89 .
Revised: For example, for ResNet18, CCTM ( CAT, CAT ) = 86 .

Original: 9% and CCTM ( DOG, DOG ) = [Table]
Revised: 5% and CCTM ( DOG, DOG ) = 92 .

Original: 2. D EFINITIONS
Revised: 2. D EFINITIONS

Original: Let w ∗ be a trained neural network model, e [Table] g., VGG19 or ResNet18.
Revised: Let w ∗ be a trained neural network model, e.g., VGG19 or ResNet18.

Original: We use the following definitions.
Revised: We use the following definitions.

Original: Definition 1 (Interference Model Set) .
Revised: Definition 1 (Interference Model Set) .

Original: Let D c be the samples of class c in a data set.
Revised: Let D c be the samples of class c in a data set.

Original: Define the gradient of class c as the average gradient that is calculated on this set:
Revised: Define the gradient of class c as the average gradient that is calculated on this set:

Original: [Equation]
Revised: [Equation]

Original: Accordingly, there are a set of class gradient directions for the model, { [Equation] [Equation]
Revised: Accordingly, there are a set of class gradient directions for the model, {∇ f ( c ) ( w ∗ ) | c = 1 , 2 , . . .

Original: [Equation] where C is the number of classes.
Revised: , C } , where C is the number of classes.

Original: An ego model of class c is generated by using a scalar α i in the class gradient direction:
Revised: An ego model of class c is generated by using a scalar α i in the class gradient direction:

Original: [Equation]
Revised: [Equation]

Original: The set, M c = { w ( c ) i | i = 1 , . . .
Revised: The set, M c = { w ( c ) i | i = 1 , . . .

Original: , m c } , is the ego model set of class c .
Revised: , m c } , is the ego model set of class c .

Original: The set union, M = ∪ Cc =1 M c , is called the ego model set .
Revised: The set union, M = ∪ Cc =1 M c , is called the ego model set .

Original: This definition is based on that each w ( c ) i is in the direction of minimizing the loss for predicting class c .
Revised: This definition is based on that each w ( c ) i is in the direction of minimizing the loss for predicting class c .

Original: Note that w ( c ) i is a sample of “ego-centric” update, which minimizes the loss for class c only.
Revised: Note that w ( c ) i is a sample of “ego-centric” update, which minimizes the loss for class c only.

Original: It therefore could cause an increase in the prediction errors for the other classes.
Revised: It therefore could cause an increase in the prediction errors for the other classes.

Original: We refer to the gradient of class c as the ego direction of the class.
Revised: We refer to the gradient of class c as the ego direction of the class.

Original: Measuring the loss on the interference models thus tells the interference between classes.
Revised: Measuring the loss on the interference models thus tells the interference between classes.

Original: Definition 2 (Interference Space) .
Revised: Definition 2 (Interference Space) .

Original: The model space { w ( c 1 ,c 2 ) | ( θ 1 , θ 2 ) ∈ Θ 1 × Θ 2 } is called the interference model space of class c 1 and c 2 , where an interference model is defined by
Revised: The model space { w ( c 1 ,c 2 ) | ( θ 1 , θ 2 ) ∈ Θ 1 × Θ 2 } is called the interference model space of class c 1 and c 2 , where an interference model is defined by

Original: [Equation]
Revised: [Equation]

Original: Define F ( c 1 ,c 2 ) =
Revised: Define F ( c 1 ,c 2 ) =

Original: { f ( w ( c 1 ,c 2 ) ) | ( θ 1 , θ 

Revised: { f ( w ( c 1 ,c 2 ) ) | ( θ 1 , θ

Original: ) ∈
Revised: ) ∈

Original: Θ 1 × Θ 2 } , which is the set of interference losses between the two classes .
Revised: Θ 1 × Θ 2 } , which is the set of interference losses between the two classes .

Original: The 3D space, Θ 1 × Θ 2 × F ( c 1 ,c 2 ) , is the loss interference space , or simply, the interference space (of class c 1 and class c 2 for model w ∗ ).
Revised: The 3D space, Θ 1 × Θ 2 × F ( c 1 ,c 2 ) , is the loss interference space , or simply, the interference space (of class c 1 and class c 2 for model w ∗ ).

Original: Proposition 1.
Revised: Proposition 1.

Original: Any interference model is a convex combination of the ego models of the two classes.
Revised: Any interference model is a convex combination of the ego models of the two classes.

Original: Proof.
Revised: Proof.

Original: Let w ( c 1 ) i and w ( c 2 )
Revised: Let w ( c 1 ) i and w ( c 2 )

Original: j be the ego model of class c 1 and c 2 , respectively.
Revised: j be the ego model of class c 1 and c 2 , respectively.

Original: According to their definition,
Revised: According to their definition,

Original: [Equation]
Revised: [Equation]

Original: where setting θ 1 = λα i and θ 2 =
Revised: where setting θ 1 = λα i and θ 2 =

Original: (1 − λ )
Revised: (1 − λ )

Original: α j finishes the proof.
Revised: α j finishes the proof.

Original:  M INIMA : F LAT OR S HARP ?
Revised:  M INIMA : F LAT OR S HARP ?

Original: Our first experiment is to understand minima sharpness of learning rate using class ego directions.
Revised: Our first experiment is to understand minima sharpness of learning rate using class ego directions.

Original: We will visualize in the interference space, Θ 1 × Θ 2 × F ( c 1 ,c 2 ) .
Revised: We will visualize in the interference space, Θ 1 × Θ 2 × F ( c 1 ,c 2 ) .

Original: We use this loss: the mistake rate for the z -axis, which is the percentage of classification mistakes on the training set to give a loss measure in the same range across different plots.
Revised: We use this loss: the mistake rate for the z -axis, which is the percentage of classification mistakes on the training set to give a loss measure in the same range across different plots.

Original: We visualize the loss of the models on the training set versus Θ 1 × Θ 2 , which is a uniform grid over [ − σ, σ ] × [ − σ, σ ] , with 19 points in each direction.
Revised: We visualize the loss of the models on the training set versus Θ 1 × Θ 2 , which is a uniform grid over [ − σ, σ ] × [ − σ, σ ] , with 19 points in each direction.

Original: This gives 361 interference models between a given class pair.
Revised: This gives 361 interference models between a given class pair.

Original: We use the ego directions of CAT-DOG (the most interfering class pair), TRUCK-CAR (with a significant level of interference), and HORSE-SHIP (with little interference).
Revised: We use the ego directions of CAT-DOG (the most interfering class pair), TRUCK-CAR (with a significant level of interference), and HORSE-SHIP (with little interference).

Original: These plots measure how sensitive the training loss changes with respect to the directions that focus on optimizing specially for individual classes and the linear combinations of these directions.
Revised: These plots measure how sensitive the training loss changes with respect to the directions that focus on optimizing specially for individual classes and the linear combinations of these directions.

Original: The center of each plot corresponds to the origin, ( θ 1 = 0 , θ 2 = 0) , at which a trained VGG19 or ResNet is located.
Revised: The center of each plot corresponds to the origin, ( θ 1 = 0 , θ 2 = 0) , at which a trained VGG19 or ResNet is located.

Original: We study the models of VGG19 and ResNet18 trained with the following optimizer setups:
Revised: We study the models of VGG19 and ResNet18 trained with the following optimizer setups:

Original: • small-lr .
Revised: • big-lr .

Original: ThisSGD optimizer uses a small learning rate 0 . 0001 .
Revised: This optimizer uses a big learning rate, 0 . 01 .

Original: It also has a momentum(rate 0 . 9 ) anda weight decay (rate 0 . 0005 )
Revised: The momentum and weight decay are the same as the small-lr optimizer

Original: Figure 2 shows
Revised: Figure 2 shows

Original: for VGG19 (top row) and ResNet18 (bottom row).
Revised: for VGG19 (top row) and ResNet (bottom row).

Original: • big-lr .
Revised: • small-lr .

Original: This  optimizer uses a big learning rate, 0 . 01 .
Revised: This SGD optimizer uses a small learning rate 0 . 0001 .

Original: The momentum  and  weight decay are the same as the small-lr optimizer.
Revised: It also has a momentum (rate 0 . 9 ) and a weight decay (rate 0 . 0005 ).

Original: • anneal-lr .
Revised: • anneal-lr .

Original: Similar to the above optimizers, but with an even bigger (initial) learning rate.
Revised: Similar to the above optimizers, but with an even bigger (initial) learning rate.

Original: A big constant learning rate 0 .
Revised: A big constant learning rate 0 .

Original: 1 leads to oscillatory training loss and poor models.
Revised: 1 leads to oscillatory training loss and poor models.

Original: We thus decay it with an initial value of 0 .
Revised: We thus decay it with an initial value of 0 .

Original: 1 using a Cosine rule (Loshchilov
Revised: 1 using a Cosine rule (Loshchilov

Original: & Hutter,
Revised: & Hutter,

Original: This is the optimizer setup used to train the models in Section 2.1.
Revised: This is the optimizer setup used to train the models in Section 2.1.

Original: The input images are transformed with RandomCrop and RandomHorizontalFlip and normalization.
Revised: The input images are transformed with RandomCrop and RandomHorizontalFlip and normalization.

Original: The batch size is 128.
Revised: The batch size is 128.

Original: The Cross Entropy loss is used.
Revised: The Cross Entropy loss is used.

Original: Each model is trained with 200 epochs.
Revised: Each model is trained with 200 epochs.

Original: The test accuracies for the models are shown in the following table.
Revised: The test accuracies for the models are shown in the following table.

Original: [Table]
Revised: [Table]

Original: This confirms that big learning rates generalize better than small ones as discovered by the community.
Revised: This confirms that big learning rates generalize better than small ones as discovered by the community.

Original: Interestingly, the anneal learning rate leads to models that generalize even much better, for which there has been no explanation to the best of our knowledge.
Revised: Interestingly, the anneal learning rate leads to models that generalize even much better, for which there has been no explanation to the best of our knowledge.

Original: Let’s first take a look at VGG19 trained with big-lr, whose interference spaces are shown at the top row of Figure 2.
Revised: Let’s first take a look at VGG19 trained with big-lr, whose interference spaces are shown at the top row of Figure 2.

Original: The loss exhibits strong sharpness in the CAT-DOG ego visualization.
Revised: The loss exhibits strong sharpness in the CAT-DOG ego visualization.

Original: From the minimum (the trained VGG19 at the center), a small step of optimizing the CAT predictions easily deteriorates the loss, in particular the red flat plateau corresponds to an accuracy on the training set down to merely 10% .
Revised: From the minimum (the trained VGG19 at the center), a small step of optimizing the CAT predictions easily deteriorates the loss, in particular the red flat plateau corresponds to an accuracy on the training set down to merely 10% .

Original: The loss change is extremely sensitive in the CAT ego direction.
Revised: The loss change is extremely sensitive in the CAT ego direction.

Original: It is similarly sensitive in all directions except near the DOG ego direction, which looks still very sensitive.
Revised: It is similarly sensitive in all directions except near the DOG ego direction, which looks still very sensitive.

Original: According to Proposition 1, any interference model in this space is a convex combination of a CAT ego model and a DOG ego model.
Revised: According to Proposition 1, any interference model in this space is a convex combination of a CAT ego model and a DOG ego model.

Original: This plot thus shows that the CAT ego is very influential even the weight of the DOG ego is large.
Revised: This plot thus shows that the CAT ego is very influential even the weight of the DOG ego is large.

Original: The visualizations in the CAR-TRUCK and HORSE-SHIP ego spaces show that the loss changes much less sensitively than for CAT and DOG when we update the model for the purpose of improving or even sacrificing the prediction accuracy of the four classes.
Revised: The visualizations in the CAR-TRUCK and HORSE-SHIP ego spaces show that the loss changes much less sensitively than for CAT and DOG when we update the model for the purpose of improving or even sacrificing the prediction accuracy of the four classes.

Original: However, close to the directions of
Revised: However, close to the directions of

Original: TRUCK ego plus negative CAR ego, and negative TRUCK ego plus CAR ego, the loss also changes abruptly.
Revised: TRUCK ego plus negative CAR ego, and negative TRUCK ego plus CAR ego, the loss also changes abruptly.

Original: If we cut the loss surface 135 degrees in the x - y axis, we end up getting a minimum that looks sharp.
Revised: If we cut the loss surface 135 degrees in the x - y axis, we end up getting a minimum that looks sharp.

Original: On the other hand, a random cut likely renders a less sharp or even flat look of the minimum.
Revised: On the other hand, a random cut likely renders a less sharp or even flat look of the minimum.

Original: The case of HORSE-SHIP is similar.
Revised: The case of HORSE-SHIP is similar.

Original: Thus whether the minimum looks flat or sharp is dependent on how the loss contour is cut.
Revised: Thus whether the minimum looks flat or sharp is dependent on how the loss contour is cut.

Original: Some care needs to be taken when we discuss minima sharpness, especially the space in which the loss is plotted.
Revised: Some care needs to be taken when we discuss minima sharpness, especially the space in which the loss is plotted.

Original: Most previous discussions on minima sharpness are based on the difference between an initial model and a trained model, or two random directions.
Revised: Most previous discussions on minima sharpness are based on the difference between an initial model and a trained model, or two random directions.

Original: Both methods have randomization effects and yet they get descent loss contours.
Revised: Both methods have randomization effects and yet they get descent loss contours.

Original: While it is amazing, the reason why random cuts render reflective loss contours is unclear.
Revised: While it is amazing, the reason why random cuts render reflective loss contours is unclear.

Original: Our guess is that most directions renders sharpness and sampling a random one is likely fine.
Revised: Our guess is that most directions renders sharpness and sampling a random one is likely fine.

Original: However, when we compare the levels of sharpness between models, random cuts may not be accurate.
Revised: However, when we compare the levels of sharpness between models, random cuts may not be accurate.

Original: Figure 2 bottom row shows for ResNet18 optimized with the big-lr optimizer.
Revised: Figure 2 bottom row shows for ResNet18 optimized with the big-lr optimizer.

Original: The loss change near the minimum is also extremely sensitive in the CAT-DOG ego space.
Revised: The loss change near the minimum is also extremely sensitive in the CAT-DOG ego space.

Original: Interestingly, for ResNet18, the loss in the DOG ego direction is more sensitive than in the CAT direction.
Revised: Interestingly, for ResNet18, the loss in the DOG ego direction is more sensitive than in the CAT direction.

Original: This seems a “transposed” effect of VGG19, because the influence of the DOG ego is stronger on the loss now.
Revised: This seems a “transposed” effect of VGG19, because the influence of the DOG ego is stronger on the loss now.

Original: For both VGG and ResNet18, the loss visualized in the CAT-DOG ego space has a clear narrow valley structure near the minimum.
Revised: For both VGG and ResNet18, the loss visualized in the CAT-DOG ego space has a clear narrow valley structure near the minimum.

Original: This kind of loss functions are known to be very challenging for gradient descent, e.g., see the Rosenbrock function also known as the Banana function (Rosenbrock, 1960).
Revised: This kind of loss functions are known to be very challenging for gradient descent, e.g., see the Rosenbrock function also known as the Banana function (Rosenbrock, 1960).

Original: In the
Revised: In the

Original: CAR-TRUCK space, the loss of ResNet18 is much less curvy up than that of VGG19.
Revised: CAR-TRUCK space, the loss of ResNet18 is much less curvy up than that of VGG19.

Original: In particular, for VGG19 it is sensitive in both the ego directions, while for ResNet18, only near the direction about 135 degrees ( x - y axis)
Revised: In particular, for VGG19 it is sensitive in both the ego directions, while for ResNet18, only near the direction about 135 degrees ( x - y axis)

Original: it is sensitive.
Revised: it is sensitive.

Original: For VGG19, the SHIP direction has lots of sensitivity.
Revised: For VGG19, the SHIP direction has lots of sensitivity.

Original: For ResNet18, the HORSE direction instead is more sensitive.
Revised: For ResNet18, the HORSE direction instead is more sensitive.

Original: Our results show the minima being flat or sharp is dependent on what spaces the loss is illustrated.
Revised: Our results show the minima being flat or sharp is dependent on what spaces the loss is illustrated.

Original: We think a better way of discussing generalization is the area of flatness around the minima in critical directions.
Revised: We think a better way of discussing generalization is the area of flatness around the minima in critical directions.

Original: Our plots in different class ego spaces show that a minimum can be a flat minimum in certain visualization spaces (e.g., ResNet18 in the CAR-TRUCK ego space), while at the same time it can look very sharp in other spaces (e.g., ResNet18 in the CAT-DOG space).
Revised: Our plots in different class ego spaces show that a minimum can be a flat minimum in certain visualization spaces (e.g., ResNet18 in the CAR-TRUCK ego space), while at the same time it can look very sharp in other spaces (e.g., ResNet18 in the CAT-DOG space).

Original: Figure 3 shows the small learning rate.
Revised: Figure 3 shows the small learning rate.

Original: This time ResNet18 is an extremely sharp minimum in all the three ego spaces.
Revised: This time ResNet18 is an extremely sharp minimum in all the three ego spaces.

Original: In a small area around the minimum in ego spaces, the loss changes dramatically.
Revised: In a small area around the minimum in ego spaces, the loss changes dramatically.

Original: Beyond that small area, the loss is invariantly high (plateau).
Revised: Beyond that small area, the loss is invariantly high (plateau).

Original: VGG19, instead, has a more smooth change of loss in a small area although in the CAT ego direction the loss changes abruptly too (which forms a cliff).
Revised: VGG19, instead, has a more smooth change of loss in a small area although in the CAT ego direction the loss changes abruptly too (which forms a cliff).

Original: This shows when the learning rate is small, the loss contour can be near non-smooth and sharp minima do not necessarily generalize worse (comparing to VGG19).
Revised: This shows when the learning rate is small, the loss contour can be near non-smooth and sharp minima do not necessarily generalize worse (comparing to VGG19).

Original: This confirms the findings by Dinh et al.
Revised: This confirms the findings by Dinh et al.

Original: and (Li et al., 2018) that there exist models that are sharp minima
Revised: and (Li et al., 2018) that there exist models that are sharp minima

Original: and yet they still generalize well.
Revised: and yet they still generalize well.

Original: In particular, ResNet18 has a better generalization than VGG19, 86.88% versus 84.99% in this case.
Revised: In particular, ResNet18 has a better generalization than VGG19, 86.88% versus 84.99% in this case.

Original: Our results show that flat minima generalize better when the learning rate is well tuned (not too small).
Revised: Our results show that flat minima generalize better when the learning rate is well tuned (not too small).

Original: However, when the learning rate is small, the minima can be sharp and they can generalize even better than less sharp ones.
Revised: However, when the learning rate is small, the minima can be sharp and they can generalize even better than less sharp ones.

Original: Finally, Figure 4 shows for the models optimized with learning rate annealing.
Revised: Finally, Figure 4 shows for the models optimized with learning rate annealing.

Original: These two models have superior generalization, with 93.87% for VGG19 and 95.15% for ResNet18.
Revised: These two models have superior generalization, with 93.87% for VGG19 and 95.15% for ResNet18.

Original: The visualization in the ego spaces show that the area of flatness is very large, especially ResNet18.
Revised: The visualization in the ego spaces show that the area of flatness is very large, especially ResNet18.

Original: Comparing to a fixed big learning rate, the models trained by annealing have a much higher level of in-sensitiveness to parameter changes in the class ego directions.
Revised: Comparing to a fixed big learning rate, the models trained by annealing have a much higher level of in-sensitiveness to parameter changes in the class ego directions.

Original: Presumably, the big initial learning rate helps establish a larger flat area.
Revised: Presumably, the big initial learning rate helps establish a larger flat area.

Original: This level of flatness has not been observed before, especially in previous experiments of learning rates.
Revised: This level of flatness has not been observed before, especially in previous experiments of learning rates.

Original: We may thus refer to minima located in a large flat terrain the lowland minima .
Revised: We may thus refer to minima located in a large flat terrain the lowland minima .

Original:  A NALYZING C LASS I NTERFERENCE 4. I NTERFERENCE FROM O NE C LASS TO THE O THERS
Revised:  A NALYZING C LASS I NTERFERENCE 4. I NTERFERENCE FROM O NE C LASS TO THE O THERS

Original: We also would like to understand the interference from one class to the others for a trained model.
Revised: We also would like to understand the interference from one class to the others for a trained model.

Original: Figure 5 shows the interference of CAT, DOG, CAR and TRUCK to all the classes.
Revised: Figure 5 shows the interference of CAT, DOG, CAR and TRUCK to all the classes.

Original: First let’s look at the CAT loss in the CAT-DOG space (first plot).
Revised: First let’s look at the CAT loss in the CAT-DOG space (first plot).

Original: It shows CAT loss increases in the cat ego direction, i.e., the gradient ascent direction, which is intuitive.
Revised: It shows CAT loss increases in the cat ego direction, i.e., the gradient ascent direction, which is intuitive.

Original: It also shows the CAT loss increases most when we minimize the DOG loss.
Revised: It also shows the CAT loss increases most when we minimize the DOG loss.

Original: This is another verification that DOG interferes CAT.
Revised: This is another verification that DOG interferes CAT.

Original: Interestingly, following the joint direction of gradient ascent directions to maximize the CAT loss and the DOG loss doesn’t increase the CAT loss much.
Revised: Interestingly, following the joint direction of gradient ascent directions to maximize the CAT loss and the DOG loss doesn’t increase the CAT loss much.

Original: In the case of CAR loss in the CAR-TRUCK space, the situation is a little different.
Revised: In the case of CAR loss in the CAR-TRUCK space, the situation is a little different.

Original: In particular, CAR loss increases significantly whether we follow the gradient descent or ascent direction of TRUCK as long as we move in the ascent direction of CAR.
Revised: In particular, CAR loss increases significantly whether we follow the gradient descent or ascent direction of TRUCK as long as we move in the ascent direction of CAR.

Original: TRUCK loss is more complicated.
Revised: TRUCK loss is more complicated.

Original: The loss increases in the joint direction of ascent directions of
Revised: The loss increases in the joint direction of ascent directions of

Original: CAR and TRUCK losses.
Revised: CAR and TRUCK losses.

Original: In addition, TRUCK loss also increases if we follow the the descent direction of CAR.
Revised: In addition, TRUCK loss also increases if we follow the the descent direction of CAR.

Original: This means minimizing the CAR loss has the effect of increasing the TRUCK loss.
Revised: This means minimizing the CAR loss has the effect of increasing the TRUCK loss.

Original: This is also a sign that CAR and TRUCK interferes.
Revised: This is also a sign that CAR and TRUCK interferes.

Original: For the other classes, their prediction losses respond more sensitively to the ego directions of CAR-TRUCK than those of CAT-DOG.
Revised: For the other classes, their prediction losses respond more sensitively to the ego directions of CAR-TRUCK than those of CAT-DOG.

Original: In the CAT-DOG space, CAR, TRUCK, PLANE, and SHIP all increase their losses in one same corner.
Revised: In the CAT-DOG space, CAR, TRUCK, PLANE, and SHIP all increase their losses in one same corner.

Original: HORSE and DEER losses both increase as we get closer to the corner where DOG loss increases; in addition, the increase of HORSE is more than DEER in this process.
Revised: HORSE and DEER losses both increase as we get closer to the corner where DOG loss increases; in addition, the increase of HORSE is more than DEER in this process.

Original: In the CAR-TRUCK space, CAT, FROG, DOG, and DEER losses have very similar shapes.
Revised: In the CAR-TRUCK space, CAT, FROG, DOG, and DEER losses have very similar shapes.

Original: This suggests these losses increase in roughly the same directions in the CAR-TRUCK space.
Revised: This suggests these losses increase in roughly the same directions in the CAR-TRUCK space.

Original: HORSE’s loss shape is also similar to these four classes, but the similarity is less.
Revised: HORSE’s loss shape is also similar to these four classes, but the similarity is less.

Original: CAR and PLANE losses have very similar shapes.
Revised: CAR and PLANE losses have very similar shapes.

Original: TRUCK and SHIP losses have a similar wing-like structure too.
Revised: TRUCK and SHIP losses have a similar wing-like structure too.

Original: CAR, PLANE,
Revised: CAR, PLANE,

Original: TRUCK and SHIP have similar loss shapes on the left side of the plots shown.
Revised: TRUCK and SHIP have similar loss shapes on the left side of the plots shown.

Original: These observations suggest that loss shapes in class ego spaces are indicative of interference.
Revised: These observations suggest that loss shapes in class ego spaces are indicative of interference.

Original: Classes that share similar loss shapes in other class ego spaces are likely to interfere.
Revised: Classes that share similar loss shapes in other class ego spaces are likely to interfere.

Original: This is going to be discussed further in the next experiment.
Revised: This is going to be discussed further in the next experiment.

Original: 4. C LASS I NTERFERENCE IN T RAINING
Revised: 4. C LASS I NTERFERENCE IN T RAINING

Original: The above experiments are for a trained model.
Revised: The above experiments are for a trained model.

Original: We were wondering whether class interference can be observed in training.
Revised: We were wondering whether class interference can be observed in training.

Original: To study this, we plot the per-class training accuracy which is the recall rate for each class.
Revised: To study this, we plot the per-class training accuracy which is the recall rate for each class.

Original: Figure 6 shows for CAT and DOG.
Revised: Figure 6 shows for CAT and DOG.

Original: The two recall rates are both highly oscillatory, especially in the beginning stage of training.
Revised: The two recall rates are both highly oscillatory, especially in the beginning stage of training.

Original: Importantly, there are many moments that one rate being high while the other being low at the same time, which we call label dance or CATDOG dance for this particular case.
Revised: Importantly, there are many moments that one rate being high while the other being low at the same time, which we call label dance or CATDOG dance for this particular case.

Original: This dancing pattern is a strong indicator that CAT and DOG interfere.
Revised: This dancing pattern is a strong indicator that CAT and DOG interfere.

Original: To further confirm this, we plot in the same figure the row of the CCTM for the training set that correspond to CAT, i.e., CCTM ( CAT, c ) , for each non-CAT class c , during the same training process.
Revised: To further confirm this, we plot in the same figure the row of the CCTM for the training set that correspond to CAT, i.e., CCTM ( CAT, c ) , for each non-CAT class c , during the same training process.

Original: As the caption of the figure shows, a rise in the DOG recall rate is often caused by a high interference of DOG to CAT.
Revised: As the caption of the figure shows, a rise in the DOG recall rate is often caused by a high interference of DOG to CAT.

Original: After some (about 118) epochs, DOG interference dominates CAT predictions errors and eventually weeds out following the other classes.
Revised: After some (about 118) epochs, DOG interference dominates CAT predictions errors and eventually weeds out following the other classes.

Original: In this phase of training (as circled in the figure), the recall rates of CAT and DOG are highly symmetric to each other (horizontally), further indicating that DOG interference is the major source of error in predicting cats and vice versa.
Revised: In this phase of training (as circled in the figure), the recall rates of CAT and DOG are highly symmetric to each other (horizontally), further indicating that DOG interference is the major source of error in predicting cats and vice versa.

Original: Figure 7 plots the “argmax” operation of the CCTM for the rows corresponding to four classes at each epoch, excluding the diagonal part.
Revised: Figure 7 plots the “argmax” operation of the CCTM for the rows corresponding to four classes at each epoch, excluding the diagonal part.

Original: The plot looks similar to music notes.
Revised: The plot looks similar to music notes.

Original: So we term this plot “dancing notes”.
Revised: So we term this plot “dancing notes”.

Original: For DOG notes, there are many pink markers at the line y = 3 , which is the class label corresponding to CAT.
Revised: For DOG notes, there are many pink markers at the line y = 3 , which is the class label corresponding to CAT.

Original: The stretched markers laying continuously is a clear sign of CAT interference to DOG.
Revised: The stretched markers laying continuously is a clear sign of CAT interference to DOG.

Original: In the CAT notes, continual red crosses also persist at y = 5 , which is the class label of DOG, showing interference of DOG to CAT.
Revised: In the CAT notes, continual red crosses also persist at y = 5 , which is the class label of DOG, showing interference of DOG to CAT.

Original: It also shows that CAT interference to DOG persists longer than the other way.
Revised: It also shows that CAT interference to DOG persists longer than the other way.

Original: For a better presentation of the results, we plot y =
Revised: For a better presentation of the results, we plot y =

Original: − 2 if no class interferes more than 0 .
Revised: − 2 if no class interferes more than 0 .

Original: The notes of CAR (class label 1) and TRUCK (class label 9) show similar duration of interference, and it appears the interference from CAT to TRUCK seems to have a close strength to the other way around.
Revised: The notes of CAR (class label 1) and TRUCK (class label 9) show similar duration of interference, and it appears the interference from CAT to TRUCK seems to have a close strength to the other way around.

Original: It is also interesting to observe that both CAR and TRUCK have interference from class labels y
Revised: It is also interesting to observe that both CAR and TRUCK have interference from class labels y

Original: = 

Revised: =

Original: and y = 8 , which correspond to PLANE and SHIP.
Revised: and y = 8 , which correspond to PLANE and SHIP.

Original: This is intuitive because these are all human made metallic crafts.
Revised: This is intuitive because these are all human made metallic crafts.

Original: It appears that the interference from PLANE to TRUCK is more often than to CAR, probably because trucks are bigger in size than cars.
Revised: It appears that the interference from PLANE to TRUCK is more often than to CAR, probably because trucks are bigger in size than cars.

Original: CAT has interference from BIRD (2) given their similar fluffy looks.
Revised: CAT has interference from BIRD (2) given their similar fluffy looks.

Original: Surprisingly, FROG (6) also interferes CAT pretty often.
Revised: Surprisingly, FROG (6) also interferes CAT pretty often.

Original: We checked the CIFAR-10 images visually and it is probably because the images are mostly close looks of the objects; in this case cats have two pointy ears which are easily confused with frogs who have their eyes positioned atop.
Revised: We checked the CIFAR-10 images visually and it is probably because the images are mostly close looks of the objects; in this case cats have two pointy ears which are easily confused with frogs who have their eyes positioned atop.

Original: Besides CAT, DOG has interference from HORSE (7) and DEER (4) because they are all four-legged.
Revised: Besides CAT, DOG has interference from HORSE (7) and DEER (4) because they are all four-legged.

Original: It is interesting to observe that
Revised: It is interesting to observe that

Original: CAT, on the other hand, almost does not have interference from HORSE, with only two or three moments of interference out of 200 epochs.
Revised: CAT, on the other hand, almost does not have interference from HORSE, with only two or three moments of interference out of 200 epochs.

Original: This means HORSE is very helpful to differentiate between CAT and DOG, which is the largest source of generalization error as we discussed in Section 2.1.
Revised: This means HORSE is very helpful to differentiate between CAT and DOG, which is the largest source of generalization error as we discussed in Section 2.1.

Original: DOG also has a little interference from BIRD (2) similar to CAT does.
Revised: DOG also has a little interference from BIRD (2) similar to CAT does.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: This paper illustrates a phenomenon called class interference of deep neural networks.
Revised: This paper illustrates a phenomenon called class interference of deep neural networks.

Original: We show it is the bottleneck of classification, which represents learning difficulty in data.
Revised: We show it is the bottleneck of classification, which represents learning difficulty in data.

Original: The proposed cross-class generalization tests, class ego directions, interference models and the study of class-wise losses in class ego directions provide a tool set for studying the generalization of trained deep neural networks.
Revised: The proposed cross-class generalization tests, class ego directions, interference models and the study of class-wise losses in class ego directions provide a tool set for studying the generalization of trained deep neural networks.

Original: The study of label dancing via the dancing notes provides a method of detecting class interference during training.
Revised: The study of label dancing via the dancing notes provides a method of detecting class interference during training.

Original: With the provided tools in these two dimensions, we hope this paper is useful to understand the generalization of deep nets, improve existing models and training methods, and understand the data better as well as the learning difficulty of recognition. 
Revised: With the provided tools in these two dimensions, we hope this paper is useful to understand the generalization of deep nets, improve existing models and training methods, and understand the data better as well as the learning difficulty of recognition. A PPENDIX

Revised: The CCTM heatmaps of GoogleNet and DLA are shown in Figure 8.

Revised: The interference between CAT and DOG is similarly high (see Figure 1 for VGG19 and ResNet18).