Original: A RE N EURONS A CTUALLY C OLLAPSED ? O N THE F INEG RAINED S TRUCTURE IN N EURAL R EPRESENTATIONS
Revised: A RE N EURONS A CTUALLY C OLLAPSED ? O N THE F INEG RAINED S TRUCTURE IN N EURAL R EPRESENTATIONS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Recent work has observed an intriguing “Neural Collapse” phenomenon in welltrained neural networks, where the last-layer representations of training samples with the same label collapse into each other.
Revised: Recent work has observed an intriguing “Neural Collapse” phenomenon in welltrained neural networks, where the last-layer representations of training samples with the same label collapse into each other.

Original: This suggests that the last-layer representations are completely determined by the labels, and do not depend on the intrinsic structure of input distribution.
Revised: This suggests that the last-layer representations are completely determined by the labels, and do not depend on the intrinsic structure of input distribution.

Original: We provide evidence that this is not a complete description, and that the apparent collapse hides important fine-grained structure in the representations.
Revised: We provide evidence that this is not a complete description, and that the apparent collapse hides important fine-grained structure in the representations.

Original: Specifically, even when representations apparently collapse, the small amount of remaining variation can still faithfully and accurately captures the intrinsic structure of input distribution.
Revised: Specifically, even when representations apparently collapse, the small amount of remaining variation can still faithfully and accurately captures the intrinsic structure of input distribution.

Original: As an example, if we train on CIFAR-10 using only 5 coarse-grained labels (by combining two classes into one super-class) until convergence, we can reconstruct the original 10-class labels from the learned representations via unsupervised clustering.
Revised: As an example, if we train on CIFAR-10 using only 5 coarse-grained labels (by combining two classes into one super-class) until convergence, we can reconstruct the original 10-class labels from the learned representations via unsupervised clustering.

Original: The reconstructed labels achieve 93% accuracy on the CIFAR-10 test set, nearly matching the normal
Revised: The reconstructed labels achieve 93% accuracy on the CIFAR-10 test set, nearly matching the normal

Original: CIFAR-10 accuracy for the same architecture.
Revised: CIFAR-10 accuracy for the same architecture.

Original: Our findings show concretely how the structure of input data can play a significant role in determining the fine-grained structure of neural representations, going beyond what Neural Collapse predicts.
Revised: Our findings show concretely how the structure of input data can play a significant role in determining the fine-grained structure of neural representations, going beyond what Neural Collapse predicts.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Much of the success of deep neural networks has, arguably, been attributed to their ability to learn useful representations , or features , of the data (Rumelhart et al., 1985).
Revised: Much of the success of deep neural networks has, arguably, been attributed to their ability to learn useful representations , or features , of the data (Rumelhart et al., 1985).

Original: Although neural networks are often trained to optimize a single objective function with no explicit requirements on the inner representations, there is ample evidence suggesting that these learned representations contain rich information about the input data (Levy & Goldberg, 2014; Olah et al., 2017).
Revised: Although neural networks are often trained to optimize a single objective function with no explicit requirements on the inner representations, there is ample evidence suggesting that these learned representations contain rich information about the input data (Levy & Goldberg, 2014; Olah et al., 2017).

Original: As a result, formally characterizing and understanding the structural properties of neural representations is of great theoretical and practical interest, and can provide insights on how deep learning works and how to make better use of these representations.
Revised: As a result, formally characterizing and understanding the structural properties of neural representations is of great theoretical and practical interest, and can provide insights on how deep learning works and how to make better use of these representations.

Original: One intriguing phenomenon recently discovered by Papyan et al.
Revised: One intriguing phenomenon recently discovered by Papyan et al.

Original: (2020) is Neural Collapse , which identifies structural properties of last-layer representations during the terminal phase of training (i.e. after zero training error is reached).
Revised: (2020) is Neural Collapse , which identifies structural properties of last-layer representations during the terminal phase of training (i.e. after zero training error is reached).

Original: The simplest of these properties is that the last-layer representations for training samples with the same label collapse into a single point, which is referred to as “variability collapse (NC1).”
Revised: The simplest of these properties is that the last-layer representations for training samples with the same label collapse into a single point, which is referred to as “variability collapse (NC1).”

Original: This is surprising, since the collapsed structure is not necessary to achieve small training or test error, yet it arises consistently in standard architectures trained on standard classification datasets.
Revised: This is surprising, since the collapsed structure is not necessary to achieve small training or test error, yet it arises consistently in standard architectures trained on standard classification datasets.

Original: A series of recent papers were able to theoretically explain Neural Collapse under a simplified model called the unconstrained feature model or layer-peeled model (see Section 2 for a list of references).
Revised: A series of recent papers were able to theoretically explain Neural Collapse under a simplified model called the unconstrained feature model or layer-peeled model (see Section 2 for a list of references).

Original: In this model, the last-layer representation of each training sample is treated as a free optimization variable and therefore the training loss essentially has the form of a matrix factorization.
Revised: In this model, the last-layer representation of each training sample is treated as a free optimization variable and therefore the training loss essentially has the form of a matrix factorization.

Original: Under a variety of different setups, it was proved that the solution to this simplified problem should satisfy
Revised: Under a variety of different setups, it was proved that the solution to this simplified problem should satisfy

Original: Neural Collapse.
Revised: Neural Collapse.

Original: Although Neural Collapse is relatively well understood in this simplified model, this model completely ignores the role of the input data because the loss function is independent of the input data.
Revised: Although Neural Collapse is relatively well understood in this simplified model, this model completely ignores the role of the input data because the loss function is independent of the input data.

Original: Conceptually, this suggests that Neural Collapse is only determined by the labels and may happen regardless of the input data distribution.
Revised: Conceptually, this suggests that Neural Collapse is only determined by the labels and may happen regardless of the input data distribution.

Original: Zhu et al.
Revised: Zhu et al.

Original: (2021) provided further empirical support of this claim via a random labeling experiment.
Revised: (2021) provided further empirical support of this claim via a random labeling experiment.

Original: On the other hand, it is conceivable that the intrinsic structure of the input distribution should play a role in determining the structure of neural net representations.
Revised: On the other hand, it is conceivable that the intrinsic structure of the input distribution should play a role in determining the structure of neural net representations.

Original: For example, if a class contains a heterogeneous set of input data (such as different subclasses), it is possible that their heterogeneity is also respected in their feature representations (Sohoni et al., 2020).
Revised: For example, if a class contains a heterogeneous set of input data (such as different subclasses), it is possible that their heterogeneity is also respected in their feature representations (Sohoni et al., 2020).

Original: However, this appears to contradict Neural Collapse, because Neural Collapse would predict that all the representations collapse into the same point as long as they have the same class label.
Revised: However, this appears to contradict Neural Collapse, because Neural Collapse would predict that all the representations collapse into the same point as long as they have the same class label.

Original: This dilemma motivates us to study the following main question in this paper:
Revised: This dilemma motivates us to study the following main question in this paper:

Original: How can we reconcile the roles of the intrinsic structure of input distribution vs. the explicit structure of the labels in determining the last-layer representations in neural networks?
Revised: How can we reconcile the roles of the intrinsic structure of input distribution vs. the explicit structure of the labels in determining the last-layer representations in neural networks?

Original: Our methodology and findings.
Revised: Our methodology and findings.

Original: To study the above question, we design experiments to manually create a mismatch between the intrinsic structure of the input distribution and the explicit labels provided for training in standard classification datasets and measure how the last-layer representations behave in response to our interventions.
Revised: To study the above question, we design experiments to manually create a mismatch between the intrinsic structure of the input distribution and the explicit labels provided for training in standard classification datasets and measure how the last-layer representations behave in response to our interventions.

Original: This allows us to isolate the effect of the input distribution from the effect of labels.
Revised: This allows us to isolate the effect of the input distribution from the effect of labels.

Original: As an illustrative example, for the CIFAR-10 dataset (a 10-class classification task), we alter its labels in two different ways, resulting in a coarsely-labeled and finely-labeled version:
Revised: As an illustrative example, for the CIFAR-10 dataset (a 10-class classification task), we alter its labels in two different ways, resulting in a coarsely-labeled and finely-labeled version:

Original: • Coarse CIFAR-10: combine every two class labels into one and obtain a 5-class task (see Figure 2 for an
Revised: • Coarse CIFAR-10: combine every two class labels into one and obtain a 5-class task (see Figure 2 for an

Original: illustration);
Revised: illustration);

Original: • Fine CIFAR-10: split every class label randomly into two labels and obtain a 20-class task.
Revised: • Fine CIFAR-10: split every class label randomly into two labels and obtain a 20-class task.

Original: We
Revised: We

Original: train standard
Revised: train standard

Original: network architectures
Revised: network architectures

Original: (e.g. ResNet, DenseNet) using SGD on these altered datasets.
Revised: (e.g. ResNet, DenseNet) using SGD on these altered datasets.

Original: Our main findings are summarized below.
Revised: Our main findings are summarized below.

Original: First, both the intrinsic structure of the input distribution and the explicit labels provided in training clearly affect the structure of the last-layer representations.
Revised: First, both the intrinsic structure of the input distribution and the explicit labels provided in training clearly affect the structure of the last-layer representations.

Original: The effect of input distribution emerges earlier in training, while the effect of labels appears at a later stage.
Revised: The effect of input distribution emerges earlier in training, while the effect of labels appears at a later stage.

Original: For example, for both Coarse
Revised: For example, for both Coarse

Original: CIFAR-10 and Fine CIFAR-10, at some point the representations naturally form 10 clusters according to the original CIFAR-10 labels (which comes from the intrinsic input structure), even though 5 ordifferent labels are provided for training.
Revised: CIFAR-10 and Fine CIFAR-10, at some point the representations naturally form 10 clusters according to the original CIFAR-10 labels (which comes from the intrinsic input structure), even though 5 ordifferent labels are provided for training.

Original: Later in training (after 100% training accuracy is reached), the representations collapse into 5 or 20 clusters driven by the explicit labels provided, as predicted by Neural Collapse.
Revised: Later in training (after 100% training accuracy is reached), the representations collapse into 5 or 20 clusters driven by the explicit labels provided, as predicted by Neural Collapse.

Original: Second, even after Neural Collapse has occurred according to the explicit label information, the seemingly collapsed representations corresponding to each label can still exhibit fine-grained structures determined by the input distribution.
Revised: Second, even after Neural Collapse has occurred according to the explicit label information, the seemingly collapsed representations corresponding to each label can still exhibit fine-grained structures determined by the input distribution.

Original: As an illustration, Figure 1 visualizes the representations from the last epoch of training a ResNet-18 on Coarse CIFAR-10.
Revised: As an illustration, Figure 1 visualizes the representations from the last epoch of training a ResNet-18 on Coarse CIFAR-10.

Original: While globally there are 5 separated clusters as predicted by Neural Collapse, if we zoom in on each cluster, it clearly consists of two subclusters which correspond to the original CIFAR-10 classes.
Revised: While globally there are 5 separated clusters as predicted by Neural Collapse, if we zoom in on each cluster, it clearly consists of two subclusters which correspond to the original CIFAR-10 classes.

Original: We also find that this phenomenon persists even after a very long training period (e.g. 1,000 epochs), indicating that the effect of input distribution is not destroyed by that of the labels, at least not within a normal training budget.
Revised: We also find that this phenomenon persists even after a very long training period (e.g. 1,000 epochs), indicating that the effect of input distribution is not destroyed by that of the labels, at least not within a normal training budget.

Original: To further validate our finding that significant input information is present in the last-layer representations despite Neural Collapse, we perform a simple Cluster-and-Linear-Probe (CLP) procedure on the representations from ResNet-18 trained on Coarse CIFAR-10, in which we use an unsupervised clustering method to reconstruct the original labels, and then train a linear classifier on top of these representations using the reconstructed labels.
Revised: To further validate our finding that significant input information is present in the last-layer representations despite Neural Collapse, we perform a simple Cluster-and-Linear-Probe (CLP) procedure on the representations from ResNet-18 trained on Coarse CIFAR-10, in which we use an unsupervised clustering method to reconstruct the original labels, and then train a linear classifier on top of these representations using the reconstructed labels.

Original: We find that CLP can achieve > 93% accuracy on the original CIFAR-10 test set, matching the standard accuracy of ResNet-18, even though only 5 coarse labels are provided the entire time.
Revised: We find that CLP can achieve > 93% accuracy on the original CIFAR-10 test set, matching the standard accuracy of ResNet-18, even though only 5 coarse labels are provided the entire time.

Original: Takeaway.
Revised: Takeaway.

Original: While Neural Collapse is an intriguing phenomenon that consistently happens in welltrained neural networks, we provide concrete evidence showing that it is not the most comprehensive description of the behavior of last-layer representations in practice, as it fails to capture the possible fine-grained properties determined by the intrinsic structure of the input distribution.
Revised: While Neural Collapse is an intriguing phenomenon that consistently happens in welltrained neural networks, we provide concrete evidence showing that it is not the most comprehensive description of the behavior of last-layer representations in practice, as it fails to capture the possible fine-grained properties determined by the intrinsic structure of the input distribution.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: The Neural Collapse phenomenon was originally discovered by Papyan et al.
Revised: The Neural Collapse phenomenon was originally discovered by Papyan et al.

Original: and has led to a series of further investigations.
Revised: and has led to a series of further investigations.

Original: A number of papers Fang et al. (2021); Lu & Steinerberger (2020); Wojtowytsch et al.
Revised: A number of papers Fang et al. (2021); Lu & Steinerberger (2020); Wojtowytsch et al.

Original: Mixon et al.
Revised: Mixon et al.

Original: ; Zhu et al.
Revised: ; Zhu et al.

Original: ; Ji et al.
Revised: ; Ji et al.

Original: ; Han et al.
Revised: ; Han et al.

Original: ; Zhou et al.
Revised: ; Zhou et al.

Original: ; Tirer &
Revised: ; Tirer &

Original: Bruna (2022); Yaras et al. (2022) studied a simplified “unconstrained feature model”, also known as “layer-peeled model”, and showed that Neural Collapse provably happens under a variety of settings.
Revised: Bruna (2022); Yaras et al. (2022) studied a simplified “unconstrained feature model”, also known as “layer-peeled model”, and showed that Neural Collapse provably happens under a variety of settings.

Original: This model treats the last-layer representations of all training samples as free optimization variables.
Revised: This model treats the last-layer representations of all training samples as free optimization variables.

Original: By doing this, the loss function no longer depends on the input data, and therefore this line of work is unable to capture any effect of the input distribution on the structure of the representations.
Revised: By doing this, the loss function no longer depends on the input data, and therefore this line of work is unable to capture any effect of the input distribution on the structure of the representations.

Original: Ergen &
Revised: Ergen &

Original: Pilanci (2021); Tirer & Bruna (2022); Weinan & Wojtowytsch (2022) considered more complicated models but still did not incorporate the role of the input distribution.
Revised: Pilanci (2021); Tirer & Bruna (2022); Weinan & Wojtowytsch (2022) considered more complicated models but still did not incorporate the role of the input distribution.

Original: Hui et al.
Revised: Hui et al.

Original: (2022) studied the connection of Neural Collapse to generalization and concluded that
Revised: (2022) studied the connection of Neural Collapse to generalization and concluded that

Original: Neural Collapse occurs only on the training set, not on the test set.
Revised: Neural Collapse occurs only on the training set, not on the test set.

Original: Galanti et al.
Revised: Galanti et al.

Original: (2021) found that
Revised: (2021) found that

Original: Neural Collapse does generalize to test samples as well as new classes, and used this observation to study transfer learning and few-shot learning.
Revised: Neural Collapse does generalize to test samples as well as new classes, and used this observation to study transfer learning and few-shot learning.

Original: Sohoni et al.
Revised: Sohoni et al.

Original: (2020) observed that the last-layer representations of different subclasses within the same class are often separated into different clusters, and used this observation to design an algorithm for improving group robustness.
Revised: (2020) observed that the last-layer representations of different subclasses within the same class are often separated into different clusters, and used this observation to design an algorithm for improving group robustness.

Original: However, it is unclear whether their observation happens in the Neural
Revised: However, it is unclear whether their observation happens in the Neural

Original: Collapse regime, while we find that fine-grained structure in representations can co-exist with Neural
Revised: Collapse regime, while we find that fine-grained structure in representations can co-exist with Neural

Original: Collapse.
Revised: Collapse.

Original: Furthermore, Sohoni et al.
Revised: Furthermore, Sohoni et al.

Original: (2020) looked at settings in which different subclasses have different accuracies and attributed the representation separability phenomenon to this performance difference.
Revised: (2020) looked at settings in which different subclasses have different accuracies and attributed the representation separability phenomenon to this performance difference.

Original: On the other hand, we find that representation separability happens even when there is no performance gap between different subclasses.
Revised: On the other hand, we find that representation separability happens even when there is no performance gap between different subclasses.

Original:  P RELIMINARIES AND S ETUP
Revised:  P RELIMINARIES AND S ETUP

Original: This section introduces the notation used throughout this paper, as well as a more detailed explanation of Neural Collapse and a clarification of the experiment setup.
Revised: This section introduces the notation used throughout this paper, as well as a more detailed explanation of Neural Collapse and a clarification of the experiment setup.

Original: For a classification task dataset, we denote it by D = { ( x k , y k ) } nk =1 ,
Revised: For a classification task dataset, we denote it by D = { ( x k , y k ) } nk =1 ,

Original: where ( x k , y k ) ∈ R d ′ ×
Revised: where ( x k , y k ) ∈ R d ′ ×

Original: [ C ] is a pair of input features and label, n is the number of samples, d ′ is the input dimension, and C is the number of classes.
Revised: [ C ] is a pair of input features and label, n is the number of samples, d ′ is the input dimension, and C is the number of classes.

Original: For a given neural network, we denote the last-layer representation by H ∈ R n × d , i.e. the hidden representation before the final linear transformation, where d is the last-layer dimensionality.
Revised: For a given neural network, we denote the last-layer representation by H ∈ R n × d , i.e. the hidden representation before the final linear transformation, where d is the last-layer dimensionality.

Original: For an original class c ∈ [ C ] , we denote the number of samples in class c by n c , and the last-layer representation of k -th sample in class c by h ( c ) k .
Revised: For an original class c ∈ [ C ] , we denote the number of samples in class c by n c , and the last-layer representation of k -th sample in class c by h ( c ) k .

Original: 3. P RELIMINARIES OF N EURAL C OLLAPSE
Revised: 3. P RELIMINARIES OF N EURAL C OLLAPSE

Original: Neural Collapse characterizes 4 phenomena, named NC1-NC4.
Revised: Neural Collapse characterizes 4 phenomena, named NC1-NC4.

Original: Here we introduce the first two which concern the structure of the last-layer representations.
Revised: Here we introduce the first two which concern the structure of the last-layer representations.

Original: NC1, or variability collapse, predicts that the variance of last-layer representations of samples within the same class vanishes as training proceeds.
Revised: NC1, or variability collapse, predicts that the variance of last-layer representations of samples within the same class vanishes as training proceeds.

Original: Formally, it can be measured by NC 1 = 1 C Tr (cid:16) Σ W Σ
Revised: Formally, it can be measured by NC 1 = 1 C Tr (cid:16) Σ W Σ

Original: [Equation]
Revised: [Equation]

Original: B (cid:17) (Papyan et al., 2020; Zhu et al., 2021), which should tend to 0 .
Revised: B (cid:17) (Papyan et al., 2020; Zhu et al., 2021), which should tend to 0 .

Original: The Σ W and Σ B are defined as
Revised: The Σ W and Σ B are defined as

Original: [Equation]
Revised: [Equation]

Original: ) and
Revised: ) and

Original: [Equation]
Revised: [Equation]

Original: where µ c = 1 n c (cid:80) n c k =1 h ( c ) k are the class means and µ G = 1 n (cid:80) nk
Revised: where µ c = 1 n c (cid:80) n c k =1 h ( c ) k are the class means and µ G = 1 n (cid:80) nk

Original: =1 h k is the global mean.
Revised: =1 h k is the global mean.

Original: NC2 predicts that the class means converge to a special structure, i.e. their normalized covariance converges to the Simplex Equiangular Tight Frame (ETF).
Revised: NC2 predicts that the class means converge to a special structure, i.e. their normalized covariance converges to the Simplex Equiangular Tight Frame (ETF).

Original: This can be characterize by
Revised: This can be characterize by

Original: [Equation]
Revised: [Equation]

Original: during training, where M is the stack of centralized class-means, whose c -th row is µ c − µ G , and
Revised: during training, where M is the stack of centralized class-means, whose c -th row is µ c − µ G , and

Original: C ∈ R C is an all-one vector in length C , I is the identity matrix.
Revised: C ∈ R C is an all-one vector in length C , I is the identity matrix.

Original: 3. E XPERIMENT S ETUP
Revised: 3. E XPERIMENT S ETUP

Original: In our experiment, we explore the role of input distribution and labels through assigning coarser or finer labels to each sample, and then explore the structure of last-layer representation of a model trained on the dataset with coarse or fine labels and see to what extent the information of original labels are preserved.
Revised: In our experiment, we explore the role of input distribution and labels through assigning coarser or finer labels to each sample, and then explore the structure of last-layer representation of a model trained on the dataset with coarse or fine labels and see to what extent the information of original labels are preserved.

Original: The coarse labels are created in the following way.
Revised: The coarse labels are created in the following way.

Original: First choose a number ˜ C which is divisible by C , and create coarse labels by
Revised: First choose a number ˜ C which is divisible by C , and create coarse labels by

Original: [Equation]
Revised: [Equation]

Original: which merges the classes whose index have the same modulus w.r.t.
Revised: which merges the classes whose index have the same modulus w.r.t.

Original: ˜ C
Revised: ˜ C

Original: and thus create ˜ C super-classes.
Revised: and thus create ˜ C super-classes.

Original: Since the original index of classes generally has no special meanings, this process should act similar to randomly merging classes 1 .
Revised: Since the original index of classes generally has no special meanings, this process should act similar to randomly merging classes 1 .

Original: We say the samples with the same coarse label belongs to the same super-class, and call the dataset ˜ D
Revised: We say the samples with the same coarse label belongs to the same super-class, and call the dataset ˜ D

Original: = { ( x k , ˜ y k ) } nk =1 the coarse dataset, which is the dataset we used to train the model.
Revised: = { ( x k , ˜ y k ) } nk =1 the coarse dataset, which is the dataset we used to train the model.

Original: Figure 2 provides an illustration of the coarse labels on CIFAR-10 with ˜ C = 5 , which we call Coarse CIFAR-10.
Revised: Figure 2 provides an illustration of the coarse labels on CIFAR-10 with ˜ C = 5 , which we call Coarse CIFAR-10.

Original: To create fine labels, we randomly split each class into two sub-classes.
Revised: To create fine labels, we randomly split each class into two sub-classes.

Original: Specifically, the fine labels are created by
Revised: Specifically, the fine labels are created by

Original: [Equation]
Revised: [Equation]

Original: where ˆ y k is the fine label of sample k and C is the number of original classes and β is a Bernoulli
Revised: where ˆ y k is the fine label of sample k and C is the number of original classes and β is a Bernoulli

Original: Variable.
Revised: Variable.

Original: This process result in a dataset ˆ D = { ( x k , ˆ y k ) } nk =1 with 2 C classes.
Revised: This process result in a dataset ˆ D = { ( x k , ˆ y k ) } nk =1 with 2 C classes.

Original: Same as before, we call ˆ D the fine dataset.
Revised: Same as before, we call ˆ D the fine dataset.

Original:  E XPLORING THE F INE -G RAINED R EPRESENTATION S TRUCTURE WITH C OARSE CIFAR- 
Revised:  E XPLORING THE F INE -G RAINED R EPRESENTATION S TRUCTURE WITH C OARSE CIFAR-

Original: In this section, we experiment with coarsely labeled datasets, using Coarse CIFAR-10 as an illustrative example.
Revised: In this section, we experiment with coarsely labeled datasets, using Coarse CIFAR-10 as an illustrative example.

Original: Specifically, the model is trained on the training set of Coarse CIFAR-10 for a certain number of steps that is sufficient for the model to converge.
Revised: Specifically, the model is trained on the training set of Coarse CIFAR-10 for a certain number of steps that is sufficient for the model to converge.

Original: We then take the last-layer representations of the model during training and explore their structure.
Revised: We then take the last-layer representations of the model during training and explore their structure.

Original: In order to make an exhaustive observation, the experiment is repeated using different learning rates and weight-decay rates.
Revised: In order to make an exhaustive observation, the experiment is repeated using different learning rates and weight-decay rates.

Original: Specifically, we choose the learning rate of training in { 10 − 1 , 10 − 2 , 10 − 3 } and weight-decay rate in { 5 × 10 − 3 , 5 × 10 − 4 , 5 × 10 − 5 } and test all 9 possible combinations of them.
Revised: Specifically, we choose the learning rate of training in { 10 − 1 , 10 − 2 , 10 − 3 } and weight-decay rate in { 5 × 10 − 3 , 5 × 10 − 4 , 5 × 10 − 5 } and test all 9 possible combinations of them.

Original: The experiment is also conducted on multiple datasets and network architectures.
Revised: The experiment is also conducted on multiple datasets and network architectures.

Original: Due to space limit, in this section we focus on experiment of ResNet-18 on Coarse CIFAR-10, where the original
Revised: Due to space limit, in this section we focus on experiment of ResNet-18 on Coarse CIFAR-10, where the original

Original: C = 10 and the number of coarse labels is chosen as ˜ C = 5 .
Revised: C = 10 and the number of coarse labels is chosen as ˜ C = 5 .

Original: For training hyper-parameters, we only report results for two representative hyper-parameter combinations, referred to as config #1 and config #2.
Revised: For training hyper-parameters, we only report results for two representative hyper-parameter combinations, referred to as config #1 and config #2.

Original: The learning rate of config #1 is set to 0 .
Revised: The learning rate of config #1 is set to 0 .

Original: 01 and weight-decay rate is 5 × 10 − 4 , and for config #2 the learning rate is 0 .
Revised: 01 and weight-decay rate is 5 × 10 − 4 , and for config #2 the learning rate is 0 .

Original: 01 and weight decay rate is 5 × 10 − 5 .
Revised: 01 and weight decay rate is 5 × 10 − 5 .

Original: Both settings reach 0 training error quickly during training.
Revised: Both settings reach 0 training error quickly during training.

Original: We defer complete experiment results to appendix.
Revised: We defer complete experiment results to appendix.

Original: As a preliminary result, we first show that under our experiment settings, Neural Collapse does happen, i.e. the representation does converge to 5 clusters and the class-means form a Simplex ETF structure.
Revised: As a preliminary result, we first show that under our experiment settings, Neural Collapse does happen, i.e. the representation does converge to 5 clusters and the class-means form a Simplex ETF structure.

Original: Specifically, we measure NC 1 and NC 2 defined in Section 3.1 for both config #1 and config #2, with C replaced by ˜ C since we are calculating it on coarse dataset.
Revised: Specifically, we measure NC 1 and NC 2 defined in Section 3.1 for both config #1 and config #2, with C replaced by ˜ C since we are calculating it on coarse dataset.

Original: The results are shown in
Revised: The results are shown in

Original: Figure 3, which matches previous results in Papyan et al.
Revised: Figure 3, which matches previous results in Papyan et al.

Original: Zhu et al.
Revised: Zhu et al.

Original: , indicating that
Revised: , indicating that

Original: Neural Collapse does happen under both settings.
Revised: Neural Collapse does happen under both settings.

Original: 4. C LASS D ISTANCE
Revised: 4. C LASS D ISTANCE

Original: We first look at the average square Euclidian distance of last-layer representations between each two original classes.
Revised: We first look at the average square Euclidian distance of last-layer representations between each two original classes.

Original: Formally, we calculate a class distance matrix D ∈ R C × C , whose entries are
Revised: Formally, we calculate a class distance matrix D ∈ R C × C , whose entries are

Original: [Equation]
Revised: [Equation]

Original: for all i, j ∈ [ C ] , where h ( c ) k represents the last-layer representation of the k -th sample of super-class u .
Revised: for all i, j ∈ [ C ] , where h ( c ) k represents the last-layer representation of the k -th sample of super-class u .

Original: Since the model is trained on coarse dataset, Neural Collapse asserts that for every original class pair i, j in the same super-class (including the case of i = j ), the class distance D i,j should be very small.
Revised: Since the model is trained on coarse dataset, Neural Collapse asserts that for every original class pair i, j in the same super-class (including the case of i = j ), the class distance D i,j should be very small.

Original: In Coarse CIFAR-10, this will result in three darks lines (let darker color represents lower value) in the heatmap of D since each super class contains two original classes, as illustrated in Figure 4 (b).
Revised: In Coarse CIFAR-10, this will result in three darks lines (let darker color represents lower value) in the heatmap of D since each super class contains two original classes, as illustrated in Figure 4 (b).

Original: For example, in Coarse CIFAR-10 the original class "airplane" and "dog" both belong to the super class "airplane & dog", therefore per Neural Collapse’s prediction, their last-layer representations would collapse to each other, making the average square distance extremely small compared to other entries.
Revised: For example, in Coarse CIFAR-10 the original class "airplane" and "dog" both belong to the super class "airplane & dog", therefore per Neural Collapse’s prediction, their last-layer representations would collapse to each other, making the average square distance extremely small compared to other entries.

Original: In contrast, if the last-layer representations perfectly reflects the distribution of input, i.e. original classes, the class distance matrix should be a diagonal matrix as shown in Figure 4 (a), because the last-layer representation of samples in each original class only collapse to the class-mean of this original class.
Revised: In contrast, if the last-layer representations perfectly reflects the distribution of input, i.e. original classes, the class distance matrix should be a diagonal matrix as shown in Figure 4 (a), because the last-layer representation of samples in each original class only collapse to the class-mean of this original class.

Original: Figures 5 and 6 display the heatmap of class distance matrix D , arranged by number of training epochs.
Revised: Figures 5 and 6 display the heatmap of class distance matrix D , arranged by number of training epochs.

Original: From the results there are two surprising observations can be made: Firstly, there are indeed three dark lines at the final stage, however, the three dark lines does not show up simultaneously, and the central line – represents the samples in the same original classes – shows up earlier.
Revised: From the results there are two surprising observations can be made: Firstly, there are indeed three dark lines at the final stage, however, the three dark lines does not show up simultaneously, and the central line – represents the samples in the same original classes – shows up earlier.

Original: The second observation is, even in the final stage where the training error is zero, the three lines can be not of the same degree of darkness, with the central line shallower, and this is especially apparent in Figure 6.
Revised: The second observation is, even in the final stage where the training error is zero, the three lines can be not of the same degree of darkness, with the central line shallower, and this is especially apparent in Figure 6.

Original: Those observations suggests that, the actual behaviour of the last-layer representations is between the cases predicted in Figure 4 (a) and (b): the input distribution and training label both have an impact on the distribution of the last-layer representations, and although the time of them to show up is different, both of them can be present even after reaching zero training error for a long time.
Revised: Those observations suggests that, the actual behaviour of the last-layer representations is between the cases predicted in Figure 4 (a) and (b): the input distribution and training label both have an impact on the distribution of the last-layer representations, and although the time of them to show up is different, both of them can be present even after reaching zero training error for a long time.

Original: These observations suggest both the existence of Neural Collapse and the inadequacy of Neural Collapse to completely describe the behaviour of last-layer representations.
Revised: These observations suggest both the existence of Neural Collapse and the inadequacy of Neural Collapse to completely describe the behaviour of last-layer representations.

Original: 4. V ISUALIZATION
Revised: 4. V ISUALIZATION

Original: In this subsection, we take a closer look at the last-layer representations of the model at the end of training by reducing the dimensionality of the last-layer representations to 2 through t-SNE (Van der
Revised: In this subsection, we take a closer look at the last-layer representations of the model at the end of training by reducing the dimensionality of the last-layer representations to 2 through t-SNE (Van der

Original: Maaten & Hinton, 2008) and visualize them.
Revised: Maaten & Hinton, 2008) and visualize them.

Original: Specifically, we visualize each super-class separately, but color the samples whose original labels are different with different colors.
Revised: Specifically, we visualize each super-class separately, but color the samples whose original labels are different with different colors.

Original: The visualization results of config #1 and config #2 are displayed in Figures 7 and 8.
Revised: The visualization results of config #1 and config #2 are displayed in Figures 7 and 8.

Original: It can be observed that, there is distinguishable difference in the distribution of original labels in both config # and config #2.
Revised: It can be observed that, there is distinguishable difference in the distribution of original labels in both config # and config #2.

Original: This suggests that, the input distribution information, i.e. the original label information, is well preserved in the last-layer representations under both settings, albeit for config #2 it seems it can not be distinguished through distance matrix in Section 4.1.
Revised: This suggests that, the input distribution information, i.e. the original label information, is well preserved in the last-layer representations under both settings, albeit for config #2 it seems it can not be distinguished through distance matrix in Section 4.1.

Original: Training extremely long.
Revised: Training extremely long.

Original: In order to explore if the fine-grained structures are still preserved even after a extremely long time of training, we further train the model with config #2 to 1,000 epochs.
Revised: In order to explore if the fine-grained structures are still preserved even after a extremely long time of training, we further train the model with config #2 to 1,000 epochs.

Original: The heatmap of the distance matrix is presented in Figure 9.
Revised: The heatmap of the distance matrix is presented in Figure 9.

Original: We also present the t-SNE visualization result, but only put the result of the first super-class in Figure 10 due to space limitation.
Revised: We also present the t-SNE visualization result, but only put the result of the first super-class in Figure 10 due to space limitation.

Original: 4. L EARNING CIFAR-10 FROM 5 C OARSE L ABELS
Revised: 4. L EARNING CIFAR-10 FROM 5 C OARSE L ABELS

Original: As the results in Section 4.2 suggest, even after the training accuracy has reached 100% for a long time, the samples within each superclass still exhibit a clear structure per their original class, and those structures act as clusters after reducing dimensionality.
Revised: As the results in Section 4.2 suggest, even after the training accuracy has reached 100% for a long time, the samples within each superclass still exhibit a clear structure per their original class, and those structures act as clusters after reducing dimensionality.

Original: Inspired by this observation, we perform a Cluster-and-Linear-Probe (CLP) test to evaluate to what extent the original class information is preserved in the last-layer representations.
Revised: Inspired by this observation, we perform a Cluster-and-Linear-Probe (CLP) test to evaluate to what extent the original class information is preserved in the last-layer representations.

Original: In CLP, we use the representations learned on Coarse CIFAR-10 to reconstruct 10 labels and run a linear probe with the reconstructed labels.
Revised: In CLP, we use the representations learned on Coarse CIFAR-10 to reconstruct 10 labels and run a linear probe with the reconstructed labels.

Original: Specifically, we first use t-SNE to reduce the dimensionality to 2 and then use KMeans to find 2 clusters of the dimensionality-reduced representations within each super-class.
Revised: Specifically, we first use t-SNE to reduce the dimensionality to 2 and then use KMeans to find 2 clusters of the dimensionality-reduced representations within each super-class.

Original: We use the clusters as reconstructed labels to do a linear probe.
Revised: We use the clusters as reconstructed labels to do a linear probe.

Original: In linear probe, we train a linear classifier on top of the previously learned representation H on the training set with reconstructed labels and evaluate the learned linear classifier on original test set.
Revised: In linear probe, we train a linear classifier on top of the previously learned representation H on the training set with reconstructed labels and evaluate the learned linear classifier on original test set.

Original: Notice that because we don’t know the mapping of reconstructed classes to true original classes, we permute each possible mapping and report the highest performance.
Revised: Notice that because we don’t know the mapping of reconstructed classes to true original classes, we permute each possible mapping and report the highest performance.

Original: We also train a linear probe with original training labels as a comparison.
Revised: We also train a linear probe with original training labels as a comparison.

Original: The results of config #1 and config #2 are shown in Figures 11 and 12 respectively.
Revised: The results of config #1 and config #2 are shown in Figures 11 and 12 respectively.

Original: In most cases, the performance of CLP on original test set is comparable with linear probe trained on true original labels or even with models originally trained on CIFAR-10.
Revised: In most cases, the performance of CLP on original test set is comparable with linear probe trained on true original labels or even with models originally trained on CIFAR-10.

Original: Notice that the representation
Revised: Notice that the representation

Original: H is obtained through the model trained with coarse labels, and the label reconstruction only uses information of H and the number of original classes.
Revised: H is obtained through the model trained with coarse labels, and the label reconstruction only uses information of H and the number of original classes.

Original: This means we can achieve a very high performance on original test set even if we only have access to almost only the information of coarse labels.
Revised: This means we can achieve a very high performance on original test set even if we only have access to almost only the information of coarse labels.

Original: This result further confirms that the input distribution can play an important role in the last-layer representations.
Revised: This result further confirms that the input distribution can play an important role in the last-layer representations.

Original:  H OW D OES S EMANTIC S IMILARITY A FFECT THE F INE -G RAINED S TRUCTURE ?
Revised:  H OW D OES S EMANTIC S IMILARITY A FFECT THE F INE -G RAINED S TRUCTURE ?

Original: In our experiments with Coarse CIFAR- each coarse label is obtained by combining two classes regardless of the semantics.
Revised: In our experiments with Coarse CIFAR- each coarse label is obtained by combining two classes regardless of the semantics.

Original: The fact that the neural network can separate the two classes in its representation space implies that the network recognizes these two classes as semantically different (even though they are given the same coarse label).
Revised: The fact that the neural network can separate the two classes in its representation space implies that the network recognizes these two classes as semantically different (even though they are given the same coarse label).

Original: In this section we explore the following question:
Revised: In this section we explore the following question:

Original: If the sub-classes in a super-class have semantic similarity, will the representations still exhibit a fine-grained structure to distinguish them?
Revised: If the sub-classes in a super-class have semantic similarity, will the representations still exhibit a fine-grained structure to distinguish them?

Original: Intuitively, if the coarse label provided is “natural” and consists of semantically similar sub-classes, it is possible that the neural network will not distinguish between them and just produce truly collapsed representations.
Revised: Intuitively, if the coarse label provided is “natural” and consists of semantically similar sub-classes, it is possible that the neural network will not distinguish between them and just produce truly collapsed representations.

Original: We take an initial step towards this question by looking at ResNet-18 trained on CIFAR-100 using the official 20 super-classes (each super-class contains 5 sub-classes) as labels.
Revised: We take an initial step towards this question by looking at ResNet-18 trained on CIFAR-100 using the official 20 super-classes (each super-class contains 5 sub-classes) as labels.

Original: Unlike randomly merging classes as we did in Section 4, the official super-classes of CIFAR-100 are natural, merging classes with similar semantics (for example, “beaver” and “dolphin” both belong to “aquatic mammals”).
Revised: Unlike randomly merging classes as we did in Section 4, the official super-classes of CIFAR-100 are natural, merging classes with similar semantics (for example, “beaver” and “dolphin” both belong to “aquatic mammals”).

Original: This offers a perfect testbed for our question.
Revised: This offers a perfect testbed for our question.

Original: We find that ResNet-18 indeed is not able to distinguish all sub-classes in its representation space, but can still produce separable representations if some sub-classes within a super-class are sufficiently different.
Revised: We find that ResNet-18 indeed is not able to distinguish all sub-classes in its representation space, but can still produce separable representations if some sub-classes within a super-class are sufficiently different.

Original: Interestingly, the notion of semantic similarity of ResNet-18 turns out to agree well with that of humans.
Revised: Interestingly, the notion of semantic similarity of ResNet-18 turns out to agree well with that of humans.

Original: Figures 13 and 14 show the t-SNE visualizations of representations from two superclasses.
Revised: Figures 13 and 14 show the t-SNE visualizations of representations from two superclasses.

Original: From the visualizations, although there are not as clear clusters as for Coarse CIFAR-10, the representations do exhibit visible separations between certain sub-classes.
Revised: From the visualizations, although there are not as clear clusters as for Coarse CIFAR-10, the representations do exhibit visible separations between certain sub-classes.

Original: In Figure 13, “bicycles” and “motorcycles” are entangled together, while they are separated from “bus”, “pickup truck”, and “train”, which is human-interpretable.
Revised: In Figure 13, “bicycles” and “motorcycles” are entangled together, while they are separated from “bus”, “pickup truck”, and “train”, which is human-interpretable.

Original: In Figure 14, “crab” and “lobster” are mixed together, which are both aquatic and belong to malacostraca, while the other three are not and have more differentiative representations.
Revised: In Figure 14, “crab” and “lobster” are mixed together, which are both aquatic and belong to malacostraca, while the other three are not and have more differentiative representations.

Original: These results confirm the intuition that the fine-grained structure in last-layer representations is affected by, or even based on, the semantic similarity between the inputs.
Revised: These results confirm the intuition that the fine-grained structure in last-layer representations is affected by, or even based on, the semantic similarity between the inputs.

Original:  THE F INE -G RAINED R EPRESENTATION S TRUCTURE ON F INE CIFAR- 
Revised:  THE F INE -G RAINED R EPRESENTATION S TRUCTURE ON F INE CIFAR-

Original: In this section, we consider finely-labeled dataset.
Revised: In this section, we consider finely-labeled dataset.

Original: We construct a fine version of CIFAR-10 with the process described in Section 3.2, and call it Fine CIFAR-10.
Revised: We construct a fine version of CIFAR-10 with the process described in Section 3.2, and call it Fine CIFAR-10.

Original: Figure 15 and Figure 16 presents the results of class distance matrices for config #1 and config #2, which are arranged by the number of training epochs.
Revised: Figure 15 and Figure 16 presents the results of class distance matrices for config #1 and config #2, which are arranged by the number of training epochs.

Original: It can be observed that at the early stage of training, there are three dark lines, which indicates the last-layer representations are converging towards 10 clusters instead of 20 first.
Revised: It can be observed that at the early stage of training, there are three dark lines, which indicates the last-layer representations are converging towards 10 clusters instead of 20 first.

Original: At the end of training, this 10 -class relationship is reserved and is especially significant in config #2, although with a lighter color.
Revised: At the end of training, this 10 -class relationship is reserved and is especially significant in config #2, although with a lighter color.

Original: This suggests that there exists fine-grained structure even at the terminal phase.
Revised: This suggests that there exists fine-grained structure even at the terminal phase.

Original: Those observations are consistent with the observations made in Section 4 and supports our conclusion.
Revised: Those observations are consistent with the observations made in Section 4 and supports our conclusion.

Original:  D ISCUSSION
Revised:  D ISCUSSION

Original: In this paper, we initiated the study of the role of the intrinsic structure of the input data distribution on the last-layer representations of neural networks, and in particular, how to reconcile it with the
Revised: In this paper, we initiated the study of the role of the intrinsic structure of the input data distribution on the last-layer representations of neural networks, and in particular, how to reconcile it with the

Original: Neural Collapse phenomenon, which is only driven by the explicit labels provided in the training procedure.
Revised: Neural Collapse phenomenon, which is only driven by the explicit labels provided in the training procedure.

Original: Through a series of experiments, we provide concrete evidence that the representations can exhibit clear fine-grained structure despite their apparent collapse.
Revised: Through a series of experiments, we provide concrete evidence that the representations can exhibit clear fine-grained structure despite their apparent collapse.

Original: While Neural Collapse is an intriguing phenomenon and deserves further studies to understand its cause and consequences, our work calls for more scientific investigations of the structure of neural representations that go beyond
Revised: While Neural Collapse is an intriguing phenomenon and deserves further studies to understand its cause and consequences, our work calls for more scientific investigations of the structure of neural representations that go beyond

Original: Neural Collapse.
Revised: Neural Collapse.

Original: We note that the fine-grained representation structure we observed depends on the inductive biases of the network architecture and the training algorithm, and may not appear universally.
Revised: We note that the fine-grained representation structure we observed depends on the inductive biases of the network architecture and the training algorithm, and may not appear universally.

Original: In our experiments on Coarse CIFAR-10, we observe the fine-grained structure for ResNet and DenseNet, but not for VGG (see appendix for extended results).
Revised: In our experiments on Coarse CIFAR-10, we observe the fine-grained structure for ResNet and DenseNet, but not for VGG (see appendix for extended results).

Original: We also note that certain choices of learning rate and weight-decay rate lead to stronger fine-grained structure than others.
Revised: We also note that certain choices of learning rate and weight-decay rate lead to stronger fine-grained structure than others.

Original: We leave a thorough investigation of such subtlety for future work.
Revised: We leave a thorough investigation of such subtlety for future work.

Original: A (N EW ) E XPERIMENT ON C OARSE CIFAR-10 WITHOUT W EIGHT D ECAY
Revised: A (N EW ) E XPERIMENT ON C OARSE CIFAR-10 WITHOUT W EIGHT D ECAY

Original: In order to study whether weight decay is crucial for the fine-grained structure, we perform an additional experiment on Coarse CIFAR-10 when weight decay is 0 .
Revised: In order to study whether weight decay is crucial for the fine-grained structure, we perform an additional experiment on Coarse CIFAR-10 when weight decay is 0 .

Original: We present the final class distance matrices (see Section 4.1) for three different learning rates  in Figure 17.
Revised: We present the final class distance matrices (see Section 4.1) for three different learning rates 0 . 1 , 0 . 01 , and 0 . 001 in Figure 17.

Original: Except for the case where the learning rate is 0 . 001 , the training error reaches 0 in other cases.
Revised: Except when the learning rate is 0 . 001 , the training error reaches 0 .

Original: We see that the fine-grained representation structure can still emerge without weight decay.
Revised: We see that the fine-grained representation structure can still emerge without weight decay.

Original: B (N EW ) E XPERIMENT ON M IXTURE OF G AUSSIANS WITH C OARSE L ABELS
Revised: B (N EW ) E XPERIMENT ON M IXTURE OF G AUSSIANS WITH C OARSE L ABELS

Original: In this section, we reproduce the fine-grained representation structure in a simple synthetic setting, i.e., classifying mixture of Gaussians using a 2-layer MLP, and perform some preliminary quantitative studies in this setting.
Revised: In this section, we reproduce the fine-grained representation structure in a simple synthetic setting, i.e., classifying mixture of Gaussians using a 2-layer MLP, and perform some preliminary quantitative studies in this setting.

Original: This allows us to preform controlled experiments by varying different characteristics of the data distribution, architecture, and algorithmic components, and it can serve as a starting point for future theoretical work.
Revised: This allows us to preform controlled experiments by varying different characteristics of the data distribution, architecture, and algorithmic components, and it can serve as a starting point for future theoretical work.

Original: Formally, we create a dataset from a mixture of Gaussians, where the input from the c -th cluster is generated from N ( µ ( c ) , I ) , and each cluster mean µ ( c ) is drawn i.i.d.
Revised: Formally, we create a dataset from a mixture of Gaussians, where the input from the c -th cluster is generated from N ( µ ( c ) , I ) , and each cluster mean µ ( c ) is drawn i.i.d.

Original: from N ( 0 , σ 2 I ) .
Revised: from N ( 0 , σ 2 I ) .

Original: The class label of each datapoint is the index of the cluster it belongs to.
Revised: The class label of each datapoint is the index of the cluster it belongs to.

Original: The larger σ 2 is, the larger the separation between each two clusters is, and the more likely it is to observe a fine-grained representation structure when given coarse labels.
Revised: The larger σ 2 is, the larger the separation between each two clusters is, and the more likely it is to observe a fine-grained representation structure when given coarse labels.

Original: We perform the same coarsening process described in Section 3.2 (by combining two classes into one super-class) and train a 2 -layer MLP on the coarsely labeled dataset.
Revised: We perform the same coarsening process described in Section 3.2 (by combining two classes into one super-class) and train a 2 -layer MLP on the coarsely labeled dataset.

Original: We measure the significance of the fine-grained structure using the ratio between {the average squared distance between representations in the same super-class but different sub-classes} and {the average squared distance between representations in the same subclass}, which we call the Mean Squared Distance Ratio (MSDR).
Revised: We measure the significance of the fine-grained structure using the ratio between {the average squared distance between representations in the same super-class but different sub-classes} and {the average squared distance between representations in the same subclass}, which we call the Mean Squared Distance Ratio (MSDR).

Original: Mathematically, it is defined as
Revised: Mathematically, it is defined as

Original: [Equation]
Revised: [Equation]

Original: where D i,j is defined in (6).
Revised: where D i,j is defined in (6).

Original: A larger MSDR means that the fine-grained structure is more pronounced, while MSDR ≈ 1 indicates no fine-grained structure.
Revised: A larger MSDR means that the fine-grained structure is more pronounced, while MSDR ≈ 1 indicates no fine-grained structure.

Original: By varying training and data-generating parameters, we investigate factors that impact the significance of the fine-grained structure.
Revised: By varying training and data-generating parameters, we investigate factors that impact the significance of the fine-grained structure.

Original: Specifically, we vary the input dimension, hidden dimension in the network, and weight decay rate, and plot how MSDR scales with σ 2 .
Revised: Specifically, we vary the input dimension, hidden dimension in the network, and weight decay rate, and plot how MSDR scales with σ 2 .

Original: The results are shown in
Revised: The results are shown in

Original: Figures 18 to 20.
Revised: Figures 18 to 20.

Original: Note that in each figure we use two different scales in the x-axes to differentiate the cases of σ 2 < 2 and σ 2 > 2 .
Revised: Note that in each figure we use two different scales in the x-axes to differentiate the cases of σ 2 < 2 and σ 2 > 2 .

Original: From the figures, we observe that both input and hidden dimensions exhibit a clear positive correlation with MSDR .
Revised: From the figures, we observe that both input and hidden dimensions exhibit a clear positive correlation with MSDR .

Original: On the other hand, the weight decay rate does not have an impact on MSDR in this setting.
Revised: On the other hand, the weight decay rate does not have an impact on MSDR in this setting.

Original: Training details.
Revised: Training details.

Original: We generate data from 8 clusters, each having 500 samples.
Revised: We generate data from 8 clusters, each having 500 samples.

Original: We train the model with gradient descent for 1 , 000 steps.
Revised: We train the model with gradient descent for 1 , 000 steps.

Original: The results are averaged over 10 runs.
Revised: The results are averaged over 10 runs.

Original: When varying one hyper-parameter, other hyper-parameters are set to their default values: d input = 512 , d hidden = 512 , weight decay = 0 .
Revised: When varying one hyper-parameter, other hyper-parameters are set to their default values: d input = 512 , d hidden = 512 , weight decay = 0 .

Original: B. M ODELING S EMANTIC S IMILARITY
Revised: B. M ODELING S EMANTIC S IMILARITY

Original: In Section 5, we showed that the emergence of fine-grained representations depends on the semantic similarity between sub-classes.
Revised: In Section 5, we showed that the emergence of fine-grained representations depends on the semantic similarity between sub-classes.

Original: Now we take a step in investigating this question by creating “similar” and “dissimilar” sub-classes in the Gaussian mixture model considered in this section.
Revised: Now we take a step in investigating this question by creating “similar” and “dissimilar” sub-classes in the Gaussian mixture model considered in this section.

Original: In particular, we use the same data-generating process described above, except that half of the super-classes will be altered so that they consist of “similar” sub-classes, and we say that the other super-classes consist of “dissimilar” sub-classes.
Revised: In particular, we use the same data-generating process described above, except that half of the super-classes will be altered so that they consist of “similar” sub-classes, and we say that the other super-classes consist of “dissimilar” sub-classes.

Original: The way to generate similar sub-classes it to first sample µ ∼ N ( 0 , σ 2 I ) and then generate two means µ ( c ) , µ ( c ′ ) ∼ N ( µ , τ 2 I ) .
Revised: The way to generate similar sub-classes it to first sample µ ∼ N ( 0 , σ 2 I ) and then generate two means µ ( c ) , µ ( c ′ ) ∼ N ( µ , τ 2 I ) .

Original: Therefore, we can vary τ 2 to control the level of similarity between similar sub-classes.
Revised: Therefore, we can vary τ 2 to control the level of similarity between similar sub-classes.

Original: We use default hyper-parameters described above, fix σ 2 = 4 , and vary τ 2 .
Revised: We use default hyper-parameters described above, fix σ 2 = 4 , and vary τ 2 .

Original: Figure 21 shows the Mean
Revised: Figure 21 shows the Mean

Original: Squared Distance Ratio for similar and dissimilar subclasses, respectively.
Revised: Squared Distance Ratio for similar and dissimilar subclasses, respectively.

Original: We see that fine-grained structure within a super-class does require sufficient dissimilarity between its sub-classes, which agrees with our observation from Section 5 on CIFAR-100.
Revised: We see that fine-grained structure within a super-class does require sufficient dissimilarity between its sub-classes, which agrees with our observation from Section 5 on CIFAR-100.

Original: C C OMPLETE C OARSE CIFAR-10 E XPERIMENT R ESULTS
Revised: C C OMPLETE C OARSE CIFAR-10 E XPERIMENT R ESULTS

Original: In the following sections, we provide extended experiment results.
Revised: In the following sections, we provide extended experiment results.

Original: As mentioned in Section 3.2, we permute learning rate in { 10 − 1 , 10 − 2 , 10 − 3 } and weight decay rate in { 5 × 10 − 3 , 5 × 10 − 4 , 5 × 10 − 5 } .
Revised: As mentioned in Section 3.2, we permute learning rate in { 10 − 1 , 10 − 2 , 10 − 3 } and weight decay rate in { 5 × 10 − 3 , 5 × 10 − 4 , 5 × 10 − 5 } .

Original: Generally, the results will be shown in a 3 × 3 table, of which each grid represents the result of one hyper-parameter combination, with each row has the same learning rate and each column has the same weight decay rate.
Revised: Generally, the results will be shown in a 3 × 3 table, of which each grid represents the result of one hyper-parameter combination, with each row has the same learning rate and each column has the same weight decay rate.

Original: In this section, we repeat the experiments in Section 4 with all learning rate and weight-decay rate combinations.
Revised: In this section, we repeat the experiments in Section 4 with all learning rate and weight-decay rate combinations.

Original: Firstly, we present the training statistics (accuracy, loss) of all hyper-parameters in
Revised: Firstly, we present the training statistics (accuracy, loss) of all hyper-parameters in

Original: Figures 22 and 23 as an reference.
Revised: Figures 22 and 23 as an reference.

Original: It can be observed that all hyper-parameter groups achieved very low training error except the first one (weight decay = 5 × 10 − 3 , learning rate = 10 − 1 ).
Revised: It can be observed that all hyper-parameter groups achieved very low training error except the first one (weight decay = 5 × 10 − 3 , learning rate = 10 − 1 ).

Original: In fact, the last two hyper-parameter combinations (learning rate = 10 − 3 , weight decay ∈ { 5 × 10 − 4 , 5 × 10 − 5 } ) didn’t achieve exactly 0 training error (their training error are < 0 .
Revised: In fact, the last two hyper-parameter combinations (learning rate = 10 − 3 , weight decay ∈ { 5 × 10 − 4 , 5 × 10 − 5 } ) didn’t achieve exactly 0 training error (their training error are < 0 .

Original: 5% but not exactly 0 )
Revised: 5% but not exactly 0 )

Original: , and the other 6 hyper-parameter combinations all achieved exactly 0 training error.
Revised: , and the other 6 hyper-parameter combinations all achieved exactly 0 training error.

Original: C. C LASS D ISTANCE
Revised: C. C LASS D ISTANCE

Original: Here we present the visualization of the heatmap of class distance matrix D which is defined in
Revised: Here we present the visualization of the heatmap of class distance matrix D which is defined in

Original: Section 4.1.
Revised: Section 4.1.

Original: We choose 4 epochs to show the trend during training.
Revised: We choose 4 epochs to show the trend during training.

Original: The results are presented in
Revised: The results are presented in

Original: Figures 24 to 27, whose epoch numbers are 20 , 120 , 240 and 349 respectively.
Revised: Figures 24 to 27, whose epoch numbers are 20 , 120 , 240 and 349 respectively.

Original: C. V ISUALIZATION
Revised: C. V ISUALIZATION

Original: In this section, we present the t-SNE visualization result of ResNet-18 on Coarse CIFAR-10 in
Revised: In this section, we present the t-SNE visualization result of ResNet-18 on Coarse CIFAR-10 in

Original: Figures 28 to 30.
Revised: Figures 28 to 30.

Original: The results are divided into three groups, each of which has the same learning rate and the format of each group is the same as Figures 7 and 8.
Revised: The results are divided into three groups, each of which has the same learning rate and the format of each group is the same as Figures 7 and 8.

Original: C. C LUSTER - AND -L INEAR -P ROBE
Revised: C. C LUSTER - AND -L INEAR -P ROBE

Original: The Cluster-and-Linear-Probe test results of ResNet-18 trained on Coarse CIFAR-10 with all hyperparameter combinations are presented in Figure 31.
Revised: The Cluster-and-Linear-Probe test results of ResNet-18 trained on Coarse CIFAR-10 with all hyperparameter combinations are presented in Figure 31.

Original: D C OMPLETE C LASS D ISTANCE R ESULT OF F INE CIFAR- 
Revised: D C OMPLETE C LASS D ISTANCE R ESULT OF F INE CIFAR-

Original: In this section we provide the visualization of the class distance matrix of Fine CIFAR-10 with all hyper-parameter combinations which has been partially displayed in Section 6.
Revised: In this section we provide the visualization of the class distance matrix of Fine CIFAR-10 with all hyper-parameter combinations which has been partially displayed in Section 6.

Original: As before multiple epochs during training are selected to display a evolutionary trend of the class distance matrices.
Revised: As before multiple epochs during training are selected to display a evolutionary trend of the class distance matrices.

Original: The results are presented in Figures 32 to whose epoch numbers are 20 , 200 and 350 respectively
Revised: The results are presented in Figures 32 to whose epoch numbers are 20 , 200 and 350 respectively

Original: E E XPERIMENT OF R ES N ET -18 ON C OARSE CIFAR- 
Revised: E E XPERIMENT OF R ES N ET -18 ON C OARSE CIFAR-

Original: In this section, we report extended experiment results on Coarse CIFAR-100.
Revised: In this section, we report extended experiment results on Coarse CIFAR-100.

Original: The same with the case of Coarse CIFAR-10, we construct CIFAR-100 through the label coarsening process described in
Revised: The same with the case of Coarse CIFAR-10, we construct CIFAR-100 through the label coarsening process described in

Original: Section 3.2 and choose ˜ C = 20 , so that every 5 original classes are merged into one super-class.
Revised: Section 3.2 and choose ˜ C = 20 , so that every 5 original classes are merged into one super-class.

Original: We repeat most of experiments in Section 4.
Revised: We repeat most of experiments in Section 4.

Original: E. C LASS D ISTANCE
Revised: E. C LASS D ISTANCE

Original: The heatmaps of distance matrices are presented in Figures 35 to 37, whose epoch number are 20 ,and 350 respectively.
Revised: The heatmaps of distance matrices are presented in Figures 35 to 37, whose epoch number are 20 ,and 350 respectively.

Original: E. C LUSTER - AND -L INEAR -P ROBE
Revised: E. C LUSTER - AND -L INEAR -P ROBE

Original: Notice that we omit the visualization result of Coarse CIFAR-100 since there are too many figures.
Revised: Notice that we omit the visualization result of Coarse CIFAR-100 since there are too many figures.

Original: We present the Cluster-and-Linear-Probe results to reflect the clustering property of last-layer representations learned on Coarse CIFAR-100.
Revised: We present the Cluster-and-Linear-Probe results to reflect the clustering property of last-layer representations learned on Coarse CIFAR-100.

Original: The CLP results are presented in Figure 38.
Revised: The CLP results are presented in Figure 38.

Original: F R ANDOM C OARSE CIFAR- 
Revised: F R ANDOM C OARSE CIFAR-

Original: In this section, as mentioned in Section 3.2, we make our experiment more complete by performing a random combination of labels on CIFAR-10 rather than using a determined coarsening process as in the main paper.
Revised: In this section, as mentioned in Section 3.2, we make our experiment more complete by performing a random combination of labels on CIFAR-10 rather than using a determined coarsening process as in the main paper.

Original: The dataset construction is almost the same as the process of assigning coarse labels described in Section 3.2, except here we randomly shuffle the class indices before coarsening them.
Revised: The dataset construction is almost the same as the process of assigning coarse labels described in Section 3.2, except here we randomly shuffle the class indices before coarsening them.

Original: The class distance matrices of three difference epochs are shown in Figures 39 to 41.
Revised: The class distance matrices of three difference epochs are shown in Figures 39 to 41.

Original: From the results we can see, although there are no longer three dark lines, for each row there are generally two dark blocks, represents the original classes belongs to the same super-class, and the same observations in
Revised: From the results we can see, although there are no longer three dark lines, for each row there are generally two dark blocks, represents the original classes belongs to the same super-class, and the same observations in

Original: Section 4 can still be made here.
Revised: Section 4 can still be made here.

Original: G E XPERIMENT WITH D ENSE N ET
Revised: G E XPERIMENT WITH D ENSE N ET

Original: We also perform our experiments with different neural network structures for completeness.
Revised: We also perform our experiments with different neural network structures for completeness.

Original: In this section, we show the result with DenseNet-121 on Coarse CIFAR-10.
Revised: In this section, we show the result with DenseNet-121 on Coarse CIFAR-10.

Original: The experiments with
Revised: The experiments with

Original: DenseNet is supportive to our observations in the main paper.
Revised: DenseNet is supportive to our observations in the main paper.

Original: G. C LASS D ISTANCE
Revised: G. C LASS D ISTANCE

Original: The class distance matrices of three epochs during training are presented in Figures 42 to 44.
Revised: The class distance matrices of three epochs during training are presented in Figures 42 to 44.

Original: G. C LUSTER - AND -L INEAR -P ROBE
Revised: G. C LUSTER - AND -L INEAR -P ROBE

Original: The Cluster-and-Linear-Probe test results are presented in Figure 45.
Revised: The Cluster-and-Linear-Probe test results are presented in Figure 45.

Original: H E XPERIMENT WITH VGG
Revised: H E XPERIMENT WITH VGG

Original: We also extend our experiments to VGG-18.
Revised: We also extend our experiments to VGG-18.

Original: Interestingly, VGG to some extent is a counter example of the observations made in the main paper: it only displays Neural Collapse, and can not distinguish different original classes within one super-class, even in an early stage of training.
Revised: Interestingly, VGG to some extent is a counter example of the observations made in the main paper: it only displays Neural Collapse, and can not distinguish different original classes within one super-class, even in an early stage of training.

Original: The reason why
Revised: The reason why

Original: VGG is abnormal requires further exploration.
Revised: VGG is abnormal requires further exploration.

Original: The class distance matrices of three epochs with VGG during training are shown in Figures 46 to 48.
Revised: The class distance matrices of three epochs with VGG during training are shown in Figures 46 to 48.

Original: It can be observed that the three dark lines appears almost at the same time and always be of nearly the same darkness.
Revised: It can be observed that the three dark lines appears almost at the same time and always be of nearly the same darkness.

Original: This represents the trend predicted by Neural Collapse (Figure 4 (a)), but rejects the prediction made by (Figure 4 (b)).
Revised: This represents the trend predicted by Neural Collapse (Figure 4 (a)), but rejects the prediction made by (Figure 4 (b)).