Original: Uncertainty-Aware Hierarchical Reﬁnement for Incremental Implicitly-Reﬁned Classiﬁcation
Revised: Uncertainty-Aware Hierarchical Refinement for Incremental Implicitly-Refined Classification

Original: Abstract
Revised: Abstract

Original: Incremental implicitly-reﬁned classiﬁcation aims at assigning hierarchical labelsto the same instance encountered at different phases.
Revised: Incremental implicitly-refined classification task aims at assigning hierarchical labels to each sample encountered at different phases.

Original: Existing methods tend to failin generating hierarchy-invariant descriptor when  novel classes are inherited fromthe old ones.
Revised: Existing methods tend to fail in generating hierarchy-invariant descriptors when the novel classes are inherited from the old ones.

Original: To address the issue, this paper explores the inheritance relationsin the process of multi-level semantic increment,and proposes an UncertaintyAware Hierarchical Reﬁnement (UAHR) scheme.
Revised: To address the issue, this paper, which explores the inheritance relations in the process of multi-level semantic increment, proposes an Uncertainty-Aware Hierarchical Refinement (UAHR) scheme.

Original: Our proposed scheme consistsof a global representation extension strategy that enhances the discrimination ofincremental representation by widening the corresponding margin distance, and ahierarchical distribution alignment strategy that reﬁnes the distillation process byexplicitly determining the inheritance relationship of the incremental class.
Revised: Specifically, our proposed scheme consists of a global representation extension strategy that enhances the discrimination of incremental representation by widening the corresponding margin distance, and a hierarchical distribution alignment strategy that refines the distillation process by explicitly determining the inheritance relationship of the incremental class.

Original: Particularly, the shifting subclasses are corrected under the guidance of hierarchicaluncertainty, ensuring the consistency of the homogeneous features.
Revised: Particularly, the shifting subclasses are corrected under the guidance of hierarchical uncertainty, ensuring the consistency of the homogeneous features.

Original: Extensive experiments on  benchmarks IIRC-CIFAR and IIRC-ImageNet-lite demonstrate thesuperiority of our proposed method over the state-of-the-art approaches.
Revised: Extensive experiments on widely used benchmarks ( i.e. , IIRC-CIFAR, IIRC-ImageNet-lite, IIRC-ImageNet-Subset, and IIRC-ImageNetfull) demonstrate the superiority of our proposed method over the state-of-the-art approaches.

Original:  Introduction
Revised:  Introduction

Original: In recent years, deep learning has made huge breakthroughs in the ﬁeld of computer vision, matching or even surpassing human performance on some image recognition tasks.
Revised: In recent years, deep learning has made huge breakthroughs in the field of computer vision, matching or even surpassing human performance on some image recognition tasks [3].

Original: However, learningmultiple tasks  in sequential data ( i e. , continual learning) remains a major challenge, which requiresmodels to have the ability to aggregate different learning objectives into a coherent whole over time.
Revised: However, learning multiple tasks [25] in sequential data ( i.e. , continual learning [20, 22]) remains a major challenge, which requires models to have the ability to aggregate different learning objectives into a coherent whole over time.

Original: When a new task comes with the increase of identiﬁed classes ( i e. , class-incremental learning), jointtraining with all old and new data is too time-consuming and labor-intensive.
Revised: When a new task comes with the increase of identified classes ( i.e. , class-incremental learning) joint training with all old and new data is too time-consuming and labor-intensive.

Original: Furthermore, mostdata from past tasks are unavailable.
Revised: Furthermore, most data from past tasks are unavailable due to the data privacy.

Original: A simple alternative is to directly ﬁne-tune the network withnew data.
Revised: To adapt rapidly to the new scenarios, previous methods [26] try to adopt a simple alternative that directly fine-tune the network with new data.

Original: In this case, the performance of the old classis severely degraded due to the bias of therepresentation and classiﬁer towards the new class, which is also known as catastrophic forgetting.
Revised: However, this may severely degrade the performance of the old class due to the bias of the feature extractor and classifier towards the new class [15], which is also known as catastrophic forgetting.

Original: To address the issue, existing methods  maintain the performance of the old class by preserving therepresentative samples ( i.e. , exemplar) and aligning the output distribution ( i.e. , knowledge distillation).
Revised: To address the issue, existing methods [16] maintain the performance of the old class by preserving the representative samples ( i.e. , exemplar) and aligning the output distribution ( i.e. , knowledge dis- tillation).

Original: The joint optimization with incremental samples strikes a balance between stability andplasticity, resulting in better predictions that fall into the label space of the old or new class.
Revised: This joint optimization with incremental samples strikes a balance between stability and plasticity, resulting in better predictions that fall into the label space of the old or new class.

Original: However, is incremental learning always a binary classiﬁcation option that distinguishes between the old and new class?
Revised: However, is incremental learning always a binary classification option that distinguishes between the old and new class?

Original: In existing setting ( e.g. , CIFAR100 in Fig.
Revised: In existing incremental learning settings ( e.g. , ImageNet in Fig.

Original: 1), each instance is arranged in a certain incremental stage in an exclusive manner.
Revised: 1(a)), each instance is arranged in a certain incremental stage in an exclusive manner.

Original: However, in real life, people’s semanticunderstanding of the same instance may be gradually enriched as the learning process proceeds.
Revised: However, in real life, people’s semantic understanding of the same instance may be gradually enriched as the learning process proceeds.

Original: Asshown inthe lower row of Fig. 1, the semantics of the same image is reﬁned from vehicles to busin the IIRC-CIFAR setting.
Revised: As shown in Fig. 1(b), the semantics of the same image is refined from Garment to miniskirt in the IIRC-ImageNet setting.

Original: This paper focuses on the incremental ability to progressively learn and maintain multi-level semantic information, which is also known asIncremental Implicitly-Reﬁned
Revised: This paper focuses on the incremental ability to progressively learn and maintain multi-level semantic information, which is also known as

Original: Classiﬁcation ( i.e. IIRC)
Revised: Incremental Implicitly-Refined Classification ( i.e. IIRC)

Original: How to discern the semantic inheritance relationship in a hierarchical incremental scenery?
Revised: How to discern the semantic inheritance relationship in a hierarchical incremental scenery?

Original: We ﬁnd that the incremental performance for multi-level recognition decreases dramatically whenapplying classical class-incremental learning methods directly to  IIRC setting.
Revised: We find that the incremental performance for multi-level recognition decreases dramatically when applying classical class-incremental learning methods directly to the IIRC setting.

Original: Our analysis ismainly attributed to the following reasons.
Revised: Our analysis is mainly attributed to the following reasons.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 2, on the one hand, although theprobability distributions of classes with inheritance relationships show a obvious consistency in theinitial phase, the incremental subclass inherited from a certain old class gradually outgroups underthe supervision of new labels.
Revised: 2, on the one hand, although the probability distributions of classes with inheritance relationships show an obvious consistency in the initial phase, the incremental subclass inherited from a certain old class gradually outgroups under the supervision of new labels.

Original: It destroys the integrity of the representation of the whole old class,thus losing the semantic relevance of the hierarchical labels.
Revised: It destroys the integrity of the representation of the whole old class, thus losing the semantic relevance of the hierarchical labels.

Original: On the other hand, some incrementalclasses inherit from none of the existing classes, leading tothe feature confusion due to the lack ofold supervision.
Revised: On the other hand, some incremental classes inherit from none of the existing classes, leading to feature confusion due to the lack of old supervision.

Original: To this end, we propose an Uncertainty-Aware Hierarchical Reﬁnement (UAHR) scheme, whichexploits the correlation of hierarchical distributions to guide the optimization of incremental representation.
Revised: To this end, we propose an Uncertainty-Aware Hierarchical Refinement (UAHR) scheme, which exploits the correlation of hierarchical distributions to guide the optimization of incremental representation.

Original: A global representation extension strategy is proposed to widen the distribution distanceamong all new classes in the embedding space, enhancing their discriminative properties.
Revised: Concretely, a global representation extension strategy is proposed to widen the distribution distance among all new classes in the embedding space, enhancing their discriminative properties.

Original: A hierarchical distribution alignment strategy is further proposed to correct the optimization of the shiftingsubclasses by aligning with the distribution of the whole superclass, ensuring the consistency ofthe hierarchical uncertainty.
Revised: Furthermore, a hierarchical distribution alignment strategy is further proposed to correct the optimization of the shifting subclasses by aligning with the distribution of the whole superclass, ensuring the consistency of the hierarchical uncertainty.

Original: Speciﬁcally, we use RBF mapping to explicitly measure the distancebetween the training samples and the class centers, quantifying the feature correlation.
Revised: In this way, we use RBF mapping to explicitly measure the distance in the feature space between the training samples and the class centers, quantifying the feature correlation.

Original: In the incremental phase, we calculate the entropy distribution of new classes in the old embedding spacefor the estimate of hierarchical uncertainty.
Revised: In the incremental phase, we calculate the entropy distribution of new classes in the old embedding space for the estimate of hierarchical uncertainty.

Original: The resulting differences and similarities are utilized toidentify the multi-level semantic relationships, reﬁning the subsequent distillation objectives.
Revised: The resulting differences and similarities are utilized to identify the multi-level semantic relationships, refining the subsequent distillation objectives.

Original: Comprehensive experimental results on IIRC-CIFAR and IIRC-ImageNet-lite datasets demonstrate thesuperiority of our method.
Revised: Comprehensive experimental results on IIRC-CIFAR, IIRCImageNet-lite, and IIRC-ImageNet-full datasets demonstrate the superiority of our method.

Original:  Related Work2.1 Class-Incremental Learning
Revised:  Related Work 2.1 Class-Incremental Learning

Original: Existing class incremental learning methods can be mainly divided into four types.
Revised: Existing class incremental learning methods can be mainly divided into four types.

Original: Regularizationbased methods preserve high-weight parameters by estimating the importance of individual modelparameters [1] [2] [3], while allowing unimportant parameters to be updated ﬂexibly to ensure thelearning ability for new classes.
Revised: Regularizationbased methods preserve high-weight parameters by estimating the importance of individual model parameters [2, 12, 24], while allowing unimportant parameters to be updated flexibly to ensure the learning ability for new classes.

Original: However, in the study of [4], it is shown that such methods havepoor generalization performance in class incremental learning. 
Revised: However, in the study of [20], it is shown that such methods have poor generalization performance in class incremental learning. [Figure]

Original: Distillation-based methods encourage the model to learn new tasks, while the model representation obtained by the current trainingdata is forced to mimic the representation of the old model.
Revised: Distillation-based methods encourage the model to learn new tasks, while the model representation obtained by the current training data is forced to mimic the representation of the old model.

Original: The new model in [5] completes knowledge distillation by matching the output of the sigmoid function with the old model.
Revised: The new model in [14] completes knowledge distillation by matching the output of the sigmoid function with the old model.

Original: LUCIR [6]mitigates forgetting by distilling the output of  softmax function with the temperature scale and L2regularization.
Revised: LUCIR [10] mitigates forgetting by distilling the output of the softmax function with the temperature scale and L2 regularization.

Original: LwM [7] reduces forgetting by matching gradient attention maps from the highestscoring labels.
Revised: LwM [7] reduces forgetting by matching gradient attention maps from the highest-scoring labels.

Original: PODNet [8] performs knowledge distillation between old and new models by retain- [Figure] ing intermediate features pooled along width and height, and controls the balance between previousknowledge and new information, resulting in a more adaptive incremental representation.
Revised: PODNet [8] performs knowledge distillation between old and new models by retaining intermediate features pooled along width and height, and controls the balance between previous knowledge and new information, resulting in a more adaptive incremental representation.

Original: Rehearsal-based methods work by storing a limited number of representative examples or adoptinggenerative model for old class samples when training new tasks.
Revised: Rehearsal-based methods work by storing a limited number of representative examples or adopting a generative model for old class samples when training new tasks.

Original: Incremental Classiﬁer Representation Learning (iCaRL) [9] retains a small number of samples for each class to approximate classcentroids and make predictions based on the nearest class average classiﬁer.
Revised: Incremental Classifier Representation Learning (iCaRL) [16] retains a small number of samples for each class to approximate class centroids and make predictions based on the nearest class average classifier.

Original: [14] generates previously observed class samples by using generative adversarial networks (GANS) [15].
Revised: [18] generates previously observed class samples by using generative adversarial networks (GANS) [9].

Original: Structurebased methods  keep the learned parameters related to the previous classes unchanged, and assignnew parameters in different forms, such as unused parameters, extra networks, etc., to learn newknowledge.
Revised: Structurebased methods [27] keep the learned parameters related to the previous classes unchanged, and assign new parameters in different forms, such as unused parameters, extra networks, etc., to learn new knowledge.

Original: DER [16] concatenates the extracted features by adding a new feature extractor in theincremental phase.
Revised: DER [23] concatenates the extracted features by adding a new feature extractor in the incremental phase.

Original: A sparsity error is adopted to encourage the model to compress parameters asmuch as possible.
Revised: A sparsity error is adopted to encourage the model to compress parameters as much as possible.

Original: Different from  existing incremental method focusing on the invariant semanticconcepts, our method is designed for the progressive understanding of the incremental semanticinformation, facilitating better feature update and retention.
Revised: Different from the existing incremental methods focusing on the invariant semantic concepts, our method is designed for the progressive understanding of the incremental semantic information, facilitating better feature update and retention.

Original:  Our Method3.1 Problem Description and Analysis
Revised:  Our Method 3.1 Problem Description and Analysis

Original: In real life, the human brain gradually enriches the semantic understanding of a object as the learningprocess.
Revised: In real life, the human brain gradually enriches the semantic understanding of an object as the learning process advances.

Original: For example, we only knew that rats were small animals when we were young, but weunderstood that rats were rodents as we grew up.
Revised: For example, we only know that rats are small animals in the young phase, but we understand that rats are rodents as we grow up.

Original: To simulate this process, we adopt the incrementalimplicitly-reﬁned classiﬁcation task (IIRC) as the fundamental task, which is arranges into N phasesin the order of steam data, denoted as { T 1 , · · · , T N }.
Revised: To simulate this process, we follow the incremental implicitly-refined classification task (IIRC), which is arranged into N phases in the order of data stream with hierarchical labels, denoted as { T 1 , · · · , T N } .

Original: Each sample can have one or two labels.
Revised: Different from the traditional incremental learning settings, the samples of incremental phases in IIRC may have one or two labels ( i.e., hierarchical labels).

Original: Inthe case of two labels, one label  is a subclass, and the other label is a superclass.
Revised: Specifically, the hierarchical labels refer to that the one label of a sample is a subclass ( i.e., hamster), and the other one is a superclass ( i.e., small animal).

Original: The number ofsamples in the superclass  is always larger than the subclass, and it increases with the number ofsubclasses.
Revised: The number of samples with superclass labels is always larger than that with subclass labels, which increases with the number of subclasses.

Original: During training, we always follow an incomplete information setup, i.e. , samples withmultiple labels are only provided with the label in the current phase.
Revised: During training, we always follow an incomplete information setup, i.e. , samples with multiple labels are only provided with the label in the current phase.

Original: In the ﬁrst phase, only  a subsetof all superclasses is used to train the model.
Revised: We only use a subset of all superclasses in the first phase to train the model.

Original: Subsequent phases contain a mixture of superclassesand subclasses.
Revised: Subsequent phases contain a mixture of superclasses and subclasses.

Original: In the training phase, the model is validated in the in-phase validation set whichonly contains current phase labels.

Original: In the evaluation phase, the model is evaluated on the post-phasevalidation set and the test set (both of them contain the complete labels).

Original: In this setting, we face two challenging problems: (1) How to judge the relationship between labelsacross phases, especially the superclass-subclass relationship?
Revised: We face two challenging problems in this setting: (1) How to judge the relationship among labels across phases, especially the superclass-subclass relationship?

Original: And how to guide the optimization ofclass distribution to fall into the distribution of  superclass exactly?
Revised: Moreover, how to guide the optimization of the novel class by means of the distribution relationship with superclass exactly?

Original: (2) How to reduce theconfusionamong the feature distributions of new classes in the incremental phase?
Revised: (2) How do we reduce the feature confusion among these novel classes in the incremental phase?

Original: Therefore, We proposetwo strategies.
Revised: Therefore, We propose two strategies.

Original: (1) Global Representation Extension.
Revised: (1) Global Representation Extension.

Original: During the new learning phase, the uncertaintybetween each class center is calculated by the RBF kernel. Then the uncertainty  is minimized fortraining.
Revised: We use the RBF kernel to calculate the uncertainty among all class centers, which is minimized for training during the incremental phase.

Original: (2) Hierarchical Distribution Alignment.
Revised: (2) Hierarchical Distribution Alignment.

Original: The uncertainty of the classes across stages arecalculated by calculating the output entropy of the new class samples with  old model.
Revised: We firstly calculate the output entropy of the new class samples with the old model, estimating the uncertainty of the classes across phases.

Original: Then it is used to discriminate the superclass-subclass relationship and the brother relationship.
Revised: Then we use the obtained uncertainty to discriminate the superclass-subclass relationships and other relationships.

Original: Finally, theoptimization direction of the new model is guided by changing the  output of the old model.
Revised: Finally, we update the old model’s output to guide the new model’s optimization direction.

Original: Thus,we can keep the  invariance of the superclass distribution.
Revised: Thus, we can keep the distribution invariance between the subclass and the superclass.

Original: 3.2 Global Representation Extension
Revised: 3.2 Global Representation Extension

Original: In the incremental phase, optimizing only the data of new classes  causes great bias in the representations and classiﬁers.
Revised: Optimization with only the data of new classes during the incremental phase causes great bias in the representation and classifier learning.

Original: To maintain the stability of the feature space, we use the  distancebetweenrepresentations as a measure of uncertainty.
Revised: To maintain the stability of the feature space, we use the representation distance as a measure of uncertainty.

Original: The optimization of new classes is achieved by comparing the output entropy with the target labels.
Revised: New class representation is optimized by comparing the output entropy with the target labels.

Original: While the stability of the old class representationsis ensured by maintaining the same output between the oldmodel andthe new model with the newsamples.
Revised: At the same time, the stability of the old class representation is ensured by maintaining the same output between the old and new models with the new samples.

Original: Speciﬁcally, we use the RBF distance [13] for thecalculation of representation extensionloss.
Revised: Specifically, we use the RBF distance [19] to calculate the representation extension loss.

Original: We use the RBF kernel to map representation to inﬁnite dimensional space for comparison,which is shown as follows,
Revised: We use the RBF kernel to map representation to infinite dimensional space for comparison, which is shown as follows,

Original: [Equation]
Revised: [Equation]

Original: where K ( ∗ ) denotes the Radial basis function kernel(RBF kernel), n b denotes the number of classescontained in the current batch, h θ ( ∗ ) denotes the  output features of the picture after global pooling,n d denotes the output dimension after global pooling, and h θ ( ∗ ) i denotes the average feature valueof a particular class i .
Revised: where K ( · ) denotes the Radial Basis Function kernel (RBF kernel [11]), n b denotes the number of classes contained in the current batch, h θ ( · ) denotes the backbone of our net, we can get the output features of the picture from it, n d denotes the output dimension after global pooling, and h θ ( · ) c denotes the average feature value of a particular class c .

Original: The symbol j in the formula is not equal to c , and j is the class number withthe smallest distance to class c in the feature space.
Revised: The symbol j represents the class number with the smallest distance to class c in the feature space, which is not equal to c .

Original: The optimization of this loss forces the extensionof the distance between the class centers, obtaining better discrimination in the class distribution.
Revised: The optimization of this loss forces the extension of the distance among the class centers, obtaining better discrimination in the class distribution.

Original: Subsequently, we use a linear layer to output the entropy of the samples to judge the class of thesesamples.
Revised: Subsequently, we use a linear layer to get the entropy of the samples, then judge the class of these samples.

Original:  BCEWithLogitsLoss is adopted for optimization, which is shown as follows,
Revised: We adopt the BCEWithLogitsLoss to optimize the learning, which is shown as follows,

Original: [Equation]
Revised: [Equation]

Original: Where Y i denotes the ground truth label of the sample i, N denotes the batch size of training, F idenotes the feature of the i th sample,  f θ  denotes the classiﬁer of the current model, note that thelabels here as well as the ﬁnal output of the model contain only the classes of  current phase.
Revised: where y i denotes the ground truth label of the sample i , N denotes the batch size of training, x i denotes the i th sample, and f θ ( · ) denotes the classifier of the current model. It should be noted that the labels here as well as the final output of the model contain only the classes of the current phase.

Original: Theoptimization of this loss allows the model to eventually judge the label of a sample by whether itsoutput is greater than 0.
Revised: The optimization of the loss allows the model to eventually judge the label of a sample by whether its output is greater than 0.

Original: And the closer  to the class center, the larger the output value.
Revised: And the closer the corresponding representation is to the class center, the larger the output value.

Original: Finally, weuse the output value of the new sample on the old model as a supervised signal.
Revised: Finally, we use the output value of the new sample on the old model as the supervised signal of the distillation process.

Original: Thus, the currentmodel learns new classes while maintaining the knowledge of the old model. As shown below:
Revised: Thus, the current model learns new classes while maintaining the knowledge of the old model, which is shown below:

Original: [Equation]
Revised: [Equation]

Original: Where h θ () denotes the backbone of the current model, f θ () denotes the classiﬁer of the currentmodel, h ∗ θ () denotes the backbone of the old model, f ∗ θ () denotes the classiﬁer head of the oldmodel, α is a scale superparameter of distillation,  n old denotes the number of classes that have beenlearned.
Revised: ) /α )) , (3) where h θ ( · ) denotes the backbone of the current model, f θ ( · ) denotes the classifier of the current model, h ∗ θ ( · ) denotes the backbone of the old model, f ∗ θ ( · ) denotes the classifier head of the old model, α is a scale hyperparameter of distillation, and n old denotes the number of classes that have been learned.

Original: x is a training sample of the current phase.
Revised: x is a training sample from the current phase.

Original: By optimizing this loss, the distribution ofoldclass features keeps stable in the  feature space.
Revised: The optimization of this loss facilitates the stability of the old class feature distribution.

Original: 3.3 Hierarchical Distribution Alignment
Revised: 3.3 Hierarchical Distribution Alignment

Original: In  incremental phase, focusing on the label relationshipsacross stages is the key to solving the IIRCproblem.
Revised: In the incremental phase, focusing on the label relationships is the key to solving the IIRC problem.

Original: The old model with ﬁxed parameters can map samples into the old feature space, while thenew subclass samples can be mapped into the space, which is close to the corresponding superclassdistribution.
Revised: Firstly, the new subclass samples can be mapped into the embedding space by the old model with fixed parameters, which is close to the corresponding superclass distribution.

Original: We calculate the uncertainty of the labels across phases by counting the entropy of newsamples on old model. Then the label relationships can be inferred from the hierarchical uncertainty.
Revised: Then the label relationships can be inferred from the hierarchical uncertainty, which is calculated by counting the entropy of new samples on the old model across phases.

Original: At the same time, we ﬂexibly adjust the distribution border, to correct the shift of subclass samplesand mitigate the confusion caused by the brother class.
Revised: At the same time, we flexibly adjust the distribution border to correct the shift of subclass samples and mitigate the confusion caused by the brother class.

Original: Speciﬁcally, as shown in Fig.
Revised: Specifically, as shown in Fig.

Original: 3, we take the new class C as an example.
Revised: 3, we take the new class C as an example.

Original: We obtain all the mappingfeatures on the old model with the new samples of class C and the output entropy as well. Thenwe count the single-label output for each phase and sample.
Revised: Firstly, we obtain all mapped features and corresponding output entropy of each old class on the old model with the new samples of class C .

Original: According to the single label matrix,we calculate the uncertainty of a single label, by counting the number ofsamples, which are judgedto belong to that single class. And then making the number divide by the number of total samples.
Revised: Then, according to the obtained output entropy, the ratio of each old class distribution is calculated. This ratio is the hierarchical uncertainty of the current added class C samples to each old class label.

Original: Then we process the binarization of uncertainty results. The label with ﬁnal result 1 has asimilar distribution to  the current speciﬁc class C (possibly a  superclass-subclass or brother relationship).
Revised: The obtained hierarchical uncertainty is binarized to determine the corresponding old class with a similar distribution (possibly superclass-subclass or brother class relationship) to the current specific class C .

Original: After completing the initial matching of uncertainty distributions, more precise distributionmatching is accomplished by the standard deviation( i.e. , std) values of the global pooling features.

Original:  We calculate the standard deviation of each similar old class with the same operation.
Revised: Finally, We calculate the standard deviation of each similar old class with the same operation.

Original: Since directsuperclass-subclass relationships have closer standard deviation values on the feature distributions.
Revised: Direct superclass-subclass relationships have closer standard deviation values on the feature distributions.

Original: So the  old classlabel with the closest std value is judged as  the superclass of the current class C, andthe rest of the classes are brother classes.
Revised: So the similar old class with the closest std value is judged as [Table] the superclass of the current class C , and the rest of the classes are brother classes.

Original: The detailed procedure is shown in Algorithm 1.
Revised: The detailed procedure is shown in Algorithm 1.

Original: After obtaining the label relationships, we use  selective distribution alignment distillation mechanism to guide the representation learning of models.
Revised: After obtaining the label relationships, we use the selective distribution alignment distillation mechanism to guide the representation learning of models.

Original: When the new class distribution falls largelywithin the old class, but there is still a partial shift.

Original: We add a margin distance to the output entropyof the corresponding old class. The margin distance has a ﬁxed value, as our hyperparameter whencorrecting the inter-hierarchical uncertainty based on output entropy. When the new class distribution has a crossover interval with the old class, but the gap of standard deviation between thedistributions is large, we need to perform the reverse optimization bysubtracting a margin distance
Revised: According to the labeling relationship, we can achieve the alignment of the distribution and mitigate the confusion caused by the brother class. On one hand, when the features of the novel sample have a superclass shift, its output entropy value of the corresponding superclass on the old model is improved by a margin distance.

Original: [Table]
Revised: Thus, the shifting samples are guided within the corresponding superclass distribution.

Original: value from the output entropy  of  the oldclass. So we can achieved the alignment of the distributionand mitigate the confusion caused by the brother class  by this strategy.
Revised: On the other hand, the output entropy value of a novel sample that lacks a superclass but falls within the old class distribution is subtracted by a margin distance.

Original: Speciﬁcally, according to the label relationships, when the current new class C is not related to theold class, the highest output entropy by the sample of class C on the old model, is subtracted by amargin value.
Revised: Specifically, according to the label relationships, when the current new class C is not related to an old class, the highest output entropy by the sample of class C on the old model is subtracted by a margin value.

Original: And when the current new class C is a subclass of the old class, the output entropyvalue of the class C sample on the old model, which is corresponding to the superclass, is added onemargin value.
Revised: Moreover, when the current new class C is a subclass of an old class, the output entropy value of the class C sample on the old model, which corresponds to the superclass, is added with one margin value.

Original: At the same time, the highest output entropy value of the non-superclass is subtractedby a margin value.
Revised: At the same time, the highest output entropy value of the non-superclass is subtracted by a margin value.

Original: Our selective distribution alignment distillation loss can be obtained by:
Revised: Our selective distribution alignment distillation loss can be obtained by:

Original: [Equation]
Revised: [Equation]

Original: where y new is the new output of the old model, after performing our hierarchical distribution alignment strategy.
Revised: where y new is the new output of the old model, after performing our hierarchical distribution alignment strategy.

Original: In the end, our complete loss is:
Revised: In the end, our complete loss is:

Original: [Equation]
Revised: [Equation]

Original: where γ is the learning weight for  the optimization of the global representation extension.
Revised: where γ denotes hyper-parameters for balancing the losses.

Original: l cls andl dis represents the same BCEwithLogitsLoss, in which the outputs are of the same magnitude, weset their ratio to 1:1 for close loss values.

Original: And in our experiments, γ is set as 10.0.
Revised: Moreover, in our experiments, γ is set as 10.0.

Original: A detaileddescription of the hyperparameter selection is shown inthe supplementary material B.2.
Revised: A detailed description of the hyperparameter selection is shown in supplementary material B.2.

Original:  Experiments 
Revised:  Experiments 4.1 Dataset and Settings

Original: 4.1 Dataset and Settings

Original: [Figure] Based on the semantic relevance of labels, CIFAR100 [18] and ImageNet [19] datasets are integratedto form datasets, where class labels are two-level hierarchy[17].
Revised: According to the semantic relevance among labels, CIFAR100 [13] and ImageNet [6] datasets are rearranged to form the two-level hierarchy datasets [1].

Original: Each label starts as a leaf node (i.e.,subclass) and similar labels are assigned a common parent node (i.e., superclass).
Revised: Each label starts as a leaf node (i.e., subclass), and similar labels are assigned a common parent node (i.e., superclass).

Original: The integrateddatasets are called IIRC-CIFAR and IIRC-ImageNet-full.
Revised: The integrated datasets are called IIRC-CIFAR and IIRC-ImageNet-full.

Original: IIRC-CIFAR.
Revised: IIRC-CIFAR.

Original: Ten superclasses are setup, each with about 4 to 8 subclasses.
Revised: Ten superclasses are set up, each with about 4 to 8 subclasses.

Original: In incremental phases, each new phase introduces 5 classesfor IIRC-CIFAR.
Revised: In incremental phases, each new phase introduces five classes.

Original: In total, IIRC-CIFAR involves 22 phases with 10 preset class ordersfor multipletests, which is calledas phase conﬁguration.
Revised: IIRCCIFAR involves 22 phases with ten preset class orders called phase configuration for multiple tests.

Original: IIRC-ImageNet-full.
Revised: IIRC-ImageNet-full.

Original: In IIRC-ImageNet-full, sixtythree superclasses are set up, and the number of subclasses that belong to one superclasses variesgreatly, from 3 to 118.
Revised: In IIRC-ImageNet-full, sixty-three superclasses are set up, and the number of subclasses that belong to one superclass varies greatly, from 3 to 118.

Original: There are a total of 35 phases with 30 classes per phase.
Revised: There are a total of 35 phases, with 30 classes per phase.

Original: 5 preset class ordersare ﬁxed for multiple tests.
Revised: Five preset class orders are fixed for multiple tests.

Original: IIRC-ImageNet-lite is a shorter, lighter version which has just ten tasks(with ﬁve tasks conﬁgurations), while referring to both collectively as  IIRC-ImageNet-full.
Revised: IIRC-ImageNet-lite is a shorter, lighter version with just ten phases (with five task configurations), which is referred as IIRC-ImageNet together with IIRC-ImageNet-full.

Original: IIRCImageNet-Subset [11], as a simpliﬁed version of the IIRC-ImageNet-full, has 10 superclasses and100 subclasses (including those have no superclass labels).
Revised: IIRC-ImageNet-Subset [21], as a simplified version of the IIRC-ImageNet-full, involves ten superclasses and 100 subclasses.

Original: The superclasses of each dataset are combined by extracting 40% samples from each subclass, whilethe corresponding subclasses are saved with 80% samples, so the superclasses and subclasses share20% samples.
Revised: The superclasses of each dataset are combined by extracting 40% samples from each subclass, while the corresponding subclasses are saved with 80% samples. That means that the superclasses and subclasses share 20% samples.

Original: When the superclass contains more than 8 subclasses, the samples of the subclassesare extracted in the proportion of
Revised: When the superclass contains more than eight subclasses, the samples of the subclasses are extracted in the proportion of

Original: ∗ 40% .
Revised: ∗ 40% .

Original: The training process follows theincomplete information setting, i.e. , if a sample has more than one label, only the label of the currentphase is provided.
Revised: The training process follows the incomplete information setting, i.e. , if a sample has more than one label, only the label of the current phase is provided.

Original: The validation set also follows the incomplete information setting.
Revised: The validation set also follows the incomplete information setting.

Original:  After a certainnumber of training phases, the overall performance is evaluated on a validation setwith completeinformation setting and a test set as well. Both of them contain samples of all seen classes and theircorresponding complete labels.[Table]
Revised: [Table] After a certain number of training phases, the overall performance is evaluated on a validation set and a test set under the complete information setting, which contains all seen class labels.

Original: Metrics.
Revised: Metrics.

Original: The evaluation metric uses the precision-weighted Jaccard similarity (PW-JS) to measurethe performance of the model on phase k after training on phase j, as follows:
Revised: The evaluation metric uses the precision-weighted Jaccard similarity (PW-JS) to measure the performance of the model on phase k after training on phase j, as follows:

Original: [Equation]
Revised: [Equation]

Original: In the formula j ≥ k , b Y ki is the prediction value of the model for i th samples of k th phase, Y ki is theground truth, and n k is the number of samples in this phase.
Revised: In the formula j ≥ k , (cid:98) Y ki is the prediction value of the model for i th samples of k th phase, Y ki is the ground truth, and n k is the number of samples in this phase.

Original: And for the evaluation of the overallperformance, we use the average precision-weighted Jaccard similarity of the model after trainingon phase j for all seen classes, as follows.
Revised: Moreover, for the evaluation of the overall performance, we use the average precision-weighted Jaccard similarity of the model after training on phase j for all seen classes, as follows:

Original: [Equation]
Revised: [Equation]

Original: 4.2 Ablation Study
Revised: 4.2 Ablation Study

Original: To prove the effectiveness of our method, we conduct a set of ablation experiments on the IIRCCIFAR dataset.
Revised: To prove the effectiveness of our method, we conduct a set of ablation experiments on the IIRCCIFAR dataset.

Original: One of the important components in our scheme is the global representation extension strategy, referred to as the GRE component.
Revised: One of the essential components in our scheme is the global representation extension strategy, referred to as the GRE component.

Original: The other is  hierarchical distribution alignmentstrategy, referred to as the HDA component.
Revised: The other is the hierarchical distribution alignment strategy, referred to as the HDA component.

Original: To verify the functionality of the GRE and HDA components, we conducted experiments in the case of phase conﬁguration 0 of IIRC-CIFAR.
Revised: To verify the functionality of the GRE and HDA components, we conducted experiments in the case of phase configuration 0 of IIRC-CIFAR.

Original: As shownin Table 1, the HDA component alone achieves a improvement of 0.6% in the last phase, whilethe GRE component alone achieve a 0.54% in the last phase.
Revised: As shown in Table 1, the HDA component alone achieves an improvement of 0.6% in the last phase, while the GRE component alone achieves 0.54% in the last phase.

Original: And with both components added,1.27% increase over baseline is achieved.
Revised: Furthermore, with both components added, a 1.27% increase over baseline is achieved.

Original: It proves that both components play a positive role in theevaluation of the overall incremental performance.
Revised: It proves that both components play a positive role in evaluating the overall incremental performance.

Original: In Fig.
Revised: In Fig.

Original: 5 (C), we show the  performance of each phase, which is evaluated over the test samples ata speciﬁc phase j, after training on that phase( R jj using Equation 6).
Revised: 6 (C), we show the incremental performance at each phase, which is evaluated over the test samples at a specific phase j, after training on that phase( R jj using Equation 6).

Original: Such cure graphs evaluatethe learning ability for new classes.
Revised: Such cure graphs evaluate the learning ability for new classes.

Original: We can see that the HDA component alone achieves a greatimprovement, whileadding the GRE component further enhances the learning abilitywith a smalloptimization on top of the HDA component.[Figure]
Revised: We can see that the HDA component significantly improves the average performance while the GRE component further enhances the learning ability on top of the HDA component.

Original: 4.3 Analysis
Revised: 4.3 Analysis

Original: The impact of global representation extension.
Revised: The impact of global representation extension.

Original: To explore the impact of  global representationextension strategy, we visualize the t-SNE features of the baseline and our method.
Revised: To explore the impact of the global representation extension strategy, we visualize the t-SNE features of the baseline and our method.

Original: We select datafrom phase 0 and phase 4 of IIRC-CIFAR withthe phase conﬁguration 0.
Revised: We select data from phase 0 and phase 4 of IIRC-CIFAR with phase configuration 0.

Original: In the upper part of Fig.4, we can see that there is an overlap among the representation distributions of the new classes inthe baseline method.
Revised: In the upper part of Fig. 4, we can observe an overlap among the representation distributions of the new classes in the baseline method.

Original: While the overlapbetween the increasing classes is weakened and  the distancebetween the increasing class feature distributions is expanded in our method.
Revised: Our method weakens the overlap and expands the distribution distance among the novel class. Fig.

Original: This proves that usingthe distance between representations as a measure of  uncertainty  is effective and the reduction ofthis uncertainty leads to theextension of the distance  between the individual class centers.
Revised: 4 proves that the usage of representation distance for the uncertainty estimate is adequate. Reducing this uncertainty leads to the distance extension between the individual class centers.

Original: The impact of the hierarchical uncertainty.
Revised: The impact of the hierarchical uncertainty.

Original: To explore the effect of hierarchical uncertainty, weﬁrst compare different uncertainty methods.
Revised: To explore the effect of hierarchical uncertainty, we first compare different uncertainty methods.

Original: A detailed version of Table 2 is presented in A.3 of Supplementary Materials.
Revised: A detailed version of Table 2 is presented in A.3 of the supplementary materials.

Original: As shown in Table.
Revised: As shown in Table.

Original: 2, the "Common Uncertainty" denotes an entropy- [Figure] based uncertainty methodwhich is widely used in the OOD [20].
Revised: 2, the "Common Uncertainty" denotes an entropybased uncertainty method widely used in the OOD [5].

Original: We use it to construct the labelrelationships.
Revised: We use it to construct label relationships.

Original: It can be seen from the Table.
Revised: It can be seen from the Table.

Original: 2 that our method achieves better experimental results,which proves that the entropy outputof the network and the standard deviation of the features havethe greatest impact on the cross-hierarchical uncertainty estimation among the statistical propertiesof the superclass and the corresponding subclass.
Revised: 2 that our method achieves better experimental results, proving that the entropy output and the standard deviation of the features exhibit the most significant impact on the cross-hierarchical uncertainty estimation among the statistical properties of the superclass and the corresponding subclass.

Revised: [Figure]

Original: To explore the effect of the hierarchical distribution alignment strategy. We visualize the t-SNEfeatures for the baseline and our method on  IIRC-CIFAR dataset with phase conﬁguration 0 by selecting data from phase 0 and phase 11.
Revised: To explore the effect of the hierarchical distribution alignment strategy, we visualize the t-SNE features for the baseline and our method on the IIRC-CIFAR dataset with phase configuration 0 by selecting data from phase 0 and phase 11.

Original: In the bottom half of Fig. 4, we can seethat there is a largeshift between the distributions of subclasses and corresponding superclasses in  baseline.
Revised: In the bottom half of Fig. 4, we can see a significant shift between the distributions of subclasses and corresponding superclasses in the baseline.

Original: Whilethisshift is greatly corrected in our method, the distribution of the increasing subclasses falls in thedistribution of the corresponding superclasses.
Revised: While our method extensively corrects this shift and promotes the distribution of the novel subclasses falling in the corresponding superclasses.

Original: It demonstrates that adjusting the optimization directionabout the feature distribution according to the labeling relationship guides the new model to betterlearn the representational relationship between the old and new classes and deepen it gradually.
Revised: It demonstrates that adjusting the optimization direction of the feature distribution according to the labeling relationship helps the new model to learn better the representation relationship between the old and new classes and deepen it gradually.

Original: 4.4 Comparison with SOTA
Revised: 4.4 Comparison with SOTA

Original: To comprehensively measure the performance of our method under IIRC setting, we reproduces theclassical methods  ER [10], AGEM [12], LUCIR [6], iCaRL-CNN [9], iCaRL-norm [9], Podnet [8],
Revised: To comprehensively measure the performance of our method under IIRC setting, we reproduces the classical methods ( i.e. , ER [17], AGEM [4], LUCIR [10], iCaRL-CNN [16], iCaRL-norm [16],

Original:  HCV [11] etc and compare with them.
Revised: Podnet [8], HCV [21]) and compare with them.

Original: Average PW-JS.
Revised: Average PW-JS.

Original: It can be seen in Fig.
Revised: It can be seen in Fig.

Original: 5 (A) that our method achieves the best until the end amongall methods withthe incomplete information in IIRC-CIFAR.
Revised: 6 (A) that our method achieves the best until the end among all methods with incomplete information in IIRC-CIFAR.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 5 (D) , our methodsalso maintains the highest scores in larger IIRC-imagenet-lite, which demonstrates the effectivenessof our methods.
Revised: 6 (D), our method also maintains the highest scores in larger IIRC-Imagenet-lite, demonstrating the effectiveness of our method.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 5 (E), we achieve the SOTA effect on the IIRC-ImageNet-Subsetdataset.
Revised: 6 (E), our method achieves the SOTA result on the IIRC-ImageNetSubset dataset.

Original:  Fig.
Revised: It can be shown in Fig.

Original: 5(F) shows that we ﬁnally achieved the highest score on IIRC-ImageNet-full.
Revised: 6(F) that our method finally achieves the highest score on IIRC-ImageNet-full.

Original: Moredetails are shown inthe A.5 of  supplementary materials.
Revised: More details are shown in A.5 of the supplementary materials.

Original: Performance on new classes.
Revised: Performance on new classes.

Original: The PW-JS values indicating the performance of  newly incrementalphase are shown in Fig.
Revised: The PW-JS values indicating the performance of the newly incremental phase are shown in Fig.

Original: 5(B).
Revised: 6(B).

Original: It can be seen that the ER algorithm and the AGM algorithm aresuperior in the ability to learn new classes.
Revised: It can be seen that the ER algorithm and the AGM algorithm are superior in learning new classes.

Original: Combined with their performance on overall phases,itis clear that they obtain an advantage on the new class at thecomplete expense of the  old class performance.
Revised: Combine with the performance in overall phases, they obtain an advantage in the new classes at the expense of the performance in old classes.

Original: Among all  algorithmsthat are beneﬁcial for suppressingthe catastrophic forgetting, ourmethod achieves the best, and outperforms existing methods inthe ability of learning new classes.
Revised: Among all beneficial algorithms for suppressing catastrophic forgetting, our method achieves the best and outperforms existing methods in learning new classes.

Original: We conduct an additional  set of confusion graph experiments where we extracted the data fromthe IIRC-CIFAR with the phase 0, 1, 2, 7, 12,  17.
Revised: We conduct an additional experimental set on the confusion matrix, where we extract the data from the IIRC-CIFAR with phases 0, 1, 2, 7, 12, and 17.

Original: They are combined and presented in Fig.
Revised: They are combined and presented in Fig.

Original: In the IIRC task, the diagonal values in the confusion matrix represents the prediction of subclasslabels, while the bottom-left corner representsthe one of  superclasses.
Revised: In the IIRC task, the diagonal values in the confusion matrix represent the prediction of subclass [Figure] labels, while the bottom-left corner represents one of the superclasses.

Original: A more accurate confusionmatrix in the lower left corner represents better label relationship maintenance between subclassand superclass.
Revised: A more precise confusion matrix in the lower left corner represents better label relationship maintenance between subclass and superclass.

Original: As shown in Figure 6, our method performs best in both the diagonal and lower leftpositions, proving the superiority of our approach in maintaining label relationships.
Revised: As shown in Fig. 5, our method performs best in both the diagonal and lower left positions, proving the superiority of our method in maintaining label relationships.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, a novel Uncertainty-Aware Hierarchical Reﬁnement schemeis proposed for  IIRC task.
Revised: This paper proposes a novel Uncertainty-Aware Hierarchical Refinement scheme for the IIRC task.

Original: A global representation extension strategy is presented to enhance the discrimination of incrementalclasses, and the misleading distillation process is reﬁned with a hierarchical distribution alignmentstrategy.
Revised: A global representation extension strategy is presented to enhance the discrimination of incremental classes, and the tricky distillation process is refined with a hierarchical distribution alignment strategy.

Original: Consequently, our method involves a multi-level semantic scenery into the incrementallearning.
Revised: Consequently, our method involves a multi-level semantic scenery in incremental learning.

Original: Experimental results show the superiority  our method in both stability and plasticity.
Revised: Experimental results show the superiority of our method in both stability and plasticity.

Revised: Acknowledgments and Disclosure of Funding

Revised: Supported by the National Key R & D Program of China under Grant 2020AAA0105701, the National Natural Science Foundation of China (NSFC) under Grants 61872327, and the Major Special

Revised: Science and Technology Project of Anhui (No. 012223665049).

Revised: This work was supported by Ant

Revised: Group through Ant Research Intern Program.

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for informationon how to answer these questions.
Revised: Please read the checklist guidelines carefully for information on how to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] or [N/A] .

Original: You are strongly encouraged to include a justiﬁcation to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justification to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] See Section 2.•
Revised: [Yes] See Section 2.

Original:  Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [No] The code and the data areproprietary.•
Revised: [No] The code and the data are proprietary.

Original:  Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [N/A] 
Revised: [N/A] 1.

Original: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the

Original: Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See Supplementary Materials.
Revised: [Yes] See Supplementary Materials.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] SeeSupplementary Materials.
Revised: [Yes] See Supplementary Materials.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] See Supplementary Materials.
Revised: [Yes] See Supplementary Materials.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)?
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Supplementary Materials.
Revised: [Yes] See Supplementary Materials.

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: [N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁ- able information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [N/A] 
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]