Margin-Based Few-Shot Class-Incremental Learning with Class-Level Overfitting Mitigation
Anonymous Author(s)
AffiliationAddress
Abstract
Few-shot class-incremental learning (FSCIL) is designed to incrementally recognize novel classes with only few training samples after the (pre-)training on baseclasses with sufficient samples, which focuses on both base-class performance andnovel-class generalization.
A well known modification to the base-class trainingis to apply a margin to the base-class classification.
However, a dilemma existsthat we can hardly achieve both good base-class performance and novel-class generalization simultaneously by applying the margin during the base-class training,which is still under explored.
In this paper, we study the cause of such dilemma for
FSCIL.
We first interpret this dilemma as a class-level overfitting (CO) problemfrom the aspect of pattern learning, and then find its cause lies in the easily-satisfiedconstraint of learning margin-based patterns.
Based on the analysis, we proposea novel margin-based FSCIL method to mitigate the CO problem by providingthe pattern learning process with extra constraint from the margin-based patternsthemselves.
Extensive experiments on CIFAR100, Caltech-USCD Birds-200-2011(CUB200), and mini ImageNet demonstrate that the proposed method effectivelymitigates the CO problem and achieves state-of-the-art performance.
Introduction
With the development of deep learning, deep neural networks gradually demonstrate superior performance on the recognition of pre-defined classes with large amount of training data [18, 10].
However,the model’s generalization capability on the downstream novel classes is much less explored and stillneeds to be improved [13, 9].
To deal with this problem, the few-shot class-incremental learning(FSCIL) task [11, 17, 2, 21, 26, 28] comes into sight.
FSCIL first (pre-)trains a model on a set ofpre-defined classes (base classes) and then generalizes the model to the incremental novel classeswith only few training samples, simulating human’s ability of continually learning novel conceptswith only few examples, and emphasizing both the performance on the pre-defined base classes andthe generalization on the downstream novel classes.
However, a dilemma is recently revealed [13, 5, 6, 7] that better loss functions, which lead to higherperformance on the pre-training data, could lead to worse generalization on the downstream tasks.
As introduced by [15] and depicted in Fig.
1, similar phenomenon also exists in the FSCIL taskthat a positive classification margin [15, 24, 8, 20] applied to the classification of the base-class(pre-)training could lead to higher base-class performance but lower novel-class performance, while anegative margin could result in lower base-class performance but increase the novel-class performance.
Although this dilemma widely exists in the tasks involving novel-class generalization such as few-shotlearning (FSL) and FSCIL, only few works [15] tried to explore its cause, and can hardly be used tohandle it.
Due to space limitation, we will provide extended related works in the appendix.
No Margin
Positive Margin
Negative Margin
Ours
[Figure]
ss
N ov e l C l a ss
In this paper, we study the cause of the dilemma in the margin-based classification for the FSCILproblem from the the aspect of pattern learning.
We find this dilemma can be understood as a classlevel overfitting (CO) problem, which can be interpreted by the fitness of the learned patterns to eachbase class.
The fitness determines how much the learned patterns are specific to some base classes orshared among classes, making the learned patterns either discriminative (tend to overfit base classes)or transferable (tend to underfit base classes) and causes the dilemma.
Based on the interpretation,we discover the cause of the dilemma lies in the easily-satisfied constraint of learning shared orclass-specific patterns.
Therefore, we further design a novel FSCIL method to mitigate the dilemmaof CO by providing the pattern learning process with extra constraint from margin-based patternsthemselves, which achieves improved performance on both base and novel classes as shown in Fig.
1,and achieves state-of-the-art performance in terms of the all-class accuracy.
Our contributions are:
• We interpret the dilemma of the margin-based classification as a class-level overfitting problemfrom the aspect of pattern learning.
• We find the cause of the class-level overfitting problem lies in the easily-satisfied constraint oflearning shared or class-specific patterns.
• We propose a novel FSCIL method to mitigate the class-level overfitting problem based on the interpretation and analysis of the cause.
• Extensive experiments on three public datasets verify the rationale of the model design, and showthat we can achieve state-of-the-art performance.
Interpreting the Dilemma of Few-Shot Class-Incremental Learning
In this section, we first describe the Few-Shot Class-Incremental Learning (FSCIL) task and thebaseline model, and then conduct experiments to analyze the dilemma.
2.1 Task and Baseline Description
The FSCIL task aims to incrementally recognize novel classes with only few training samples.
Basically, the model is first (pre-)trained on a set of base classes with sufficient training samples (a.k.a.base session), then confronted with novel classes with limited training samples (a.k.a. incrementalsession), and finally required to recognize test samples from all encountered classes.
Specifically, given the base session dataset D
= { ( x i , y i ) } n 0 i =1 with the label space Y 0 , the model istrained to recognize all | Y 0 | classes from Y 0 by minimizing the loss
[Equation]
where L ( · , · ) is typically a cross-entropy loss, ϕ ( · ) is the predictor which is composed of a backbonenetwork f ( · ) for feature extraction and a linear classifier, represented as ϕ ( x ) =
W ⊤ f ( x ) whereϕ ( x ) ∈ R N 0 × 1 , W ∈ R d × N 0 and f ( x ) ∈ R
d × 1 .
Typically, f ( x ) and the W are L 2 normalized [26].
When the k th incremental session comes, the model needs to learn from its training data D k
={ ( x i , y i ) } n k i =1 .
The weight of the classifier will be extended to represent the novel label space
Y kimported by this session, represented as W = { w 01 , w 02 , ..., w 0 | Y 0 | } ∪ ... ∪ { w k 1 , ..., w k | Y k | } where w kjdenotes the weight of the classifier corresponding to the j th class of the k th session.
A strong baseline [26] is to freeze model’s parameters to avoid the catastrophic forgetting broughtby the finetuning on novel-classes.
For the incremental sessions (i.e., k > 0), the average of thefeatures extracted from the training data will be used as the classifier’s weight [26] (a.k.a.
prototype)as
w kj = 1 n jk (cid:80) n jk i =1 f ( x i ) , where n jk denotes the number of training samples in the class j for thesession k .
As this baseline focuses on the base-class training, in this paper, the term training , if nototherwise stated, refers to the base-class training.
Finally, the performance of the k th session will beobtained by classifying the test samples from all (cid:80) ki =
| Y i | encountered classes.
2.2 Margin-Based Classification
A well known modification to base-class training loss (Eq. 1) is to integrate a margin [15, 8, 24] as
[Equation]
where w y i refers to the classifier weight for class y i , τ is typically set to 16.0 and m is the margin.
As analyzed in [15], empirically a dilemma exists that a positive margin could improve the baseclass performance but harm the novel-class generalization, and reversely, a negative margin couldcontribute to the novel-class performance but decrease that of the base classes as shown in Fig.
Similar phenomenon has been observed in other works such as [13] that a better loss function for thepre-training task could harm the generalization on downstream tasks.
2.3 Interpretation of Class-Level Overfitting from Pattern Learning View
Experiments are conducted on the CIFAR100 [14] dataset and reported in Fig.
2. CIFAR100 contains100 classes in all.
As split by [21], 60 classes are chosen as base classes, and the remaining 40 classes(with 5 training samples in each class) are chosen as novel classes 1 .
Experiments are conducted onthe last incremental session, where all 100 classes are involved.
From Fig.
2 (left), we can see that as the margin increases, the base-class accuracy increases whilethe novel-class accuracy decreases, which is consistent with [15] and validates the dilemma exists.
Compared with the well-known overfitting between the training and testing data, such dilemma,although all validated on testing data, is more like the overfitting to base classes instead of samples.
Therefore, we term it as class-level overfitting (CO) .
Additionally, the balance is reached when nomargin is added, i.e., FSCIL cannot be improved by simply applying the margin.
For such dilemma,[15] gave an explanation by the degraded mapping from novel to base classes.
However, it couldhardly be used to develop methods for handling such dilemma.
In this section, we go a step further toexplain this phenomenon from the aspect of pattern learning for developing methods to handle it.
A pattern denotes a part of information that the model extracts from the input, which is a finer-grainedlevel of analyzing the model’s behavior.
As studied in the interpretability of deep networks [27, 1],each channel in the feature extracted by deep networks could correspond to a certain pattern of theinput 1 , which can be viewed as to compose the base and novel classes [29].
Therefore, we conductexperiments on feature channels to study the patterns learned by applying different margins.
2.3.1 Class-Level Overfitting Interpreted by Pattern Fitness
to Base Classes
Pattern’s fitness to each base class.
We first evaluate the sparsity of the base-class patterns, whichis measured by the L 1 norm of each feature vector.
As the extracted features are L 2 normalized, thesmaller the L 1 norm is, for each feature, the sparser the patterns with high activation are.
Resultsare plotted in Fig.
2 (mid), where we can see a consistent decrease in L 1 when the margin increases,which means the model needs less activated patterns (channels) to represent each base class.
As thenumber of activated patterns decreases, the effectiveness of each activated pattern must increase toaccount for the performance increase in the base-class pre-training in Fig.
2 (left).
Therefore, wehold that as the margin increases, the patterns learned by the model could fit each base class better.
Pattern fitness measured by the template-matching score.
To further verify the fitness increase,we view each pattern as a semantic template and measure its matching score to each base class.
Asanalyzed in [27] and [1], each pattern can be understood as a template [3] for the model to match theinput (so that each class would has its own set of templates for recognition), and the activation can beviewed as the matching score.
Therefore, we could know how much all patterns fit (match) each classby finding the most important patterns for each class and compare their activation.
As analyzed in[27] and [29], patterns (channels) with higher weights in the classification layer are more important,and the most important ones dominate the model decisions.
Therefore, given an input, we select itsmost important patterns by the top classification weights of its ground-truth class, and record theaverage activation on these patterns.
The Mean value of such Top Activation across all samples isdenoted as MTA in Fig.
2 (mid).
As can be seen, as the margin increases, MTA increases consistently,which further verifies patterns’ increase in fitting each base class brought by the margin increase.
Better pattern fitness, worse pattern transferability.
As each pattern could fit a correspondingbase class better, its discriminability increases accordingly, but could it be transferred across classes?
To answer this question, we test the transferability of patterns.
Since classes are related (e.g., cat andtiger), transferable patterns activated in one class could also be activated in other classes (e.g., felidpatterns).
Therefore, we first find important patterns for each base class by the classification weights,then record activation of these patterns on other classes, and measure the transferability of patternsby the mean value of such other-class-activation.
The results are plotted in Fig.
2 (right).
As can beseen, the transferability consistently decreases when the margin increases.
Combine this result with
Fig.
2 (mid), we hold that patterns tend to be less transferable when they fit each base class better.
Discussion.
The fitness also reflects the how much the given pattern is specific to a base class.
Imagine the extreme situation where each base-class only needs one pattern for representation, thefitness would reach its upper bound to make such pattern thoroughly specific to the correspondingclass.
Therefore, we interpret that the higher the margin is, the more specific (overfitting) the patternsare to each base class, which makes patterns more discriminative and less transferable.
Meanwhile,the lower the margin is, the more the patterns could be shared between classes (underfitting), makingpatterns more transferable but less discriminative.
The CO dilemma lies in that patterns can hardly beboth class-specific and shared among classes by simply applying the classification margin.
2.3.2 Inherent Class Relations Lead to the Change in Pattern’s Base-Class Fitness
Pattern’s fitness negatively influences class relations.
In Fig.
2 (right), we also plot the classrelations w.r.t.
the margins.
The class relations are measured by the average of cosine similaritiesbetween every two classes’ prototypes.
As can be seen, the relation drops as the margin grows, inconsistent with the trend of the patterns’ transferability.
This is rationale because the if two prototypesshare some patterns, the activation of the corresponding channels will be similar, making the cosinesimilarity larger.
As the transferability of patterns is negatively related to pattern’s base-class fitness,we hold that the class relations are also negatively related to the base-class fitness.
Inherent class relations influence pattern’s fitness.
The margin applied to the classificationdirectly modifies the decision boundary between every two classes, and the decision boundary isrelated to the relationship between every two classes.
Therefore, we study how the class relationinfluences the pattern’s fitness to base classes.
Specifically, given 60 base classes, for the modeltrained without margins, we first calculate the cosine similarity between every two different classes,which gives 60 × (60 - 1) / 2 = 1,750 relations denoted as R 0 , which represents the inherent relationsbetween all classes.
Similarly, we calculate 1,750 relations for the model trained with positive and negative margins respectively, denoted as R pos and R neg .
Then we calculate D pos
=
R pos − R 0 and
D neg =
R neg − R 0 .
Finally, we sort R 0 from small to large, and use the sorted index to arrange D negand D pos .
As plotted in Fig.
3, the blue dots denote D neg (left) or D pos
(right), and the blue curve aredots fitted by multinomial functions.
We can see that the change in class relations is positively relatedto the inherent class relations for the negative margins, while is negatively related to the inherentclass relations for the positive margins, especially for index larger than 1,000.
Since class relations are negatively related to pattern’s fitness to base classes, the more the classrelation increases, the less the patterns could fit the corresponding base class.
Therefore, the resultscan be understood that the more inherently similar two classes are, by applying a negative margin,the less the patterns could fit the given base class, i.e., the more the patterns are shared by givenclasses, making these classes’ representations more similar; by applying a positive margin, the morethe patterns could fit the given base classes, making these classes’ representations more dissimilar.
Conclusion and Discussion.
Therefore, as shown in Fig. 4, we interpret the pattern learning processas follows.
Given a negative margin, the decision boundary between two classes are confused, makingmore samples fall into the overlapping region between two classes.
This makes it possible to learnpatterns shared by these two classes and hinder the pattern from fitting the given base class, andmakes patterns more transferable but less discriminative.
The more similar inherently two classesare (i.e., larger overlapping region in Fig. 4), the more shared patterns can be learned (e.g., morepatterns could be shared between cats and tigers than cats and air-planes), therefore making theseclasses’ representations more similar.
On the contrary, given a positive margin, the decision boundarybetween two classes should be well separated, pushing the model to learn patterns fitting (i.e., specificto) each class, which are more discriminative but less transferable.
The more similar inherently twoclasses are, the harder the learning is and the larger the training loss will be, therefore making thepatterns fit each class more and making these classes’ representations more dissimilar.
Mitigating the Dilemma of Few-Shot Class-Incremental Learning
In this section, we first analyze the cause of the dilemma in margin-based classification based onthe interpretation from the pattern learning view, then we propose our method to mitigate the COdilemma based on the analysis.
The framework is shown in Fig.
3.1 Analysis on the Cause of Class-Level Overfitting
Based on the above analysis, we can find the learning of negative-margin-based patterns looselyconstrains the given pattern to be shared by both classes.
However, such constraint is easily satisfiedby ineffective patterns such as edges or corners, which could lead to the low discriminability ofnegative-margin-based patterns.
Similarly, the learning of positive-margin-based patterns looselyconstrains the given pattern to be specific to the given class.
However, such constraint could besatisfied by easily finding patterns sharing no information with other classes, such as finding somecomplex texture only specific to the given class, which could lead to the low transferability of positivemargin-based patterns.
Such easy-constraint problems push patterns to be only class-specific orshared among classes, making pattern effective in one scenario ineffective in other ones.
Therefore,the key to mitigate the CO dilemma is to provide extra constraint to the pattern learning process.
3.2 Mitigating Class-Level Overfitting by Providing Extra Constraint
Negative-Margin-based (NM) patterns are more transferable and class-shared, while Positive-Marginbased (PM) patterns are discriminative and class-specific.
These characteristics are similar tothe behavior of features from the shallow and deep layers of deep networks [25, 30].
Generally, patterns by as or , while the features encode high patterns few classes therefore
Specifically, given an feature extractor f ( · ) , we add one more layer to construct the PM feature as
[Equation]
where F ( · ) is the PM feature extractor, g ( · ) is typically composed of a fully-connected layer, a 211 batch-normalization layer [12] and an activation layer.
During training, two classification loss isapplied to both F ( · ) and f ( · ) with positive and negative margins respectively, represented as
[Equation]
i , y
[Equation]
[Equation]
where w Pj ∈ R 1 ×
d P denotes the classifier weights for F ( · ) corresponding to class j , m neg denotesthe negative margin for f ( · ) , m pos is the positive margin for F ( · ) , and λ is typically set to 1.0.
Given this modification, the NM patterns f ( · ) are utilized to construct the discriminative PM patterns
F ( · ) .
Since g ( · ) is relatively too simple to capture complex patterns, f ( · ) will be pushed to bediscriminative (validated in Fig.
7), instead of casually learning ineffective patterns shared betweensimilar classes such as edges or corners.
Similarly, as F ( · ) are built from the transferable f ( · ) by thesimple g ( · ) , the transferability could be maintained by pushing PM patterns to fit the correspondingbase class by transferable information, which improves the transferability of F ( · ) (validated in Fig.
During testing, given the improved f ( · ) and F ( · ) , the final feature would be their concatenation.
3.3 Further Mitigation of Class-Level Overfitting by Integrating Class Relations
Based on the architecture design, we propose to further mitigate the CO problem through boostingthe discriminability and tranferability of NM and PM patterns by integrating the class relations.
For easy understanding, we first introduce the modification to the classification for f ( · ) , and laterintroduce that for F ( · ) .
To modify the margin-based classification from a single margin to marginsrelated to class relations, we first move the margin from the ground-truth logit to all other logits as
[Equation]
(6) where A is the adjacency matrix between all classes, m ( · ) maps the adjacency value between twoclasses to a margin.
Given this modification, if the margin is set to the original fixed value as Eq.
2,the decision boundary between class y i and other classes remains the same, therefore the new loss in
Eq.
6 could be an approximation of the original loss in Eq. 2.
For measuring class relations, we choose to utilize the adjacency matrix of all classes.
As both thefeature and the classifier’s weights are L 2 normalized, the adjacency can be measured by the cosine
[Table]
[Table]
similarity between every two class prototypes as A ij = cos ( P i , P j ) , where P i is the prototype forclass i , which is typically set to w i of the classifier.
If two classes are identical, the cosine similarity would reach its upper bound to 1.0.
Meanwhile, theaverage of class relations reflects a global margin that is effective for most classes.
Therefore, wedesign to interpolate the margin from a global effective value to a pre-defined upper bound value as
[Equation]
where m upper is a hyper-parameter controlling the margin for the upper-bound class relation (i.e.,two identical classes), m ave is another hyper-parameter controlling the margin for the average classrelations, set to the same value as that in Eq. 2, and A ave is the average class relations calculated as A
[Equation]
For F ( · ) , we adopt the same modification as f ( · ) , which replaces f ( · ) with F ( · ) in Eq. 6, and replaces
A with A P in Eq.
6 where A Pij = cos ( w Pi , w Pj ) .
Therefore, the final training objective is
[Equation]
where m p ( · ) and m n ( · ) are two interpolate functions as Eq.
7 with positive and negative hyperparameters ( m Pupper , m Pave ) and ( m upper , m ave ) respectively.
In experiments (Fig. 9), we find it beneficial to have m upper < m ave and m Pupper > m Pave , whichmeans for classes with higher similarities, the relation mapping module enables the model to learnmore shared (transferable) patterns by applying smaller negative margins, and learn more classspecific (discriminative) patterns by applying larger positive margins.
Moreover, due to the connectionbetween NM and PM patterns, the improved transferability in NM patterns would be transmitted to
PM patterns and vice versa, as validated in Fig.
9, which further mitigates the CO dilemma.
After the base-session training, the feature extractor F ( · ) and f ( · ) will be applied to the training dataof each session to obtain the extended classifier weight W = { w 01 , w 02 , ..., w 0 | Y 0 | }∪ ... ∪{ w k 1 , ..., w k | Y k | }by averaging extracted features, and the final performance of each session will be obtained based onthe classification of all the encountered classes’ test samples.
Experiments
In this section, we first introduce the experiment settings, then compare the proposed method withthe state-of-the-art methods, and finally report the ablation study for the effectiveness of each design.
4.1 Datasets
Datasets include CIFAR100 [14], Caltech-UCSD Birds-200-2011 (CUB200) [23] and miniImageNet [22] as listed in Tab. 1 following the split in [21].
For details, please refer to the appendix.
[Table]
4.2 Implementation Details
The implementation is based on CEC’s code [26], and our code will be released.
For CIFAR100, weset d P =256, set m ave =
-0.2, set m upper =
-0.5, and we have m Pave =0.1 and m Pupper =0.2.
For CUB200,we scale the learning rate of the backbone network to 10% of the global learning rate since thepre-training of the backbone is adopted [28, 26], and set d P to 8192.
Then we have m ave
=-0.2and m upper =
-0.25 and m Pave =0.3 and m Pupper =0.6.
For mini ImageNet, we set d P to 4096, and havem ave =
-0.2 and m upper =
-0.5 and m Pave =0.1 and m Pupper =0.2.
All models are optimized by the SGDoptimizer with momentum set to 0.9.
Please refer to the appendix for details.
4.3 Comparison with the State-of-the-Art
Comparisons with the state-of-the-art works are listed in Tab. 2 and Fig.
6, where we can achievestate-of-the-art performance on all three datasets.
Specifically, we can first see that our method, as aprototype-based method (e.g., CEC [26]), could significantly outperform finetune-based methods(e.g.
, iCaRL [17]).
This is because the few-shot training data could not provide sufficient informationfor novel-class learning, therefore directly freezing parameters on novel classes could reduce thecatastrophic-forgetting problem brought by the finetuning.
Our method also outperforms otherprototype-based methods, this is because the core of the prototype-based methods lies in the metriclearning [19].
As empirically proved by current works [20, 24, 8, 15], applying the margin-basedclassification could effectively improve the embedding space learned by metric-based methods.
Sinceour method handles the difficulty of applying margin-based classification (i.e., class-level overfitting)to the FSCIL task, our method could beat these prototype-based ones.
For detailed numbers of
CIFAR100 and mini ImageNet, please refer to the appendix.
4.4 Ablation Study of Modules
The ablation study of each module is reported in Tab. 3, where g ( · ) denotes adding another layer,
Margin denotes the applicant of margin-based classification, and Relation refers to the relationmapping of the margin.
We study from three aspects: Base -class, Novel -class, and Overall accuracyof the last incremental session.
From Tab. 3, we can see that
• Simply applying another layer cannot consistently improve the performance .
• The designed architecture could mitigate class-level overfitting .
Compared with experiments in Fig.
2 where no Overall performance improvements can be obtained by simply adding margins, theperformance here is clearly improved by adding margins on the designed architecture.
Moreover, the
overall improvements originate from not only the improved Base performance, but also the boosted
Novel performance, demonstrating the mitigation of class-level overfitting (CO) problem.
• Relation mapping could further improve performance by mitigating class-level overfitting .
4.5 Verification of Class-Level Overfitting Mitigation
Mitigating the class-level overfitting by the architecture design.
We first compare our methodwith the baseline method which applies the margin directly to the backbone feature in Fig.
7 and
8, so as to validate the mitigation of CO.
In Fig.
7, we apply the negative margin to the backbone(NM) feature while applying no margin to the PM feature, and see the accuracy of the NM featureagainst the baseline.
As can be seen, while keeping the improvements on novel classes (mid), ourmethod (orange) maintains the base-class accuracy (right), in contrast to a sharp decrease of thebaseline (blue) when the margin decreases, which makes our NM feature significantly outperformsthe baseline in terms of the overall accuracy (left).
Similarly, in Fig.
8, our PM feature also maintainsa higher performance on novel classes when the margin increases.
These experiments validate themitigation of the CO dilemma: by implicitly guiding the learning of the PM and NM features by eachother, the transferability or discriminability drop on them can be mitigated respectively.
Further mitigating class-level overfitting by relation mapping.
To validate the relation mappingmodule, in Fig.
9 we fixed the average margin (i.e., m ave =-0.2 in Fig.
9.1st and m Pave =0.3 in
Fig.
9.2nd) and experiment with different upper margins (i.e., m upper in Fig.
9.1st and m Pupper in
Fig.
9.2nd).
Similar to Fig.
7, when conducting experiments on one branch, the hyper-parameters ofthe other branch are fixed.
We can see the relation mapping module can indeed help the learning, asthe upper margins are significantly different from the average margin.
We also plot the NM and PMperformance on base classes with negative margins in Fig.
9(3rd), and those on novel classes withpositive margins in Fig.
9(4th).
We can see the NM and PM performance are improved simultaneously,which validates the improvements brought by the extra guidance of each features.
Therefore, therelation mapping module can also help to mitigate the class-level overfitting problem by building
PM features with more transferable patterns, and vice versa.
Moreover, it is interesting to find theperformance is improved when m upper < m ave and m Pupper > m Pave , which means for classes withhigher similarities, the relation mapping module enables the model to learn more shared patternsby applying negative margins with larger absolute value, and learn more class-specific patterns byapplying larger positive margins.
Also, this is consistent with the results in Fig.
Conclusion
In this paper, we focus on the dilemma in the margin-based classification for FSCIL.
We first interpretthe dilemma as a class-level overfitting problem from the aspect of pattern learning, then find thecause of this problem lies in the easily-satisfied constraint of learning shared or class-specific patterns.
Based on the analysis, we design a method to mitigate the dilemma by constructing PM patterns from
NM patterns, and mapping class relations into class-specific patterns.
Extensive experiments on threepublic datasets validate the effectiveness and outstanding performance of the proposed method.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[Yes](d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A](b)
Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[No]Experiments are conducted based on the code released by CEC (CVPR’21), whichspecified the random seed for re-implementation and comparison.
Therefore, we followthe same random seed to conduct experiments and compare with other methods withoutthe error bar.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[Yes]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[Yes]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]