MetaMask: Revisiting Dimensional Confounder for Self-Supervised Learning
Anonymous Author(s)
AffiliationAddress email
Abstract
As a successful approach to self-supervised learning, contrastive learning aims tolearn invariant information shared among distortions of the input sample.
Whilecontrastive learning has yielded continuous advancements in sampling strategyand architecture design, it still remains two persistent defects: the interference oftask-irrelevant information and sample inefficiency, which are related to the recurring existence of trivial constant solutions.
From the perspective of dimensionalanalysis, we find out that the dimensional redundancy and dimensional confounderare the intrinsic issues behind the phenomena, and provide experimental evidenceto support our viewpoint.
We further propose a simple yet effective approach
MetaMask , short for the dimensional Mask learned by Meta -learning , to learnrepresentations against dimensional redundancy and confounder.
MetaMask adoptsthe redundancy-reduction technique to tackle the dimensional redundancy issueand innovatively introduces a dimensional mask to reduce the gradient effects ofspecific dimensions containing the confounder, which is trained by employinga meta-learning paradigm with the objective of improving the performance ofmasked representations on a typical self-supervised task.
We provide solid theoretical analyses to prove MetaMask can obtain tighter risk bounds for downstreamclassification compared to typical contrastive methods.
Empirically, our methodachieves state-of-the-art performance on various benchmarks.
Introduction
A fundamental idea behind self-supervised learning is to learn discriminative representations fromthe input data without relying on human annotations.
Recent advances in visual self-supervisedlearning [1, 2, 3, 4] demonstrate that unsupervised approaches can achieve competitive performanceover supervised approaches by introducing sophisticated self-supervised tasks.
A representativelearning paradigm is contrastive learning [5, 6, 7, 8, 9], which aims to learn invariant information fromdifferent views (generated by data augmentations) by performing instance-level contrast, i e., pullingviews of the same sample together while pushing views of different samples away.
Various[10, 11, 12]techniques have been explored to address the problem of trivial solutions to self-supervised contrastivelearning, e.g., constant representations.
However, state-of-the-art methods still suffer from two crucialdefects, including the interference of task-irrelevant information and sample inefficiency.
The trivialsolution problem is related to such defects, e.g., a small number of negative samples or a largeproportion of task-irrelevant information may lead the model to learn constant representations.
Werevisit the current self-supervised learning paradigm from the perspective of dimensional analysis
[Figure]
O u r M e t hod S i m C L
[Figure]
and argue that the intrinsic issues behind the defects of self-supervised learning are the dimensionalredundancy and dimensional confounder 1 .
From the perspective of information theory, each dimension holds a subset of the representation’sinformation entropy.
The dimensional redundancy denotes that multiple dimensions hold overlappinginformation entropy.
To tackle this issue, Barlow Twins [14] takes advantage of a neuroscienceapproach, i.e., redundancy-reduction [15], in self-supervised learning.
It makes the cross-correlationmatrix of twin embeddings close to the identity matrix, which is conceptually simple yet effectiveto address the dimensional redundancy issue.
Figure 1 (a) supports the superiority of redundancyreduction in addressing the dimensional redundancy problem.
Since Barlow Twins encouragesthe dimensions of learned representations to model decoupled information, it naturally avoids thecollapsed trivial solution that outputs the same vector for all images.
However, only reducing dimensional redundancy still suffers from the problem of dimensionalconfounder.
The dimensional confounder indicates a set of dimensions that solely contain “harmful”information and further degenerates the performance of the model, e.g., the dimensions simplycapturing chaotic and worthless background information from the input.
To prove the existence ofsuch a dimensional confounder, we conduct motivating experiments from two perspectives.
It is statedin [14] that Barlow Twins keeps improving as the dimensionality of the projection head increases,since it acts as a dimensionality bottleneck to constrain the dimensionality of the representationto capture more and more task-irrelevant information.
Counterintuitively, our observation fromthe motivating experiments is in stark contrast with the conclusion in [14].
As shown in Figure 1(b), the performance of Barlow twins as a function of the dimensionality presents a convex patternon either CIFAR-10 or ImageNet.
After reaching the peak, the performance gradually decreasesas the dimensionality increases until it collapses.
We reckon that since the task-relevant semanticinformation is limited, as the information captured in different dimensions is gradually decoupled,more task-irrelevant noisy information is encoded into the representation, i.e., the dimensionalconfounder increases, so the performance of the learned representation is getting worse until collapse.
[Figure]
ByondBaseline
Baseline
BelowBaseline
[Figure]
I n f e r e n ce acc u r ac y
CIFAR73.
[Figure]
ByondBaseline
Baseline
BelowBaseline
[Figure]
ImageNet
Following the experimental protocol of [14, 12], we evaluate the performance on CIFAR-10 and
ImageNet by adopting KNN prediction and linear probing, respectively, which further proves the general existence of dimensional confounder.
To further validate our inference, we randomly mask several dimensions by frankly setting the maskedvalues to 0 , and then we evaluate the performance of the masked representation.
The experimentalresults are demonstrated in Figure 2, where every single point denotes the result of a sampled mask.
The representations with some specific masked dimensions can indeed achieve better performancethan the unmasked representation, and such an observation is consistent on a small-scaled dataset,e.g., CIFAR-10, and a large-scaled dataset, e.g., ImageNet.
The dimensional mask rate that canimprove the performance of the model on ImageNet is on average higher than that on CIFAR-10.
Weunderstand that this phenomenon depends on an ingredient: the representation learned on CIFAR-10models discriminative information for 10 categories in 512 dimensions, while the representation uses2048 dimensions to model information for 1000 categories on ImageNet, so the dimension-to-classratio on CIFAR-10 ( 51 . 2 ) is much higher than that on ImageNet ( 2 . 048 ).
Therefore, the presence ofdimensional confounder naturally remains at a low level on ImageNet since supporting 1000 -categoryclassification requires a large amount of heterogeneous discriminative information.
This empiricalevidence further supports our assumption of the dimensional confounder.
To this end, we intuitively propose a simple yet effective approach MetaMask , short for the Metalearned dimensional Mask , to adjust the impacts of different dimensions in a self-paced manner.
In order to learn representations against dimensional redundancy and confounder, we regard thecross-correlation matrix constraint of Barlow Twins as a dimensional regularization term and furtherintroduce a dimensional mask, which simultaneously enhances the gradient effect of the dimensionscontaining discriminative information and reduces that of the dimensions containing the confounderduring training.
We employ the meta-learning paradigm to train the learnable dimensional mask withthe optimization objective of improving the performance of masked representations on typical selfsupervised tasks, e.g., contrastive learning.
Consequently, the dimensional mask incessantly adjuststhe weights of different dimensions with respect to their gradient contribution to the optimization, so asto make the model focus on the acquisition of discriminative information, which can address the trivialsolution caused by treating all dimensions equally in optimization, especially for high-dimensionalrepresentations.
Empirically, we conduct head-to-head comparisons on various benchmark datasets,which prove the effectiveness of MetaMask.
Concretely, the contributions of this paper are four-fold:
• We present a timely study on the adaptation of the state-of-the-art self-supervised modelson downstream applications and identify two critical issues associated with the learnedrepresentation, i
e., dimensional redundancy and dimensional confounder. • We propose a novel approach, i.e., MetaMask, to tune the gradient effects of differentdimensions in a self-paced manner during optimization according to their improvements ona specific self-supervised task.
• Theoretically, we provide sound theorems and corresponding proofs to support that Meta-
Mask achieves relatively tighter risk bounds for downstream classification compared to typical contrastive methods.
• We conduct extensive experiments to demonstrate the superiority of MetaMask over bench-
mark methods on downstream tasks.
Related Work
Self-supervised learning achieved impressive success in the field of representation learning without human annotation by constructing auxiliary self-supervised tasks.
Typically, self-supervisedapproaches can be categorized as either generative or discriminative [16, 6, 10].
Conventional generative approaches bridge the input data and latent embedding by building a distribution over them, andthe learned embeddings are treated as image representations [17, 18, 19, 20, 21, 22].
However, sucha learning paradigm learns the distribution at the pixel-level, which is computationally expensive.
The detailed information required for image generation may be redundant for representation learning.
For discriminative methods, contrastive learning achieves state-of-the-art performance on variousdownstream tasks [4, 23, 24, 25, 26, 27, 28, 29, 11].
Deep InfoMax [1] explores to maximize themutual information between the input and output of an encoder.
CPC [3] adopts noise-contrastiveestimation [30] as the contrastive loss to train the model by maximizing the mutual informationof multiple views, which is deduced by using the Kullback-Leibler divergence [31].
CMC [5] and
AMDIM [2] employs contrastive learning on the multi-view data.
SimCLR [6] and MoCo [7, 32]use large batch or memory bank to enlarge the amount of available negative features.
To sampleinformative negatives, DebiasedCL[8] and HardCL [9] design sampling strategies by using positiveunlabeled learning methods [33, 34].
Motivated by [35], [36] provides an information theoreticalframework for self-supervised learning and proposes an information bottleneck-based approach.
However, these methods rely on a large number of negative samples to learn valuable representationswhile avoiding trivial solutions, i.e., constant representation.
SimSiam [12] and BYOL [10] jointlywaive negatives and avoid training degradation by using the techniques of asymmetric siamesenetwork and stop-gradient.
SSL-HSIC [37] provides a unified framework for negative-free contrastivelearning based on Hilbert Schmidt Independence Criterion [38].
The representation learning paradigmemployed by current contrastive methods is contrary to the coding criteria proposed by [15] and[39], respectively.
Therefore, Barlow Twins [14] proposes to constrain the cross-correlation matrixbetween the features to be close to the identity matrix, which can effectively reduce the dimensionalredundancy.
Yet, such an approach still suffers from the dimensional confounder issue, and we thuspropose a new meta-learning-based method to attenuate the over-learning of dimensional confounderby learning a mask to filter out irrelevant information.
Preliminaries
Formally, we suppose the input multi-view dataset as X = (cid:110) x ji (cid:12)(cid:12)(cid:12) i ∈ (cid:74) 1 , N ∗ (cid:75) , j ∈ (cid:74) 1 , M (cid:75) (cid:111) , where
N ∗ denotes the number of samples, and M denotes the number of views.
Therefore, x ji representsthe j -th view of the i -th sample following x ji ∼ X .
The multiple views are built by performingidentically distributed data augmentations [5, 6, 14], i.e., { t j } Mj =1 ∼ T .
3.1 Contrastive Learning
We recap the preliminaries of contrastive learning [6], which aims to learn an embedding that jointlymaximizes agreement between views of the same sample and separates the views of different samplesin the latent space.
Given a minibatch of N examples sampled from the multi-view dataset X ,positives represent the pairs { x ji , x j ′ i } where j, j ′ ∈ (cid:74) 1 , M (cid:75) , and negatives represent { x ji , x j ′ i ′ } wherei ̸ = i ′ .
Conventional contrastive methods feed the input x ji into a view-shared encoder f θ ( · ) to learnthe representation h ji , which is mapped into a feature z ji by a projection head g ϑ ( · ) , where θ and ϑare the network parameters.
f θ ( · ) and g ϑ ( · ) are trained by using the contrastive loss [3, 6]:
[Equation]
where d ( · ) is a discriminating function to measure the similarity of the feature pairs, 1 [ i ̸
= i ′ ∨ j ̸ = j ′ ]denotes an indicator function equalling to 1 if i ̸
= i ′ or j ̸ = j ′ , and τ is a temperature parameter
[Figure]
valued by following [6].
In inference, the projection head g ϑ ( · ) is discarded, and the representationh ji is directly used for downstream tasks.
3.2 Dimensional Redundant Reduction
Following the principle proposed by [15], [14] introduces a redundancy-reduction objective function:
[Equation]
where λ is a positive hyperparameter valued by following [14], and C is the cross-correlation matrixcomputed between the outputs of the encoder along the batch dimension:
[Equation]
where k and k ′ index the dimension of features.
C is a square matrix with the size of D , whichdenotes the dimensionality of the projected feature z .
The value range of the elements in C is [ − 1 , 1] ,i.e., complete anti-correlation to complete correlation.
Methodology
Our goal is to learn discriminative representations against dimensional redundancy and dimensionalconfounder from multiple views by performing self-supervised learning.
To this end, we propose
MetaMask to automatically determine the optimal dimensionality of the representation by decouplinginformation into different dimensions and filtering out those dimensions with irrelevant information.
MetaMask addresses the dimensional redundancy issue by adopting a redundancy-reduction regularization.
To tackle the dimensional confounder problem, MetaMask learns a dimensional mask toadjust the gradient weights of dimensions according to their contributions to improving a specificself-supervised task during optimization, which is achieved in a meta-learning manner.
MetaMask’s architecture is shown in Figure 3.
Specifically, the sample x ji is first fed into the encoderto obtain its representation h ji .
Then, we build a learnable dimensional mask M = (cid:8) ω
k (cid:12)(cid:12) k ∈ (cid:74) 1 , D (cid:75) (cid:9) ,which is introduced to assign a weight to each dimension of the representation h ji by
[Equation]
where ˜ h ji denotes the masked representation, and · ⊗ · is an element-wise Hadamard product function.˜
h ji is then fed into two separate projection heads g drr ( · ) and g clϑ ( · ) , respectively.
g clϑ ( · ) plays the samerole as in conventional contrastive learning methods, i.e., the contrastive loss L contrast in Eq.
(1) isapplied to the output features of g clϑ ( · ) .
For the projection head g drr ( · ) , the redundancy-reductionloss L drr in Eq.
(2) is performed to its output features to tackle the dimensional redundancy.
In training, the encoder f θ ( · ) , the projection heads
g drr ( · ) and g clϑ ( · ) , as well as the dimensional mask
M are trained in a meta-learning manner consisting of two steps.
In the first regular training step, wetrain f θ ( · ) , g drr ( · ) and g clϑ ( · ) by jointly minimizing the redundancy-reduction and contrastive losses,which is formalized by
[Equation]
where α is a coefficient that controls the balance between L drr and L contrast , and we evaluate theinfluence of this hyper-parameter in Section 6.
In the second meta-learning-based step, we update M by using the second-derivative technique [40]to solve a bi-level optimization problem.
We encourage M to mask (reduce the gradient weights of)the specific dimensions containing task-irrelevant information, which are regarded as dimensionalconfounder, so that the encoders can better explore the discriminative information in training.
Tothis end, M can be updated by computing its gradients with respect to the performance of f θ ( · ) andg clϑ ( · ) .
The corresponding performance is measured by using the gradients of f θ ( · ) and g clϑ ( · ) duringthe back-propagation of contrastive loss.
Formally, we update the dimensional mask M by
[Equation]
where X ′ represents a minibatch sampled from the training dataset X , ϑ trial and θ trial represent thetrial weights of the encoders and projection heads after one gradient update using the contrastive lossdefined in Equation 1, respectively.
We formulate the updating of such trial weights as follows:
[Equation]
where ℓ θ and ℓ ϑ are learning rates.
The intuition behind such a behavior is that we perform aderivative over the derivative (Hessian matrix) of the combination { θ, ϑ } to update M , i.e., thesecond-derivative trick.
Specifically, we compute the derivative with respect to M by using a retainedcomputational graph of { θ, ϑ } and then update M by back-propagating the derivative as Equation 6.
We train M only based on the performance of contrastive learning for two reasons: 1) the dimensionalredundancy reduction is a regularization term with strong constraints and generally stable convergence,and it is independent of downstream tasks; 2) such a meta-learning-based training approach thatpromotes contrastive learning performance empowers our method to constrain the upper and lowerbounds of the cross-entropy loss on downstream tasks, which ensures that M can partially mask thegradient contributions of dimensions containing task-irrelevant information and further promote theencoder to focus on learning task-relevant information, which is demonstrated in Section 5.
The two steps for updating f θ ( · ) , g drr ( · ) , and g clϑ ( · ) and updating M are iteratively repeated untilconvergence.
The training pipeline is detailed by Algorithm 1.
Theoretical Analyses
Current contrastive methods follow the fundamental assumption of multi-view learning (Assumption1 in [35], Assumption 1 in [36], and Assumption 4.1 in [41]), i.e., views (or distortions) of a same
Input: Multi-view dataset X , minibatch size N , and a hyperparameter α .
Initialize The neural network parameters: θ for f θ ( · ) , ϑ for g clϑ ( · ) , ϑ drr for g drr ( · ) , and M = (cid:8) ω
k (cid:12)(cid:12) k ∈ (cid:74) 1 , D (cid:75) (cid:9) .
The learning rates: ℓ θ and ℓ ϑ , etc. repeatfor t -th training iteration do
Iteratively sample a minibatch X ′ with N examples from X .
# regular training step, fix M
[Equation]
# compute trial weights and retain computational graph , fix θ and ϑ
[Equation]
# meta training step using second derivative
[Equation]
end for until θ , ϑ , ϑ drr , and M converge.
sample holds the invariant discriminative label-relevant information.
Theorem 4.2 in [41] furtherproposes the guarantees for general encoders in the learning paradigm of regular contrastive learning,which proves that the contrastive loss in the self-supervised learning stage can constrain the upperand lower bounds of the cross-entropy loss in the supervised learning stage for downstream tasks.
Toelaborate the behavior of our MetaMask, we extend this Theorem to build a connection between theproposed masked representation and downstream performance by
Theorem 5.1.
(Connecting Masked Representations to Downstream Cross-Entropy Loss).
Underthe general assumption of self-supervised multi-view learning, when M is optimal, for any coupledh , ˜ h ∈ R , L crossentropy ( h ) for downstream classification can be bounded by L contrast (cid:16) g clϑ
[Equation]
˜ h
(cid:17)(cid:17)in self-supervised learning:
[Equation]
where M denotes the quantity of negative samples, D denotes the dimensionality of the representation,h denotes the unmasked representation, ˜
h denotes the masked representation obtained by Equation4, y denotes the target label, Φ ( ·| y ) =
E P ( y ) (cid:2) E P ( ·| y ) δ (cid:0) · , E P ( ·| y )
(cid:1)(cid:3) represents the conditionalvariance function where δ ( · , · ) is a discriminating function to measure the difference between theterms, e.g., δ ( · , · ) =
∥ [ · ] − [ · ] ∥ 2 in low-dimensional embedding space or δ ( · , · ) = − log (cos ( · , · ))
=− log [ · ] × [ · ] ||·||×||·|| in high-dimensional embedding space, ⌊·⌉ k is a function acquiring k -th dimensionvector, log (cid:0) MD (cid:1) is a constant, and O (cid:16) M − 12 (cid:17) is the approximation error’s order.
Theorem 5.1 states that reducing the risk of contrastive learning can improve the performance ondownstream tasks, which further supports our intuition to make the model focus on the acquisitionof discriminative information by learning a dimensional mask with the objective of improving theperformance of masked representations on contrastive learning is theoretically sound.
Refer to
Appendix A.1 for the elaboration of MetaMask’s behavior to mask the gradient contribution ofspecific dimensions containing confounder information.
Hence, we derive
Theorem 5.2.
(Guarantees for Reduced Conditional Variance of Masked Representations).
When
M is optimal, for any coupled h , ˜ h ∈ R , given label y , the conditional variance of ˜ h is reduced:
[Equation]
Theorem 5.2 states that given the label y , the masked representation ˜ h has smaller conditional variancethan h in contrastive learning.
We bring Theorem 5.2 into Theorem 5.1 to derive a conclusion:
our
[Table]
approach can better bound the downstream classification risk, i
[Table]
e., the upper and lower bounds ofsupervised cross-entropy loss obtained by MetaMask are tighter than typical contrastive learningmethods.
Please refer to Appendix A.2 for the corresponding proofs.
Experiments
6.1 Benchmarking MetaMask with Various Backbones
Implementations.
For the comparisons demonstrated in Table 1, we uniformly set the batch size as
[Table]
and we adopt a network with the 5 convolutional layers in AlexNet [44] as conv and a network with2 additional fully connected layers as fc .
We straightforwardly adopt the same data augmentationsin CMC [5], and the memory bank [45] is adopted to facilitate calculations by retrieving 4,096 pastnegative features during training.
Following the experimental principle of linear probing, we removethe projection head and re-train an MLP as the classification head for the learned representation.
For the experiments in Table 2, the batch size is valued by 512, and ResNet-18 [46] is used as thebackbone encoder.
We adopt the data augmentation and other experimental settings following [14].
Note that the KNN classifier is adopted for evaluation.
Discussion of the comparisons using conv and fc encoders.
Table 1 shows the comparisons on fourbenchmark datasets, and the corresponding positive or negative margins are reported.
We observethat MetaMask beats the best prior methods by a significant margin on most downstream tasks, whichdemonstrates that MetaMask can efficiently improve the performance of the learned representationwhen the self-supervision is insufficient, i.e., the limited batch size and weak encoder.
Discussion of the comparisons using the ResNet encoder.
We further perform classificationcomparisons using a strong encoder, i.e., ResNet.
Table 2 shows that MetaMask can consistentlyimprove the state-of-the-art methods, which indicates that the proposed method has strong adaptabilityto different encoders.
The experimental evidence proves our assumption of dimensional confounder.
Thus, given a fixed dimensionality of the projection head, MetaMask can indeed improve thediscriminativeness of the learned representation by masking the dimensional confounder.
6.2 Validation of the Robustness of MetaMask against Dimensional Confounder
We conduct experiments to prove that MetaMask is able to mitigate the negative impact of thedimensional confounder on the discriminativeness of the learned representations, and the results areshown in Figure 4.
The intriguing outcome demonstrates two merits of the proposed method: 1)
[Table]
[Figure]
ResNet18.
∗ denotes MetaMask using a trick of fixed learning rate instead of the cosine annealing strategy.
even if the dimensionality of the projection head changes, MetaMask is consistently robust to thedimensional confounder, i.e., the model maintains relatively consistent performance; 2) MetaMaskcan alleviate the negative impact of curse of dimensionality [47, 48], e.g., given a fixed number ofexamples, as the dimensionality of the representation increases during training, the performance of themodel on downstream tasks instead degenerates.
Concretely, Figure 2 demonstrates that regardless ofwhether representation learning suffers from the curse of dimensionality, dimensionality confoundingfactors are widely present in learned representations, and hence, the defined dimensional confounderis a more generalized issue of the curse of dimensionality.
Refer Appendix A.3 for the furtherdiscussion.
Moreover, we observe a surprising result that MetaMask ∗ achieves better performance.
We provide an understandable explanation: the essence of the curse of dimensionality is that a networkwith a large number of parameters is prone to over-fitting to the training data.
Therefore, comparedwith the annealing strategy, a fixed and relatively large learning rate can prevent the model fromfalling into a local optimum (over-fitting) during optimization.
While, our further experiments showthat MetaMask ∗ collapses abruptly at larger dimensional settings, e.g., 16382, which demonstratesthat the trick of using a fixed learning rate can only unsteadily improve the performance of MetaMask,but this behavior is not essential.
Figure 4 shows that under the different experimental settings, vanilla
MetaMask consistently counteracts the negative effects of the dimensional confounder.
Conclusion and Future Discussion
From the dimensional perspective, we first clarify that the dimensional redundancy and confounderissues are the essence of the current self-supervised learning paradigm’s defects.
Multiple motivatingexperiments are conducted to support our viewpoint, and the intriguing results are opposite tothe traditional conclusions.
To jointly tackle the dimensional redundancy and confounder issues,we propose a heuristic learning approach, called MetaMask, which leverages a second-derivativetechnique to learn a dimensional mask in order to adjust the gradient weights of different dimensionsduring training.
The proposed theoretical analyses demonstrate that MetaMask obtains relativelytighter risk bounds for downstream classification compared to conventional contrastive methods.
From the experiments, we observe a consistent improvement in the performance of our method overstate-of-the-art methods on benchmark datasets.
One of our critical contributions is proposing a heuristic learning paradigm for self-supervised learning: regarding the dimensional redundancy reduction as a regularization term and further exploringto mask dimensions containing the confounder.
In the current methodology, we train the learnabledimensional mask with the objective of improving the performance of masked representations ona specific self-supervised task , i.e., contrastive learning, and provide the theoretical analysis toprove that such a learning approach can indeed prompt the model focus on acquiring discriminativeinformation.
However, contrastive learning is immutable for the self-supervised task in MetaMask.
It is commonly acknowledged that masked image modeling (MIM) methods [50, 51, 52] achieveimpressive performance in self-supervised learning.
Although the theoretical contribution bridgingthe gap between MIM loss and cross-entropy loss on downstream tasks has not yet emerged, we stilldeem combining MIM approaches with our proposed paradigm is attractive to the community.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section 7.
(c) Did you discuss any potential negative societal impacts of your work?
[No] Since
this work presents a general technique to tackle the dimensional confounder for self-
supervised learning, we did not see any particular foreseeable ethical consequence.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section 5.(b)
Did you include complete proofs of all theoretical results?
[Yes] See Appendix A.1and Appendix A.2.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex-
perimental results (either in the supplemental material or as a URL)?
[Yes] See the
supplementary files.(b)
Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Section 6, Appendix A.4, and the supplementary files.
(c) Did you report error bars (e g., with respect to the random seed after running experi-
ments multiple times)?
[No] The experimental results provided by benchmark methods
do not contain such error bars.
We also follow this principle, and we reckon the reason
behind such a behavior is that the comparisons are conducted in balanced datasets, so
the difference between results is not excessively large.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Section 1 and Appendix A.4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] See Section 1, Section2, Section 5, and Section 6.
(b) Did you mention the license of the assets?
[No] The assets we adopted are totally
established and public to the community.
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
See the supplementary files.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[No] The datasets are all public to the community.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[No] The datasets have been discussed, validated,
and used by many researchers.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional ReviewBoard (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]