Dataset Distillation using Neural Feature Regression
Anonymous Author(s)
AffiliationAddress email
Abstract
Dataset distillation aims to learn a small synthetic dataset that preserves most ofthe information from the original dataset.
Dataset distillation can be formulatedas a bi-level meta-learning problem where the outer loop optimizes the metadataset and the inner loop trains a model on the distilled data.
Meta-gradientcomputation is one of the key challenges in this formulation, as differentiatingthrough the inner loop learning procedure introduces significant computation andmemory costs.
In this paper, we address these challenges using neural Feature
Regression with Pooling (FRePo), achieving the state-of-the-art performance withan order of magnitude less memory requirement and two orders of magnitudefaster training than previous methods.
The proposed algorithm is analogous totruncated backpropagation through time with a pool of models to alleviate varioustypes of overfitting in dataset distillation.
FRePo significantly outperforms theprevious methods on CIFAR100, Tiny ImageNet, and
ImageNet-1K. Furthermore,we show that high-quality distilled data can greatly improve various downstreamapplications, such as continual learning and membership inference defense.
Introduction
Knowledge distillation [1] is a technique in deep learning to compress knowledge for easy deployment.
Most previous works focus on model distillation [2, 3] where the knowledge acquired by a largeteacher model is transferred to a small student model.
In contrast, dataset distillation [4, 5] aims tolearn a small set of synthetic examples preserving most of the information from a large dataset suchthat a model trained on it can achieve similar test performance as one trained on the original dataset.
Distilled data can accelerate model training and reduce the cost of storing and sharing a dataset.
Moreover, its highly condensed and synthetic nature can also benefit various applications, such ascontinual learning [5–8], neural architecture search [5, 7] and privacy-preserving tasks [9, 10].
Dataset distillation was first studied by Maclaurin et al.
[11] in the context of gradient-based hyperparameter optimization and subsequently Wang et al.
[4] formally proposed dataset distillation as a newtask.
Dataset distillation can be naturally formulated as a bi-level meta-learning problem.
The innerloop optimizes the model parameters on the distilled data (meta-parameters), while the outer looprefines the distilled data with meta-gradient updates.
One key challenge in dataset distillation is computing the meta-gradient.
Several methods [4, 11–13]compute it by back-propagating through the unrolled computation graph, but they often suffer fromhuge compute and memory requirement [14], training instability [15, 16], and truncation bias [17].
To avoid unrolled optimization, surrogate objectives are used to derive the meta-gradient, suchas gradient matching [5, 7, 18], feature alignment [8, 19], and training trajectory matching [20].
Nevertheless, a surrogate objective may introduce its own bias [19], and thus, may not accuratelyreflect the true objective.
An alternative is using kernel methods, such as Neural Tangent Kernel(NTK)
[21], to approximate the inner optimization [22, 23].
However, computing analytical NTK formodern neural network can be extremely expensive [22, 23].
cup bottle can plate clock lamp couch bed chair wardrobe telephone television bicycle motorcycle pickup_truck bus tank rocket mountain cloud sea plain road forest orange pear apple maple_tree oak_tree palm_tree bee beetle crab lion tiger wolf (a) CIFAR espresso pizza basketball rugby ball maypole ladybug fly monarch koala
African elephant go-kart sports car school bus bullet train lifeboat obelisk flagpole water tower triumphal arch barn magnetic compass teapot wok backpack rocking chair (b) Tiny ImageNet tench english springer cassette player church french horn garbage truck golf ball parachute english foxhound golden retriever rhodesian ridgeback old English sheepdog australian terrier dingo
Samoyed shih-tzu (c) ImageNette and ImageWoof
Even with an accurate meta-gradient dataset distillation still suffers from various types of overfitting.
For instance, the distilled data can easily overfit to a particular learning algorithm [4, 13, 20], acertain stage of optimization [13, 19], or a certain network architecture [5, 7, 20, 22, 23].
Meanwhile,the model can also overfit the distilled data during training, which is the most common cause ofoverfitting when we train on a small dataset.
All these kinds of overfitting impose difficulties on thetraining and general-purpose use of the distilled data.
We propose an efficient meta-gradient computation method and a “model pool” to address theoverfitting problems.
The bottleneck in meta-gradient computation arises due to the complexity ofinner optimization, as we need to know how the inner parameters vary with the outer parameters [24].
However, the inner optimization can be pretty simple if we only train the last layer of a neural networkto convergence while keeping the feature extractor fixed.
In this case, computing the prediction on thereal data using the model trained on the distilled data can be expressed as a kernel ridge regression(KRR) with respect to the conjugate kernel [25].
Hence, computing the meta-gradient is simplyback-propagating through the kernel and a fixed feature extractor.
To alleviate overfitting, we proposeto maintain a diverse pool of models instead of periodically training and resetting a single model as inprior work [7, 13, 18].
Intuitively, our algorithm targets the following question: what is the best datato train the linear classifier given the current feature extractor?
Due to the diverse feature extractorswe use, the distilled data generalize well to a wide range of model distributions.
Summary of Contributions
• We propose an effective method for dataset distillation.
Our method, named neural Feature
Regression with Pooling (FRePo), achieves state-of-the-art results on various benchmark
datasets with a 100x reduction in training time and a 10x reduction in GPU memory requirement.
Our distilled data looks real (Figure 1) and transfers well to different architectures.
• We show that FRePo scales well to datasets with high-resolution images or complex labelspace.
We achieve 7.5% top1 accuracy on ImageNet-1K [26] using only one image per class.
The same classifier obtains only 1.1% accuracy from a random subset of real images.
Theprevious methods struggle in this task due to large memory and compute requirements.
• We demonstrate that high-quality distilled data can significantly improve various downstreamapplications, such as continual learning and membership inference defense.
Method
2.1 Dataset Distillation as Bi-level Optimization
Suppose we have a large labeled dataset T “ ␣ p
[Equation]
, ` x |
[Equation]
|
[Equation]
y | T | ˘( with | T | image and labelpairs.
Dataset distillation aims to learn a small synthetic dataset S “ ␣ p x 1 , y 1 q , . . .
, ` x | S | , y | S | ˘( 71 that preserves most of the information in T .
We train several neural networks parameterized by
[Figure]
θ on the dataset S and then compute the validation loss L p
A lg p θ, S q , T q on the real dataset T ,where A lg p θ, S q is the neural network parameters optimized by a learning algorithm
A lg with themodel initialization θ and distilled dataset S as its inputs.
The validation loss L p
A lg p θ, S q , T q is anoisy objective with the stochasticity coming from random model initialization and inner learningalgorithm.
Thus, we are interested in minimizing the expected value of this loss, which we denote itas F p S q .
We formulate the dataset distillation as the following bi-level optimization problem.
[Equation]
In this bi-level setup, the outer loop optimizes the distilled data to minimize F p S q , while the innerloop trains a neural network using the learning algorithm, A lg , to minimize the training loss on thedistilled data S .
From the meta-learning perspective, the task is defined by the model initializationθ , and we want to learn a meta-parameter S that generalizes well to different models sampled fromthe model distributions P θ .
During learning, we optimize the meta-parameter S by minimizing themeta-training loss F p S q .
In contrast, at meta-test time, we train a new model from scratch on S andevaluate the trained model on a held-out real dataset.
This meta-test performance reflects the qualityof the distilled data.
2.2 Dataset Distillation using Neural Feature Regression with Pooling (FRePo)
The outer-level problem can be solved using gradient-based methods of the form S Ð S
´ α ∇ S F p S q ,where α is the learning rate for the distilled data and ∇ S F p S q is the meta-gradient [27].
For aparticular model θ , the meta-gradient can be expressed as ∇ S L p
A lg p θ, S q , T q .
Computing thismeta-gradient requires differentiating through inner optimization.
If A lg is an iterative algorithmlike gradient descent, then backpropagating through the unrolled computation graph [14] can be asolution.
However, this type of unrolled optimization introduces significant computation and memoryoverhead, as the whole training trajectory needs to be stored in memory (Figure 2(b)).
Traditionally, these issues are alleviated with truncated backpropagation through time (TBPTT)[28–30].
Instead of backpropagating through an entire unrolled sequence, TBPTT performs backpropagation for each subsequence separately.
It is efficient because its time and memory complexity scalelinearly with respect to the truncation steps.
However, truncation may yield highly biased gradientsthat severely impact training.
To mitigate this truncation bias [14], we consider training only the toplayer of a network to convergence .
The key insight is that the data helpful for training the outputlayer can also help train the whole network.
Thus, we decompose the neural network into a featureextractor and a linear classifier.
We fix the feature extractor at each meta-gradient computation andtrain the linear classifier to convergence before updating S .
After that, we adjust the feature extractorby training the whole network on the updated distilled data.
We note that similar two-phase procedurehas been studied in the context of representation learning [31].
Meta-Gradient Computation: If we consider the mean square error loss, then the optimal weightsfor the linear classifier have a closed-form solution.
Moreover, since the feature dimension is typically Require: : a dataset; data Initialization
Initialization converged do 2: Ż θ
M .
3: Ż dataset p q
4: Ż Compute
5: Ż
S :
Y
Ð Y
Y
L 6: Ż 7: Ż : end while larger than the number of distilled data, we can use kernel ridge regression (KRR) with a conjugatekernel [25] rather than solving the weights explicitly [12]. The resulting meta-training loss (Eq. 2) issimilar to that used in KIP [22, 23], but we use a more flexible kernel rather than NTK.
[Equation] [Equation]
) where p X t , Y t q and p X s , Y s q are the inputs and labels of the real data T and distilled data S respectively.
The Gram matrix between real inputs and distilled inputs is denoted as K θX t X s P R | T |ˆ| S | ,while the Gram matrix between distilled inputs is denoted as K θX s X s P R | S |ˆ| S | .
λ controls the regularization strength for KRR.
Let us denote the neural network feature for a given input X and modelparameter θ as f p X, θ q P R N ˆ d , where N is the number of input and d is the feature dimension 1 .
The conjugate kernel is defined by the inner product of the neural network features.
Thus, the two Gram matrices are computed as follows:
[Equation]
Now, computing the meta-gradient ∇ S L p
A lg p θ, S q , T q is just back-propagating through the conjugate kernel and a fixed feature extractor, which is very efficient and takes even fewer operationsthan computing the gradient for the network’s weights.
Moreover, we decouple the meta-gradientcomputation from the model online update.
Hence, we can train the online model using any optimizer,and the distilled data will be agnostic to the specific learning algorithm choice.
Our proposed methodis similar to 1-step TBPTT in that we compute the meta-gradient at each step while performing theonline model update.
Unlike the conventional 1-step TBPTT, we compute the meta-gradient using a
KRR output layer to mitigate truncation bias, illustrated in Figure 2(a).
Model Pool: As discussed in Section 1, there are various types of overfitting in dataset distillation.
Several techniques have been proposed to alleviate such problem, such as random initialization [4],periodic reset [13, 5, 7], and dynamic bi-level optimization [19].
These techniques share the sameunderlying principle: the model diversity matters.
Thus, we propose to maintain a “model pool”filled with diverse set of parameters obtained from different number of training steps and differentrandom initializations.
Unlike the previous methods that periodically training and resetting a singlemodel, FRePo randomly sample a model from the pool at each meta-gradient computation and updateit using the current distilled data.
However, if a model has been updated more than K steps, wereinitialize it with a new random seed.
From the meta-learning perspective, we maintain a diverseset of meta-tasks to sample from and avoid sampling very similar tasks at each consecutive gradientcomputation to avoid overfitting to a particular setup.
Pool Diversity: We can increase the regularization strength by increasing the diversity of the modelpool by setting a larger K , using data augmentation when training the model on the distilled data, orusing models with different architectures.
To keep our method simple, we use the same architecturefor all models in the pool and do not use any data augmentation when training the model on thedistilled data.
Thus, our model pool only contains models with different initialization, at differentoptimization stages, and trained at different time-step of the distilled data.
Related Work
Unrolling in Bi-Level Optimization: One way to compute the meta-gradient is to differentiatethrough the unrolled inner optimization [4, 11–13].
However, this approach inherits several difficultiesof the unrolled optimization, such as: 1) large computation and memory cost [14]; 2) truncationbias with short unrolls [17]; 3) exploding or vanishing gradients with long unrolls [15]; 4) chaoticand poorly conditioned loss landscapes with long unrolls [16].
In contrast, our method considersapproximating the inner optimization with kernel ridge regression instead of unrolled optimization.
Surrogate Objective: To avoid unrolled optimization, several works turn to surrogate objectives.
DC [5], DSA [7], and DCC [18] formulate the dataset distillation as a gradient matching problembetween the gradients of neural network weights computed on the real and distilled data.
In contrast,
DM [8] and CAFE [19] consider the feature distribution alignment between the real and distilled data.
Moreover, MTT [20] shows that knowledge from many expert training trajectories can be distilled toa dataset by using a training trajectory matching objective.
Nevertheless, surrogate objectives mayintroduce new biases and thus, may not accurately reflect the true objective.
For example, gradientmatching approaches [5, 7, 18] only focus on short-range behavior and may easily overfit to a biasedset of samples that produce dominant gradients [19, 20].
Closed-form Approximation: An alternative way to cirvumvent unrolled optimization is to find aclosed-form approximation to the inner optimization.
Based on the correspondence between infinitelywide neural networks and kernel methods, KIP [22, 23] approximates the inner optimization with
NTK [21].
In this case, the meta-gradient can be computed by back-propagating through the NTK.
However, computing NTK for modern neural networks is extremely expensive.
Thus, using NTKfor dataset distillation requires thousands of GPU hours and sophisticated implementation of thedistributed kernel computation framework [23].
Dataset Distillation
4.1 Implementation Details
We compare our method to four state-of-the-art dataset distillation methods [7, 8, 20, 23] on variousbenchmark datasets [26, 32–37].
We train the distilled data using Algorithm 1 with the same setof hyperparameters for all experiments except stated otherwise.
We use a wider architecture thanprevious works [7, 8, 20] because the KRR component does not behave well when the featuredimension is low, resulting in a significant performance drop for our method.
Results on the originalarchitecture are included in Appendix C.6.
We evaluate each distilled data using five random neuralnetworks and report the mean and standard deviation.
For the sake of brevity, we provide moreimplementation details and ablation studies in the appendix.
4.2 Standard Benchmarks
Distillation Performance: We first evaluate our method on six standard benchmark datasets.
Welearn 1, 10, and 50 images per class for datasets with only ten classes, while we learn 1 and 10images per class for CIFAR100 [34] with 100 classes, Tiny ImageNet [35] with 200 classes, and
CUB-200 [37] with 200 fine-grained classes.
As shown in Table in 1, we achieve the state-of-the-artperformance in most settings despite the hyperparameter may be suboptimal.
Our method performsexceptionally well on datasets with a complex label space when learning few images per class.
Forexample, we improve the CIFAR100, Tiny ImageNet, and CUB-200 in one image per class settingfrom 24.3%, 8.8%, and 2.2% to 28.7%, 15.4%, and 12.4%, respectively.
Figure 4 shows that ourdistilled images look real and natural though we do not directly optimize for this objective.
Weobserve a strong correlation between the test accuracy and image quality: the better the image quality,the higher the test accuracy.
Our results suggest that a highly condensed dataset does not need to bevery different from the real dataset as it may just reflect the most common pattern in a dataset.
Wealso report the KRR predictor’s test accuracy using the feature extractor trained on the distilled data.
When the dataset is as simple as MNIST [32], the KRR predictor achieves similar performance asthe neural network predictor.
In contrast, for more complex datasets, the KRR predictor consistentlyoutperforms the neural network predictor, with the most significant gap being 3.7% for Tiny ImageNetin the one image per class setting.
[Table]
[Figure]
T i
[Figure]
e p e r S
[Figure]
MTTDSADMFRePo
Model Width
P ea k G P U
M e m o r
y U s a g e ( G
[Figure]
[Figure]
Label Learning: A similar trend can also be observed for label learning.
When the dataset is simpleand has only a few classes, label learning may not be necessary.
However, it becomes crucial forcomplex datasets with many labels, such as CIFAR100 and Tiny-ImageNet (See more details in
Appendix).
Similar to the teacher label in the knowledge distillation [1], we observe that the distilledlabel also encodes the class similarity.
We identify three typical cases in Figure 4d.
The first groupconsists of highly confident labels with a much higher value for one class than other classes (largemargin), such as sunflower, bicycle, and chair.
In contrast, the distilled labels in the second groupare confident but may get confused with some closely-related classes (small margin).
For instance,the learned label for "girl" has almost equally high values for the girl, woman, man, boy, and baby,suggesting that these classes are very similar and may be difficult for the model to distinguish themapart.
The last group contains distilled labels with low values for all classes, such as bear, beaver, andsquirrel.
It is often hard for humans to recognize the distilled images in such a group, suggesting thatthey may be the challenging classes in a dataset.
Training Cost Analysis: Figure 3a, 3b shows that our method is significantly more time-efficientthan the previous methods.
When learning one image per class on CIFAR100, FRePo reaches asimilar test accuracy (23.4%) to the second-best method (24.0%) in 38 seconds, compared to 3805seconds for MTT, which is roughly two orders of magnitude faster.
Moreover, FRePo achieves 92%of its final test accuracy (26.4% out of 28.7%) in only 385 seconds.
As shown in Figure 3c, ouralgorithm takes much less time to perform one gradient step on the distilled data.
Thus, we canperform more gradient steps in a fixed time.
Furthermore, Figure 3d suggests that our algorithm hasmuch less GPU memory requirement.
Therefore, we can potentially use a much larger and morecomplex model to take advantage of the advancement in neural network architecture.
[Table]
[Figure]
-Architecture
Generalization: We also evaluate the distilled data from CIFAR10 on a widerange of architectures which it has not seen during training, including AlexNet [38], VGG [39],and ResNet [40].
Table 2 shows that our method outperforms previous methods on all unseenarchitectures.
Instance Normalization (IN) [41], as the vital ingredient in several methods (DSA,
DM, MTT), seems to hurt the cross-architecture transfer.
The performance degrades a lot when nonormalization (NN) is applied (Conv-NN, AlexNet) or using a different normalization, like Batch
Normalization (BN) [42].
It suggests that the distilled data generated by those methods encode theinductive bias of a particular training architecture.
In contrast, our distilled data generalize well tovarious architectures, including those without normalization (Conv-NN, AlexNet).
Note that Figure1, 4 also indicate that our distilled data encode less architectural bias as the distilled images looknatural and authentic.
A simple idea to further alleviate the overfitting of a particular architectureis to include more architectures in the model pool.
However, the training may not be stable as themeta-gradient computed by different architectures can be very different.
4.3 ImageNet
High Resolution ImageNet Subset To understand how well our method performs on high-resolutionimages, we evaluate it on ImageNette and ImageWoof datasets [36] with a resolution of 128x128.
Welearn 1 and 10 images per class on both datasets and report the performance in Table 3 and visualizesome distilled images in Figure 1.
As shown in Table 3, we outperform MTT on all settings andachieve much better performance when we distill ten images per class on a more difficult dataset
ImageWoof.
It suggests that our distilled data is better at capturing the discriminative features foreach class.
Figure 1 shows that our distilled images look real and capture the distinguishable featureof different classes.
For the easy dataset (i.e., ImageNette), all images have clear different structures,while for ImageWoof, the texture of each dog seems to be crucial.
Resized ImageNet-1K: We also evaluate our method on a resized version of ILSVRC2012 [26] witha resolution of 64x64 to see how it performs on a complex label space.
Surprisingly, we can achieve7.5% and 9.7% Top1 accuracy using only 1k and 2k training examples, compared to 1.1% and 1.4%using an equally-sized real subset.
[Table]
Application5.1 Continual Learning
Continual learning (CL)
[43] aims to address the catastrophic forgetting problem [43–45] when amodel learns sequentially from a stream of tasks.
A commonly used strategy to recall past knowledgeis based on a replay buffer
[Table]
which stores representative samples from previous tasks [46–49]
Sincesample selection is an important component of constructing an effective buffer [48–51], we believedistilled data can be a key ingredient for a continual learning algorithm due to its highly condensednature.
Several works [6–8, 52] have successfully applied the dataset distillation to the continuallearning scenario.
Our work shows that we can achieve much better results by using a better datasetdistillation technique.
We follow Zhao and Bilen [8] that sets up the baseline based on GDumb [49] which greedily storesclass-balanced training examples in memory and train model from scratch on the latest memory only.
In that case, the continual learning performance only depends on the quality of the replay buffer.
Weperform 5 and 10 step class-incremental learning [53] on CIFAR100 with an increasing buffer sizeof 20 images per class.
Specifically, we distill 400 and 200 images at each step and put them intothe replay buffer.
We follow the same class split as Zhao and Bilen [8] and compare our method torandom [49], herding [54, 55], DSA [7], and DM [8].
We use the default data preprocessing anddefault model for each method in this experiment as we find it gives the best performance for eachmethod.
We use the test accuracy on all observed classes as the performance measure [8, 48].
Figure 5 shows that our method performs significantly better than all previous methods.
The finaltest accuracy for all classes for our method (FRePo) and the second-best method (DM) are 41.6%,33.9% in 5-step learning, and 38.0%, 34.0% in 10-step learning.
However, we notice that for FRePo,distilling 2000 images in a continual learning setup achieves a similar test accuracy (41.6%) asdistilling only 1000 images from the whole dataset (41.3% from Table 1).
In addition, performancedrops as we perform more steps.
It suggests that FRePo considers all available classes to derivethe most condensed dataset.
Splitting the data into multiple groups and performing independentdistillation may generate redundant information or fail to capture the distinguishable features.
5.2 Membership Inference Defense
Membership inference attacks (MIA) aim to infer whether a given data point has been used to trainthe model or not [56–58].
Ideally, we want a model to learn from the data but not memorize it topreserve privacy.
However, deep neural networks are well-known for their ability to memorize allthe training examples, even on large and randomly labeled datasets [59].
Several methods have beenproposed to defend against such attacks by either modifying the training procedure [60] or changingthe inference workflow [61].
This section shows that the distilled data contain little informationregarding sample presence in the original dataset.
Thus, instead of training on the original datasets,training on distilled data allows privacy preservation while retaining model performance.
We consider three distilled data generated by DSA [7], DM [8] and FRePo.
We perform five popular"black box" MIA provided by Tensorflow Privacy [62] on models trained on the real data or the datadistilled from it.
The attack methods include a threshold attack and four model-based attacks usinglogistic regression (LR), multi-layer perceptron (MLP), random forest (RF) and K-nearest neighbor(KNN).
The inputs to those attack methods are ground-truth labels, model predictions, and losses.
Tomeasure the privacy vulnerability of the trained model, we compute the area under the ROC curve(AUC) of an attack classifier.
Following prior work, [56, 63], we keep a balanced set of trainingexamples (member) and test examples (non-member) with 10K each to maximize the uncertainty of
[Table]
[Figure]
MIA.
Thus, the random guessing strategy results in a 50% MIA accuracy.
We conduct experimentson MNIST and FashionMNIST with a distillation size of 500.
For space reasons, we provide moreimplementation details and results in appendix.
As shown in Table 4, all models trained on the distilled data preserve privacy as their attack AUCsare closed to random guessing.
However, we observe a small drop in test accuracy compared tothe model trained on the full dataset, which is expected as we only distill 500 examples instead of10,000 examples.
Compared to the model trained on an equally sized subset of the original data, themodel trained on distilled data results in much better test performance.
Figure 5c, 5d demonstrate thetrade-off between test accuracy and attack effectiveness as measured by ROC AUC.
It shows thatearly stopping can be an effective technique to preserve privacy.
However, we will still be under high
MIA risk if we perform early stopping by monitoring the validation loss.
In contrast, training a modelon the distilled data does not have this problem as the attack AUCs keep at a very low level regardlessof training steps.
Conclusion
We propose neural Feature Regression with Pooling (FRePo) to overcome two challenges in datasetdistillation: meta-gradient computation and various types of overfitting in dataset distillation.
Weobtain state-of-the-art performance on various datasets with a 100x reduction in training time anda 10x reduction in GPU memory requirement.
The distilled data generated by FRePo looks realand natural and generalizes well to a wide range of architectures.
Furthermore, we demonstrate twoapplications that take advantage of the high-quality distilled data, namely, continual learning andmembership inference defense.
Broader Impact “Synthetic data”, in the broader sense of artificial data created by generative models,can help researchers understand how an otherwise opaque learning machine “sees” the world.
Therehave been concerns regarding the risk of fake data.
This paper explores a new research directionin generating synthetic data only for downstream classification tasks.
We believe this work canprovide additional interpretability and potentially address the common concerns in machine learningregarding training data privacy.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .
• Did you include the license to the code and datasets?
[No] The code and the data areproprietary.
• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[No]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[No]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[No]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]