SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 1: Test accuracies  $(\%)$  on SVHN, CIFAR-10, and CIFAR-100. "IPC" denotes the number of images per class. "#Params" denotes the total number of budget parameters. The best results and the second-best result are highlighted in bold and underline, respectively.
IPC
#Params	SVHN	CIFAR-10	CIFAR-100
1
30.72k	10
307.2k	50
1536k	1
30.72k	10
307.2k	50
1536k	1
30.72k	10
307.2k	50
1536k
Coreset	Random	14.6 ±1.6	35.1 ±4.1	70.9 ±0.9	14.4 ±2.0	26.0 ±1.2	43.4 ±1.0	4.2 ±0.3	14.6 ±0.5	30.0 ±0.4
Herding	20.9 ±1.3	50.5 ±3.3	72.6 ±0.8	21.5 ±1.3	31.6 ±0.7	40.4 ±0.6	8.4 ±0.3	17.3 ±0.3	33.7 ±0.5
Input-sized parameterization	DC	31.2 ±1.4	76.1 ±0.6	82.3 ±0.3	28.3 ±0.5	44.9 ±0.5	53.9 ±0.5	12.8 ±0.3	25.2 ±0.3	-
DSA	27.5 ±1.4	79.2 ±0.5	84.4 ±0.4	28.8 ±0.7	52.1 ±0.5	60.6 ±0.5	13.9 ±0.3	32.3 ±0.3	42.8 ±0.4
DM	-	-	-	26.0 ±0.8	48.9 ±0.6	63.0 ±0.4	11.4 ±0.2	29.7 ±0.2	43.6 ±0.4
CAFE+DSA	42.9 ±3.0	77.9 ±0.6	82.3 ±0.4	31.6 ±0.8	50.9 ±0.5	62.3 ±0.4	14.0 ±0.2	31.5 ±0.2	42.9 ±0.2
TM	58.5 ±1.4	70.8 ±1.8	85.7 ±0.1	46.3 ±0.8	65.3 ±0.7	71.6 ±0.2	24.3 ±0.2	40.1 ±0.4	47.7 ±0.2
KIP	57.3 ±0.1	75.0 ±0.1	80.5 ±0.1	49.9 ±0.2	62.7 ±0.3	68.6 ±0.2	15.7 ±0.2	28.3 ±0.1	-
FRePo	-	-	-	46.8 ±0.7	65.5 ±0.4	71.7 ±0.2	28.7 ±0.1	42.5 ±0.2	44.3 ±0.2
Parameterization	IDC	68.1 ±0.1	87.3 ±0.2	90.2 ±0.1	50.0 ±0.4	67.5 ±0.5	74.5 ±0.1	-	-	-
HaBa	69.8 ±1.3	83.2 ±0.4	88.3 ±0.1	48.3 ±0.8	69.9 ±0.4	74.0 ±0.2	33.4 ±0.4	40.2 ±0.2	47.0 ±0.2
FreD	82.2 ±0.6	89.5 ±0.1	90.3 ±0.3	60.6 ±0.8	70.3 ±0.3	75.8 ±0.1	34.6 ±0.4	42.7 ±0.2	47.8 ±0.1
Entire original dataset	95.4 ±0.1	84.8 ±0.1	56.2 ±0.3
Increment of decoded instances	IDC	×5	×5	×5	×5	×5	×5	-	-	-
HaBa	×5	×5	×5	×5	×5	×5	×5	×5	×5
FreD	×16	×8	×4	×16	×6.4	×4	×8	×2.56	×2.56
Binary Mask
Baseline (TM)
Ours (FreD)
(a) Class: Airplane
Figure 6: Visualization of the binary mask, condensed image from TM, and the transformed images of FreD on CIFAR-10 with IPC=1 (#Params=30.72k). Binary masks and trained synthetic images are all the same size. The size of mask and baseline image were enlarged for a better layout.
(b) Class: Automobile
Ours (FreD)
(c) Class: Horse
Ours (FreD)
However, PCA, being a data-driven transform function, cannot be practically utilized as a method for dataset distillation. Please refer to Appendix E.1 for supporting evidence of this claim.
Figure 5a shows the sum of  $\eta$  in descending order in both the spatial and frequency domains. First, we observe that as the number of selected dimensions in the frequency domain increases, frequency domain is much faster in converging to the total variance than the spatial domain. This result shows that the EVR-based dimension selection in the frequency domain is more effective than the selection in the spatial domain. In terms of training, Figure 5b also shows that the performance of a dataset constructed with a very small number of frequency dimensions converges rapidly to the performance of a dataset with the entire frequency dimension.
We evaluate the efficacy of FreD on various benchmark datasets, i.e. SVHN [13], CIFAR-10, CIFAR-100 [8] and ImageNet-Subset [6, 3, 4]. Please refer to Appendix D for additional experimental results on other datasets. We compared FreD with both methods: 1) a method to model  $S$  as an input-sized variable; and 2) a method, which parameterizes  $S$ , differently. As the learning with input-sized  $S$ , we chose baselines as DD [20], DSA [24], DM [25], CAFE+DSA [19], TM [3], KIP [14] and FRePo [27]. We also selected IDC [7] and HaBa [10] as baselines, which is categorized as the parameterization on  $S$ . We also compare FreD with core-set selection methods, such as random selection and herding [22]. We use trajectory matching objective (TM) [3] for  $\mathcal{L}_{DD}$  as a default although FreD can use any dataset distillation loss. We evaluate each method by training 5 randomly initialized networks from scratch on optimized  $S$ . Please refer to Appendix C for a detailed explanation of datasets and experiment settings.
Table 2: Test accuracies (%) on CIFAR-10 under various dataset distillation loss and cross-architecture. "DC/DM/TM" denote the gradient/feature/trjectory matching for dataset distillation loss, respectively. We utilize AlexNet [9], VGG11 [18], and ResNet18 [5] for cross-architecture.
IPC
#Params	DC	DM	TM
2
61.44k	11
337.92k	51
1566.72k	2
61.44k	11
337.92k	51
1566.72k	2
61.44k	11
337.92k	51
1566.72k
ConvNet	Vanilla	31.4 ±0.2	45.3 ±0.3	54.2 ±0.6	34.6 ±0.5	50.4 ±0.4	62.0 ±0.3	50.6 ±1.0	63.9 ±0.3	69.8 ±0.5
w/ IDC	35.2 ±0.5	53.8 ±0.4	56.4 ±0.4	45.1 ±0.5	59.3 ±0.4	64.6 ±0.3	56.1 ±0.4	60.9 ±0.4	71.1 ±0.4
w/ HaBa	34.1 ±0.5	49.9 ±0.5	58.9 ±0.2	37.3 ±0.1	56.8 ±0.1	64.4 ±0.4	56.8 ±0.4	69.5 ±0.3	73.3 ±0.2
w/ FreD	45.3 ±0.5	55.8 ±0.4	59.8 ±0.5	55.9 ±0.4	61.3 ±0.8	66.6 ±0.6	61.4 ±0.3	70.7 ±0.5	75.5 ±0.2
Average of Cross-Architectures	Vanilla	22.0 ±0.9	29.2 ±0.9	34.1 ±0.6	21.5 ±2.2	39.5 ±1.1	52.6 ±0.7	33.1 ±1.1	43.9 ±1.4	55.0 ±1.0
w/ IDC	28.7 ±1.2	35.4 ±0.6	40.2 ±0.7	37.3 ±1.1	50.5 ±0.6	61.3 ±0.5	42.5 ±1.5	48.7 ±1.8	61.5 ±1.0
w/ HaBa	25.4 ±0.9	31.4 ±0.7	35.5 ±0.9	30.1 ±0.6	47.0 ±0.5	60.1 ±0.6	46.4 ±1.0	55.8 ±1.8	64.0 ±0.9
w/ FreD	37.3 ±0.9	37.4 ±0.7	42.7 ±0.8	48.1 ±0.7	57.3 ±0.8	65.0 ±0.7	49.7 ±1.0	60.1 ±0.7	69.1 ±0.7
Table 3: Test accuracies  $(\%)$  on ImageNet-Subset  $(128\times 128)$  under IPC=2 (#Params=983.04k).
Model	ImgNette	ImgWoof	ImgFruit	ImgYellow	ImgMeow	ImgSquawk
TM	55.2 ±1.1	30.9 ±1.3	31.8 ±1.6	49.7 ±1.4	35.3 ±2.2	43.9 ±0.6
w/ IDC	65.4 ±1.2	37.6 ±1.6	43.0 ±1.5	62.4 ±1.7	43.1 ±1.2	55.5 ±1.2
w/ HaBa	51.9 ±1.7	32.4 ±0.7	34.7 ±1.1	50.4 ±1.6	36.9 ±0.9	41.9 ±1.4
w/ FreD	69.0 ±0.9	40.0 ±1.4	46.3 ±1.2	66.3 ±1.1	45.2 ±1.7	62.0 ±1.3
Table 4: Test accuracies (%) on 3D MNIST.
IPC
#Params	1
40.96k	10
409.6k	50
2048k
Random	17.2 ±0.5	49.6 ±0.7	60.3 ±0.7
DM	42.5 ±0.9	58.6 ±0.8	64.7 ±0.5
w/ IDC	51.9 ±1.5	54.0 ±0.5	56.8 ±0.3
w/ FreD	54.9 ±0.5	62.9 ±0.5	66.6 ±0.7
Entire dataset	78.7 ±1.1
Performance Comparison. Table 1 presents the test accuracies of the neural network, which is trained on  $S$  inferred from each method. FreD achieves the best performances in all experimental settings. Especially, when the limited budget is extreme, i.e. IPC=1 (#Params=30.72k); FreD shows significant improvements compared to the second-best performer: 12.4%p in SVHN and 10.6%p in CIFAR-10. This result demonstrates that using the frequency domain, where information is concentrated in specific dimensions, has a positive effect on efficiency and performance improvement, especially in situations where the budget is very small. Please refer to Appendix D for additional experimental results on other datasets.
Qualitative Analysis. Figure 6 visualizes the synthetic dataset by FreD on CIFAR-10 with IPC=1 (#Params=30.72k). In this setting, we utilize 64 frequency dimensions per channel, which enables the construction of 16 images per class under the same budget. The results show that each class contains diverse data instances. Furthermore, despite of huge reduction in dimensions i.e.  $64 / 1024 = 6.25\%$ , each image contains class-discriminative features. We also provide the corresponding binary masks, which are constructed by EVR value. As a result, the low-frequency dimensions in the frequency domain were predominantly selected. It supports that the majority of frequency components for image construction are concentrated in the low-frequency region [1, 21]. Furthermore, it should be noted that our EVR-based mask construction does not enforce keeping the low-frequency components, what EVR only enforces is keeping the components with a higher explanation ratio on the image feature. Therefore, the constructed binary masks are slightly different for each class. Please refer to Appendix D.10 for more visualization.
Compatibility of Parameterization. The parameterization method in dataset distillation should show consistent performance improvement across different distillation losses and test network architectures. Therefore, we conduct experiments by varying the dataset distillation loss and test network architectures. In the case of HaBa, conducting an experiment at  $\mathrm{IPC} = 1$  is structurally impossible due to the existence of a hallucination network. Therefore, for a fair comparison, we basically follow HaBa's IPC setting such as  $\mathrm{IPC} = 2,11,51$ . In Table 2, FreD shows the highest performance improvement for all experimental combinations. Specifically, FreD achieves a substantial performance gap to the second-best performer up to  $10.8\% \mathrm{p}$  in training architecture and  $10.6\% \mathrm{p}$  in cross-architecture generalization. Furthermore, in the high-dimensional dataset cases, Table 3 verifies that FreD consistently outperforms other parameterization methods. These results demonstrate that the frequency domain exhibits high compatibility and consistent performance improvement, regardless of its association with dataset distillation objective and test network architecture.
Figure 7: Test accuracies  $(\%)$  on CIFAR-10-C.
Table 5: Test accuracies (\%) on ImageNet-Subset-C. Note that the TM on ImgSquawk is excluded because the off-the-shelf synthetic dataset is not the default size of  $128 \times 128$ .
#Params	Model	ImgNette-C	ImgWoof-C	ImgFruit-C	ImgYellow-C	ImgMeow-C	ImgSquawk-C
491520 (IPC=1)	TM	38.0 ±1.6	23.8 ±1.0	22.7 ±1.1	35.6 ±1.7	23.3 ±1.1	-
w/ IDC	34.5 ±0.6	18.7 ±0.4	28.5 ±0.9	36.8 ±1.4	22.2 ±1.2	26.8 ±0.5
w/ FreD	51.2 ±0.6	31.0 ±0.9	32.3 ±1.4	48.2 ±1.0	30.3 ±0.3	45.9 ±0.6
4915200 (IPC=10)	TM	50.9 ±0.7	30.9 ±0.7	32.3 ±0.8	45.6 ±1.0	30.1 ±0.5	44.4 ±1.8
w/ IDC	40.4 ±1.0	21.9 ±0.3	32.2 ±0.7	39.6 ±0.5	23.9 ±0.8	40.5 ±0.7
w/ FreD	55.2 ±0.8	33.8 ±0.8	35.7 ±0.6	47.9 ±0.4	31.3 ±0.9	52.5 ±0.8
3D Point Cloud Dataset. As the spatial dimension of the data increases, the required dimension budget for each instance also grows exponentially. To validate the efficacy of FreD on data with dimensions higher than 2D, we assess FreD on 3D point cloud data, 3D MNIST. Table 4 shows the test accuracies on the 3D MNIST dataset. FreD consistently achieves significant performance improvement over the baseline methods. This confirms the effectiveness of FreD in 2D image domain as well as 3D point cloud domain.
Robustness against Corruption. Toward exploring the application ability of dataset distillation, we shed light on the robustness against the corruption of a trained synthetic dataset. We utilize the following test datasets: CIFAR-10.1 and CIFAR-10-C for CIFAR-10, ImageNet-Subset-C for ImagNet-Subset. For CIFAR-10.1 and CIFAR-10-C experiments, we utilize the off-the-shelf synthetic datasets which are released by the authors of each paper. We report the average test accuracies across 15 types of corruption and 5 severity levels for CIFAR-10-C and ImageNet-Subset-C.
Figure 7 and Table 5 show the results of robustness on CIFAR-10-C and ImageNet-Subset-C, respectively. From both results, FreD shows the best performance over the whole setting which demonstrates the superior robustness against corruption. We want to note that IDC performs worse than the baseline in many ImageNet-Subset-C experiments (see Table 5) although it shows performance improvement on the ImageNet-Subset (see Table 3). On the other hand, FreD consistently shows significant performance improvement regardless of whether the test dataset is corrupted. It suggests that the frequency domain-based parameterization method shows higher domain generalization ability than the spatial domain-based parameterization method. Please refer to Appendix D.5 for the results of CIFAR-10.1 and detailed results based on corruption types of CIFAR-10-C.
To explain the rationale, corruptions that diminish the predictive ability of a machine learning model often occur at the high-frequency components. Adversarial attacks and texture-based corruptions are representative examples [11, 23]. Unlike FreD, which can selectively store information about an image's frequency distribution, transforms such as factorization or upsampling are well-known for not preserving frequency-based information well. Consequently, previous methods are likely to suffer a decline in predictive ability on datasets that retain class information while adding adversarial noise. In contrast, FreD demonstrates relatively good robustness against distribution shifts by successfully storing the core frequency components that significantly influence class recognition, regardless of the perturbations applied to individual data instances.
Collaboration with Other Parameterization. The existing method either performs resolution resizing in the spatial domain or uses a neural network to change the dimension requirement of the spatial domain. On the other hand, FreD optimizes the coefficient of the frequency domain dimension and transforms it into the spatial domain through the inverse frequency transform. Therefore, FreD can be applied orthogonally to the existing spatial domain-based parameterization methods. Table 6 shows the performance of different parameterizations applied to HaBa. From the results, we observed that FreD further enhances
Table 6: Test accuracies  $(\%)$  of each collaboration on CIFAR-10.
IPC
#Params	2
61.44k	11
337.92k
TM	50.6 ±1.0	63.9 ±0.3
w/ HaBa	56.8 ±0.4	69.5 ±0.3
w/ IDC & HaBa	61.3 ±0.3	70.9 ±0.4
w/ FreD & HaBa	62.3 ±0.1	72.9 ±0.2
the performance of HaBa. Furthermore, it is noteworthy that the performance of HaBa integrated with FreD is higher than the combination of IDC and HaBa. These results imply that FreD can be well-integrated with spatial domain-based parameterization methods.
4https://www.kaggle.com/datasets/daavoo/3d-mnist
Effectiveness of Binary Mask  $M$ . We conducted a comparison experiment to validate the explained variance ratio as a criterion for the selection of frequency dimensions. We selected the baselines for the ablation study as follows: Low-pass, Bandstop, High-pass, Random, and the magnitude of the amplitude in the frequency domain. We fixed the total budget and made  $k$  the same. Figure 8a illustrates the ablation study on different variations of criterion for constructing  $M$ . We skip the high-pass mask because of its low performance:
14.32% in IPC=1 (#Params=30.72k) and 17.11% in IPC=10 (#Params=307.2k). While Low-pass and Amplitude-based dimension selection also improves the performance of the baseline, EVR-based dimension selection consistently achieves the best performance.
(a) Binary mask  $M$
(b) Frequency transform  $\mathcal{F}$
Figure 8: Ablation studies on (a) the binary mask  $M$ , and (b) the frequency transform  $\mathcal{F}$ .
Effectiveness of Frequency Transform  $\mathcal{F}$ . We also conducted an ablation study on the frequency transform. Note that the FreD does not impose any constraints on the utilization of frequency transform. Therefore, we compared the performance of FreD when applying widely used frequency transforms such as the Discrete Cosine Transform (DCT), Discrete Fourier Transform (DFT), and Discrete Wavelet Transform (DWT). For DWT, we utilize the Haar wavelet function and low-pass filter instead of an EVR mask. As shown in Figure 8b, we observe a significant performance improvement regardless of the frequency transform. Especially, DCT shows the highest performance improvement than other frequency transforms. Please refer to Appendix D.8 for additional experiments and detailed analysis of the ablation study on frequency transform.
Budget Allocation. Dataset distillation aims to include as much information from the original dataset as possible on a limited budget. FreD can increase the number of data  $|F|$  by controlling the dimension budget per instance  $k$ , and FreD stores the frequency coefficients selected by EVR as  $k$ -dimensional vector. For example, with a small value of  $k$ , more data can be stored i.e. large  $|F|$ . This is a way to increase the quantity of instances while decreasing the quality of variance and reconstruction. By utilizing this flexible trade-off, we can pick the balanced point between quantity and quality to further increase the efficiency of our limited budget. Figure 9 shows the performance of selecting the dimension budget per channel under different budget situations. Note that, for smaller budgets i.e.  $\mathrm{IPC} = 1$  ( $\# \mathrm{Params} = 30.72\mathrm{k}$ ), increasing  $|F|$  performs better. For larger budget cases, such as  $\mathrm{IPC} = 50$  ( $\# \mathrm{Params} = 1536\mathrm{k}$ ), allocating more dimensions to each instance performs better i.e. large  $k$ . This result shows that there is a trade-
off between the quantity and the quality of data instances depending on the budget size.
Figure 9: Ablation study on budget allocation of FreD with DM. Dashed line indicates the performance of DM.
This paper proposes a new parameterization methodology, FreD, that utilizes the augmented frequency domain. FreD selectively utilizes a set of dimensions with a high variance ratio in the frequency domain, and FreD only optimizes the frequency representations of the corresponding dimensions in the junction with the frequency transform. Based on the various experiments conducted on benchmark datasets, the results demonstrate the efficacy of utilizing the frequency domain in dataset distillation. Please refer to Appendix G for the limitation of the frequency domain-based dataset distillation.
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No.2021R1A2C200981613). ※ MSIT: Ministry of Science and ICT
[1] Nasir Ahmed, T_Natarajan, and Kamisetty R Rao. Discrete cosine transform. IEEE transactions on Computers, 100(1):90-93, 1974.
[2] Yoshua Bengio, Yann LeCun, et al. Scaling learning algorithms towards ai. Large-scale kernel machines, 34(5):1-41, 2007.
[3] George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A Efros, and Jun-Yan Zhu. Dataset distillation by matching training trajectories. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4750-4759, 2022.
[4] George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A Efros, and Jun-Yan Zhu. Generalizing dataset distillation via deep generative prior. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3739-3748, 2023.
[5] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[6] Jeremy Howard. A smaller subset of 10 easily classified classes from imagenet and a little more french. URL https://github.com/fastai/imagenette, 2019.
[7] Jang-Hyun Kim, Jinuk Kim, Seong Joon Oh, Sangdoo Yun, Hwanjun Song, Joonhyun Jeong, Jung-Woo Ha, and Hyun Oh Song. Dataset condensation via efficient synthetic-data parameterization. In International Conference on Machine Learning, pages 11102-11118. PMLR, 2022.
[8] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[9] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84-90, 2017.
[10] Songhua Liu, Kai Wang, Xingyi Yang, Jingwen Ye, and Xinchao Wang. Dataset distillation via factorization. arXiv preprint arXiv:2210.16774, 2022.
[11] Yuyang Long, Qilong Zhang, Boheng Zeng, Lianli Gao, Xianglong Liu, Jian Zhang, and Jingkuan Song. Frequency domain model augmentation for adversarial attack. In European Conference on Computer Vision, pages 549-566. Springer, 2022.
[12] Michael Mathieu, Mikael Henaff, and Yann LeCun. Fast training of convolutional networks through ffts. arXiv preprint arXiv:1312.5851, 2013.
[13] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
[14] Timothy Nguyen, Roman Novak, Lechao Xiao, and Jaehoon Lee. Dataset distillation with infinitely wide convolutional networks. Advances in Neural Information Processing Systems, 34:5186-5198, 2021.
[15] Harry Pratt, Bryan Williams, Frans Coenen, and Yalin Zheng. Fconn: Fourier convolutional neural networks. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2017, Skopje, Macedonia, September 18-22, 2017, Proceedings, Part I 17, pages 786-798. Springer, 2017.
[16] Jorge Rebaza. A first course in applied mathematics. John Wiley & Sons, 2021.
[17] Noveen Sachdeva and Julian McAuley. Data distillation: A survey. arXiv preprint arXiv:2301.04272, 2023.
[18] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
[19] Kai Wang, Bo Zhao, Xiangyu Peng, Zheng Zhu, Shuo Yang, Shuo Wang, Guan Huang, Hakan Bilen, Xinchao Wang, and Yang You. Cafe: Learning to condense dataset by aligning features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12196-12205, 2022.
[20] Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, and Alexei A Efros. Dataset distillation. arXiv preprint arXiv:1811.10959, 2018.
[21] Ye Wang, Miikka Vilermo, and Leonid Yaroslavsky. Energy compaction property of the mdct in comparison with other transforms. In Audio Engineering Society Convention 109. Audio Engineering Society, 2000.
[22] Max Welling. Herding dynamical weights to learn. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 1121-1128, 2009.
[23] Chenglin Yang, Adam Kortylewski, Cihang Xie, Yinzhi Cao, and Alan Yuille. Patchattack: A black-box texture-based attack with reinforcement learning. In European Conference on Computer Vision, pages 681-698. Springer, 2020.
[24] Bo Zhao and Hakan Bilen. Dataset condensation with differentiable siamese augmentation. In International Conference on Machine Learning, pages 12674-12685. PMLR, 2021.
[25] Bo Zhao and Hakan Bilen. Dataset condensation with distribution matching. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 6514-6523, 2023.
[26] Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. Dataset condensation with gradient matching. arXiv preprint arXiv:2006.05929, 2020.
[27] Yongchao Zhou, Ehsan Nezhadarya, and Jimmy Ba. Dataset distillation using neural feature regression. arXiv preprint arXiv:2206.00719, 2022.