Hyperbolic Dataset Distillation
Authors: Wenyuan Li
Guang Li
Keisuke Maeda
Takahiro Ogawa
Miki Haseyama
Venue: NeurIPS 2025 poster ()
Keywords: Dataset Distillation
Hyperbolic Space
Distribution Matching
Primary area: deep_learning
TL;DR: This paper introduces hyperbolic space into dataset distillation for the first time.

Abstract: To address the computational and storage challenges posed by large-scale datasets in deep learning, dataset distillation has been proposed to synthesize a compact dataset that replaces the original while maintaining comparable model performance. Unlike optimization-based approaches that require costly bi-level optimization, distribution matching (DM) methods improve efficiency by aligning the distributions of synthetic and original data, thereby eliminating nested optimization. DM achieves high computational efficiency and has emerged as a promising solution. However, existing DM methods, constrained to Euclidean space, treat data as independent and identically distributed points, overlooking complex geometric and hierarchical relationships. To overcome this limitation, we propose a novel hyperbolic dataset distillation method, termed HDD. Hyperbolic space, characterized by negative curvature and exponential volume growth with distance, naturally models hierarchical and tree-like structures. HDD embeds features extracted by a shallow network into the Lorentz hyperbolic space, where the discrepancy between synthetic and original data is measured by the hyperbolic (geodesic) distance between their centroids. By optimizing this distance, the hierarchical structure is explicitly integrated into the distillation process, guiding synthetic samples to gravitate towards the root-centric regions of the original data distribution while preserving their underlying geometric characteristics. Furthermore, we find that pruning in hyperbolic space requires only 20\% of the distilled core set to retain model performance, while significantly improving training stability. Notably, HDD is seamlessly compatible with most existing DM methods, and extensive experiments on different datasets validate its effectiveness. To the best of our knowledge, this is the first work to incorporate the hyperbolic space into the dataset distillation process. The code is available at https://github.com/Guang000/HDD.

Hyperbolic Dataset Distillation
Wenyuan Li Hokkaido University wenyuan@lmd. ist hokudai...... ac jp
Guang Li' Hokkaido University guangālmd. ist. cett..ooudii.cm ac. jp
Keisuke Maeda Hokkaido University maeda@lmd. ist www.hhiii............. jP.
Takahiro Ogawa
Hokkaido University
ogawa0lmd. ist hokudai.ac. ac.
Miki Haseyama
Hokkaido University
mhaseyama@lmd. ist. ac.oudai.a......
Abstract
To address the computational and storage challenges posed by large-scale datasets in deep learning, dataset distillation has been proposed to synthesize a compact dataset that replaces the original while maintaining comparable model performance. Unlike optimization based approaches that require costly bi-level optimization, distribution matching (DM) methods improve efficiency by aligning the distribu- tions synthetic and original data, thereby eliminating nested optimization. DM achieves high computational efficiency and has emerged However, existing DM methods, constrained to Euclidean space, treat data as independent and identically distributed points, overlooking complex geometric and hierarchical relationships. To overcome this limitation, we propose a novel hyperbolic dataset distillation method, termed HDD. Hyperbolic space, character- ized by negative curvature and exponential volume growth with distance, naturally models hierarchical and tree-like structures. HDD embeds features extracted by a shallow network into the Lorentz hyperbolic space, where the discrepancy be- tween synthetic and original data measured by the hyperbolic (geodesic) distance between their centroids. By optimizing this distance, the hierarchical structure is explicitly integrated into the distillation process, guiding synthetic samples gravitate towards the root-centric regions of the original data distribution while preserving their underlying geometric characteristics. Furthermore, we find that pruning in hyperbolic space requires only 20% of the distilled core set to retain model performance, while significantly improving training stability. Notably, HDD is seamlessly compatible with most existing DM methods, and extensive experi- ments on different datasets validate its effectiveness. To the best our knowledge, this is the first work to incorporate the hyperbolic space into the dataset distillation process. The code is available at battps://gae tigt..b... com coom/Guaag000///
promising solution.
Introduction
Recently, deep neural networks (DNNs) have demonstrated outstanding performance across wide range of tasks. However, the continuous performance improvement has led increasingly large datasets, which in turn have escalated storage costs and computational demands, emerging as key
"Correspondence to: Guang Li <guang @ Imd hokudaiaoouuii...ppp
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

An Example of Dataset Tree
Lorrntte Myovebbl De Spaces
Leaf Node
Hypertilic Space
9
origin
Embedding
Hanrccical branches
Node
'cat' CIFAR-10 dataset
Figure 1: An example of hierarchical representation in hyperbolic space using the 'Cat' class from the CIFAR-10 10 Dataset. Hyperbolic space naturally encodes hierarchical structures. In this context, samples located near the root node often represent the category prototype more effectively, while those situated at higher hierarchical levels (closer to the leaf nodes) tend to contain noisier or specific information, such as object parts or less visible angles.
bottlenecks in the further advancement of deep learning. To address this issue, dataset distillation (DD) has been proposed [56]. By condensing the information of the original dataset, DD synthesizes significantly smaller artificial dataset while striving to achieve comparable model performance. Beyond this, DD has also been widely applied in various domains, such as neural architecture
search (88 14 42 52), continual learning [19 60). and privacy protection 13 30 31.
To avoid the bi-level optimization problem of the DD methods, matching- dataset distillation methods have been proposed. Currently, they can be broadly classified into three categories: gradient matching [69]. trajectory matching 11 and distribution matching 688 64 70). The first ap- proaches can be collectively referred to as optimization-diin these methods have achieved promising performance, their reliance on expensive optimization or nested gradients often incurs high computational costs, which hinders their scalability and broader application. In contrast, Zhao al. proposed a distribution matching approach, which mitigates the need for expensive optimization by aligning the feature distributions encoded by neural networks from both the original and synthetic datasets, thereby reducing computational overhead [68. Despite its advantages, distribution matching methods generally underperform pppimization-drivee approaches in terms of final model accuracy.
dataset distillation methods. Although
47 and moment matching (68, 64]. The central challenge lies in defining an effective metric to quantify the distributional discrepancy between the original and synthetic datasets. Point-wise matching is performed Euclidean space by comparing feature representations using Mcan Squared Error (MSE) on a per-sample basis. However, MSE primarily focuses on local alignment (e.g.. pixel-wise similarity within samples) and tends to overlook the global semantic structure embedded in high- high-dimensional manifolds. In contrast, moment matching employs Maximum Mean Discrepancy (MMD) as metric, which enables effective measurement of overall distribution differences in a Reproducing Kernel Hilbert Space (RKHS). Although both MSE and MMD attempt to reduce the distribution gap between original and synthetic datasets, they overlook a critical aspect: the hierarchical (or tree-like) structure inherent in dataset samples [59] 46). as illustrated in Figure Under the hierarchy, the significance of samples series-llwwe-leed
Distribution matching is typically divided into instance-level (point-wise) matching
55
samples (closer to the root) tend to better represent the category prototype, whereas higher-level samples (closer to the leaves) often carry more irrelevant noisy information [22] 20). Treating all samples as independent and identically distributed (i.i.d)) when using MSE or MMD may thus degrade distillation performance.
To address the above-mentioned limitation, we introduce hyperbolic space as the distribution space for samples and propose novel hyperbolic dataset distillation (HDD) method. Unlike Euclidean and Hilbert spaces, hyperbolic space characterized by negative curvature, whose geometric constraints offer continuous approximation of hierarchical tree-like structures, effectively capturing complex hierarchical relationships [12 15). hyperbolic space, the centroid of data distribution is the point that minimizes the total of squared hyperbolic distances to all sample points. Due to the unique

---

geometric properties of hyperbolic space, higher-level samples exert less influence on the centroid, naturally biasing it toward lower-level samples that are more representative of category prototypes. Nevertheless, the centroid still integrates the influence of all samples, which allows it to encode the overall geometrie structure of the dataset. Based on this observation, we propose to match the distribution centroids of the original and synthetic datasets in hyperbolic space. This strategy aims to minimize distributional discrepancies, particularly concerning lower-level (prototype-like) samples, while also preserving the global geometric structure of the dataset [25]. The motivation of this study is that samples within a dataset contribute unequally to the overall representation depending on their hierarchical level, and the distillation process should be designed to reflect this imbalance. Notably, HDD is fully compatible with most existing dataset distillation methods. To the best of our knowledge, this is the first work to introduce hyperbolic space into the dataset distillation framework.
To summarize, our contributions are as follows:
We propose hyperbolic dataset distillation (HDD), novel method that incorporates hyper- bolic geometry into dataset distillation enable hierarchical sample weighting, effectively capturing semantic structures at multiple levels. Additionally, HDD aligns the global ge- ometric distributions of the original and distilled datasets by matching their centroids in hyperbolic space.
We analyze the contributions of samples at different hierarchical levels to the overall training loss, providing insights into their respective roles during distillation.
Extensive experiments on diverse benchmarks, including Mashion-MNIST SVHN, CIFAR- 10, CIFAR-100, and TinylmageNet, demonstrate the effectiveness of our method. Addition- ally, our model also performs well in cross-architecture experiments.
Furthermore, we apply hierarchical pruning to the original dataset by utilizing only the pruned subset for distribution alignment. Empirical results show that merely 20% of the original data suffices to preserve performance, underscoring the efficacy of hierarchical structuring within hyperbolic space.
Related Works
Dataset Distillation.
Existing DD methods can be broadly categorized into three categories: gradient
matching, trajectory matching, and distribution matching [34] 29 63 4T. Gradient matching [69 67 seeks to preserve critical information by minimizing the discrepancy between the gradients induced by synthetic and original samples during model training. Trajectory matching [4] 14
32 33 parameters. Distribution matching 68
achieves fine-grained knowledge transfer by aligning the training trajectories of network improves the representational capacity of synthetic samples by aligning their statistical distributions with those of original data in feature or activation
64 70
61 and decoupling optimization-based methods (62 53 49 have been proposed, accelerating advancements in the field of dataset distillation. In this work, we introduce hyperbolic space into dataset distillation by leveraging its inherent negative curvature to impose the tree-like hierarchy of the original dataset onto synthetic data, thereby offering a novel perspective to address the fundamental challenges in dataset distillation.
spaces, Recently, generative-basee dataset distillation [8 50 36 37
Hyperbolic Machine Learning. Hyperbolic space naturally encodes hierarchical data, which has attracted considerable interest in machine learning. It was first widely adopted in graph neural networks 66] 17 2 to more effectively capture hierarchical and complex graph structures. In computer vision and multimodal tasks. hyperbolic geometry has also been applied to metric learn- ing [16 45 40), generation g 3 recognition [24), and segmentation [L. As fully hyperbolic architectures have matured, hyperbolic-based vision methods have become increasingly sophisticated. In this work, we introduce hyperbolic space into dataset distillation for the first time, leveraging its hierarchical properties to assign differentiated weights to samples.
3

---

3 Method
3.1 Preliminaries
Problem Definition. Consider large-scale original dataset R { ) where riahl represents the i-th sample instance from the original dataset, & represents the corresponding label of the sample rigall in the original dataset, and [R] is the total number of samples in the original dataset. The goal of dataset distillation is to construct a significantly smaller synthetic dataset
j=1' where " represents the j-th synthetic sample instance, 17" represents the corresponding label of the synthetic sample syg and [S] is the total number of samples in the synthetic dataset, with [] < [R]. Such that a model trained on S (denoted 0yyn) exhibits performance comparable to one trained on R (denoted Oreal) when evaluated on previously unseen samples. Formally, let Pr denote the true data distribution and a loss function (e.g., cross-entropy), then the optimal synthetic dataset is obtained by minimizing the discrepancy in performance between Osyn and Oral as follows:
= {(6)", ")"))
8 = arg min
||e (p), (p)calp)) Pp
where (p, () represents sample pair drawn from the true data distribution Pr, with denoting the data instance and t its corresponding label.
To tackle Eq. T previous optimization- methods have primarily focused on two key strategies.
One approach refines through meta-learning, while the other aligns gradients or parameters between S and R. Nevertheless, both strategies necessitate bi-level optimization structure, which is computationally demanding due to the need for nested gradient computations. In contrast, DM [68 introduces distribution matching as more efficient alternative by aligning the feature distributions between and R. Within this framework, the optimization of the condensed dataset is typically categorized into instance-level matching and moment matching. Instance level matching overlooks the global semantic structure the data, making a suboptimal choice. In contrast, moment matching is formulated as follows:
R
IS Σ [S]
S* = -arg min Esq-Poo
SQ real
S )
(2)
RR
3=1
where σo Pag represents a feature extractor randomly sampled from the distribution Peo (typically instantiated by a randomly initialized DNN without the final linear classification layer).
Hyperbolic Geometry. In hyperbolic geometry, the n-dimensional hyperbolic space is formally defined as a Riemannian manifold (M", .KK endowed with a constant negative curvature K < O, where M denotes the underlying manifold and gk is the Riemannian metric that characterizes its geometric structure. To facilitate efficient and numerically stable computations, we adopt the Lorentz model L 9L) which embeds the hyperbolic space into (n + 1)-dimensional Minkowski space. Here, L represents the set of points satisfying the constraint (x, x) - = 1/K, and the metric tensor is given by 9к diag([11, the Lorentzian manifold can be defined as follows:
L: { X
(x,
1 K
> 00.
Each point x E L& can be expressed as a vector x = [xg x. where v, > is referred to as the time component and v. E R" as the spatial component. The Lorentzian inner product is defined as:
(x,y)c := E,3i + x Ve-
Although several isometrically equivalent models exist hyperbolic geometry, such as the Poincaré ball, the Klein model, and the upper half-space model, our work primarily utilizes the Lorentz model due to its analytical tractability and improved numerical behavior. The relevant details are explained in detail in Appendix A.

---

3.2 Hyperbolic Dataset Distillation for Distribution Matching
Given the original dataset R {(rta)) ( and the synthetic dataset for update
(where || < [R), we first encode the data through a frozen pre-trained encoder Φ { +
generating corresponding feature vectors nreal and
" 2 as follows:
1 o(riea)), 1 o(s)"").
(5)
This process projects both original samples rf and synthetic samples s) into Euclidean feature space parameterized by 6. Subsequently, we map each sample from both the original and synthetic datasets to the hyperbolic space via the exponential map, yielding the hyperbolic embeddings zreal and 2)" respectively, as follows:
real
real cxPpo (u.eaa) = cosh [1 Po +
(6)
K
|u ||
2/ exP (1))t)) = cash (u po
Sinh(V-Klue
Here, |||||= v (w; v) denotes the norm induced by the Minkowski inner product, and po represents the base point in the hyperbolic space, which is defined as:
po 1.-
0,0,
where K < 0 denotes the curvature of the hyperbolic space.
To facilitate subsequent analysis, collect all hyperbolic embeddings of the original and synthetic datasets into two sets:
ZToal = 22
,
real [ Jim1
Z
22 syn syn [S]
1--1
Here, Zreal and - denote the sample points in the hyperbolic space corresponding the real and synthetic samples, respectively. Unlike distribution matching methods in Euclidean space, in hyperbolic space, the distributional center of each embedded dataset characterized by its Riemannian (Karcher) mean. We define their Riemannian means in the Lorentz model as:
R
S
real
arg min ZEER
(2, real)
arg min ZEER
(2,
syn 2
(10)
where denotes point in the Lorentzian hyperbolic space Li over which the Riemannian mean is optimized, and the Lorentzian hyperbolic distance d, on the upper-sheet hyperboloid model is
dr, (m, n) = V-K acosh(- KK (m,, n)ec; m, L
(11)
and ( denotes the Minkowski inner product, shown in Eq.
To mitigate the extra computational overhead introduced by iterative procedures, we employ the centroid approximation approach proposed by Law et al. (25 which can be expressed as follows:
c=V-K
where
(12)
(2,2)c1++
Here, z, E Bd+1 denotes the input vectors in Lorentzian hyperbolic space, z is their Euclidean average. The curvature constant K < 0 reflects the negative curvature of the hyperbolic space. A small is added for numerical stability.
Finally, we define the distribution matching loss as the Lorentzian hyperbolic distance between the two means follows:
A
Lind = A dd (zreal zeyu) = V-K acosh (greal 20ym)2)
(13)
5

---

Real Dataset
Euciddean space
Geodesic Distance
Fneenker
Top down parspective
EXP (vtt)
Goodessic Distance
Eyo levell perspective
Eoverts
Synthatic
* x The commui af reallsynthii dataser, n Eudidkan space +/+ The Frichas maan a mallsyrtharic dasavoti Hyperbolls space
update
The
space
sysychatic
Figure 2: The framework of hyperholic dataset distillation. The proposed method leverages exponen- tial mapping to embed the dataset into hyperbolic space, enabling a hierarchical representation where samples at different levels assigned varying weights reflect their significance within the global geometry. Centroids of both the original and synthetic datasets are then computed in the hyperbolic space, and the geodesic distance between them is used to quantify the distributional discrepancy. This hyperbolic distance serves as a loss term to iteratively update the synthetic dataset, encouraging it to better align with the class-specific prototypes of the original data.
where À is the gradient scaling factor. In hyperbolic space, the centroid distribution is close to the origin, resulting a very small distance between the centroids of the original dataset and the synthetic dataset. Additional parameters are required for amplification, detailed in Appendix B.
Based this loss, objective in distribution matching is reformulated as minimizing the Lorentzian hyperbolic distance between the Riemannian means of the original and synthetic datasets:
S = arg min [Adz( (zreal sny")..
(14)
Beq ~Po
As illustrated in Fig. the framework of HDD is presented. It is compatible with broad range of existing distribution matching frameworks.
3.3 Loss Contribution of Samples at Different Levels
In hyperbolic space, samples embedded at lower levels tend to better represent category prototypes. When calculating the centroid, hyperbolic space can effectively assign different weights to relatively lower-level and higher-level samples, meaning their contributions to the centroid vary in influence. To gain explicit insight into how each sample influences the alignment between the original dataset R and the synthetic dataset S in hyperbolic space, we adopt tangent space approximation centered the origin o E Lince the centroids of the sets (greal and gay") are near the origin, this approximation reasonably effective. For rreal and z respectively, define the hyperbolic radius (distance to the origin) of each sample as:
T (o real
8, do "
(15)
and let the corresponding normalized tangent vectors at the origin be
u, / cosh r; o,
ey = syn - cosh O.
(16)
These vectors satisfy ue My E T,L% (tangent space at the origin), i.e., they lie the tangent space at the origin and satisfy (u, 0) (2) 1 0.

---

To capture the radial influence of each sample, we introduce the scalar weight function (the derivation process is in Appendix C):
( K
(17)
sinh( V d)
which is strictly decreasing in d. d represents the distance from the corresponding point to the reference point, which is defined as the origin in this context. This reflects that samples closer to the origin (i.e., with smaller hyperbolic norm) contribute more strongly the Fréchet mean in the tangent space.
Under the tangent-space approximation of the Fréchet mean condition (i.c., the first-order optimality condition for the squared distance sum), the logarithmic maps of the centroids can be approximated
R
SS]
Logo real
w(r)) us,
Logo
w w(1)) U/.
(18)
j=1 This yields the approximate loss function as the Euclidean distance between the two Nog-mapped centroids in the tangent space:
R wo(r)) du
ISI Σ
Lapprox dr. (zreal, 25
(19)
This formulation explicitly reveals the per-sample contribution to the overall loss: each sample affects the direction of the weighted log-mean, with its impact modulated by the scalar weight w(r). Specifically, central samples (closer to the origin) receive higher weights, while peripheral samples (with larger hyperbolic radius) contribute less. This reflects natural attenuation of influence in hyperbolic geometry and enhances stability by reducing the effect of outliers. Furthermore, we also explain this phenomenon from the perspective of gradients. For details, please refer to Appendix D.
4 Experiments
4.1 Experimental Setup
Dataset. We evaluated HDD on several standard benchmark datasets, including Fashion-MNIST (58), SVHN [43, CIFAR-10 (23], CIFAR-100 [23,, and the larger-scale TinylmageNet (26). Additionally, for hybrid architecture experiments, we utilized the Woof subset of ImageNet [0. which features higher resolution images. Please refer to Appendix for detailed information about the datasets used.
Network Architectures. For our primary experiments, we adopt the same convolutional network (ConvNet 277 architecture as used in DC 69), DM 668, and IDM (70 to extract feature repre- sentations. This ConvNet consists of three sequential modules, each comprising a convolutional layer, instance normalization, ReLU activation, and stride-2 average pooling layer. To evaluate arostterciee generalization, we follow the protocol in DM and conduct experiments using ConvNet, AlexNet, VGG1 and ResNet18 (The results can be found in Appendix F). For hybrid architecture experiments, we adopt the architectural configuration proposed in Dance [64]
Implementation Details. Our hyperparameter settings follow the design of the DM 68 IDM 70 and Dance [64] architectures. We adopt the differentiable siamese augmentation 67 enhancement method used in prior works. The synthetic dataset is learned using SGD. For DM with HDD, we train for 20,000 iterations, while for IDM with HDD and Dance with HDD, we train for 10,000 iterations. For all experiments, we set the batch size to 256. Additionally, for different experiments, we use distinct hyperbolic curvature K, gradient scaling factor X, and synthetic image learning rate r, as detailed in Appendix G. All experiments are conducted on one RTX A6000 Ada GPU, except for Section 4.5.
4.2 Main Results
In the main results section, we established a comprehensive set of baseline methods to evaluate model performance. For core set selection approaches, we employed Random Selection (7). Herding (57), K- Center [88,, and Forgetting [54]. Within the category of optimization-basee methods, we incorporated
7

---

Table Comparison of different methods on the FashionMNIST, SVHN, CIFARIO, and CIFAR
datasets with IPC = 1, 10, and 50.
Method
FashonMNIST
CIFARIO
CIFAR
(a) (8)
0017
0.17
0.02
Rankon
514458
738407
52,5+077
14.61.
35.144.1
709+09
14.4200
260+122
434=1.0
4.240.3
14.6
300404
71.1107
21.5112
3166007
K-Geneer
669118
54.71155
68.3108
21.0::.5
8.310.3
3051003
Forgeting
12.145.6
19,5+1.5
25.3+110
23.5-1.1
4.5+0.
15.1-0.2
30.+004
69)
70.5::06
62.3400
$3.504
51.211.4
62.310.3
18.350.5
44:9:005
53.0-0.5
128403
35.2:0.3
70
84.6+0.3
58.7+0.3
27.
79.2+0.5
8444004
6-0.7
52.1+0.
60.6-0.5
139403
10. 30.5-0.3
428+00
77.1+00
83.0+044
$4.8+04
42.6+3.3
81
10.3+1.1
46.14-0
55.5-0.6
129403
27.8-003
17.+0.3
CATU-DSA
73.710.7
83.0.0.3
58.210.3
77,9106
31.108
50910.
62.3104
1401103
76.21.0.8
83.3102
34,020.7
5441005
30.520.3
G-VBSM
54.3:0.3
18.7:002
45.1104
32.0+1.2
$4.2-088
34.8-0.5
70.7:06
814400
62610.3
68.5106
60.2:0.5
114403
430004
840400
88.800
25
2504
18.790:2
5033403
133900
774+0-3
8244002
34.5-0.1
45.2+0055
57.1+003
67.2-6.1
44.-00.3
46.5+04
Whele 785102
IDM
K183002
364103
678.002
$40100
87.60.1
47.010.1
61.3101
69.7.0.
253102
4544:0.1 489103
93.540.1
96440.1
84.8-001
562-003
DC 1691. DSA 667, and DCC 28 For distrbbution-matching methods, our baselines included CAFE (55], CAFE+DSA (55), DataDAM(47]], as well as DM (68] and IDM [70]. Additionally, we have also considered the decoupling optimization method G- G-VBSM [49]. Detailed descriptions of these baseline methods are provided in Appendix H. For DM, IDM, and HDD, each experiment is conducted three times, and the mean and standard deviation are reported.
Table presents a comparative evaluation of our method against prior approaches on Fashion- MNIST 588 SVHN 43 CIFAR-10 [23], and CIFAR-100 [23] The results for TinyImageNet [26] are provided in Appendix IDM augmented with HDD, which exploits the hierarchical inductive bias of hyperbolic space, consistently outperforms the baseline IDM across all benchmarks. Under the IPC setting, IDM with HDD achieves classification accuracies of 78.5% on MNSH (+1.1%), 67.8% SVHN (+2.5%), 47.0% on CIFAR-10 (+1.8%), and 25.3% on CIFAR- 100 (+3.2%), demonstrating its superiority in low-data regimes. With IPC = 10, the proposed method attains 61.3% accuracy CIFAR- 10, 4.0% improvement over IDM. Under IPC = 50, it yields gains of 2.4%, 2.5% and 2.4% on SVHN, CIFAR- 10, and CIFAR- 100, respectively. Furthermore, DM with HDD also exhibits notable enhancements relative to DM: on SVHN, accuracy increases by 3.1 % (IPC= 1) and 2.3% (IPC= = 10), and on CIFAR-10 10 (IPC = 1) by 2.3%. We present some of the distilled images in Appendix K.
4.3 Hierarchical Pruning
To validate the efficacy of hyperbolic-space- aware hierarchical pruning, we conducted the pruning experiments on CIFAR-10 (IPC = 10) by comparing DM with HDD against IDM with HDD across varying pruning rates. Specifically, given a batch of the original CIFAR-10 dataset
Table 2: The distillation accuracy of CIFARIO
(IPC= 10) for different pruning ratios.
Pruning Ratio
DM
DM HDD
IDM with
95%
48.9 18.661066
49.6.10.5
39.1
D = [(r;, 4,, ) where x; denotes the time component of sample we sort all samples
80%
48. 7+0.2
50.2+02
60,3+0.3
50%
8:0.2
50.3:0.1
60.92022
0%
48.5:00.6
50.3+003
61.350.1
in descending order of xt and remove the top
a% of samples exhibiting the highest time component, with pruning ratios a E {95%, 80%, 50%).
Formally, the retained subset is defined as
D' = (ri, 4s E D rank(a)) > [a N7),
(20)
where rank(z;) denotes the position of of in the descending-sorte list.
Table presents the matching accuracy after hierarchical pruning: both DM and DM with HDD require only 20% of the original training set to maintain performance, while IDM with HDD likewise preserves the vast majority of its performance with just 20% of data. This observation demonstrates that lower-level samples possess greater representativeness also observed that excessively small sample sizes still lead to performance degradation, indicating that higeer-level samples also influence the centroid. Furthermore, Figs. (a) and (b) depict the accuracy trajectories throughout the distillation process under various pruning ratios for HDD-DM
hyperbolic space. However, we
8

---

2074
twatish
(a) DM with HDD
(b) IDM with HDD
Figure 3: Distillation accuracy variations of CIFAR-IO (IPC = 10) during the distillation process with different pruning rates.
and HDD-IDM, respectively. Notably, after pruning, the accuracy curves exhibit significantly reduced fluctuations during later training stages, demonstrating markedly enhanced training stability.
4.4 Hybrid Architecture Experiment
Table 3: Comparison of different methods on the CIFARIO, CIFARIOO, and Image Woof datasets.
Method
CIFAR1O
CIFAR100
ImageWonf
IFC
10
50
10
50
10
Rano (3))
0.02
0.2
2
10
0.11
1.10
65.560.2
50.260.3
406:20
D'M
55.260.4
723:003
150300
47.1
59940.2
09:000.3
24.620.1
15.72044
51.3204
DSDM 15
45.0:04
06.5+0.3
758-003
10.5:0.2
46.246.3
$4.0-122
MBD
63
45.3 45.3:0.3
63.520.2
69 00.5
26. 26.2300.3
42.42662
50 50.9:0.7
Dance
47:2:003
70.2460.2
76 26.3:0111
26, 23:22
19.746.1
$2
27 1802
46 2:02
Dance with HDD
46.8:0.3
76:8::022
77.1
27.7::0.3
50.23:0.3
$3.928.1
27.6:02
46630.1
Whole Dataset
$4.8390.1
56,240.3
67.0:13
To evaluate HDD's ' scalability, we ran additional experiments with the Hybrid Dance 64 architecture
that alternates between cross-entropy and distribution matching optimization. We compared our
proposed Dance with HDD method with leading distribution matching methods (IID [1 DSDM
35
and M3D (65) as well as state-of-the-ae approaches from other domains (DATM 21, RDED 53
D'M (50), and the comprehensive experimental results are summarized in Table 3 On CIFAR-10
10
with IPC = 50, Dance with HDD improves over the original Dance by 0.8%. On CIFAR- with IPC
=1, 1, it outperforms both Dance and M3D by 1.5%. Remarkably, at IPC 10 on CIFAR-100, Dance with HDD within 6% of training on the whole dataset. When scaling up to higher resolutions, our method still leads: on ImageWoot it gains 0.5% IPC 1 and 0.4% at IPC 10 compared to Dance. In addition, we also conducted our experiments on another hybrid architecture, DSDM [35]; please refer to Appendix J.
4.5 Runtime and GPU Memory Usage
We evaluate the computational overhead of DM with HDD relative to the baseline DM on the CIFAR-10 dataset. All experiments in this section are conducted on an RTX 4090. DM with HDD uses the same settings as DM (e.g., batch size, input image resolution), matching those in the main experiments. For runtime, we run 1,000 iterations and report the time per 100 iterations by dividing the total by 10.
Table 4: Runtime and GPU Memory Usage CIFAR-10
Rautim:
HDD
DM Mansry
Monnry
3,502Mi8
3,502MB8
10
3,66618
1,632MB8
3.88MIB
3,002MBB
As shown in Table 4 across IPC 1/10/50 on CIFAR-10, adding HDD increases runtime from 4.9-5.4s to 6.7-7.1s, while GPU memory overhead negligible. The effect stable across IPC levels, indicating modest, largely constant-time cost without inflating memory.
4.6 Ablation Study
We conducted an ablation study on different curvature values K within the DM framework on CIF As shown the Table although the curvature K slightly affects the final accuracy, the variation is modest, and HDD consistently outperforms the Euclidean baseline. For example,

---

when IPC 10, DM with HDD at curvatures IK1=1/3 and IK1=5 still outperforms plain DM by 1.1% and 1.5% percentage points, respectively. Note that the original DM is unaffected by curvature (its curvature is fixed at 0).
4.7 Discussion
The original CIFAR-10 data and the distilled synthetic sets with HDD were both projected onto the Poincaré ball for visualization; their centroids almost perfectly align. The essence of HDD lies in replacing the densely tree- structured distribution of the original dataset
Table 5: Accuracy of DM with HDD at different curvature values.
INC
Mathed
IRI
DM
2542003
with sparse tree-structured representation. As shown in Figs. (a) and (c), although the num- ber of samples in the synthetic dataset is signif- icantly smaller, it still approximately captures
tmo
270102
204300
267.02
376112
256113
TM
48.5006
30 DM witwhh HUD
40 ait0)
503:03
50.12011
50 DM AMWwi RDD
63 D503
632200
631502
02 F20
the distributional trajectory of the original dataset. The synthetic dataset tends to be denser in regions where the original data is dense and sparser in regions where the original data is sparse. However, we also observe a tendency of the synthetic samples to concentrate closer to the root node (i.e., toward the center), as illustrated in Fig. (b). Despite the presence of pronounced edge accumulation the original dataset (i.e., a large number of samples located near the boundary), the synthetic samples are noticeably "attracted" toward the direction of the root node, As shown in Fig. (d), although the synthetic dataset contains fewer samples overall, it exhibits higher concentration of points near the root node compared to the original dataset.
DM with HDD
IDM with HDD
IPC10
(c)
IPCSS
400
Sample points orighna
Sample po nts the synthtic contr wwt.ooigglllooooo catase!
the synthtlic daraset
Figure 4: After distillation with DM with HDD and IDM with HDD, the distributions of the original and synthetic datasets in the Poincaré hyperbolic space are visualized.
5 Conclusion and Future Works
In this study, we introduce hyperbolic space into dataset distillation for the first time and propose novel hyperbolic dataset distillation method, termed HDD, Leveraging the negative curvature of hyperbolic geometry, HDD effectively captures the hierarchical structure inherent in real-world datasets. By aligning the centroids of the original and synthetic datasets in hyperbolic space, ensure that the synthetic data preserves the underlying geometric properties of the original data. Crucially, due the varying influence of samples from different hierarchical levels the centroid, the loss function naturally emphasizes contributions from lower-level (prototype) samples. This inductive bias enhances the preservation of class prototype distributions, thereby improving the quality of distillation. Currently, distribution metrics from information theory (e.g.. KL divergence) and optimal transport theory (e.g., Wasserstein distance) have been extensively utilized in dataset distillation to enhance model performance. However, the application of these methods in hyperbolic dataset distillation remains unexplored, which presents a promising direction for future research to extend these methodologies into Non-Euclidean-Be dataset distillation.
Acknowledgments
This research was supported part by JSPS KAKENHI Grant Numbers JP23K11211, JP23K21676. JP24K02942, JP24K23849 and JP25K21218.
10

---

References
[1] Mina Ghadimi Atigh, Julian Schoep. Erman Acar, Nanne Van Noord, and Pascal Mettes. Hyperbolic image segmentation. In /EEE/CVE Conference on Computer Vision and Pattern Recognition, pages 4453-4462, 2022.
[2] Gregor Bachmann, Gary Bécigneul, and Octavian Ganea. Constant curvature graph convolu- tional networks. In International Conference on Machine Learning, pages 486-496, 2020.
[3] Ahmad Bdeir, Kristian Schwethelm, and Niels Landwehr. Fully hyperbolic convolutional neural networks for computer vision. arXiv preprint arXiv:2303.15911 2023.
[4] George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei Efros, and Jun-Yan Zhu. Dataset distillation by matching training trajectories. In /EEE/CVF Conference Computer Vision and Pattern Recognition, pages 4750 4759, 2022.
[5] Ines Chami, Zhitao Ying, Christopher Ré, and Jure Leskovec. Hyperbolic graph convolutional neural networks. Advances in Neural Information Processing Systems, 32, 2019.
[6] Dingfan Chen, Raouf Kerkouche, and Mario Fritz. Private set generation with discriminative information. Advances in Neural Information Processing Systems, 35:1 15:1446784469, 2022.
[7] Yutian Chen, Max Welling, and Alex Smola. Super-samples from kernel herding, arXiv preprint arXiv: 2012.
[8] Seunghyuk Cho, Juyong Lee, and Dongwoo Kim. Hyperbolic vae via latent gaussian distribu- tions. Advances in Neural Information Processing Systems, 36:569-588, 2023.
[9] Justin Cui, Ruochen Wang, Si Si, and Cho-Jui Hsieh. Scaling up dataset distillation to imagenet- Ik with constant memory. In International Conference Machine Learning, pages 2023.
[10] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255, 2009.
[11] Wenxiao Deng, Wenbin Li, Tianyu Ding, Lei Wang, Hongguang Zhang, Kuihua Huang, Jing Huo. and Yang Gao. Exploiting inter-sample and inter-feature relations in dataset distillation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17057-17066, 2024.
[12] Karan Desai, Maximilian Nickel, Tanmay Rajpurohit, Justin Johnson, and Shanmukha Ra- makrishna Vedantam. Hyperbolic image-text representations. International Conference on Machine Learning, pages 7694-7731, 2023.
[13] Tian Dong, Bo Zhao, and Lingjuan Lyu. Privacy for free: How does dataset condensation help privacy? In International Conference on Machine Learning, pages 5378-5396, 2022.
[14] Jiawei Du, Yidi Jiang, Vincent YF Tan, Joey Tianyi Zhou, and Haizhou Li. Minimizing the accumulated trajectory enror to improve dataset distillation. In JEEE/CVF Conference Computer Vision and Pattern Recognition, pages 3749-3758, 2023.
[15] songwei Ge, Shlok Mishra, Simon Korbblith, Chun-Liang Li, and David Jacobs. Hyperbolic contrastive learning for visual representations beyond objects. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6840-6849, 2023.
[16] Mina Ghadimi Atigh, Martin Reslel, and Pascal Mettes. Hyperbolic busemann learning with ideal prototypes. Advances Neural Information Processing Systems, 34:103-115 2021.
[17] Albert Gu, Frederic Sala, Beliz Gunel, and Christopher Ré. Learning mixed-curvature rep- resentations product spaces. In International Conference on Learning Representations, 2018.

---

[18] Jianyang Gu, Saeed Vahidian, Vyacheslav Kungurtsev, Haonan Wang, Wei Jiang, Yang You, and Yiran Chen. Efficient dataset distillation via minimax diffusion. In JEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15793-15803, 2024.
[19] Jianyang Gu, Kai Wang, Wei Jiang, and Yang You. Summarizing stream data for memory- constrained online continual learning. In AAAI Conference on Artificial Intelligence, pages 12217-12225, 2024.
[20] Yunhui Guo, Youren Zhang, Yubei Chen, and Stella X Yu. Unsupervised feature learning with emergent data-driven prototypicality. In IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 23199-23208, 2024.
[21] Ziyao Guo, Kai Wang. George Cazenavette, Hui Li, Kaipeng Zhang, and Yang You. To- wards lossless dataset distillation via difficult--alinned trajectory matching. In International Conference on Learning Representations, 2024.
[22] Valentin Khrulkov, Leyla Mirvakhabova, Evgeniya Ustinova, Ivan Oseledets, and Victor Lem- pitsky. Hyperbolic image embeddings. In IEEE/CVF Conference Computer Vision and Puttern Recognition, pages 6418-6428, 2020.
[23] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[24] Hyeongjun Kwon, Jinhyun Jang, Jin Kim, Kwonyoung Kim, and Kwanghoon Sohn. Improving visual recognition with hyperbolical visual hicrarchy mapping, In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17364-17374, 2024.
[25] Mare Law, Renjie Liao, Jake Snell, and Richard Zemel. Lorentzian distance learning for hyperbolic representations. In International Conference on Machine Learning, pages 3672- 3681, 2019.
[26] Yann Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7,7)33, 2015.
[27] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. IEEE, 86(1 11:2278-222224 1998.
[28] Saehyung Lee, Sanghyuk Chun, Sangwon Jung, Sangdoo Yun, and Sungroh Yoon. Dataset condensation with contrastive signals. In International Conference on Machine Learning, pages 12352-12364 2022.
[29] Shiye Lei and Dacheng Tao. comprehensive survey to dataset distillation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(1):17-32, 2023.
[30] Guang Li, Ren Togo, Takahiro Ogawa, and Miki Haseyama. Soft-label anonymous gastric x-ray image distillation. In IEEE International Conference on Image Processing, pages 305-309, 2020.
[31] Guang Li, Ren Togo, Takahiro Ogawa, and Miki Haseyama. Compressed gastric image generation based on soft-label dataset distillation for medical data sharing. Computer Methods and Programs in Biomedicine, 227:107189, 2022.
[32] Guang Li, Ren Togo, Takahiro Ogawa, and Miki Haseyama. Dataset distillation using parameter pruning. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, 2023.
[33] Guang Li, Ren Togo, Takahiro Ogawa, and Miki Haseyama. Importance-aware adaptive dataset distillation. Neural Networks, 2024.
[34] Guang Li. Bo Zhao, and Tongzhou Wang. Awesome dataset distillation. httpss//giithub Com//Gung000 Awesome Baliaagoo/Aeeeceeetttttee------s
2022.
[35] Hongcheng Li, Yucan Zhou, Xiaoyan Gu, Bo Li, and Weiping Wang. Diversified semantic distribution matching for dataset distillation. ACM International Conference on Multimedia, pages 7542-7550, 2024.
12

---

[36] Longzhen Li, Guang Li, Ren Togo, Keisuke Maeda, Takahiro Ogawa, and Miki Haseyama. Generative dataset distillation: Balancing global structure and local details. In JEEE/CVF Conference Computer Vision and Pattern Recognition Workshops, pages 7664-7671, 2024.
[37] Longzhen Li, Guang Li, Ren Togo, Keisuke Maeda, Takahiro Ogawa, and Miki Haseyama. Generative dataset distillation based on self-knowledge distillation. In IEEE International Conference Acoustics, Speech, and Signal Processing, pages 1-5, 2024.
[38] Mingzhuo Li, Guang Li, Jiafeng Mao, Takahiro Ogawa, and Miki Haseyama. Diversity generative dataset distillation based on diffusion model with self-adaptive memory. In IEEE International Conference Image Processing. 2024.
[39] Mingzhuo Li, Guang Li, Jiafeng Mao, Linfeng Ye, Takahiro Ogawa, and Miki Haseyama. Task-specific generative dataset distillation with sefficullt-guidedd sampling. In JEEE/CVF International Conference on Computer Vision Workshops, 2025.
[40] Fangfei Lin, Bing Bai, Kun Bai, Yazhou Ren, Peng Zhao, and Zenglin Xu. Contrastive multi-view hyperbolic hierarchical clustering. arXiv preprint PrXiv:2205.026112 2022.
[41] Ping Liu and Jiawei Du. The evolution of dataset distillation: Toward scalable and generalizable solutions. arXiv preprint arXiv:2502.05673, 2025.
[42] Dmitry Medvedev and Alexander D D'yakonov. Learning to generate synthetic training data using gradient matching and implicit differentiation. In International Conference on Analysis of Images, Social Networks and Texts, pages 138-150, 2021.
[43] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. In Neural Information Processing Systems Workshops, 2011.
[44] Timothy Nguyen, Roman Novak, Lechao Xiao, and Jaehoon Lee. Dataset distillation with infinitely wide convolutional networks. Advances in Neural Information Processing Systems, 14:51865519,, 2021.
[45] Avik Pal, Max van Spengler, Guido Maria D' Amely di Melendugno, Alessandro Flaborea, Fabio Galasso, and Pascal Mettes. Compositional entailment learning for hyperbolic vision-language models, arXiv preprint arXiv:2410.06912. 2024.
[46] Sameera Ramasinghe, Violetta Shevchenko, Gil Avraham, and Ajanthan Thalaiyasingam. Accept the modality gap: An exploration the hyperbolic space. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27263-27272 2024.
[47] Ahmad Sajedi, Samir Khaki, Ehsan Amjadian, Lucy Z Liu, Yuri A Lawryshyn, and Konstanti- nos N Plataniotis. Datadam: Efficient dataset distillation with attention matching. IEEF/CVF International Conference on Computer Vision, pages 17097-17107, 2023.
[48] Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. arXiv preprint
00480, 2017.
[49] Shitong Shao, Zeyuan Yin, Muxin Zhou, Xindong Zhang, and Zhiqiang Shen. Generalized large-scale data condensation via various backbone and statistical matching. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16709-16718, 2024.
[50] Duo Su, Junjie Hou, Weizhi Gao, Yingjie Tian, and Bowen Tang. D 4: Dataset distillation via disentangled diffusion model. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5809-5818, 88, 2024.
[51] Duo Su, Junjie Hou, Guang Li, Ren Togo, Rui Song, Takahiro Ogawa, and Miki Haseyama. Generative dataset distillation based on diffusion model. In European Conference on Computer Vision Workshops, 2024.
[52] Felipe Petroski Such, Aditya Rawal, Joel Lehman, Kenneth Stanley, and Jeffrey Clune. Genera- tive teaching networks: Accelerating neural architecture search by learning to generate synthetic training data. In International Conference on Machine Learning, pages 9206-9216, 2020.
13

---

[53] Peng Sun, Bei Shi, Daiwei Yu, and Tao Lin. On the diversity and realism of distilled dataset: An efficient dataset distillation paradigm. In IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 9390-9399, 2024.
[54] Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey Gordon. An empirical study of example forgetting during deep neural network learning. arXiv preprint wrw.vv:812..00333
2018.
[55] Kai Wang, Bo Zhao, Xiangyu Peng, Zheng Zhu, Shuo Yang. Shuo Wang, Guan Huang, Hakan Bilen, Xinchao Wang, and Yang You. Cafe: Learning to condense dataset by aligning features. In IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 2022.
[56] Tongzhou Wang. Jun- Yan Zhu, Antonio Torralba, and Alexei A. Efros. Dataset distillation. arXiv preprint arXiv: 1811.10959 2018.
[57] Max Welling. Herding dynamical weights to learn. In International Conference on Machine Learning, pages 1121-1128, 2009.
(58) Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for henchmarking machine learning algorithms. arXiv preprint arXiv: (tci::/70..000 078.07777, 2017.
[59] Jiexi Yan, Lei Luo, Cheng Deng, and Heng Huang. Unsupervised hyperbolic metric learning. In IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 12465-12474 2021.
[60] Enneng Yang, Li Shen, Zhenyi Wang. Tongliang Liu, and Guibing Guo. An efficient dataset condensation plugin and its application continual learning. Advances in Neural Information Processing Systems, 36, 2023.
[61] Linfeng Ye, Shayan Mohajer Hamidi, Guang Li, Takahiro Ogawa, Miki Haseyama, and Kon- stantinos N. Plataniotis. Information-guided diffusion sampling for dataset distillation. In Advances in Neural Information Processing Systems Workshops, 2025.
[62] Zeyuan Yin, Eric Xing, and Zhiqiang Shen. Squeeze, recover and relabel: Dataset condensation imagenet scale from a new perspective. Advances in Neural Information Processing Systems, 2023.
[63] Ruonan Yu, Songhua Liu, and Xinchao Wang. A comprehensive survey dataset distillation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(1):150-170, 2023.
[64] Hansong Zhang, Shikun Li, Fanzhao Lin, Weiping Wang. Zhenxing Qian, and Shiming Ge. Dance: Dual-view distribution alignment for dataset condensation. arXiv preprint arXiv:2406.01063 2024.
[65] Hansong Zhang, Shikun Li, Pengju Wang, Dan Zeng, and Shiming Ge. M3d: Dataset condensa- tion by minimizing maximum mean discrepancy. In AAAI Conference on Artificial Intelligence, pages 9314-9322 2024.
[66] Yiding Zhang, Xiao Wang, Chuan Shi, Xunqiang Jiang, and Yanfang Ye. Hyperbolic graph attention network. IEEE Transactions on Big Data, 8(6):1990-170: 2021.
[67] Bo Zhao and Hakan Bilen. Dataset condensation with differentiable siamese augmentation. In International Conference on Machine Learning, pages 12674-12685, 2021.
[68] Bo Zhao and Hakan Bilen. Dataset condensation with distribution matching. In JEEE/CVF Winter Conference on Applications of Computer Vision, pages 6514-6523, 2023.
[69] Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. Dataset condensation with gradient matching. arXiv preprint arXiv:2006.0m 05529, 2020.
[70] Ganlong Zhao, Guanbin Li, Yipeng Qin, and Yizhou Yu. Improved distribution matching for dataset condensation. In JEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1856-7865, 2023.
14

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA]-
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1 (1-2 sentence) justificatin right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked While "[Yes] " is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providded a proper justification is given (e.g., "error bars expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
the checklist one of the factors in their evaluation.
reported because it would be too computationally
"[No] " or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer often more nuanced, so please just use your best, judgment and write justification elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly articulate the research claims. The theoretical part is supported in Section 3 of the main text and Appendices. A, B, C, and D. The validation of performance improvements is substantiated by the experimental data in Section 4 of the main text.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
15

---

Answer: [Yes]
Justification: We have outlined the limitations of the framework in the conclusion section and discussed potential directions for future work.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: We provide complete explanations and proofs in Section 3 of the main text and Appendices B, C, and D.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
16

---

The proposed architecture is fully elaborated in Section 3 of the manuscript, while Section 4 presents the detailed experimental configurations. The code is available at https: https:///gtthuee com Saamm//aang000/F10
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (c.g.. with an open-soure dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The code is available atlttpe://giiuubb.cm///aagg///
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
mublic/guides
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https httpl//i//guidde//deeeububbbeeee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
17

---

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes)
Justification: The detailed experimental setup, including hyperparameters, fully described in Section 4 of the main text and Appendix G.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: mean.
All numerical values the paper are provided with the standard error of the
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative eтror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
18

---

Justification:
In Section 4, we specify the models of the computing resources we utilized.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
The authors of this study have thoroughly reviewed the NeurIPS Code of Ethics and have made every effort maintain and preserve anonymity throughout this research.
Justification:
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: In this study, we propose novel research perspective for dataset distillation, which may yield positive impacts including:
Reducing computational resource requirements for deep learning.
Decreasing energy consumption associated with large-scale training.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
19

---

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks. mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
This study proposes a novel dataset distillation model, whose risk is not significantly higher compared to previous dataset distillation models. Additionally, the model does not include a generative component, and there is minimal risk of the research results being misused. Therefore, consider this item not applicable
Justification:
study.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The datasets, models, etc. involved the paper have been properly cited.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperssiippswiiiieea. thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new datasets, and the source code will be made publicly available upon acceptance of the manuscript.
20

---

Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This study does not involve crowdsourcing experiments or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This study does not involve human subject research or crowdsourcing and therefore does not require IRB approval.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
21

---

Justification: This study does not involve large language models (LLMs) in its core method- ology, data processing, or experimental design. The research is based on dataset distillation techniques without any L.LM-related components. Therefore, no declaration of L.M usage is required for this submission.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https://neurip) /0tp:////////eeeeeeee//////////0 for what should or should be described.
22

---

Appendix
A Complementary Details of the Lorentz Hyperbolic Space
A.1 Tangent Space T&C
In the Lorentz model, hyperbolic space C is realized as a sheet of the two-sheeted hyperboloid in g=+1 with Minkowski metric. For any point x [ E L, the tangent space captures all possible instantaneous directions x. It is defined by
TxC= {v 1 (x, v)c = 03.
(21)
This tangent space inherits the Lorentzian metric, and any tangent vector v has norm
| v (v, v)
(22)
which is strictly positive, ensuring that tangent vectors are purely spatial and providing the metric foundation for the exponential map.
A.2 Exponential and Logarithm Maps
The exponential map pushes vectors in the tangent space onto the manifold, yielding local Euclidean- like parametrization. Let K V-K. For v TxC. define
sinh |vlx)
expx (v) cosh (s<|\vllx) x
(23)
K <||v|x
This formula satisfies expx (0) = x and ensures that the interpolation curve is a geodesic of constant curvature. The inverse (logarithm map) brings point y back to the tangent space:
Logx(y)= arccosh K () - (xx,))xxx)
(24)
V - ((x,))))
A.3 Bijection between the Lorentz and Poincaré Ball Models
For many applications (especially visualization), it is convenient to switch to the Poincaré ball. Given a Lorentz point x [r), we map it to the unit ball ||p|| via
p
(25)
+ K 20g
Conversely, for any p e IR" with ||||| < 1, set a =
|||| and recover
1 + |p|2
2 p
(26)
One verifies that the reconstructed x satisfies =2? + ||r = 1/ and de 0.
B Centroid Convergence Toward the Origin
Given finite sample [ Lg define the Fréchet functional
F(P) = Σ de (p.P... 1=1
(27)
Using the Riemannian logarithm Logp: L. Tp one obtains
N
FF()) = - Σ
Logp(p:),
Logp(pi) E.LL
(28)
i=1
23

---

so that the unique Fréchet mean p satisfies
Logp- (p:) = 0.
(29)
Since L. has constant curvature K < O, each map + || (p:)(P:)l is strictly convex along geodesics, ensuring single global minimizer. We choose the origin Po to be the unique fixed point maximal compact subgroup of Tsom(T_k)) ), whose stabilizer is isomorphic to ( A
comparison-theoren theorre argument then shows
Logpo (p:) = (po, Pi) 2 Logp whenever (po, P.) 2 (p) (30)
forcing the solution of Σ; (P.) = 0 to lie radially closer Pu than the Euclidean centroid. Moreover, as | increases, the lower bound on the second-derivative of 1 - || Log,, (p:) along any geodesic 7 grows, making this radial bias toward Po even more pronounced. This results in the centroids of both the original dataset and the synthetic dataset being biased towards po. while the distance between them relatively small.
C Hierarchical Weight
In the hyperholoid model of constant sectional curvature K < 0, one introduces the scale parameter [K] and radius R 1/K, that the ambient space is
; {x E R=+1 (z,x)ı - -R² zo > 0},
(31)
where L denotes the Minkowski inner product of signature (-+...+)) The geodesic distance between two points q E HH% is given by
dx (p. 9) = R arccosh - the 1))
1
(32)
arccosh K (p, r.).
k
In particular, choosing the basepoint o = (R, o........ 0) and writing ra = dx xo, one has
1
arccosh - K (o, (0)
(33)
cosh (x):)));;))))))) = - (0, x:)
(34)
122
The logarithmic map at o takes the form
Logo(x;)
K sinh(krs)
(x: cosh((rr)))) o).
(35)
Defining
w(r:) = sinh(kr;))
W
(36)
u x; cosh((rr:) Logo(x;) w(rx) uq.
(37)
one obtains
(38)
D Gradient Contributions in the Lorentz Model of Hyperbolic Space
Given N sample points [ C L. their Fréchet mean (centroid) ş is defined by
arg min
Σ
d(x,
(39)
FEER
so that the objective (loss) is
L(x) = Σ
(rrosh(((xx,p))
(40)
24

---

To study how a single point "pulls" on set
I - p).
(41)
cosh(d(<x.p) 2 1.
A standard derivation shows
Vx d(,, =-2 arcosh(()
(p ())),
(42)
v1=1
and hence the magnitude of this pull is proportional to arcosh(t)
f(t) V
(43)
Asymptotic Behavior.
Near the "origin" (t (1-+11 1+). Since arcosh(t) (2(t-1) and VP-1 2 1), we have
arcosh(t) 1.
(())
(44)
vp
Thus, points very close to x exert almost the maximal pull of magnitude 1. Near the boundary + oo). Using (r) ln(2() and vi2 1 gives
f(t) In(21) 0,
(45)
so points very far from contribute almost no pull. Monotonicity. Differentiating
('(1) V - arcosh(()
(46)
(t2 3/32
we note that for all <>> 1,
Vt2 <t t and arcosh(t) > 1
arcosh(t) > V 1
(47)
so the numerator is negative while the denominator is positive. Hence
f'( <0 Vt>1,
(48)
ie.. f() is strictly decreasing oo).
Since f(t) decreases from to 0 as runs from 1 + to co, points closest the current centroid exert the largest gradient pull, whereas points near the hyperbolic boundary (very far away) exert the smallest pull.
E
Benchmark Datasets
We validate our hyperbolic dataset distillation method using six benchmark datasets: Fashion-
MNIST 588, SVHN 43), CIFAR-10 (3), CIFAR-100 23 Tiny ImageNet (66,, and Image-
Woof 10
FaShionMNIST is drop-in replacement for the classic MNIST dataset, comprising 70,000 grayscale images of size 28 x 28 pixels across 10 apparel categories (e.g., T-shirt/top, sneaker) with 60,000/1,000 train/test split.
SVHN contains approximately 600,000 real-world 32 x 32 RGB digit crops (0-9) collected from Google Street View images. is partitioned into training (73,257), testing (26,032), and extra set of 531 111 samples for data augmentation.
CIFAR- consists of 60,000 32 x 32 color images evenly distributed over 10 object classes (airplane, car, bird, cat, deer, dog, frog, horse, ship, truck). There are five training batches of 10000 images each and one test batch, with exactly 1000 images per class.
25

---

CIFAR-100 (building on CIFAR-10) contains 60,000 32 x 32 color images in 100 fine classes (600 images each) grouped into 20 coarse superclasses. Each fine class has 500/100 train/test split, enabling hierarchical and fine-grained classification studies.
Tiny ImageNet is subset of the ILSVRC-2012 challenge, selecting 200 classes and resizing all images to 64 64 pixels. It provides 100,000 images (500 train, 50 val, 50 test per class), offering a mid-scale benchmark between CIFAR and full ImageNet.
visually similar dog breeds drawn from ImageNet (e.g., Beagle, Samoyed, Golden Retriever). It contains 9,025 training and 3,929 validation images, with optional noisy-label variants, and is commonly used to benchmark tin-grained recognition models.
Image Woof omaf a challenging subset
F
Cross-architecture Generalization
cross-architecture generalization capability serves as a critical metric for evaluating the effectiveness of dataset distillation, where significant performance degradation across different architectures is deemed unacceptable. To assess this capability, we evaluated our method by testing its performance on ConvNet, AlexNet, VGGI1, and ResNet18. As demonstrated in Table both DM with HDD and IDM with HDD exhibit robust adaptability across diverse architectures. Compared with baseline DM and IDM methods, the HDD -enhnnend approach demonstrates superior generalization strength and more stable performance while maintaining architectural compatibility.
Table 6: The distillation accuracy of CIFAR-10 (IPC = 10) for cross-architecture generalization.
Model
ConvNet
AlexNet
VGG11
ResNet18
DSA
67
52.1
35,9
43.240.5
35.9
KIP
44
47.6+0.9
24.443.9
42.1+0.4
36.8+1.00
DM
68
48.9
38.8+0.5
42.1+0.4
41.2+1111
IDM
70
53.0+0.3
44.6+0.8
47.
44.6
DM with HDD
50.3+0.3
46.310.4
45.7+0.3
40.2+0.4
IDM with HDD
61.3:00.1
57.2+:..3
58.610.4
56.8+10.3
G Hyperparameter Details
For different experiments, use distinct hyperbolic curvature K, gradient scaling factor X, and synthetic image learning rate r, as shown Table Table For the hyperbolic curvature K, we set it between 0.2 and 3. For the gradient scaling factor X, refer the loss in Hilbert space and ensure that the hyperbolic distance loss maintains the same order magnitude as the Hilbert space loss through A. We make minor adjustments to the synthetic image learning rate while respecting the original method.
H Details of Baseline Methods
Dataset Condensation (DC) 169 achieves this objective by learning synthetic dataset that, when used alongside the large dataset to train a deep network, results comparable weight gradients.
Differentiable Siamese Augmentation (DSA) 67 enables learning synthetic training sets by applying identical random transformations both real and synthetic data during training while supporting gradient backpropagation through differentiable augmentations.
Dataset Condensation with Contrastive signals (DCC) (28 enhances dataset condensation by matching summed gradients across all classes (unlike glass-wise matching in DC) and optimizing synthetic data with contrastive signals. It stabilizes training via kernel velocity tracking and bi-level warm-up, improving fine-gri mme-grained classification.
Condense dataset by Aligning FFatures (CAFE) 55
condenses data by aligning layer-wise features between real and synthetic data, explicitly encoding discriminative power into synthetic clusters, and adaptively adjusting SGD steps via a bi-level optimization scheme.
26

---

Table 7: Hyperparameter details of DM with HDD and IDM with HDD.
DM with HDD
IDM with HDD
Dataset
IPC
1/K
1/KK
À
40
0.5
FNSST
10
1
40
60
50
80
0.2
1
10
120
0.5
SVHN
10
50
120
50
1
100
120
0.2
1
80
0.5
CIFAR 10
10
100
50
80
120
0.2
1
10
60
0.5
CIFAR 100
10
2
100
80
0.2
50
2
120
100
0.6
80
0.5
TinyImageNet
10
100
0.5
50
120
0.6
Table 8: Hyperparameter details Dance with HDD.
DM with HDD
Dataset
IPC
1/K
1
1.8
20
0.02
CIFAR-10
10
0.2
40
0.2
50
2
60
0.5
1
2
40
0.02
CIFAR-100
10
1.5
0.1
50
2
120
0.5
1
0.6
100
0.1
ImageWoof
10
0.5
120
0.1
Dataset Distillation with Attention Matching (DataDAM) generates synthetic images by aligning the spatial attention maps real and synthetic data, produced across various layers set of randomly initialized neural networks.
Distribution Matching (DM) 68 is the first use maximum mean discrepancy to optimize synthetic data to match the distribution of the original data.
Improved Distribution Matching (IDM) 70 enhances DM by addressing feature imbalance through Partitioning and Expansion augmentation, and correcting invalid MMD estimation using enriched remi-trained model embeddings and class-aware distribution regularization, resulting in more accurate feature alignment and improved performance.
Generalized Various Backbone and Statistical Matching (G-VBSM) 49 is novel framework for generalized dataset condensation, comprising three key components: data densification enhances intra- class diversity by ensuring linear independence within each class; generalized statistical matching captures patch- and channel-level convolutional statistics without gradient updates for effective syn- thesis; and generalized backbone matching enforces consistency across diverse backbones, boosting generalization. Together, they enable efficient and robust generalized matching.
Difficult-AAlgneed Trajectory Matching (DATM) 21 dynamically adjusts the difficulty of synthetic data (matching the early or late training trajectories the teacher network) to adapt the scale of the synthetic datasct- small datasets correspond simple modes (carly trajectories), while large
27

---

datasets correspond to complex modes (late trajectories). This approach achieves lossless dataset distillation for the first time.
Realistic, Diverse, and Efficient Dataset Distillation (RDED) 5 is a non-optimization-hann dataset distillation method that enhances realism by cropping realistic patches from original images and improves diversity by stitching these patches into new synthetic images, achieving high efficiency and superior performance on large-scale, high-resolution datasets.
Dataset Distillation via Disentangled Diffusion Model (D*M) [50] leverages a disentangled diffusion model with novel training-timm matching strategy to efficiently distill high-resolution, realistic datasets while improving crossarcchtecttree generalization and reducing computational costs.
Inter-sample and Inter-feature Relations in Dataset Distillation (IID) [1] introduces two key constraints improve distribution matching: a class centralization constraint to enhance intra-class feature clustering, and a covariance matching constraint to accurately align feature distributions by considering both mean and covariance, even with limited synthetie samples.
Diversified Semantic Distribution Matching (DSDM) [35] distills datasets by aligning the semantic cearbbutioons--ppeeeeeteteee as Gaussian prototypes and covariance matrices-of distilled data with those of original data.
Minimizing the Maximum Mean Discrepancy (M3D) 65 enhances DM-based dataset conden- sation by aligning not only the first but also higher-ordet moments of feature distributions through kernel-based Maximum Mean Discrepancy, enabling accurate distribution matching with theoretical guarantees and strong performance across diverse datasets.
Dual-view distribution AligNment for dataset CondEnsation (DANCE) 66 introduces dual- view approach to dataset condensation by leveraging expert models: it performs pseudo long-term distribution alignment via convex combination of initialized and trained models align inner-class distributions without persistent training, and applies distribution calibration using expert models to mitigate inter-class distribution shift and preserve class boundaries.
I Results on TinyImageNet
Table 9: Comparison TinylmageNet with different IPCs.
Method
IPC = (0.2%)
IPC 10 (2%)
IPC = 50 (10%)
Random
1,4+00.1
5.0+10.2
15.0+10.4
Herding 57
2.8++0.2
6.3++0.2
16.7+-0.3
K-Center
48
1.6++0.2
5.1+0.11
15.0++0.3
Forgetting
[54
1.6:0.22
5.1
15.0+:0.1
DC [69]
5.340.1
12.940.1
12.740.4
DSA 667
5,710.1
16.3+0.2
5.110.2
DataDAM 47
8.3
18.7#0.3
28.7-10.3
MTT
6.2400.4
17.3+0.2
26.5+0033
IDM 70
10.1
21.91006
26,9+0.2
IDM with HDD
11.910..2
22.4+10.3
27.8+0..3
Whole Dataset
37.61.0.6
We compare IDM with HDD against DC DSA 67 DataDAM 47.. MTT
and IDM [70]
on TinylmageNet, as shown in Table Our method achieves superior performance at both IPC and IPC 10. Furthermore, compared to IDM, IDM with HDD demonstrates improvements 1.8%, 0.5%, and 0.9% at IPC = IPC = 10, and IPC = 50, respectively.
Experiments on an Alternative Hybrid Architecture (DSDM)
evaluated the performance of HDD on the hybrid architecture DSDM 35 using the CIFAR-10
dataset. As shown in Table 10 when IPC = 1, DSDM with HDD achieved a 2.6% performance gain
28

---

compared to the original DSDM; when IPC = 10, DSDM with HDD showed an improvement of 0.8%.
Table 10; Accuracy comparison of DSDM with/without HDD.
Method
IPC
Accuracy (%)
DSDM
43.8 + 0.2
DSDM with HDD
46.4 0.3
DSDM
10
65.8 + 0.3
DSDM with HDD
10
66.6 + 0.4
DSDM
50
75.8 + 0.2
DSDM with HDD
50
76.0 + 0.2
K Visualization of Distilled Images
We showcased a portion of the synthetic dataset distilled through HDD. Figure displays the FashionMNIST samples synthesized using the DM with HDD at IPC = 50, while Figure shows the analogous SVHN outputs under identical conditions. Figures 3 and 4 correspond to CIFAR-10: Figure 7 (a) and (b) depict the IDM with HDD results at IPC and IPC 10, respectively, and
Figure demonstrates the IPC = 50 case. Figure 9 extends this analysis to CIFAR- presenting
IDM with HDD distillations at IPC = (a). IPC = 10 (b), and IPC = 50 (c). Finally, Figure TO
illustrates the Image ImageWoof distilled samples obtained via the Dance with HDD at IPC (a) and IPC
10 (b).
ftt
Figure 5: The distilled images of MNSST with IPC 50 using DM with HDD.
Figure 6: The distilled images of SVHN with IPC 50 using DM with HDD.
29

---

(a)
(b)
Figure 7: The distilled images of CIFAR-10 with IPC 1 (a) and IPC 10 (b) using IDM with HDD.
Figure 8: The distilled images of CIFAR-10 with IPC 50 using IDM with HDD.
30

---

(
(b)
(c)
Figure 9: The distilled images of CIFAR-100 with IPC (a), IPC 10 (b), and IPC 50 (c) using IDM with HDD.
31

---

(a)
(b)
Figure 10: The distilled images of Woof with IPC (a) and IPC 10 (b) using Dance with HDD.
32


Reviews:
[{"review_id": "KH2xnrp4Mb", "reviewer": "Reviewer_9awn", "summary": "Dataset distillation compresses a large dataset into a small set of synthetic examples that achieve performance comparable to training on the full original data. In dataset distillation methods, distribution matching algorithms align feature distributions between real and synthetic dataset. This paper aims to leverage the hierarchical structure inherent in dataset samples, and introduce a novel hyperbolic data distillation (HDD) method. The objective in distribution matching is reformulated as minimizing the Lorentzian hyperbolic distance between the Riemannian means of the original and synthetic datasets. Experiments show that the proposed HDD method is easy to add on top of two existing method across 5 different datasets leading to better performance improvements.", "questions": "Main questions are listed in weaknesses. To summarize, I suggest the authors to clarify how the cross entropy loss of IDM and DANCE is added to HDD, whether as hyperbolic or euclidean? I also suggest the authors to visualize the samples of synthetic and real datasets that are closer to the origin and therefore are more _important_ for distribution matching. Finally, I want to know the authors' perspective on how a class-wise constraint can be added to HDD? \n\n**Minor**:\n- What is IPC? IPC of synthetic dataset is referred to multiple times in the paper but is not explained.\n- duplicate citations ([23],[24])", "limitations": "The authors discuss limitations adequately.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "## Strengths\n\n**Novelty:** To my knowledge, this is the first work to explore dataset distillation through the lens of hyperbolic geometry. The formulation of distribution matching loss in hyperbolic space is simple but sound. \n\n**Interesting observations:** The observation that samples closer to origin contribute more strongly to the Fréchet mean of the dataset is interesting. The authors equate this to reducing outlier contribution (line 213-214). This is also validated in Table 2 where removing 80% samples exhibiting highest curvature does not impact the performance on CIFAR-10 dataset. \n\n**Empirical Effectiveness:** The authors mention that HDD can be *plugged* into existing distribution matching methods and achieves meaningful gains over over using the methods alone. The experiments show results across distribution matching methods like DM[62], IDM[64] and hybrid methods like DANCE[58] (alternates between cross entropy and distribution matching).\n\n## Weaknesses\n\n**How can HDD augment existing distribution matching methods?** \n\n- Firstly, It is straightforward to understand how HDD loss can replace maximum mean discrepancy loss (MMD) the DM[62] paper. However, there are some missing implementation details with IDM[64] and DANCE[58]. Since both methods have an additional cross entropy loss, is the cross entropy loss still in euclidean space or in hyperbolic space? What is reason for choosing one or the other?\n- The paper only shows two distribution matching methods, the original DM[62] and IDM[64]. Is this generalizable to other distribution matching methods like CAFE [50], M3D[59], IID[12]? Is it as simple as replacing the MMD loss with HDD loss or does it require additional tuning?\n\n**Visualizations**: Which samples contribute the most/least in synthetic dataset? I suggest the authors to also visualize such samples. Do samples closer to origin look more prototypical than samples farther away suggested in Figure 1.\n\n**Class wise contributions:** Figure 4 shows that samples clustering is quite disperse. This will affect class-wise separability. How does class-aware regularization (similar to IDM or IID) look like in hyperbolic space? While this doesn't affect the novelty introduced in the paper, I'm curious how a hyperbolic equivalent of class centralization constraint (compact clustering in IID) look like and how would it impact how samples look like in the space?", "was_revised": true, "final_justification": "The authors have addressed my concerns, so I keep my original rating. I suggest the authors to add proposed changes to the final version", "rebuttal": "## Response to Reviewer 9awn\n\nWe sincerely thank you for the thoughtful review and for highlighting the novelty and empirical value of our approach. Below, we provide detailed answers to each of your questions and comments.\n\n---\n\n### **Clarification 1 – About the Cross-Entropy Term in IDM and DANCE**\n\nIn our current implementation, **the cross-entropy (CE) loss remains in Euclidean space**. The logits produced by standard convolutional networks live naturally in Euclidean tensor space, and the CE loss is well defined there. By leaving this component unchanged, **we guarantee full compatibility with existing dataset-distillation baselines** and avoid introducing additional confounding factors. Although a hyperbolic softmax layer is conceptually possible, it would require redesigning the classifier head and decision boundaries, which we consider outside the scope of this study.\n\n---\n\n### **Clarification 2 – Generalisability to Other Distribution-Matching Methods**\n\nFor methods such as CAFE that align layer-wise activations, the objective differs substantially from our global centroid-matching formulation; an effective combination would therefore demand a dedicated re-design of the loss. M3D focuses on higher-order moment matching and can, in principle, benefit from our hyperbolic formulation as well, but the resulting optimisation landscape would become considerably more complex.  \n\nTo substantiate the claim of broad applicability, **we have already integrated HDD into DSDM [1]**, which is a hybrid algorithm that alternates between CE and distribution matching. The table below summarises the results; **adding HDD yields consistent gains across all “images-per-class” (IPC) settings on CIFAR-10**.\n\n| Method            | IPC | Accuracy (%) |\n|-------------------|----:|-------------:|\n| DSDM              |   1 | 43.8 ± 0.2   |\n| **DSDM + HDD**    | **1** | **46.4 ± 0.3** |\n| DSDM              |  10 | 65.8 ± 0.3   |\n| **DSDM + HDD**    | **10** | **66.6 ± 0.4** |\n| DSDM              |  50 | 75.8 ± 0.2   |\n| **DSDM + HDD**    | **50** | **76.0 ± 0.2** |\n\n---\n\nWe also conducted the IPC-10 experiment on CIFAR-10 using IID (DM version). We only replaced the MMD component, setting curvature $K= -1$ and the scaling factor $\\lambda$ to 20. Our experimental results are as follows:\n\n---\n\n| Method        | Accuracy (%) |\n|---------------|-------------:|\n| IID (DM)      | 55.1 ± 0.1 |\n| **IID + HDD** | **55.8 ± 0.2** |\n\n---\n\nDue to time constraints, experiments that substitute the MMD term with HDD in IID for additional settings are still running, and we will report the corresponding results in the revised manuscript.\n\n---\n\n### **Clarification 3 – Prototypicality of Samples Near the Hyperbolic Origin**\n\nBecause conference policy prevents us from embedding raw CIFAR-10 images directly in the rebuttal, we instead list the identifiers of **the five samples closest to the origin** and **the five farthest** after 10,000 optimisation steps of IDM (all from the *horse* class).\n\n- **Nearest samples:** `data_batch_3_img6963`, `data_batch_4_img455`, `data_batch_4_img6399`, `data_batch_2_img4897`, `data_batch_4_img5922`  \n- **Farthest samples:** `data_batch_1_img1394`, `data_batch_5_img897`, `data_batch_1_img2701`, `data_batch_3_img3910`, `data_batch_2_img6227`\n\nWhen we visualise these images locally, **near-origin samples are centred and uncluttered**, whereas far-origin samples are partially occluded, truncated, or mixed with background objects. This qualitative difference supports our claim that hyperbolic geometry naturally emphasises prototypical examples.\n\n---\n\n### **Clarification 4 – Adding Class-Wise Regularisation in Hyperbolic Space**\n\nThe visualization results for IDM are already provided in our paper, specifically in the two panels on the right side of Figure 4. In addition, we are also visualizing IID. However, due to conference policy restrictions, we are unable to directly include the corresponding images. We will include the visualization results in the revised version.\n\nWe agree that encouraging intra-class coherence without compromising the global hierarchy is an important direction. We have revisited the IID approach: when IID introduces class‑centric constraints, directly applying it in hyperbolic space may be unsuitable, because **those constraints can compromise the dataset’s hierarchical structure**. So we propose a different angle: **the overall hyperbolic distance can be decomposed into radial and angular components**. We can strip out the radial component—since forcibly shortening it might damage the hierarchy—while retaining the angular component to impose intra‑class constraints. Next, we provide the specific definition of angular loss for two points. Let a Lorentz-model point be written as $\\(x=(x_0,\\mathbf{x})\\)$ with $\\(x_0>0\\)$ and $\\(-x_0^{2}+\\|\\mathbf{x}\\|^{2}=-1\\)$. For two points $\\(x\\)$ and $\\(y\\)$ we extract their spatial parts and normalise them,\n\n$$\n\\hat n_x = \\frac{\\mathbf{x}}{\\|\\mathbf{x}\\|}, \\quad\n\\hat n_y = \\frac{\\mathbf{y}}{\\|\\mathbf{y}\\|},\n$$\n\nthen compute the directional cosine $\\(\\cos\\theta = \\hat n_x \\cdot \\hat n_y\\)$. The **angular loss** is given by\n\n$$\nL_{\\text{angle}} = 1 - \\cos\\theta = 2 \\sin^{2}\\Bigl(\\tfrac{\\theta}{2}\\Bigr),\n$$\n\nwhich equals 0 when directions coincide and 2 when they are opposite. \n\n---\n\n### **Minor Comments**\n\n* We will explicitly define **IPC** as “**images per class**” at its first appearance.  \n* The duplicate citations [23] and [24] refer to the same source; we will merge them in the final manuscript.\n\n---\n\nWe hope that these detailed clarifications convincingly address your concerns. Thank you again for your positive assessment and constructive suggestions.\n\n---\n\n**Reference**\n\n[1] *Diversified Semantic Distribution Matching for Dataset Distillation*."}, {"review_id": "2n4Sstr9Sr", "reviewer": "Reviewer_gver", "summary": "This paper introduces Hyperbolic Dataset Distillation (HDD), a drop-in modification to existing distribution-matching methods (DM, IDM, Dance) that embeds real and synthetic features into Lorentz hyperbolic space and minimizes the geodesic distance between their centroids. The key insight is that hyperbolic geometry inherently emphasizes low-level “prototype” samples, which helps retain class semantics and hierarchical structure during distillation. The authors demonstrate that HDD consistently improves performance across five datasets (Fashion-MNIST, CIFAR-10, CIFAR-100, SVHN, Tiny-ImageNet) for a variety of IPC (images per class) settings and enables “hierarchical pruning,” where only 20% of the real data is used to distill without hurting final accuracy.", "questions": "1. How sensitive is HDD to the curvature K? Are there stability issues for very negative values?\n\n2. Have you tried using more sophisticated centroids as opposed to just tangent-space approximation of the Fréchet mean? In fact, due to some recent work on differentiating through the Fréchet mean [a], you may actually be able to perform the operation with the true mean (although this will likely be considerably slower). \n\n### References\n\n[a] Differentiating through the Fréchet Mean. https://arxiv.org/pdf/2003.00335", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "## Strengths\n\n1. The main idea of doing centroid matching in Lorentz space is simple, intuitive, and reasonably novel in the distillation setting. The use of hyperbolic geometry biases the synthetic data toward prototypical examples and aligns well with the distribution-matching objective.\n\n2. The improvements are consistent across multiple datasets and baselines. For instance, adding HDD to IDM yields a 4% absolute gain on CIFAR-10 at IPC=10 (from 57.3% to 61.3%). Similar trends hold across other datasets, including CIFAR-100 and Tiny-ImageNet. Standard deviation are given and in most cases the proposed method yields a statistically significant result.\n\n3. The hierarchical pruning results are strong. The fact that the authors can discard 80% of the real dataset and still get the same final accuracy is surprising and interesting. The pruning also appears to stabilize training.\n\n## Weaknesses\n\n1. The paper lacks any ablation over curvature ($K$) or the weight $\\lambda$ on the centroid loss. These hyperparameters likely matter quite a bit, especially given the numerical instability of hyperbolic operations, and should be studied.\n\n2. No runtime or memory comparison is given. Log/exp maps and acosh are not cheap, and it’s unclear whether this adds significant overhead over the Euclidean baseline.\n\n3. Although the idea is new in the context of dataset distillation, it’s not fundamentally new in general. Centroid matching is well-known, and using a different geometry for it is not that far from existing work.\n\n4. The scope is fairly limited: experiments top out at Tiny-ImageNet (64×64 resolution), and it’s unclear whether the method scales to larger datasets like ImageNet-1k or non-image domains.\n\n## Verdict\n\nThis is a clean and reasonably well-executed paper that introduces a simple yet effective modification to DM-style dataset distillation. While the originality is limited and runtime/memory comparisons are missing, the empirical results are solid and the implementation is relatively straightforward. I recommend a weak accept rating.", "was_revised": true, "final_justification": "Fundamentally, the paper is a clean and reasonably well-executed work that introduces a simple yet effective modification to DM-style dataset distillation. After the rebuttal, I still believe the originality is fairly limited, but am happy that the authors have presented runtime/memory comparisons in their rebuttal and would recommend these be included in the final paper. I also very much like the addition of a comparison between the tangent space mean and the Fréchet mean (I also highly recommend the authors include this in the final version). Overall, my opinion is that rebuttal has improved the paper, but fundamentally I still maintain my weak accept rating.", "rebuttal": "## Response to Reviewer gver\n\nWe sincerely appreciate your thoughtful review and constructive comments on our submission. Below, we provide detailed answers to each of your questions and comments.\n\n---\n\n### **Clarification 1 – Ablation on Curvature $K$ and Loss-Weight $\\lambda$**  \n\nWe conducted an ablation study on different curvature values $K$ within the DM framework on CIFAR-10; the results are presented below:\n\n---\n\n| IPC   | Method       | $\\|K\\|$ = 1/3       | 0.5             | 1               | 2               | 5               |\n|-------|--------------|----------------|------------------|------------------|------------------|------------------|\n| 1     | DM           | 26.4 ± 0.3     | 26.4 ± 0.3      | 26.4 ± 0.3      | 26.4 ± 0.3      | 26.4 ± 0.3      |\n|       | DM + HDD     | 27.0 ± 0.2     | 28.8 ± 0.3      | 28.7 ± 0.2      | 27.6 ± 0.2      | 28.6 ± 0.2      |\n| 10    | DM           | 48.5 ± 0.6     | 48.5 ± 0.6      | 48.5 ± 0.6      | 48.5 ± 0.6      | 48.5 ± 0.6      |\n|       | DM + HDD     | 49.6 ± 0.3     | 49.9 ± 0.1      | 50.3 ± 0.3      | 50.1 ± 0.1      | 50.0 ± 0.2      |\n| 50    | DM           | 62.2 ± 0.5     | 62.2 ± 0.5      | 62.2 ± 0.5      | 62.2 ± 0.5      | 62.2 ± 0.5      |\n|       | DM + HDD     | 63.0 ± 0.3     | 63.1 ± 0.1      | 63.2 ± 0.4      | 63.1 ± 0.2      | 62.7 ± 0.1      |\n\nAlthough $\\|K\\|$ slightly affects the final accuracy, **the variation is modest, and HDD consistently surpasses the Euclidean baseline**.\n\n---\n\nUnlike the curvature hyperparameter, the inter‑centroid distances in hyperbolic space are extremely small because of its unique geometry, so we introduce a scaling factor. **As long as this factor stays within a reasonable range, it exerts little influence on the final results.** We choose its value empirically: by inspecting the raw loss values of the original method and selecting a factor that brings our (scaled) loss to the same order of magnitude. Of course, this value can also be determined automatically. To illustrate this, we run experiments with DM as the baseline while keeping the curvature fixed at  -1. Here, the initial loss denotes the loss after multiplying by the scaling factor when the number of distribution‑matching iterations is zero (the original DM uses no scaling). Our experimental results are as follows:\n\n| IPC = 10 | $\\lambda$ | Initial Loss | Accuracy (%) |\n|---------:|--:|-------------:|-------------:|\n|  DM | –  | 25 – 28 | 48.5 ± 0.6 |\n| DM + HDD | 10 | 3 – 4   | 50.3 ± 0.2 |\n| DM + HDD | 20 | 6 – 7   | 50.3 ± 0.3 |\n| DM + HDD | 40 | 12 – 14 | 50.2 ± 0.3 |\n| DM + HDD | 80 | 24 – 27 | 50.3 ± 0.2 |\n\n| IPC = 50 | $\\lambda$ | Initial Loss | Accuracy (%) |\n|---------:|--:|-------------:|-------------:|\n|   DM | –  | 5 – 7  | 62.2 ± 0.5 |\n| DM + HDD | 40 | 5 – 7   | 63.0 ± 0.3 |\n| DM + HDD | 80  | 11 – 13 | 63.2 ± 0.4 |\n| DM + HDD | 120 | 17 – 19 | 63.3 ± 0.2 |\n| DM + HDD | 160 | 25 – 27 | 62.9 ± 0.2 |\n\nBecause hyperbolic inter-centroid distances are inherently small, we multiply them by $\\lambda$ to match the scale of the original DM loss. As long as $\\lambda$ stays within a reasonable range, the accuracy remains virtually unchanged, confirming the **insensitivity of HDD to this scaling factor**.\n\n---\n\n### **Clarification 2 – Runtime and Memory Overhead**\n\nWe benchmarked 100 training iterations on a single RTX 4090 GPU with batch size = 256 on CIFAR-10.\n\n| IPC | DM Runtime | DM + HDD Runtime | DM Memory | DM + HDD Memory |\n|-----|------------|------------------|-----------|-----------------|\n| 1   | 4.9 s      | 6.7 s            | 3522 MiB  | 3522 MiB        |\n| 10  | 5.0 s      | 6.8 s            | 3626 MiB  | 3632 MiB        |\n| 50  | 5.4 s      | 7.1 s            | 3888 MiB  | 3922 MiB        |\n\n**Memory usage for the two methods is nearly identical**, while HDD requires more runtime, but the overhead remains within an acceptable range.\n\n---\n\n### **Clarification 3 – Originality**\n\nTo the best of our knowledge, this is **the first work to integrate hyperbolic geometry into dataset distillation** and to **tailor the distribution-matching objective explicitly to the space’s curvature**. We further observe that the synthetic samples naturally mimic the hierarchical structure of the real data and gravitate toward the class centroid. Exploiting this property, we prune redundant real samples without any loss of accuracy. We have deliberately kept the method simple, **no additional training tricks are introduced**, but we still **achieve notable accuracy gains over strong baselines on multiple datasets**. This simplicity enhances both reproducibility and extensibility. Further, we list some possible optimization directions: the framework can be generalized to (a) pseudo-Riemannian manifolds, (b) information-theoretic metrics in hyperbolic space, and (c) higher-order moment matching on hyperbolic manifolds, opening new avenues for more powerful and versatile dataset-distillation methods.\n\n---\n\n### **Clarification 4 – Scalability to Larger Datasets and Non-Image Domains**\n\nWe have applied HDD to an ImageNet subset, ImageWoof, with high resolution (224 × 224), and verified the effectiveness of our method (Table 3). The results indicate that HDD can scale beyond tiny images, and we are planning to extend experiments to other modalities like text and speech.\n\n---\n\n### **Clarification 5 – Sensitivity and Stability at Extreme Curvatures**\n\nWe conducted experiments on the extreme values of $\\|K\\|$​​ (including both extremely high curvature and extremely low curvature), and the results are as follows:\n\n| IPC   | Method       | No Curvature     | $\\|K\\|$ = 0.01 (Low Curvature) | $\\|K\\|$ = 100 (High Curvature) |\n|-------|--------------|------------------|----------------------------|----------------------------|\n| 1     | DM           | 26.4 ± 0.3       | -                          | -                          |\n| 1     | DM + HDD     | -                | 26.6 ± 0.2                 | 27.8 ± 0.2                 |\n| 10    | DM           | 48.5 ± 0.6       | -                          | -                          |\n| 10    | DM + HDD     | -                | 48.6 ± 0.3                 | 49.9 ± 0.2                 |\n| 50    | DM           | 62.2 ± 0.5       | -                          | -                          |\n| 50    | DM + HDD     | -                | 62.5 ± 0.3                 | 63.1 ± 0.1                 |\n\nWe observe that when the curvature $\\|K\\|$ is very small, the performance gap between HDD and the original DM shrinks, which aligns with the theoretical expectation that hyperbolic space degenerates into Euclidean space as the curvature approaches zero. Conversely, when $\\|K\\|$ becomes very large, our method remains effective.\n\n---\n\n### **Clarification 6 – Exact vs. Tangent-Space Fréchet Mean**\n\nThe vast majority of hyperbolic geometry methods in machine learning employ tangent space (or other approximate) centroids, as they are numerically stable and have consistently proven effective in practice. This is particularly true for dataset distillation frameworks, where centroids are invoked thousands of times per training run, making runtime efficiency critically important. We conducted experiments using the exact centroid computation method from the paper you provided. For the differential approximation, we perform 20 iterations to compute its centroid. In both centroid‑computation methods, the curvature is fixed at  -1. The runtime (for 100 iterations) and results on CIFAR-10 are as follows:\n\n| IPC | DM (Accuracy, Time)     | HDD (Tangent)             | HDD (Exact FM)           |\n|-----|--------------------------|----------------------------|---------------------------|\n| 1   | 26.4 ± 0.3, 4.9 s        | 28.7 ± 0.2, 6.7 s         | 28.4 ± 0.4, 8.6 s        |\n| 10  | 48.5 ± 0.6, 5.0 s        | 50.3 ± 0.3, 6.8 s         | 50.4 ± 0.2, 9.5 s        |\n| 50  | 62.2 ± 0.5, 5.4 s        | 63.2 ± 0.4, 7.1 s         | 63.3 ± 0.3, 9.8 s        |\n\nIt can be seen that the two approaches produce nearly identical results, yet the computational burden rises markedly. Therefore, we consider the tangent‑space approximation to be the more cost‑effective option. We will add a note on this point in the revised version. Thank you for your valuable suggestion!\n\n---\n\nWe hope that these detailed clarifications address all of your concerns. Thank you again for your valuable suggestions, which have helped us strengthen the manuscript."}, {"review_id": "CrYYSmpre3", "reviewer": "Reviewer_RBVi", "summary": "This study proposes Hyperbolic Dataset Distillation (HDD), a dataset distillation based on hyperbolic space, instead of Euclidean space-based dataset distillation. Since hyperbolic space naturally implies a hierarchical structure, we expect it to model the complex geometry and hierarchical relationship of datasets better than existing methods. As a specific methodology, this study proposes a distribution matching objective over hyperbolic space using Lorentz hyperbolic space. Experimental results show the effectiveness of HDD.", "questions": "1. The experimental results (Table 1) show that adding HDDs improves performance, but I'm curious about the rationale for this.\n\n2. In this manuscript, it is mentioned that the closer to the centroid, the better the representation of category prototypes because it encodes the overall geometric structure of the dataset. However, it is difficult to accept the claim because it is difficult to judge which image is higher-level or lower-level when looking at the visualization of distilled images in the Appendix. I would like to know what hyperbolic space the distilled images should be visualized in and the interpretation of it.\n\n3. I'm also curious about the results of HDD on a benchmark dataset with a hierarchy structure.\n\n4. If a hierarchy structure exists in the optimized synthetic dataset, it is able to prune along the hierarchy after training. If this direction is correct, I would be interested to see a comparison with MDC [1], as well.\n\n[1] Multisize Dataset Condensation", "limitations": "See weaknesses and questions", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n1. This manuscript is well written and easy to read.\n\n2. To my knowledge, this is the first study in the field of dataset distillation to discuss optimization in hyperbolic space rather than Euclidean space, which is interesting and significant because it brings new perspectives and possibilities to the field of dataset distillation.\n\n3. This study proposes a reasonable and logical methodology using hyperbolic geometry. It is also easy to apply because HDD can be implemented with the materials of prior research without additional materials.\n\n**Weaknesses**\n1. My major concern is the originality and novelty of the work. As the authors note in Related works, the direction of introducing hyperbolic spaces into machine learning has been well studied. The proposed HDD is a methodology that replaces the Euclidean nature of vanilla DM with a hyperbolic nature. This work appears to be a combination of existing techniques, applying hyperbolic geometry to dataset distillation. Therefore, I think that this study lacks originality and novelty because HDD has not been modified for dataset distillation.\n\n2. Lack of a clear rationale for applying hyperbolic space to dataset distillation. I was expecting to read about what problems Euclidean space DM has and how introducing hyperbolic space can solve them, but the manuscript only has statements to support this, and no theoretical or experimental evidence (only performance comparison).\n\n3. Although hyperbolic space has the advantage of an inherent hierarchical structure, there are no experimental results for benchmark datasets that contain a hierarchical structure. This makes it difficult to verify that synthetic datasets created with HDDs actually contain a hierarchical structure well.", "was_revised": true, "final_justification": "This paper introduces hyperbolic space instead of Euclidean space to the field of dataset distillation for the first time and proposes a dataset distillation objective through Riemannian mean matching. These ideas have been experimentally proven to show consistent performance improvements despite being intuitive and simple methodologies.\n\nHowever, there is a lack of logical and solid grounds for introducing hyperbolic space to dataset distillation. Additionally, the analysis of the reasons for the performance improvement of the proposed methodology compared to previous studies has not been logically proven. Although these issues were discussed during the rebuttal period, clear and sufficient answers were not provided. In the absence of such evidence, I believe that the proposed simple methodology is merely a well-combined version of already widely known concepts.\n\nTherefore, I propose a borderline rejection of the paper, as it requires further refinement in its core aspects.", "rebuttal": "## Response to Reviewer RBVi\n\nWe sincerely thank you for your insightful review and for acknowledging the clarity of our manuscript and the potential of introducing hyperbolic geometry into dataset distillation.  Below, we provide detailed responses to your concerns.\n\n---\n\n### **Clarification 1 – Originality**\n\nTo the best of our knowledge, this is **the first study to integrate hyperbolic geometry into dataset distillation and to tailor the distribution‑matching objective explicitly to the curvature of the space**. We not only introduce a new distillation objective, matching the Fréchet means of two data distributions in hyperbolic space, but also **systematically examine how samples at different hierarchical levels contribute to the loss**, and we propose a hierarchy‑aware pruning strategy. Previous hyperbolic research has focused mainly on graph networks, metric learning, or generative models; in contrast, our framework targets the specific pain point in dataset distillation where hierarchical alignment and structure are often overlooked, as differentiated in our related work overview. In summary, we inject a hyperbolic hierarchical inductive bias into the distillation process for the first time, **without adding any extra training tricks, yet still achieve notable accuracy gains across multiple datasets** compared with baseline methods. This simplicity makes our approach easy to reproduce and extend. Further, we list some possible optimization directions: the framework can be generalized to (a) pseudo-Riemannian manifolds, (b) information-theoretic metrics in hyperbolic space, and (c) higher-order moment matching on hyperbolic manifolds, opening new avenues for more powerful and versatile dataset-distillation methods.\n\n---\n\n### **Clarification 2 – Motivation for Hyperbolic Space**\n\n**Limitations of Euclidean discrepancy measures.**  \nEuclidean MSE / MMD treats samples as i.i.d. points, weighting high-level (fine-detail or noisy) samples the same as low-level (prototype) samples and thereby diluting distillation effects.\n\n**Why negative curvature?**  \nHyperbolic space ($K < 0$) expands exponentially in volume, embedding tree structures continuously. By matching centroids in this space, we align the hierarchical structure between real and synthetic data. Samples at larger geodesic distance naturally contribute less, **amplifying prototypes while suppressing noise**.\n\n**Experimental evidence.**  \nOur pruning study (Table 2 in the paper) confirms that removing samples farther from the hyperbolic origin has little impact, indicating that those points indeed contain less critical information.\n\n---\n\n### **Clarification 3 – Hierarchical Properties of Benchmark Datasets**\n\n**The benchmark datasets we use also exhibit hierarchical structure**, which can be quantified by **Gromov δ‑hyperbolicity** [1]. Gromov δ‑hyperbolicity measures how close a metric space is to an ideal negatively curved space. Values near 0 indicate that the data are inherently more compatible with tree‑like or negatively curved geometry, whereas values approaching 1 suggest a stronger tendency toward Euclidean geometry. In feature space, common benchmark datasets have Gromov δ‑hyperbolicity of roughly as shown below:\n\n| Dataset      | δ-hyperbolicity |\n|--------------|----------------:|\n| CIFAR-10     | ≈ 0.25 |\n| CIFAR-100    | ≈ 0.23 |\n| TinyImageNet | ≈ 0.21 |\n\nThese figures demonstrate that **these widely used benchmarks indeed lean toward negatively curved spaces and possess a certain degree of hierarchy**.\n\n---\n\n### **Clarification 4 – Rationale for Performance Improvements**\n\nAs we explained in **Clarification 2**, our choice is grounded in two main considerations:\n\n(a) By aligning centroids to match the hierarchical structure of the data, we minimize the overall distributional gap between synthetic and real samples. As Figure  4 shows, the synthetic samples closely trace the distribution of the originals.\n\n(b) The hierarchical weighting of centroids in hyperbolic space encourages the synthetic samples toward class prototypes while suppressing boundary noise.\n\n---\n\n### **Clarification 5 – Visualisation and Interpretation**\n\nDue to conference policy, images cannot be uploaded. We therefore list IDs of the five CIFAR-10 *Horse* images **nearest to** and **farthest from** the hyperbolic origin after 10,000 IDM iterations:\n\n- **Nearest samples:** `data_batch_3_img6963`, `data_batch_4_img455`, `data_batch_4_img6399`, `data_batch_2_img4897`, `data_batch_4_img5922`  \n- **Farthest samples:** `data_batch_1_img1394`, `data_batch_5_img897`, `data_batch_1_img2701`, `data_batch_3_img3910`, `data_batch_2_img6227`\n\nWhen viewed locally, **near-origin samples are visually cleaner and centred**, whereas far-origin samples are cluttered or partially occluded, supporting our interpretation of hyperbolic radius as a hierarchy indicator.\n\n---\n\n### **Clarification 6 – Benchmark Dataset with a Hierarchy Structure**\n\nAs we explained in **Clarification 3**, we have confirmed that the benchmark dataset we used also exhibits a hierarchical structure. In the future, we plan to extend HDD to other data modalities, such as language, with even richer hierarchical structures.\n\n---\n\n### **Clarification 7 – Comparison with Multisize Dataset Condensation**\n\nMultisize Dataset Condensation (MDC) collapses the N individual condensation runs that would normally be required for different target set sizes into a single process. It does so by adding an adaptive subset loss on top of the standard gradient‑matching loss, allowing the full synthetic set and any of its subsets to converge toward the real data simultaneously and eliminating the “subset performance collapse.” In other words, **MDC permits informational overlap between a subset and the complete set**. Our method, in contrast, **attaches explicit hierarchical information to every synthetic sample**. Trimming the synthetic dataset, therefore, means deliberately discarding part of that hierarchy, which causes a pronounced performance drop. The results of the experiment we conducted are as follows：\n\n| Scenario (CIFAR-10)                  | Accuracy (%) |\n|--------------------------------------|--------------|\n| IPC 1                             | 28.7 ± 0.2 |\n| IPC 10 → IPC 1             | 21.6 ± 0.8 |\n| IPC 50 → IPC 1             | 17.0 ± 0.6 |\n\nWe observed that **the larger the IPC in the original set, the less information remains after trimming**. Specifically, cutting an IPC‑50 set down to IPC‑1 performs worse than cutting an IPC‑10 set down to IPC‑1, while a native IPC‑1 set, with no trimming at all, achieves the best results by a wide margin.\n\n---\n\nWe hope that these detailed clarifications address all of your concerns. Thank you again for your valuable suggestions, which have helped us strengthen the manuscript.\n\n---\n\n**Reference**\n\n[1] *Hyperbolic Image Embeddings*."}, {"review_id": "AXi2LgqPQe", "reviewer": "Reviewer_arwV", "summary": "This paper investigates an issue with dataset distillation, where synthesized data in distribution matching method are trained under the setting of independent and identical distribution and fail to capture the complex geometric and hierarchical relationship. To address this problem, the authors propose a new dataset distillation strategy called Hyperbolic Dataset Distillation method (HDD). HDD embeds data into hyperbolic space and optimizes geodesic distance between the centroids of synthetic and original data, naturally capturing tree-like and hierarchical relationships. This approach allows hierarchical pruning and it is compatible with most existing distribution matching methods.", "questions": "1. How is the curvature associated with sample $i$ computed?\n2. The authors mention that distribution matching methods improve efficiency, but optimization-based methods typically achieve higher performance. Is it feasible to apply HDD to optimization-based methods? If so, could the authors provide experimental results?\n3. Could the authors provide more concrete evidence that the performance gains from HDD are indeed due to the presence of hierarchical structure in the original data? For the standard benchmark datasets used in the paper, the assumption of hierarchical relationships is not verified.", "limitations": "In Sec. 5, the authors mention a limitation that more advanced distribution alignment objectives, such as KL divergence, are not explored in hyperbolic space, which could potentially improve performance further.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\nThis paper is well-motivated, and well-organized. The authors provide comprehensive experiments on various datasets such as CIFAR-10, CIFAR-100, TinyImageNet. They provide insightful analysis of the experimental results and a detailed method description.\n\n**Weakness**\n1. The encoder used to extract features before hyperbolic embedding. This may lead to the loss of important hierarchical information or cause feature collapse, where synthetic samples concentrate around the origin without preserving meaningful structure.\n2. The effectiveness depend on tuning of curvature $K$ and scaling factor $\\lambda$. \n3. Some definitions are not clear: (a) In line 265, the definition of $K_i$ is ambiguous; (b) The correct presentation of Eq. (1) should be: $ \\mathcal{S}^* = \\arg \\min_{\\mathcal{S}} E_{(x, t) \\sim P_T} || \\ell(\\theta_{\\text{syn}}(x), t) - \\ell(\\theta_{\\text{real}}(x), t) || $. Eq. (2) contains a similar mistake.\n4. Some notations are abused. For example, the authors define $x$ as data instance, $t$ as label and $s$ as synthetic data with super script $syn$ in Sec 3.1 Problem Definition. However, in Sec 3.1 Hyperbolic Geometry, $x_t$ and $x_s$ are redefined as time component and spatial component. It is recommended to revise the notation to ensure consistency and clarity.", "was_revised": true, "final_justification": "I thank the authors for their detailed and thoughtful rebuttal. The response fully resolves my main concern. I believe the paper is good enough to be accepted, and I raised my score accordingly.", "rebuttal": "## Response to Reviewer arwV\n\nWe sincerely appreciate your thoughtful review and constructive comments on our submission. Below, we provide detailed responses to your concerns.\n\n---\n\n### **Clarification 1 - Frozen Encoder: No Feature Collapse, Robust at Extreme Curvatures**\n\nWe extract image features using a frozen network, whose weights remain unchanged during distillation; therefore, **the features do not collapse into trivial representations** (i.e., the synthesized samples are not forced to cluster around the origin).\nMoreover, our extensive experiments confirm that no feature collapse has been observed to date. Finally, we tested extreme values of $\\|K\\|$, and the results show that HDD maintains robust performance even in these edge cases, as presented below:\n\n| IPC   | Method       | No Curvature     | $\\|K\\|$ = 0.01 (Low Curvature) | $\\|K\\|$ = 100 (High Curvature) |\n|-------|--------------|------------------|----------------------------|----------------------------|\n| 1     | DM           | 26.4 ± 0.3       | -                          | -                          |\n| 1     | DM + HDD     | -                | 26.6 ± 0.2                 | 27.8 ± 0.2                 |\n| 10    | DM           | 48.5 ± 0.6       | -                          | -                          |\n| 10    | DM + HDD     | -                | 48.6 ± 0.3                 | 49.9 ± 0.2                 |\n| 50    | DM           | 62.2 ± 0.5       | -                          | -                          |\n| 50    | DM + HDD     | -                | 62.5 ± 0.3                 | 63.1 ± 0.1                 |\n\nWe observe that when the curvature $\\|K\\|$ is very small, the performance gap between HDD and the original DM shrinks, which aligns with the theoretical expectation that hyperbolic space degenerates into Euclidean space as the curvature approaches zero. Conversely, when $\\|K\\|$ becomes very large, our method remains effective.\n\n---\n\n### **Clarification 2 – Ablation on Curvature $K$ and Loss-Weight $\\lambda$**  \n\nWe conducted an ablation study on different curvature values $K$ within the DM framework on CIFAR-10; the results are presented below:\n\n---\n\n| IPC   | Method       | $\\|K\\|$ = 1/3       | 0.5             | 1               | 2               | 5               |\n|-------|--------------|----------------|------------------|------------------|------------------|------------------|\n| 1     | DM           | 26.4 ± 0.3     | 26.4 ± 0.3      | 26.4 ± 0.3      | 26.4 ± 0.3      | 26.4 ± 0.3      |\n|       | DM + HDD     | 27.0 ± 0.2     | 28.8 ± 0.3      | 28.7 ± 0.2      | 27.6 ± 0.2      | 28.6 ± 0.2      |\n| 10    | DM           | 48.5 ± 0.6     | 48.5 ± 0.6      | 48.5 ± 0.6      | 48.5 ± 0.6      | 48.5 ± 0.6      |\n|       | DM + HDD     | 49.6 ± 0.3     | 49.9 ± 0.1      | 50.3 ± 0.3      | 50.1 ± 0.1      | 50.0 ± 0.2      |\n| 50    | DM           | 62.2 ± 0.5     | 62.2 ± 0.5      | 62.2 ± 0.5      | 62.2 ± 0.5      | 62.2 ± 0.5      |\n|       | DM + HDD     | 63.0 ± 0.3     | 63.1 ± 0.1      | 63.2 ± 0.4      | 63.1 ± 0.2      | 62.7 ± 0.1      |\n\nAlthough $\\|K\\|$ slightly affects the final accuracy, **the variation is modest, and HDD consistently surpasses the Euclidean baseline**.\n\n---\n\nUnlike the curvature hyperparameter, the inter‑centroid distances in hyperbolic space are extremely small because of its unique geometry, so we introduce a scaling factor. **As long as this factor stays within a reasonable range, it exerts little influence on the final results.** We choose its value empirically: by inspecting the raw loss values of the original method and selecting a factor that brings our (scaled) loss to the same order of magnitude. Of course, this value can also be determined automatically. To illustrate this, we run experiments with DM as the baseline while keeping the curvature fixed at  -1. Here, the initial loss denotes the loss after multiplying by the scaling factor when the number of distribution‑matching iterations is zero (the original DM uses no scaling). Our experimental results are as follows:\n\n| IPC = 10 | $\\lambda$ | Initial Loss | Accuracy (%) |\n|---------:|--:|-------------:|-------------:|\n|  DM | –  | 25 – 28 | 48.5 ± 0.6 |\n| DM + HDD | 10 | 3 – 4   | 50.3 ± 0.2 |\n| DM + HDD | 20 | 6 – 7   | 50.3 ± 0.3 |\n| DM + HDD | 40 | 12 – 14 | 50.2 ± 0.3 |\n| DM + HDD | 80 | 24 – 27 | 50.3 ± 0.2 |\n\n| IPC = 50 | $\\lambda$ | Initial Loss | Accuracy (%) |\n|---------:|--:|-------------:|-------------:|\n|   DM | –  | 5 – 7  | 62.2 ± 0.5 |\n| DM + HDD | 40 | 5 – 7   | 63.0 ± 0.3 |\n| DM + HDD | 80  | 11 – 13 | 63.2 ± 0.4 |\n| DM + HDD | 120 | 17 – 19 | 63.3 ± 0.2 |\n| DM + HDD | 160 | 25 – 27 | 62.9 ± 0.2 |\n\nBecause hyperbolic inter-centroid distances are inherently small, we multiply them by $\\lambda$ to match the scale of the original DM loss. As long as $\\lambda$ stays within a reasonable range, the accuracy remains virtually unchanged, confirming the **insensitivity of HDD to this scaling factor**.\n\n---\n\n### **Clarification 3 - Clarification of Notation and Equations**\n\nRegarding $K_i$ in line 265, what we originally meant was the temporal component of sample features embedded under a fixed curvature. We apologize for the resulting misunderstanding and will amend this in the revised manuscript. Equations (1) and (2) themselves are correct and follow a written format similar to DANCE [1], whereas the formulation you suggested aligns more with other distribution‑matching methods, so we will adopt the format in the revised version. Thank you also for your keen observation: we will rectify the notation issue and replace  $x_t$ and $x_s$ with $p_t$ and $p_s$.\n\n---\n\n### **Clarification 4 - Per-Sample Curvature**\n\nAll samples were **consistently embedded in a hyperbolic space with identical curvature**. We apologize once again for the confusion regarding the definition of $K_i$  in line 265.\n\n---\n\n### **Clarification 5 - Can HDD Help Optimisation-Based DD?**\n\nHDD cannot boost optimization‑based methods on its own, but it can indirectly enhance their performance when used within a hybrid framework. As shown in Table  3 of the original paper, incorporating HDD into DANCE [1] yields performance improvements. We also provide results obtained by combining HDD with another hybrid approach, DSDM [2]. The table below summarises the results; **adding HDD yields consistent gains across all IPC settings on CIFAR-10**.\n\n| Method            | IPC | Accuracy (%) |\n|-------------------|----:|-------------:|\n| DSDM              |   1 | 43.8 ± 0.2   |\n| **DSDM + HDD**    | **1** | **46.4 ± 0.3** |\n| DSDM              |  10 | 65.8 ± 0.3   |\n| **DSDM + HDD**    | **10** | **66.6 ± 0.4** |\n| DSDM              |  50 | 75.8 ± 0.2   |\n| **DSDM + HDD**    | **50** | **76.0 ± 0.2** |\n\n---\n\n### **Clarification 6 - Evidence of Hierarchical Structure in Benchmarks**\n\nFirst, **the benchmark datasets we use also exhibit hierarchical structure**, which can be quantified by **Gromov δ‑hyperbolicity** [3]. Gromov δ‑hyperbolicity measures how close a metric space is to an ideal negatively curved space. Values near 0 indicate that the data are inherently more compatible with tree‑like or negatively curved geometry, whereas values approaching 1 suggest a stronger tendency toward Euclidean geometry. In feature space, common benchmark datasets have Gromov δ‑hyperbolicity of roughly as shown below:\n\n| Dataset      | δ-hyperbolicity |\n|--------------|----------------:|\n| CIFAR-10     | ≈ 0.25 |\n| CIFAR-100    | ≈ 0.23 |\n| TinyImageNet | ≈ 0.21 |\n\nThese figures demonstrate that **these widely used benchmarks indeed lean toward negatively curved spaces and possess a certain degree of hierarchy**.\n\n---\n\nWe also visualized the labels of the five samples nearest to the origin and the five farthest from it among the 256 raw 'Horse' images (one batch) at the 10,000th training iteration of IDM on CIFAR-10. However, due to conference policy restrictions, we are unable to provide the images directly. You may inspect the corresponding images by downloading this publicly available dataset.  We apologize for any inconvenience this limitation may cause and appreciate your understanding.\n\n- **Nearest samples:** `data_batch_3_img6963`, `data_batch_4_img455`, `data_batch_4_img6399`, `data_batch_2_img4897`, `data_batch_4_img5922`  \n- **Farthest samples:** `data_batch_1_img1394`, `data_batch_5_img897`, `data_batch_1_img2701`, `data_batch_3_img3910`, `data_batch_2_img6227`\n\nWe observe that **samples farther from the origin are noticeably messier**, either showing cluttered scenes with both people and horses or only partial views such as a horse’s head, whereas those nearer to the origin are visually cleaner, free of distracting objects, and present the horse’s full outline.\n\n---\n\nWe hope that these detailed clarifications address all of your concerns. Thank you again for your valuable suggestions, which have helped us strengthen the manuscript.\n\n---\n\n**References**\n\n[1] *DANCE: Dual-View Distribution Alignment for Dataset Condensation*  \n[2] *Diversified Semantic Distribution Matching for Dataset Distillation*  \n[3] *Hyperbolic Image Embeddings*"}]

Author rebuttal:

Meta-review: This paper proposes hyperbolic dataset distillation. Distribution matching is currently restricted to Euclidean space and this paper overcomes that gap in literature. After the review and rebuttal stage, the paper received an average rating of 4, based on one accept (5), two weak accepts (4), and one weak reject (3).

Regarding the strengths of the paper, the reviewers are in agreement. All positively not the novelty of the paper, the writing, and the empirical outcomes. Reviewer 9awn (score 5) asked a few prodding questions on the generalization capabilities of the method, for which the authors provide extra empirical results. Reviewer gver (score 4) has multiple good points regarding missing information on ablations and scalability. Here too, the authors provide the required answers, backed up with empirical outcomes. Reviewer arwV (score 4) had very similar concerns. Lastly, reviewer RBVi (score 3) pointed out in the final internal discussions that they had concerns regarding a lack of theoretical analyses. They find the final response by the authors interesting and insightful. While the reviewer would still like to know more precisely whether Riemannian mean matching indeed corresponds to the distribution matching (beyond bounds). However, they point out that based on the novelty of the paper, they are not opposed to acceptance.

As such, the AC follows the reviewer consensus and votes for acceptance. The strengths of the paper are clear and the weaknesses mostly concerned technical questions and additional insights/ablations, which have largely been addressed.
Decision: Accept (poster)