FairDD: Fair Dataset Distillation
Authors: Qihang Zhou
ShenHao Fang
Shibo He
Wenchao Meng
Jiming Chen
Venue: NeurIPS 2025 poster ()
Keywords: Dataset Distillation
Dataset Condensation
Primary area: applications
TL;DR: We propose FairDD to mitigate the bias in condensed datasets with a promising tradeoff between accuracy and fairness.

Abstract: Condensing large datasets into smaller synthetic counterparts has demonstrated its promise for image classification. However, previous research has overlooked a crucial concern in image recognition: ensuring that models trained on condensed datasets are unbiased towards protected attributes (PA), such as gender and race. Our investigation reveals that dataset distillation fails to alleviate the unfairness towards minority groups within original datasets. Moreover, this bias typically worsens in the condensed datasets due to their smaller size. To bridge the research gap, we propose a novel fair dataset distillation (FDD) framework, namely FairDD, which can be seamlessly applied to diverse matching-based DD approaches (DDs), requiring no modifications to their original architectures. The key innovation of FairDD lies in synchronously matching synthetic datasets to PA-wise groups of original datasets, rather than indiscriminate alignment to the whole distributions in vanilla DDs, dominated by majority groups. This synchronized matching allows synthetic datasets to avoid collapsing into majority groups and bootstrap their balanced generation to all PA groups. Consequently, FairDD could effectively regularize vanilla DDs to favor biased generation toward minority groups while maintaining the accuracy of target attributes. Theoretical analyses and extensive experimental evaluations demonstrate that FairDD significantly improves fairness compared to vanilla DDs, with a promising trade-off between fairness and accuracy. Its consistent superiority across diverse DDs, spanning Distribution and Gradient Matching, establishes it as a versatile FDD approach.

FairDD: Fair Dataset Distillation
Qihang Zhou', Shenhao Fang", Shibo Hel, Wenchao Meng, Jiming Chen Zhejiang University
{zghang, 22460454, s18he, wmengzju, cjm]@zju.edu edu. cn
Abstract
Condensing large datasets into smaller synthetic counterparts has demonstrated promise for image classification. However, previous research has overlooked a crucial concern in image recognition: ensuring that models trained on condensed datasets are unbiased towards protected attributes (PA), such as gender and race. Our investigation reveals that dataset distillation fails to alleviate the unfairness towards minority groups within original datasets. Moreover, this bias typically worsens in the condensed datasets due to their smaller size. To bridge the research gap, we propose a novel fair dataset distillation (FDD) framework, namely FairDD, which can be seamlessly applied to diverse matching-based DD approaches (DDs), requiring no modifications to their original architectures. The key innovation FairDD lies in synchronously matching synthetic datasets to PA-wise groups of original datasets, rather than indiscriminate alignment to the whole distributions vanilla DDS, dominated by majority groups. This synchronized matching allows synthetic datasets to avoid collapsing into majority groups and bootstrap their balanced generation to all PA groups. Consequently, FairDD could effectively regularize vanilla DDs to favor biased generation toward minority groups while maintaining the accuracy of target attributes. Theoretical analyses and extensive experimental evaluations demonstrate that FairDD significantly improves fairness compared vanilla DDs, with a promising trade-off between fairness and accuracy. Its consistent superiority across diverse DDs, spanning Distribution and Gradient Matching, establishes versatile FDD approach. Code is available hhttpss //github. com/ www.agg/haagaa///
Introduction
Deep learning has witnessed remarkable success in computer vision, particularly with recent break- throughs in vision models [45] 28, 47, 333 (7).. Their vision backbones, such as ResNet [20) and
ViT (16), are data-hungry models that require extensive amounts of data for optimization. Dataset Distillation (DD) [60 67 69
7 & 71] provides a promising solution to alleviate this data requirement by condensing the original large dataset into more informative and [0] Despite its appeal, existing researches focus on ensuring that models trained on condensed datasets perform comparable accuracy to those trained on the original dataset
6 58
32 12
38
18
22
smaller counterparts 42
in terms of target attributes (TA) [13 40 56]. However, they have overlooked enabling the fairness of trained models with respect protected attributes (PA).
Unfairness typically arises from imbalanced sample distributions among PA in the empirical training datasets. When the original datasets suffer from the PA imbalance, the corresponding datasets condensed by vanilla DDs inherit and amplify this bias in Fig. [(e] Since vanilla DDs tend to cover TA distribution for image classification, and result, naturally leads to more synthetic samples located in majority groups compared to minority groups w.r.t. PA, as shown in Figs. [(] [(b) [(c]
and I(d) In this case, these condensed datasets retain the imbalance between protected attributes.
"Equal contribution. Corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

A
2
DM
5
S
5
6
X
3
9
8
DM
DM
DM
DM
(d) CelebA (Each row denotes one class)
0
0
0
0
Y
2
2
2
2.
2
2
3
3
5
$
5
5
6
Terress
8
8
FaiMDD-DM
Dai
FairDD+DM
Eacrsoneasl
ChiNGes
(a) C-MNIST (FG). (b) C-FMNIST (BG).
(c) CIFARIO-S.
(e) Fairness comparison(PPC-5)
Visualization comparison on S at IPC = 10 for diverse datasets. FairDD successfully mitigates the bias from original datasets (a) foreground digital color, (b) background color, (c) foreground and background grayscale (d) real-world bias. (e) vanilla DDs exacerbate the unfairness.
Figure 1:
thereby rendering the model trained on them unfair. Moreover, the reduced size of the condensed datasets typically amplifies the bias present in the original datasets, especially when there is a significant gap in size between the original and condensed datasets, such as image per class (IPC) 1000 vs. 10. Therefore, it is worthwhile to broaden the scope of DDs to encompass both TA accuracy and PA fairness. Recent works [4 41) attempt to address the class/TA-level long-tailed phenomenon [7I] and spurious correlations [13 to improve the classification performance [56] 59].
but the exploration on visual fairness is still blank.
To bridge this gap, we propose FairDD, a novel FDD framework that achieves PA fairness in models trained on condensed datasets, even when the original data exhibit PA imbalance. Note that FDD addresses deta- fairness: how to generate a distilled dataset that is inherently unbiased, agnostic to the downstream model. Instead, traditional fairness literatures focus on model-level fairness: how to train a fair model given a dataset. These two perspectives approach fairness from distinct yet complementary directions. FDD requires simultaneously maintaining TA accuracy and improving PA fairees.. It is challenging, as the algorithm must properly balance the emphasis across all groups reducing the dominance of majority groups while maintaining their TA distributional coverage, and preserving minority groups to mitigate PA bias.
FairDD tackles this challenge by (1) partitioning the empirical training distribution into different groups according to PA and decomposing the single alignment target of vanilla DDs into PA-wise sub- targets. (2) synchronously matching synthetic samples to these PA groups, which equally bootstraps synthetic datasets to each PA group without involving the specific group size. In doing so, we reformulated the optimization objectives of vanilla DDs into fairDD-style versions. This allows FairDD to mitigate the effect of imbalanced PA on the generation of S and prevents S from collapsing into the majority group. In Fig. [(d] FairDD synthesizes more male samples (highlighted by blue squares) within an attributive class originally dominated by females. Meanwhile, FairDD could also achieve the comprehensive coverage of the entire distribution for TA accuracy. We provide theoretical guarantee that FairDD could improve PA fairness while maintaining TA accuracy.
Extensive experiments demonstrate that framework effectively mitigates the unfaimess in datasets of highly diverse bias. FairDD substantially improves data fairness trained on condensed datasets compared to various vanilla DDs. FairDD demonstrates its versatility across diverse DDs, including Distribution Matching (DM) and Gradient Matching (GM) Our main contributions are as follows:
"W do not apply FairDD to Trajectory Matching (TM) because it would require additional model trajectories trained minority groups. prone overlitting due to their limited sample sizes.

---

To the best of our knowledge, our research is the first attempt to incorporate visual fairness into DDs explicitly. We reveal that vanilla DDs fail to mitigate the bias in original datasets and may exacerbate it due to the limited synthetic samples, leading to severe PA bias in the model trained by the resulting condensed dataset.
We introduce novel FDD framework called FairDD, which proposes synchronized match- ing to align synthetic samples to all PA groups partitioned from the original data distribution. This allows the generated synthetic samples to be agnostic PA imbalance of original datasets while maintaining the overall distributional coverage of TA.
Extensive empirical experiments demonstrate that FairDD is generalist to significantly mitigate the unfairness of vanilla DDs. Its consistent superiority is observed across various DDs, including DM and GM.
2 Related Work
Dataset distillation Dataset distillation has been broadly applied to many important fields 30 21
179. The first work [60] attempts to formulate dataset distillation as a bi-level optimization problem.
However, the two folds of the optimization process are time-consuming. Neural tangent kernel (23 is utilized to obtain the closed form of the inner loop [44] (37 (73). Some previous works propose achieve comparable even better performance, including matching-based
surrogate objectives
methods like GM 770 67 31, DM 669
58
68 TM 6 [2], soft label learning 54 62 [2], and
41 attempt to mitigate bias to improve classification accuracy on the TA without considering protected attributes (PA), staying within the traditional setting
factorization [27 15 35 31, Recent works 14
of DD 13 7 56 59) The work [33 mitigates sample-wise bias by assigning higher weights to samples located in low-density regions of the original data distribution, while they neglect fairness concerning PA. It fails to guarantee that the alignment objective is unbiased across all attribute groups, nor does it ensure adequate distribution coverage. Instead, our methods have fairness alignment objective to facilitate unbiased data distillation; In addition, we provide a theoretical proof to guarantee the distribution coverage for TA. This makes FairDD with a good balance between fairness and accuracy. We provide a performance comparison Appendix
Visual fairness Current literature on fairness aims to train a model that outputs fair logits under class-imbalanced datasets [5]. According to the stage of bias mitigation, the research field of
fairness algorithm 3 can be classified into three branches: Pre- Pre-processing [[] 39 46 [31], In-
processing M 24 64 66 |6] 61 25 65], and Post- processon [9]. They learn fair representations 57 48). Fairness-aware synthetic data generation
without anvolving information condensation (53 55
serves as pre-processing for fairness. They frame fairness mitigation as a data-to-data translation problem, and utilize generative models (63] to produce fairer datasets with respect to protected groups [46, 55). However, they do not consider the aspect of information condensation. Instead, our work aims to reduce bias in condensed datasets by: (1) ensuring that the information from the original datasets is effectively distilled into the condensed datasets, and (2) simultancously mitigating both the inherent bias of the original dataset and the bias exacerbated by vanilla dataset condensation. Once the data is condensed, we can train a fair model without any further human intervention.
3 Preliminaries
Dataset Distillation. Given a vast dataset T = [2;, 1 DDs aim to condense original dataset T into a smaller dataset S = (x., y.) M via distillation algorithm Alg with nerual networks, parameterized by 0. Randomly initialized classification network Is should maintain the same empirical risk whether it is trained on S or TT
S argmin Alg(S,T,))
Eg [Elos S)] = Bg ([(e: T)),
where V and E(-) represent the parameter space and loss function, respectively. The pioneering work (60) formulates Alg as a bi-level optimization problem. However, such an optimization process time-consuming and unstable. Recent works circumvent it and propose surrogate matching objectives to achieve comparable and even better performance. This research line is collectively referred to as the DMF, and our paper primarily studies one-stage GM 770 67 and DM (69 58 68 We leave it for future exploration.
3

---

Visual Fairness Visual faimess is an important field to mitigate discrimination against minority groups. Group faimness requires statistical disparity to different groups in terms of PA, such as race and gender. This means that an ideal fair model should make independent predictions between TA and PA. One of the common fairness criteria is equalized odds (EO), which computes the prediction accuracy of PA conditioned on TA, to evaluate the level of conditional independence between PA and TA. We use two types of difference of equalized odds DEOM and DEO, from the worst and averaged levels. Formally, given the PA set A = {01, az..... Op J. where p denotes the number of protected attributes. DEOM and AEOA [26] can be formulated mathematically as follows:
DEOM max max WEY ar,ay EAicc,
|P y/Y y, a.) P(ý y/Y y, a,}l
DEOA vEY
mean
max Mgn E.Ako s
|P- = v/Y = y, A= = P(K y/Y = y, A a;)l
A Close Look at Dataset Distillation
A unified perspective for Data Match Framework. The essence of the DMF lies in choosing the target signs of original samples that effectively represent their characteristies for image recognition, and then aligning these signals ((x; is typically the key information from feature extraction or optimization process using a randomly initialized network parameterized by 0. For example, GM aligns the gradient information produced by T with that of the condensed S. Instead, DM matches the embedding distributions of T and S. As for these approaches in DMF. we can unify the optimization objective as L(S: T):
proxy task optimize the condensed dataset. The target signal
L(S;0, T) = EBEN D(BlOanT, (2; 0), El (2; 0));
(1)
where E. El (2;; 0)] E RC and Eiox-s, (1; (x: E RC are represented expectation vectors of the target signs T and S, respectively. D(.,-) is a distance function. In DMF, MSE is adopted in DM and DREAM, and MAE is used IDC.
Why do vanilla DDs fail to mitigate PA imbalance? Given the dataset T = {(1i,, Vi, ai) ),)),r a, E A, let us define the class-level sample ratio R, = (rai , .r...... 20) ). where ry = 1T 1/(T, and represents the cardinal number of a set. Current DDs' paradigms focus on preserving TA representativeness for image recognition. Here, we decompose the whole expectation into the expectation of PA-wise groups, i.e, ElØenT, (z; 0)] can be rewritten as follows:
LazeA ry "y:BB( Blo (z; 0)], and thus Eq.
((S:0,T) Eyey D(En,eA r El
(a; 0)], Elo. (x;
(2)
From Eq. the optimization objective of class y is weighted by the sample ratio ree from different
groups. When T suffers from PA imbalance, e.g., r > Eijg Ti the majority group indexed by i contributes more to the alignment compared to minority groups. In other words, S tends to produce more samples belonging to group for the total loss minimization. The objective of vanilla DDs suffers from PA imbalance within T.
Next, we further study how the resulting S is affected by sample ratio r of different groups To this end, we assume that the optimization process could reach the optimal solution for each class, and as a result, the final resulting S satisfies the condition that the derivative of the objective function with
aD(EafEA " Blo Eloo., Tg. (m:0)). (m:())) = 0. Now, let's
respect to Biox~s, (z; 0) equals O, i.e.,
05/0r= (x:80]
delve into the specific distance metrics used in vanilla DDs, where the most commonly used metrics
are MAE, MSE, and cosine distance. We could compute the optimal point of E[ (z; 0)] could reach under these metrics:
Bioa-s, (z: = Σ
r E
2n (z;0)),
(3)
a.cAA
Where A is a constant shared across all groups, equal to for MAE and MSE, and equal to |R| (x:0]]]
1Enee Rio AnT, (2:0)]11z for the cosine loss. Eq. 3 presents that the expectation of synthetic sam- ples B[dnss, (r;; 0) ultimately converges to an average on expectations of all PA groups, weighted by their respective sample ratios r This indicates that vanilla DDs naturally favor majority groups, causing S to shift towards them and inherit their biases.

---

When original datasets suffer from PA imbalance, e.g., TVy > Ei the unfaimesss of the synthetic dataset stems from two different aspects: 1) The majority group renders synthetic samples to locate its region from Eq. 2) According to Eq. the large sample quantities of the majority group contribute more to the total loss. As result, minority groups experience higher loss during testing, which limits the model to represent them accurately. These factors prompt us to reduce the impact of PA imbalance on the generation of S.
5 FairDD
Overview In this paper, we propose a novel FDD framework that achieves both PA fairness and TA accuracy for the model trained on its generation S, regardless of whether the original datasets exhibit PA fairness. As illustrated in Fig. 2 FairDD first partitions the dataset into different groups w.r.t PA and then introduces an effective synchronized match- ing to equally align S with each group within T. Com- pared to vanilla DDs, which pull the synthetic dataset to- ward the majority group in the synthetic dataset, FairDD pro- poses group-level synchro- nized alignment, in which each group attracts the synthetic data toward itself, thus forcing it to move farther from other groups. This synchronized pull prevents the generation from collapsing into majority groups (fairness) and ensures class-level distribu- tional coverage (accuracy).
Vanilla
FDirDD
T
Ty
T
+1 alignment
+() aligument
[(Eleho-r (v; 6)), Bios..s, (()))i)
En.ca D(E (a:, ()), E/earss, (1;)
S Syathesized dataset
Original dataset
o(-)
Space of target signals
Figure 2: The overview of FairDD. FDirDD first groups target signals of T and then proposes to align S (random initialization) with respective group centers. With this synchronized matching, S is simultaneously pulled by all group centers in batch. This prevents the condensed dataset S from being biased towards the majority group, allowing it to better cover the distribution of T.
Synchronized matching As mentioned in Sec. 4 vanilla DDs fail to mitigate PA imbalance and even amplify the discrimination. The relation behind the failure is that the majority group dominates the generation direction of S and leads the resulting S inheriting the PA imbalance, i.e., preference to fitting to the majority group. To avoid the synthetic samples collapsing into the majority group, we decompose the single target (dominated by the majority group) into PA-wise sub-targets, and simultancously align& S with these sub-targets, without incorporating the specific sample ratio of each group into the optimization objective. The samples assigned one group have the same PA within the same class label. In this way, we obtain the unified objective function of FairDD:
(4) The reformulation forms synchronized matching, where different sub-tan sub-targets attempt pull S into their corresponding PA regions. Each PA group holds equal importance in generating S, ultimately converging balanced (fair) status. Subsequently, we present theoretical analysis illustrating how FairDD effectively mitigates PA imbalance and aligns TA distribution.
L (S00, 0,T) EyEr E.EAA D(B[6, ~TT (æ; 0),, Eld...s, (a; 0)).
Theorem 5.1. For any PA set A, network parameters and target signs s(-), LFair T) could mitigate the influence of PA imbalance of original datasets on generating synthetic samples. Especially when D(-) is MAE or MSE, synchronized matching ensures that the signal expectation of S is situated at the center of the expectation across all PA groups within T.
Proof. We assume that Eldass, (x; 0)] could reach the optimal solution for each class. Hence, have θL FF 0, Ty) /0/ ( Sows, (1;; 0) = 0:
E Eidb (x: 0)
A
Σ
Blg 21 (z; 0)]
(5)
a.EA
5

---

According to Eq. 5 the resulting Elδe~ss, (z; 8)] are independent on the sample ratio Ry, indicating the corresponding unaffected by Ry. As a result, the condensed will not be dominated by majority groups that happened in vanilla DDs. All PA centers contribute equally to the generation
of S, which succeeds in mitigating the PA imbalance of TT Especially when D(-) is MAE or MSE, the expectation of target signs of S is equal to the arithmetic mean of centers of all PA groups. This shows that S generated by FairDD is not biased towards any groups.
Although we mitigate the bias inheritance in vanilla DDs by synchronously aligning S to fine-grained PA-wise groups, it is also crucial to investigate whether DDair 7) (synchronized matching) ensures that the resulting S achieves comprehensive distributional coverage for T. As mentioned above, L(S; 8, T) matches S and T in global view to fully cover T's distribution. Below, we provide theoretical guarantee that Fair-DD(S 0,;0, T) could maintain comprehensive coverage compared to 01 T) when ((.,-) )) is a convex distance function, commonly used in diverse DDs
Theorem 5.2. For any PA set A and target signs (o(-), DrairDD(S. 0. (.) is the upper bound of vanilla unified objective C(S; 0, T),, i.e. LFarrDD(S;0, 0, T) 2 L(S: 0, T),, when D(,,,) is convex. Optimizing FD (S; 'T) can guarantee the comprehensive distribution coverage for TT
The proof is given in Appendix D ( 0, T) serves as the upper bound of L(S; 0, T), mean- ing that minimizing Crair DD (5; 0, T) ensures the minimization of L(S; T.). Hence, optimizing S in FairDD can guarantee the distributional coverage by bounding L(S; 0, 'T) tailored for accuracy.
6 Experiment
6.1 Experiment Setup
Datasets Comprehensive experiments are conducted on publicly available datasets with diverse types of bias, including foreground bias (FG), background bias (BG), combined BG & FG bias, and real-world bias. The evaluated datasets include synthetic datasets: C-MNIST (FG), C-MNIST (BG), FMNISS (FG), Colored-FMNIS. (BG), and CIFARIO-S (BG & FG), as well as real-world datasets: CelebA, UTKFaae, and BFFHQ. For more details on these datasets, please refer to Appendix B and C We also explore Tiny-ImageNet-S and ImageNet Subsets-S with the same operations as those performed on CIFARIO for CIFARIO-S.
Baselines & Evaluation metrics FairDD is a general fairness framework applicable to diverse DDs in DMF. We apply FairDD to diverse DMF approaches including DM method DM [69] and GM methods DC [70]. IDC (DC version) 277 and DREAM (DC version) [36. To provide an DEOM [0, 100) and D [o, 100] to measure the worst and average fairness levels. Also, we report accuracy(t) to assess the model's prediction of TA. We also provide comparison with MTT in Appendix R Sometimes, we will abuse DM+FairDD and FairDD for clarification.
overall evaluation for model bias toward PA,
Implementation details We default BR of 0.9 for all synthetic original datasets to induce signifi- cant PA skew. In Table 16 we conduct the ablation study on BR. All baselines are reproduced using official implementations. FairDD doesn't introduce extra hyperparameters or learnable parameters. Experiments are conducted on PoTcch 2.0.0 with single NVIDIA RTX 3090 24GB GPU.
6.2 Main results
We use distilled datasets S from different DDs to train and evaluate ConvNet with the same parameters, and then report the corresponding fairness and accuracy. Random refers to sampling defined IPC from the original dataset to create smaller datasets. Besides, Whole means train the model using the entire training dataset without distillation or sampling.
D significantly improves the fairness of vanilla DDs We provide comprehensive fairness comparisons across various DDs, including DM and DC. As illustrated in Table vanilla DDs fail to mitigate the bias present in the original datasets and even exacerbate unfairness towards biased groups In C-MNIST (FG), the distilled datasets from DM suffer from severe unfairness at IPC=10 compared
Emperical experiments show FairDD also can cover the TA distributions when D(:-) is not convex.

---

Table 1: Fairness comparison on diverse IPCs.
Methods Dataset
IPC
Random
DM
DM-FuirDD DEOO
DC+FuidD DEON
DC
IDC
IDC+FaiDD DREAM
+RarDD
Whole
150
DE00.
DEL0 DEO. DE00
DEO, DEO
C-MNIST
10
100.0
98.72
100.0
99.96
17.04
7.95
19.885
65.61
26.75
11.96
100.0
91.45
12.24
6.64
98.99
78.71
11.88 7.21
(FG)
50
100,0
99.58
100.0
91.68
10.05
5.46
46.99
20.55
18,42
8.86
65.34
34,91
9.18
5,94
52,03
26.63
18.37
7.50
10.10
5.89
100
100.0
88.64
99.36
66.38
8.17
4.86
45.27
17:45
22.32
9.49
64.36
35.82
11.88
6.21
69.30
33.30
11.88
6.88
C-MNIST
10
100,0
99.11
100.0
99.97
13.42
6.77
100.0
73.60
20.66
9.94
100.0
88.30
18.61
7.50
100.0
52.06
15.31
6.83
(BG)
100
100.0
100.0
89.07
99.77
10000
100.0
52.23
97.85
8.98
6.60
4.31
62,66
60.06
20.87
20.29
32.58
10.40
9.90
93.05
63.24
27.79
42.23
19.66
12:24
8.05
6.32
64.15
44.88
22,86
23.30
20.41
16.33
9.04
7.80
5.78
C-FMNIST
100.0
99.18
99.05
26.87
199.40
78.96
24.01
100.0
97.27
32.33
16.80
100.0
95.17
42.00
(FG)
100
50
100.0
100.0
94.85
94.61
100.0
100.0
85,11
96
/6
23.83
24.92
12.75
13.74
99.55
(9.
39.33
66.15
67.002 02.
56.68
46.67
23.07
21.48
100.0
100.0
79.10
81.93
48.33
40.00
17.37
17.43
97.333
99.67
70.10
K3.27 47.67
74,00
40.40
22.33
77.220
41.72
C-FMNIST
10
100.0
99.
100.0
99.68
33.05
19.72
100.0
92.91 61.75
34.88
100.0
99
42.00
23.80
100.0
94.70 36.00
2.3.50
(BG)
50
100,0
98.52
100.0
99.71
24.50
14.47
100.0
75.41
44.60
25,25
100.0
95.60
78.00
34.50
100.0
$8,40
34,00
23.70 91,40 51.68
100
100.0
96.05
100.0
93.88
21.95
13.33
99.770
73.38
231.48
100.0
77.00
36.00
100.0
83,90
40.00
23.20
10
25.04
8.29
59.20
39.31
31.75
8.73
112.23
27.35 35
22.08
8.22
80.70
48.38
19.90
5.28
51.80
31
20.80
7.77
CIFARIO-S
57.11
55.70
18.28
71.46
45.81
11.21
92.00
29.00
9.10
56.80
36.19
14.70
6.53
33.17
100
66.49
43.16
73.81
55,10
14.77
5.89
66..09
48.64
32.70
11,26
92.
60.93
62.80
28.18
82.30
48.12
12.10
6,066
10:48
9.20
28.8
9.37
15.48
14.16
5.29
34.85
34.48
8,36
4.49
40075
36.70
9.20
5.36
CelebA
50
22.88
20.32
40.26
38.81
14.08
9.87
24.89
23.83
14.33
12.92
56.74
46.50
22.57
15.15
43.57
38.53
23.62
14.29
24.85
24.16
100
18.67
18.01
42.63
41.12
10.93
6.65
29.00
27.52
18.16
17.04
50.99
42,66
28.27
17.63
52.51
39.34
24.87
15.36
26.00
18.47
55.40
34,87
37.40
21.60
+33.80
26.80
35.00
20,
66
$2.00
30.80
34.40
23.13
+16.00
29.87
32.40
24.93
UTKÉacce
50
40.60
25.27
43.60
33.13
23.60
17.27
38,
227
27.90
20.86
60
30.73
30.20
22.40
38.20
27.73
29,80
23.13 39.00
24.00
100
50.00
30.07
34.60
20.33
30,60
23.87
42,80
31.13
23.53
32.87
27.40
21.13
10
19.44
16.72
44.32
34.76
15.84
10.68
(47.04
38.92
37.84
28.84
58.00
51.92
21.28
12.44
60.96
52.76
25.52
19.36
BFTHQ
37.12
26.84
19.76
59.
51.24
52.24
42.48
70.64
13.92
10.88
60.04
31.28
30,00
66.40 55.20
100
43.12
36.20
65.36
53.12
17.52
13.32
660.5
49.76
54.72
46.64
66.24
60.68
14.96
8.20
65.76
59.50
10.24
7.12
Table 2: Accuracy comparison on diverse IPCs.
Table 3: Cross-arch. comparison.
Methods Datasets
IPC
Randomi
Acc.
DM
+FairDD
Aue:
DC
+FairDD Aue:
IDC
+FairDD DREAM (Whoe
Auc:
Medied
Choss
1000
DMsFriDD
7o0
ConnNet
10300
91.08
56.54
10:05
5.46
96.55
C-MNIST
(FG)
50
10
47.38
67.41
30.75
25.01
56.84
78.01
96.58
94.61
96.79
71.11
90.54
91.61
92.68
93.23
90.62
53.06
90.39
58.55
96.77
95.67
97.11
91.02
75.04
94,04
94.59
95.16
97.71
C-NNIST (FG)
Nol1
AletNee
YOGH
99.20
100.0
1030
56.50
00.25
05.23
64.02
50.05
10.35
9.55
8,40
6.36
5,50
4.63
96.12
96.00
97.13
199.0
C-MNIST (BG)
100
50
10
27.95
45.52
67.28
23.10
47.74
19.87
96.86
97.33
94.88
665.91
88.53
90.20
92.20
90.84
92.73
62.09
86,14
89.66
95.29
95.84
94.84
89:24
79.81
90.70
93.54
93.20
97.80
CAEMNNST (BG)
Coen/Nll
AleeNer
YOGH1
1030
100.0
10000
00.77
99:75
99.71
43.11
36.27
27.77
31,60
34,50
30,60
14.36 78.57
14.11
14.47 79:07
76.14
C-FMNIST (FG)
50
10
42.48
32,80
333.35
19.94
82.11
77.09
660.77
69.08
75.83
76.01
14.008
61.45
$0.80
79.66
65.60
49.72
77.24
78.79
82.94
Ro
Moa
10000
10.25
33.50
14.73
14.36
77.15
75.11
55.31
$7.99
83.25
66.884
74.91
66.37
$0.28
68.25
CoenNel
75.13
55.70
45.02
18.28
7.35
55.54
AlcoNia
75:30
52.57
56.00
15.84
5,12
49.16
C-FMNIST
10
24.
34.92
123.26
36.27
79.07
71.10
147.32
60.58
75.80
68.51
37.55
46.20
73.72
72.67
53.62
45. 30
71.56
72,80
77.97
CIFARIO
ReeNetl
YGGH
70.23
11.51
4.16
5.14
$2.65
50.95
44.87
80.63
62..7
71.76
48.61
73.18
53.32
73.00
Moo
122.04
51.67
40.59
15.27
5.44
$2.90
CIFARIO-S
10 23.60
36.46
137.88
45.02
58.84
45.17
33.888
41,28
49.26
41.82
48.30
47.26
56.40
57,84
57.59
$5.09
61.85
58,40
69.78
CdsbA
CrowNet
AlcoNes
VOG11
26.03
32.51
00:26
31.62
36.51
61.61
63.10
61.57
9.38
16:08
9.87
5775
6.32
62:65
64.34
6k.00
39.31
61.33
12.73
51.74
17.27
56.98
57.14
RoNNot
10
$4,51
661.79
64.37
19
57.63
661.19
63.54
64.38
66.26
Mean
[31.10
30.25
02.40
9.78
6.58
64.15
ColebA
55.99
64.61
68.50
60.11
59.89
60.75
66.89
64.62
68.26
74.09
Com/Nel
43:60
32.15
63.94
13,00
17.21
71.75
60.62
66
68.84
622.53
61.89
61.04
67.24
62.58
UTKo0e
VOG11
AlcoNe
45.40
33.40
21,43
21.96
69:36
67.14
10
46.62
665.
66.92
158.52
60.01
667.05
67.85
67.75
67.68
RoNoll
UTKFaw
50
59,70
68. 94
71.75
69,00
70.28
69.82
69.95
71.97
71.12
78.67
Mean
[47.68
31.63
65.97
31,30
20.09 68.76
63.87
71.27
73.70
66.88
67.65
72.75
69.43
70.13
ConnNat
60.88
50.56
65:23
19:76
14.96 69.60
10
$7.40
664.0
65.46
622.22
63,30
65.52
68.70
64.32
63.94
AlesNes
35.96
12:95
68:71
BFFHQ
61,78
65.
69.00
61162
68.04
70.64
70.50
63.01
66.60
71,40
RoNNil
96.55
46.06
62.00
1811
14,34 65.47
100
62.94
66.20
73.74
67.40
68.72
63.16
70.50
62.74
Mon
57.71 46.47
6199
31.41
14.55 67.34
to Whole, with DEOM and DEO, reaching 100.0 and 99.96 vs. 10.10 and 5.89. In some cases, Random presents better fairness than vanilla DDs, particularly when dealing with complex objects like CelebA. This suggests that while vanilla DDs effectively condense information into smaller samples, their inductive bias, which favors the majority group, worsens the fairness to the minority group. However, when FairDD is applied to vanilla DDs, there is a significant improvement in fairness performance, with DEOM dropping substantially from 100.0 to 17. 17.04, and DEO, decreasing from 99.96 to 7.95 in C-MNIST (FG). This indicates that FairDD's synchronized matching ensures the equal treatment of each group, effectively mitigating the bias that vanilla DDs exacerbate. FDirDD further reduces the bias originally present in the original datasets. For example, DC + FairDD outperforms Whole in C-FMNIST (FG) and CIFAR10-S, as well as in the real-world dataset CelebA, achieving the overall improvement on DEOM and DEO, metrics. Similar performance gains are also observed in other baselines.
7

---

FairDD maintains the comparable and even higher accuracy than vanilla DDs A faimess framework must maintain TA aceuracy in addition to improving fairness across PA groups. We report the TA accuracy of FairDD in comparison to other baselines in Table 2 Compared to Random, training the model by vanilla DDs yields better performance. This shows that vanilla DDs capture the informative patterns of majority groups, improving their TA accuracy. However, by focusing on dominant patterns in majority groups, they neglect the important patterns in minority groups within the training datasets. Thus, their representation coverage is limited. In contrast, FairDD proposes synchronized matching to push the S to cover each group, and as a result, the generated S retains key patterns of all groups and achieves comprehensive coverage. For example, DM obtains 25.01 at IPC = 10 on C-MNIST (FG), and its accuracy boosts to 94.61 when applying FairDD. In real-world CelebA. FairDD obtains comparable performance for DC and presents superiority over vanilla DDs. These demonstrate that FairDD could mitigate the bias without compromising accuracy.
Generalization to diverse architectures Here, we investigate the cross-model generalization of FairDD, where ConvNet is used to condense datasets, and we evaluate S on other architectures, including AlexNet, VGGI and ResNet18. We compare DM and FairDD across four datasets at IPC = 500 evaluating performance against BG, FG, BG & FG, and real-world biases. As shown in Table among these architectures, FairDD achieves DEOM of 10.05, 10.35. 9.55, and 8.40 on C-MNIST (FG), DEO, of 14.47, 14.11, 14.36, and 14.96 on C-FMNIST (BG), and accuracy of 58.84, 49.16, 52.65, and 50.93 on CIFAR I0-S. These steady results suggest that S generated by FairDD) is not restricted to the model used for distillation but generalizes well across diverse architectures. Additionally, with the model capacity increasing, the model generally tends to be more fair to all groups. However, the accuracy sometimes decreases, such as when it drops from 58.84 (ConvNet) to 50.93 (ResNet18) 8) in CIFARIO-S and from 68.50 (ConyNet) to 61.80 (ResNet 18) in CelebA. We assume that while increased attention from larger models can lead to accuracy gains for minority groups it may limit the representations for majority groups at certain levels. The accuracy gains for minority groups may be smaller than the accuracy losses for majority groups, particularly in larger models that have limited potential improvement in recognizing minority groups
Discussion FairDD employs a bias-free alignment objective, where the expectation of the distilled data is unbiased across all groups, as shown in Eq. S To ensure faimess during optimization, each PA group contributes equally to the total loss, with gradients that are independent of group sample sizes. This design leads to balanced sample generation across all protected groups. Regarding TA, the loss used in vanilla DDs is designed to align with TA-wise classification accuracy. In FairDD, as proven in Thm. T.2. the loss function serves as an upper bound on the loss of vanilla DDs. This implies that minimizing the FairDD objective ensures distributional coverage across TA. As result, models trained on FairDD- maintaining stable performance on majority groups. The main takeaways of FairDD are summarized as follows: Fairness-Aligg Aligned Objective: The distilled dataset should be constructed such that its expected representation is unbiased across all attribute groups, aligning with fairness principles at the objective level; Equal Group Contribution: encouraging balanced optimization and preventing group-specific bias during training; Distributional Coverage of TA: It is essential to ensure that the distilled dataset maintains adequate distributional coverage over the TA, supporting both fairness and classification accuracy.
datasets achieve higher accuracy on minority PA groups, while
Each group should contribute equally to the overall loss,
6.3 Result Analysis
Visualization analysis on fairness and accuracy To intuitively present the effectiveness of FairDD, we train Pg using S of C-MNIST (FG) distilled by DM and FairDD, and then extract the features from the test dataset. Different colors paint these resulting features according to PA and TA, respectively. As shown in Figs. 3aa and 3(b) features with the same PA tend to form cluster, indicating that the model trained on DM is sensitive to PA and thus failing to guarantee fairness among all PA. In contrast with DM, the feature distributions in Fig. ((b) exhibit nearly complete overlaps across all PA. shows that the model trained on FairDD is agnostic to PA and does not exhibit bias towards these PA. Besides the PA fairesss, we also study the feature distribution from the TA perspective. Fig. 3(c) shows that features belonging to one TA scatter and fail to provide compact representations for one class. The failure of DM can be attributed to model bias toward PA. Combined with Fig. 3(a] it can be observed that PA has a stronger influence on the feature distribution compared to TA, As result, PA-wise representations are tightly clustered, but representations from the same TA are divided into
8

---

(a) I-SNE of DM. (b) PA I-SNE FairDD. (c) TA L-SNE of DM. (d) TA I-SNE of FairDD.
Figure 3: T-SNE visualization towards test features. Color represents distinct PA groups in (a) and (b), and TA labels in (c) and (d). In (a), DM shows obvious distinctiveness towards different PA. But (b) shows DM+FairDD eliminates the recognition of PA. In (c) and (d), DM+FairDD enables compact TA representations, but DM tends to cluster features with the same PA.
Table 5: Ablation on fair extractor.
Table 6: Ablation on initialization at IPC = 50.
Methods
JPC
DM+FRarDD
DM-LW
DM+LIF
Methods
Init.
DM
DM-FairDD DED,
DEOM DEO, Auc. DE0gg
DEO,
Dutaee
DE0M
DEO,
DE0g
C-MNIST
94.61
100.0
23.76
22.75
C-NNIST
Proportion
100.
91.68
56.84
10.05
5.46
96.58
(PG)
96.58
48.28
1000
91.57 56.23
(PG)
Balanced
1000
96.41
9.80
C
71.10
21.54
21.74
C-PNNIST (BC)
Proportion Balanced
1000
99.71
79.07
(BG)
50
24.50
14.47
79.07
100.0
99.75
33.13
1000
99.66
36.23
100.0
99.62
30.94
24.85
14.19
79.98
CIFARIO-S
31,75
8.73
45.17
61.21
42.15
37.26
77,83
59.22
43.47
CIFARIOSS
Propunion
75.13
55.70
45.02
18.28
7.35
58.84
50
18.28
7,35 58.84
00:73
41.84
36,88
75.35
38.51
43.68
Balanced
76.21 52,31
45.97
19.19
6.51
58.82
PA-wise parts. In contrast, FairDD proposes synchronized matching effectively mitigates this by treating each PA group equally within one TA. The equal treatment allows different PA groups within the same TA to cluster more easily, leading to more compact representations that benefit capturing class semantics in Fig. 3(d) These results highlight the superiority of FairDD in improving PA fairness and TA accuracy. Additional analysis on computation overhead and representation coverage on generation are provided in Appendix and F We also visualize on S generation Appendix G
Exploring the scalability on arge datasets To further evaluate the scalability of FairDD, we conduct experiments on large datasets such as ImageNet Subset and Tiny-ImageNet. To introduce bias, we apply the same procedure used for CIFARIO-S, resulting in biased versions: ImageNet Subset-S and Liny-ImaeNNet-.. The results are shown in Table As observed, FairDD outperforms the vanilla DDs on these datasets, demonstrating its superior scalability.
Table 4: Scalability on ImageNet-serie datasets at IPC = 10.
ImageNet Subset S
Tmy--IugoNNe-S
Nete
Fruit
Meow
Squawk DED,
Yollow
Methods
DED,
Ace.
D100
DBO
Acc.
DBUs
DED
Acce.
DBUM
DEO,
Avc.
DE0
Acc.
D150s
DBU,
Avc.
DBOM
DEU, Acc.
DM
60.05
16.15
12.24 60.08
30.88
18.66
11.04
27.27
22.44
56.08
30.83
25.01
52.04
35.02
34.07
60.02
40.52
32.85
55.78
12:26
7.30
DM-FaiDD
32,05
15.26
45.61
32.09
19.66
22.28
32.07
13.64
22.84
40.04
16.02
26.05
32,05
19.68
35.49
36,04
15.28
38.82
46.21
7.55
7.78
40.68
45.64
23.63
47.60
30.06
54.43
10:33
9.09
De-FuirDD
34.09
13.56
44.83
46.06
16.36
22.34
35.25
14.88
22,88
40.80
19.04
21.98
34.00
13.12
35,00
47.20
13.44
38.92
48.45
6.56
9.65
6.4 Ablation Study
Ablation on fair extractor General DDs treat the extractor as non-linear transformation, where the randomly initialized extractor either does not require training or only updates parameters after a few iterations. Here, we investigate whether vanilla DDs can mitigate bias when the extractor is fair to PA. We employ two fairness approaches LW and LfF to train the extractor fairly to PA (43] Then, we use the extractor for condensation, resulting in DM+LW and DM+LfF models. From [5] we can observe that DM+FairDD still outperforms DM+LW and DM+LfF on both fairness and accuracy across FG, BG, and FG&BG. Although they use fair extractor towards PA, which helps provide a balanced feature space, Eq, 3 illussrates that vanilla DDs shift synthetic datasets toward the majority group. This biased shift still causes S to inherit the bias of the original dataset during the condensation. In contrast, FairDD agnostic to whether the extractor is fair and consistently mitigates the bias in the condensed dataset.

---

Table 7: Ablation on lairness-wware learning.
DC
DCFFiiDD
DCAMF
DC-DF
Dinaset
Acc.
DEO-M DEO-A
Aec. DEO-M DEO-A
Acc.
DEO-M DEO-A Acc. DROM DEO-A
CMNIST-BG (65991
100.00
73.60
19084
20.66
9,94
667.0
99.50
70.60
82.45
76.44
42.05
CIFARIO-S
(7.888
42,25
27.35
(11822
22,08
8.22
19.47
35,86
22.04
(40.00
27.20
10.29
Ablation on initialization of synthetic images The initialization of S determines the prior in- formation obtained by DDs. We examine the effect of different initialization using three strategies: random: randomly drawing samples from the original datasets to initialize S; Noise: using noise obeying the standard normal distribution for initialization; and balanced: initializing with the equal number of each group. In 6 DEOM and A metries of DM suffer from the bias present in the original dataset across these strategies. Especially in balanced, we keep the synthetic dataset without group imbalance, vanilla DDs still inherit the imbalance from the original dataset. This again demonstrates the disadvantage of vanilla DDs when condensing biased datasets, In contrast, FairDD achieves robust performance in fairness and accuracy.
Ablation on fairness-aware learning in vanilla DDs In this work, we explore Fairness-aware learning for vanilla models through distillation fairness (DF) and model training fairness (MF). During distillation, DF assigns weights to each group inversely proportional to its sample proportion to achieve fairness-aware learning, while keeping the rest of the training procedure unchanged. MF, on the other hand, keeps the distillation process unchanged and applies fairness-award learning only during the model training stage. We apply DF to the DC framework and obtain DC+DF, and similarly incorporate MF into DC to derive DC+ME. As illustrated in Table applying these fairness regularizations can alleviate distillation bias to some extent; however, they do not ensure that the alignment objective remains unbiased across all attribute groups, nor do they guarantee comprehensive coverage of the target attribute distribution. Consequently, the distilled data of certain minority groups may be lost due to biased distillation.
We provide summary to guide the reader through the appendix in Appendix A We conduct performance comparison with [13 in Appendix E and more ablation study on weighting mechanism in Appendix H. additional experiments on CelebA in Appendix I computation overhead in Appendix[]] attribute missing in test dataset in Appendix KJ ablation study on the biased ratio of original datasets in Appendix L group label noise and missing in Appendix M balanced original dataset Appendix NJ nuanced PA groups in Appendix [ imbalanced
groups in Appendix PP] exploration on vision transformer as the backbone in Appendix Q
comparison with MTT in Appendix R
Conclusion
This is the first work to introduce attribute fairness into the field of dataset distillation and to systematically provide a theoretical analysis of why vanilla dataset distillation fails to mitigate attribute bias. To address the problem, we propose unified fair dataset distillation framework called FairDD, broadly applicable to various DDs in DMF. FairDD requires no modifications to the architectures of vanilla DDs and introduces an pasy-to-implemnt yet effective attribute-wise matching. This method mitigates the dominance of the majority group and ensures that synthetic datasets equally incorporate representative patterns with all protected attributes from both majority groups and minority groups. By doing so, FairDD guarantees the fairness of synthetic datasets while maintaining their representativeness for image recognition. We provide extensive theoretical analysis and empirical results to demonstrate the superiority of FairDD.
Since FairDD relies on PA's prior information to conduct attribute- wise matching, is valuable explore the scenario where PA is unavailable [4 A potential solution is generate pseudo-labels to guide FairDD through self-supervised learning or unsupervised learning.
Limitations
Broader Impacts This paper aims to improve data efficiency and enhance data faimess in modern machine learning, fully compliant with legal regulations. Since training a fair model from scripts with extensive data is time-consuming, our work in providing a fair condensed dataset for effective model training can have significant societal impacts. We hope our research raises attention achieving fairness and accuracy for dataset distillation in academia and industry.
10

---

Acknowledgments
This work was supported by NSFC 62088101 Autonomous Intelligent Unmanned Systems and NSFC U23A20326. We thank Joey Tianyi Zhou for feedback on the draft. We also thank Ninghao Liu for helpful discussion.
References
[1] Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna Wallach, A reductions approach to fair classification. In International Conference on Machine Learning. pages 60-69. PMLR, 2018.
[2] Wael Alghamdi, Shahab Asoodeh, Hao Wang, Flavio P Calmon, Dennis Wei, and Karthikeyan Natesan Ramamurthy. Model projection: Theory and applications to fair ma- chine learning. In 2020 IEEE International Symposium Information Theory (ISIT), pages 2711-2716. IEEE, 2020.
[3] Rachel KE Bellamy, Kuntal Dey, Michael Hind, Samuel C Hoffman, Stephanie Houde, Kalapriya Kannan, Pranay Lohia, Jacquelyn Martino, Sameep Mehta, Aleksandra Mojsilović, et al. Ai fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias. IBM Journal of Research and Development, 63(4/5):4-1, 2019.
[4] Ondrej Bohdal, Yongxin Yang, and Timothy Hospedales. Flexible dataset distillation: Learn labels instead of images. arXiv preprint arXiv:2006. 08572, 2020.
[5] Simon Caton and Christian Haas. Fairness in machine learning: survey, ACM Computing Surveys, 56(7):1-38, 2024.
[6] George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A Efros, and Jun- Yan Zhu. Dataset distillation by matching training trajectories. Proceedings of the IEEE/CVF Confer- ence Computer Vision and Pattern Recognition, pages 4750-4759, 2022.
[7] Xuechao Chen, Wenchao Meng, Peiran Wang, and Qihang Zhou. Distributed boosting: An enhancing method on dataset distillation. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, CIKM '24, page 3689-3693, New York, NY, USA, 2024. Association for Computing Machinery.
[8] Xuechao Chen, Wenchao Meng, Peiran Wang, and Qihang Zhou. Distributed boosting: An enhancing method on dataset distillation. CIKM '24, page 3689-3693, New York, NY, USA, 2024. Association for Computing Machinery.
[9] Xuxi Chen, Yu Yang, Zhangyang Wang, and Baharan Mirzasoleiman. Data distillation can be like vodka: Distilling more times for better quality. arXiv preprint arXiv:2310.0660 2023.
[10] Ming-Yo Yu Chung, Sheng-Yea Yen Chou, Chia-Mu Yu, Pin-Yu Chen, Sy-Yen Kuo, and Tsung- Yi Ho. Rethinking backdoor attacks on dataset distillation: kernel method perspective. arXiv preprint arXiv:231 111. 1. 16646, 2023.
[11] Elliot Creager, David Madras, Jömn Henn-K Jacobsen, Marissa Weis, Kevin Swersky, Toniann Pitassi, and Richard Zemel. Flexibly fair representation learning by disentanglement, In International conference machine learning, pages 1436-1445. PMLR, 2019.
[12] Justin Cui, Ruochen Wang, Si Si, and Cho-Jui Hsieh. Scaling up dataset distillation to imagenet-
Ik with constant memory. arXiv preprint arXiv:2211.10532 2022.
[13] Justin Cui, Ruochen Wang, Yuanhao Xiong, and Cho-Jui Hsieh. Ameliorate spurious correla- tions in dataset condensation. arXiv preprint arXiv:2406.06600 2024.
[14] Justin Cui, Ruochen Wang, Yuanhao Xiong, and Cho-Jui Hsich. Mitigating bias in dataset distillation. arXiv preprint S11::24000066000 2024.
[15] Zhiwei Deng and Olga Russakovsky, Remember the past: Distilling datasets into addressable memories for neural networks. arXiv preprint arXiv:2206.0200 2022.

---

[16] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. ArXiv, abx/2010.11929, 2020.
[17] Yunzhen Feng, Shanmukha Ramakrishna Vedantam, and Julia Kempe. Embarrassingly simple dataset distillation. The Twelfth International Conference on Learning Representations, 2023.
[18) Ziyao Guo, Kai Wang, George Cazenavette, Hui Li, Kaipeng Zhang, and Yang You. To- wards lossless dataset distillation via difficulty-aligned trajectory matching. arXiv preprint arXiv:2310.05773, 2023.
[19] Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29, 2016.
[20] Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. 2016 IEEE Conference Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2015.
[21] Yang He, Lingao Xiao, Joey Tianyi Zhou, and Ivor Tsang. Multisize dataset condensation. In The Twelfth International Conference on Learning Representations, 2024.
[22] Yang He and Jocy Tianyi Zhou. Data-independent module-aware pruning for hierarchical vision transformers. In The Twelfth International Conference on Learning Representations, 2024.
[23] Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and generalization in neural networks. Advances in neural information processing systems, 31, 2018.
[24] Heinrich Jiang and Ofir Nachum. Identifying and correcting label bias machine learning. In International Conference Artificial Intelligence and Statistics, pages 702-712. PMLR, 2020.
[25] Sangwon Jung, Sanghyuk Chun, and Taesup Moon. Learning fair classifiers with partially annotated group labels. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 10348-10357, 2022.
[26] Sangwon Jung, Donggyu Lee, Taeeon Park, and Taesup Moon. Fair feature distillation for visual recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12115-12124 2021.
[27] Jang-Hyun Kim, Jinuk Kim, Seong Joon Oh, Sangdoo Yun, Hwanjun Song, Joonhyun Jeong, Jung-Woo Ha, and Hyun Oh Song. Dataset condensation via efficient synthetic-data param- eterization. In International Conference on Machine Learning, pages 11102 1102-11116 11118. PMLR, 2022.
[28] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitchead, Alexander Berg, Wan- Yen Lo, al. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4015-4026, 2023.
[29] Yann LeCun, Corinna Cortes, Chris Burges, et al. Mnist handwritten digit database, 2010.
[30] Dong Bok Lee, Seanie Lee, Joonho Ko, Kenji Kawaguchi, Juho Lee, and Sung Ju Hwang. self-supervised dataset distillation for transfer learning. In The Twelfth International Conference on Learning Representations, 2023.
[31] Hae Beom Lee, Dong Bok Lee, and Sung Ju Hwang. Dataset condensation with latent space knowledge factorization and sharing. arXiv preprint arXiv:2208. 10494, 2022.
[32] Saehyung Lee, Sanghyuk Chun, Sangwon Jung, Sangdoo Yun, and Sungroh Yoon. Dataset condensation with contrastive signals. International Conference on Machine Learning, pages 123641 PMLR, 2022.
12

---

[33] Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language- image pre-training for unified vision- -anguaga understanding and generation. In International conference on machine learning, pages 12888-12900. PMLR, 2022.
[34] Evan Zheran Liu, Behzad Haghgoo, Annie S. Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. ArXiv, abs/2107.09044. 2021.
[35] Songhua Liu, Kai Wang, Xingyi Yang. Jingwen Ye, and Xinchao Wang. Dataset distillation via factorization. arXiv preprint arXiv:2210. 2022.
[36] Yanqing Liu, Jianyang Gu, Kai Wang. Zheng Hua Zhu, Wei Jiang, and Yang You. Dream: Effi- cient dataset distillation by representative matching, 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 17268-1727, 2023.
[37] Noel Loo, Ramin Hasani, Alexander Amini, and Daniela Rus. Efficient dataset distillation using random feature approximation. Advances in Neural Information Processing Systems, 35:13877-1389,, 2022.
[38] Noel Loo, Ramin Hasani, Mathias Lechner, Alexander Amini, and Daniela Rus. Understanding reconstruction attacks with the neural tangent kernel and dataset distillation. arXiv preprint arXiv:2302.01428, 2023.
[39] Christos Louizos, Kevin Swersky, Yujia Li, Max Welling, and Richard Zemel. The variational fair autoencoder. arXiv preprint arXiv: 151 2015.
[40] Yao Lu, Jianyang Gu, Xuguang Chen, Saeed Vahidian. and Qi Xuan. Exploring the impact of dataset bias on dataset distillation. ArXiv, abs/2403. 2024.
[41] Yao Lu, Jianyang Gu, Xuguang Chen, Saeed Vahidian, and Qi Xuan. Exploring the impact of dataset bias on dataset distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7656-7663, 2024.
[42] Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. ACM computing surveys (CSUR), 54(6):1-35, 2021.
[43] Jun Hyun Nam, Hyuntak Cha. Sungsoo Ahn, Jacho Lee, and Jinwoo Shin. Learning from failure: Training debiased classifier from biased classifier. ArXiv, abs/2007.02560 .22611, 2020.
[44] Timothy Nguyen. Roman Novak, Lechao Xiao, and Jachoon Lee. Dataset distillation with infinitely wide convolutional networks. Advances in Neural Information Processing Systems, 34:5186-5198, 2021.
[45] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07/ 07.000799933 193, 2023.
[46] Novi Quadrianto, Viktoriia Sharmanska, and Oliver Thomas. Discovering fair representations in the data domain. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8227-8236, 2019.
[47] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning. pages 8748 8763. PMLR, 2021.
[48] Harsh Rangwani, Sumukh K Aithal, Mayank Mishra, and R. Venkatesh Babu. Escaping saddle points for effective generalization on class-imbalanced data. ArXiv, abs/2212.13822 2022.
[49] Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. ArXiv, 110/191 2019.
13

---

[50] P. Sattigeri, S. C. Hoffman, V. Chenthamarakshan, and K. R. Varshney. Fairness gan: Generating datasets with fairness properties using a generative adversarial network. IBM Journal of Research and Development, 63(4/5):3:1334) 2019.
[51] Prasanna Sattigeri, Samuel C Hoffman, Vijil Chenthamarakshan, and Kush R Varshney. Fairness gan: Generating datasets with fairness properties using a generative adversarial network. IBM Journal of Research and Development, -3/5)33--, 2019.
[52] Xinyi Shang, Peng Sun, and Tao Lin. GIFT: Unlocking full potential of labels in distilled dataset at near-zero cost. In The Thirleenth International Conference Learning Representations, 2025.
[53] Shivashankar Subramanian, Afshin Rahimi, Timothy Baldwin, Trevor Cohn, and Lea Frermann. Fairness-aware class imbalanced learning. In Conference on Empirical Methods in Natural Language Processing, 2021.
[54] Ilia Sucholutsky and Matthias Schonlau. Soft-label dataset distillation and text dataset distilla- tion. In 2021 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEE, 2021.
[55] Davoud Atace Tarzanagh, Bojian Hou, Boning Tong, Qi Long, and Li Shen. Fairness-aware class imbalanced learning on multiple subgroups. Proceedings of machine learning research, 216:2123-2133, 2023.
[56] Saeed Vahidian, Mingyu Wang. Jianyang Gu, Vyacheslav Kungurtsev, Wei Jiang, and Yiran Chen. Group distributionally robust dataset distillation with risk minimization. arXiv preprint arXiv:2402.04677 2024.
[57] Robin Vogel, Mastane Achab, Stéphan Clémençon, and Charles Tillier. Weighted empirical risk minimization: Sample selection bias correction based on importance sampling. ArXiv, abs/2002.05144 155,, 2020.
[58] Kai Wang, Bo Zhao, Xiangyu Peng, Zheng Zhu, Shuo Yang, Shuo Wang. Guan Huang, Hakan Bilen, Xinchao Wang, and Yang You. Cafe: Learning to condense dataset by aligning features. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 2219-112200 2022.
[59] Shaobo Wang, Yantai Yang, Qilong Wang, Kaixin Li, Linfeng Zhang, and Junchi Yan. Not all samples should be utilized equally: Towards understanding and improving dataset distillation, 2024.
[60] Tongzhou Wang, Jun- Zhu, Antonio Torralba, and Alexei A Efros. Dataset distillation. arXiv preprint arXiv: 10959, 2018
[61] Zeyu Wang. Klint Qinami, Ioannis Christos Karakozis, Kyle Genova, Prem Nair, Kenji Hata, and Olga Russakovsky. Towards faimess in visual recognition: Effective strategies for bias mitigation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8919 8928, 2020.
[62] Lingao Xiao and Yang He. Are large-scale soft labels necessary for large-scale dataset distil- lation? In The Thirty-eighth Annual Conference Neural Information Processing Systems, 2024.
[63] Depeng Xu, Shuhan Yuan, Lu Zhang, and Xintao Wu. Fairgan: Fairness-aware generative adversarial networks. 2018 IEEE International Conference on Big Data (Big Data), pages 570-575, 2018.
[64] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogrigue7, and Krishna P Gummadi. Fairness constraints: Mechanisms for fair classification. In Artificial intelligence and statistics, pages 962-970. PMLR, 2017.
[65] Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork, Learning fair representa- tions. In International conference machine learning, pages 325-333. PMLR, 2013.
14

---

[66] Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 335-340, 2018.
[67] Bo Zhao and Hakan Bilen. Dataset condensation with differentiable siamese augmentation. In International Conference on Machine Learning, pages
PMLR, 2021.
[68] Bo Zhao and Hakan Bilen. Synthesizing informative training samples with gan. In NewrIPS 2022 Workshop Synthetic Data for Empowering ML Research, 2022.
[69] Bo Zhao and Hakan Bilen. Dataset condensation with distribution matching. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 6514-6523, 2023,
[70] Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. Dataset condensation with gradient matching. arXiv preprint 2xXiv:2006.05929, 2020.
[71] Zhenghao Zhao, Haoxuan Wang, Yuzhang Shang, Kai Wang, and Yan Yan. Distilling long-tailed datasets, 2024.
[72] Qihang Zhou, Guansong Pang, Yu Tian, Shibo He, and Jiming Chen. Anomalyclip: Object- agnostic prompt learning for zero-shot anomaly detection, In The Twelfth International Confer- ence on Learning Representations, 2023.
[73] Yongchao Zhou, Ehsan Nezhadarya, and Jimmy Ba. Dataset distillation using neural feature regression, arXiv preprint SoXiv:220660071199 2022.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction precisely reflect the contribution and scope of this paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have created a separate "Limitations" section in our paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
16

---

Justification:
We provide a correct proof for our theoretical results in Appendix D
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide a detailed illustration our proposed algorithm and baselines in the Appendix B and Appendix C
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
17

---

Answer: [No]
Justification:
We will make our code and dataset available once the paper is accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
lc for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ /pippe.cc/pullic/guuii wwwpc//ggue////eeeeceoocc CoodeSubeiiieeee ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provide full details in Section 6
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We report the average results across five runs.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
18

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes] We point out the specific compute resources in Section 6
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttps://e http://ppp....pp.....//cccccttta
Answer: [Yes]
Justification: Our paper obeys the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the broader impacts of paper.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
19

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper poses such risks
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We respect the Licenses for existing assets that we use.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppswpptseeee www.rreswthhhe....c. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
20

---

Answer: [NA]
Justification:
We will release new assets proposed in our paper once the paper is accepted.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects, adapt
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
21

---

the paper does not use LLM to impact the core methodology, scientific rigorousness, or originality of the research.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy hattps://neurips.. 225p:///e////oeeeeeee/////2////0 for what should or should be described.
22

---

A
Appendix summary
We summarize the appendix contents follows:
Dataset details: Appendix
B
Dataset statistics: Appendix
Proof of the theorem: Appendix
Additional performance comparison: Appendix
Visualization analysis on representation coverage: Appendix F
Visualization analysis on S generation: Appendix
Ablation study on weighting mechanism: Appendix
H
Additional experiments on CelebA: Appendix
Computation overhead: Appendix J
Attribute missing in test dataset: Appendix
Ablation study on biased ratio of original datasets: Appendix
Group label noise and missing labels: Appendix
Balanced original dataset: Appendix
Nuanced PA groups: Appendix 0
Imbalanced PA groups: Appendix
P
Exploration using Vision Translormer as the backbone: Appendix Q
Comparison with MTT: Appendix R
More visualizations: Appendix
B
Datasets
Comprehensive experiments have been conducted on publicly available datasets of diverse biases, including foreground bias (FG), background bias (BG), BG FG bias, and real-world bias. C-MNIST (FG) is a variant of MNIST [29 used to evaluate model fairness, where the handwriting numbers in each class are painted with ten different colors. To correlate the TA (digital number) and PA (color) within the training dataset, each training class is predominantly associated with one color according to the same biased ratio (BR), while the remaining samples are evenly painted with the other nine colors. BR is the ratio of the majority group samples the total samples across all groups. For the test dataset, we evenly paint the numbers for each class with ten colors to test the model bias trained S. C-MNIST (BG) adopts the same operation the background and keeps the foreground unchanged. Colored-FMNIST (FG) is the modified version of Fashion-Ne MNIIST originally aiming to classify object semantics. Like C-MNIST (FG), color the objects for the training and test datasets. ColOSed-FMNIST (BG) paints the background similarly to C-MNIST (BG). CIFAR 10-S (BG & FG) introduces PA by applying grayscale not 10 samples. Following (6]. grayscale portion of the training images, correlating TA and PA among different classes. For fairness evaluation, we duplicate the test images, apply grayscale to the copies, and add them the test dataset. We also test FairDD on the real-world facial dataset CelebA, a widely used fairness dataset. We follow the common practice of treating attractive attribute as TA and gender as PA (evaluations on other attributes refer to Appendix IT.
C Datasest statistics
In this section, we provide detailed statistics for all datasets used in the manuscript for reproduction. As shown in Table we present the target attribute (TA), protected attribute (PA), the sample number of the training set, the sample number of the test set, and the BR in the training set. Additionally, all test sets are balanced, with equal sample sizes across groups. We also report the condensed ratio at IPC 10, 50, and 100, which computed by the ratio the condensed dataset size the training set size.
23

---

Table 8: Statistics for all datasets used our paper.
Daiasets
TA
PA
TA number
PA number
Training set size
Test set size
BR in Training set
BR Test sel
10
Condensed miio
50
100
C-MNIST (FG)
Digita sumber
Digiol color
10
60000
10000
0.90
bolaane
00.17%
0.83%
1.67%
CANIST
sumber
Backgroand
66000
10000
0.90
bolance
0.1755
0.83%
FMNNSTT (PG)
Object
Object color
63000
10000
0.90
bolance
0.17%
0.83%
1.6%%
C-FMNIST
cargoay
Bockgroand
60000
10000
0.90
bilancz
0.17%
0.83%
CIFARIO-S Object comgory
Gnescale
0.90
bolance
0.20%
Attractive
162770
classt::
belance
0/0124
0.0615)
clase0:
UTKfnee Age
Race
20813
1200
class 0.35
belance
0.1498
0.72%
1.44%
class2: 11.3
BATNQQ Age
Gender
2
19200
1000
class:: 0.955
classl: 0.995
balance
0.10%
0.50%
1.04%
D Proof of the theorem
Theorem D.I. For any PA set A and target signs Φ0(+), (P; is the upper bound of
vanilla unified objective [(S; 0, T), i.e., DF (S:0, T) Optimizing (F O) can guarantee the comprehensive distribution coverage for T.
L(S: T),, when D(:-)) is convex.
Proof. L(S;0,T) =
Σ
D D(E (2;
(2; 0).)
WEY
Σ
D
Σ
ry
El
El (2; 0)] Blo
yEY
6.EA
Σ
2 D Bi TT (x; 0)] 18 (x;0))
(6)
VCy AEEA
D(E TT
(x; 0)], E El (z; (z; 0)])
(7)
NEY 0,EA
(S.07)
Eq. 6 is obtained according to Jensen Inequality, and Eq. 7 is given since group ratios are smaller than one. (F T) serves as the upper bound of L(S; T), meaning that minimizing (S; T) ensures the minimization of C(S: T). Hence, optimizing S in FairDD can guarantee the distributional coverage by bounding L(S; 0, T) tailored for accuracy.
E
Additional performance comparison
Table 9: Performance comparison.
DM DEO-M
DM-FairDD DEO-M
Dataset
Acc.
DED-A
Acc.
DEO-A
Acc.
DEO-M
DEO-A
MNIST-F
25.01
100.00
99.96
94.61
17.04
7.95
90.42
34.77
18.46
CIFARIO-S
37.88
59.20
39.31
45.17
31.75
8.73
41.42
48.20
22.48
Since the work does not release its code, we follow the implementation details presented in their paper. We conduct experiments CMNIST-FG and CIFARI0-S using DM at IPC=10. Their method can mitigate the distillation bias to some extent. However, this approach does not guarantee that the alignment objective is unbiased across all attribute groups, nor does it ensure adequate distribution coverage. Instead, our methods have a fairness alignment objective to facilitate unbiased data distillation; In addition, we provide theoretical proof guarantee the distribution coverage for TA. We summarize the three main advantages of FairDD over theirs:
Better preservation of target attribute representations: Their approach assigns higher weights to samples in low-density regions of the data distribution. However, these samples may lie on the periphery of the data manifold and carry low information. As result, the distilled dataset presents the patterns with low information and hinders the distillation of informative patterns. contrast, our method explicitly aligns the centroids of each group (defined by protected and target attributes) between the distilled and original datasets. This ensures that
24

---

each group in the distilled dataset preserves representative and informative patterns, thereby maintaining the semantic integrity of the original distribution.
Better support for protected attribute fairness: Their weighting strategy can still underrepre- sent minority groups if those groups are dense in the data manifold. This results in biased generation against minority group attributes. In contrast, our method is agnostic to sample density and directly aligns groups across both PA. This ensures fair representation for all attribute groups, regardless of their density, and effectively mitigates PA-related bias.
Stronger theoretical foundations: Their work primarily relies on empirical evaluation and does not provide a theoretical explanation for why distilled datasets inherit biases from the original data, or how their method mitigates these biases. In contrast, we offer a formal theoretical analysis that explains why dataset distillation naturally inherits bias from the source data. Furthermore, we provide provable guarantees on both target attribute accuracy and protected attribute balance.
F Visualization analysis on representation coverage
(a) The S distribution of generated by DM.
(b) The S distribution of generated by FairDD.
Figure 4: Feature coverage comparison on TA between DM and DM+FairDD. We visualize features extracted by de on training and synthetic datasets. One class is highlighted and the remaining classes are transparent. The S generated by DM and FairDD are marked by stars in (a) and (b).
We investigate whether the FairDD effectively covers the whole distribution of the original datasets. For this purpose, we first feed the original training set into the randomly initialized network used in the distillation to extract the corresponding features. Subsequently, we use the same network to extract
features of the distilled dataset S from DM and FairDD. As shown in Fig. 4(a) the synthetic samples in vanilla DDs almost locate the majority group for optimizing the original alignment objective. In this case, vanilla DDs neglect to condense the key patterns of minority groups. This leads to the information loss of minority groups in S. FairDD achieves overall coverage for both majority and minority groups in Fig. 4(b) This is because FairDD introduces synchronized matching to reformulate the distillation objective for aligning the PA-wise groups rather than being dominated by the majority group like vanilla DDs. doing so, FairDD avoids collapsing into the majority group and retains informative patterns from all groups
G Visualization analysis on S generation
We aim to investigate whether FaiDD renders the expectation of S locate the center among all groups, as clarified in Eq. 5 If the clarification holds, S should contain all PA at IPC = because the expectation of S is equal S when IPC =1. We visualize S at IPC=1 on C-MNIST (FG), where each class (digital number) is dominated by one color, and the rest is colored by the rest nine colors. As shown in Fig. 5 the generated by FairDD combines all colors from PA groups. This suggests that FairDD can effectively incorporate all PA into resulting S, indirectly validating the Theorem 5.1 Meanwhile, we observe that the majority groups dominate vanilla DDs according to Eq. where the resulting S contains the colors from the corresponding majority groups.
25

---

Synthetic datuset
Original dataset
Synthetic dataset
3
O
O
O
3
FairDD
Vanilla DD,
9
9
9
Figure 5: Visualization S at IPC=1 for FairDD and vanilla DDs. Left is the condensed dataset using FairDD, which incorporates different PA, i.e., foreground colors. Right is the condensed dataset using vanilla DDs, where each class presents the same color as the corresponding majority group.
Table 10: Ablation diverse weighting mechanisms.
Methods Dataset
FrirDD-GoupDRO D60M DEO, Acc. DEOx DEO, Acc. DEOM DEO, Acc. DE0 DEO, Ace.
DM+FairDD
FairD--IW
FaiDD--LDAM
C-MNIST
10
17.04
7,95
94.61
15.44
8.79
94.18
23.33
9.38
94.06
20.19
10.41
92.73
(FG)
50
10.05
5.46
96.58
12.50
6.49 96.26
12.03
6.60
96.32
17.91
7,90 94.43
C-FMNIST
16.38
74.50
(BG)
50
24.92
13.74
79.07
68.45
36.86
77.23
69.70
36.23
77.21
75.35
36.50
71.23
CIFARIO-S
10
31.75
8,73
45.17
48.27
37.41
38.14
49.88
36.17
39.27
44.85
34.53
38.21
50
18.28
7.35
58.84
63.22
46.96
46.41
59.20
44.47
47,60
65.29
44.20
47.07
H
Ablation on weighting mechanism
Our approach treats groups separately, similar to the weighting mechanism used in the traditional fairness field. Here, explore diverse weighting mechanisms based proposed group-
wise alignment strategy: (1) FairDD+IW weights groups by inverse proportion to their respective
sample size
(2) FairDD+LDAM adopts a soft exponential weighting 1-917
1-18
53 (3)
FairDD+GroupDRo optimizs the group with the maximum alignment loss instead of simultaneous
alignment of all groups [49]. As illustrated in Table IO, DM FairDD outperforms other weighting mechanisms in terms of both fairness and accuracy. We attribute the inferior performance of FairDD+IW and FairDD+LDAM to the excessive penalization of groups with larger sample sizes. Penalizing groups based on sample cardinality reintroduces an unexpected bias related to group size in the information condensation process. This results in large groups receiving smaller weights during alignment, placing them in a weaker position and causing synthetic samples to deviate excessively from large (majority) groups. Consequently, majority patterns become underrepresented, ultimately hindering overall performance. On the other hand, FairDD+GroupDRO shows that inadequate alignment also makes it difficult to equally represent each group. The success of FairDD lies in making each group equally contribute to the total alignment, mitigating the effects of imbalanced sample sizes across all groups. Meanwhile, FairDD performs synchronized alignment to enable the expectation of S to locate the expectation over all group centers of T. Hence, FairDD can be generally applied to datasets with highly varied biases.
I More attributes Analysis on CelebA
We explore additional facial attributes in CelebA to further demonstrate the robustness of FairDD. To this end, regard gender the PA, and young. big_nose, and blond_hail hair the TA, which results in CelebAy. CelebA,, CelebA,, and respectively. We also exchange the PA and TA for CelebA, resulting in CelebA" h The performance is reported on faimess and accuracy in Tables 11 and 12
Computation overhead
In this section, investigate the computational efficiency of FairDD. The only computational difference between FairDD and vanilla DDs is that FairDD replaces the whole alignment with group-level alignment.
Assume we have m real samples and n synthetic samples with G attributes in a batch. For DM, the computational complexity of group-level alignment involves computing the group center. FairDD
26

---

Table 11: Fairness comparison different attributes.
Table 12: Accuracy comparison.
Methods
IPC
DM
DM+FarDD
DC
DC+FairDD
Whole
Methods Dataset
LPC
DM
+FairDD
DC
+FairDD
Whole
Dataset
DEOu
DEO,
DEOx
DEO
DEOu
DEO+
(EOM DEO, DEOu DEO
Acc.
Acc.
Acc.
Acc.
Acc.
10
34.18 31.49
13.30
10.38
20.58
19.26
10.86
8,55
10 62.34
63.79
55.91
56.99
CelchA,
50
46.90
41.13
12,90
8.21
27.98
28.18
14.69
11.26
25.40 16.02
ColebA
50 63.59
67.33
59.87
59.42
75.99
100
37.84
27.76
24.26
19.03
13661
100666.68
69.90
63.53
61.59
10
45.57
45.13
15.63
13.47
18.17
16.81
7.54
6.34
10 (57.46
59.50
[52.91
54.67
Celeb/s,
50
51.91 51.13
14,44
12.01
23.85
22.34
20.58
16.87
34.48
25.50
CelebAs
50
58.71
62.39
56.35
55.46
66.80
100
52.75
51,27
8.03
6.10
24.48
23.53
12,15
11.00
100600.30
64.34
$7.65
57.15
ColebAn 10] 17.01
9.56
7.76
6.02
12:44
8.01
9.25
7.31
15.53
11.56
CelebAs.
10 63.64
64.86
$8.04
57.55
75.33
ColobA
30.28
20.76
12.70
8.28
25.94
15.11
16.78
46.67
26.11
ColobA
110 17.66
79.71
12.00
75.03
79.44
has complexity of G * O(m/G) O(n). In contrast, the computational complexity of vanilla DDs is O(m) + O(). we ignore GPU parallelism, the computational complexity should be the same. However, since GPU parallelism is highly efficient for large batches, it results in G O(m/G) > O(m), raising additional time consumption in FairDD. As for DC, the additional time consumption comes from two parts: one is the backward pass for gradients, and the other is compute the average of the gradients. FairDD incurs additional memory consumption twice due to the above-mentioned GPU parallelism.
Therefore, our additional memory overhead not related to the dataset scale but to the group number of the dataset. We evaluate the impact of the number of groups training time (min) and peak GPU memory consumption (MB). As shown in Table 13 FairDD requires more time than vanilla DDs on C-MNIST (FG), and the time increases the number of groups (PA) grows. This phenomenon is particularly noticeable in DC because DC suffers from GPU parallelism twice. Regarding GPU memory usage, FairDD incurs obvious additional overhead compared to vanilla DDs.
Table 13: Comparison of computation overhead on FairDD and vanilla DDs.
Group number
(vanilla DD)
(FairDD)
(FairDD)
(FairDD)
(FairDD)
(FairDD)
T (min)
G (MB)
T (min)
G (MB)
T (min)
G (MB)
T (min)
GMB)
T (min) G (MB)
T (min) G (MB)
DC
70
2143
94
2345
128
2369
152
2393
181.8
2419
210
2443
DM
26.2
1579
31.75
1579
33.2
1579
35.2
1579
36,5
1579
36,9
1579
Here, we further supplement the overhead analysis with respect to image resolutions. We conduct experiments on CMNIST, CelebA (32), CelebA (64), and CelebA (96) on DM and DC at IPC=10. DM and DC align different signals, which would bring different effects. As illustrated in Table 14 it can be observed that FairDD + DM does not require additional GPU memory consumption but does necessitate more time. The time gap increases from 0.42 minutes to 1.79 minutes as input resolution varies (e.g., CelebA 32 x 32, CelebA 64 x 64, and CelebA 96 x 96); however, the gap remains small. This can be attributed to FairDD performing group-level alignment on features, which is less influenced by input resolution. FairDD + DM requires no additional GPU memory consumption. Its additional time depends on both input resolutions. As for DC, FairDD requires additional GPU memory and time.
Table 14: Comparison of computation overhead for IPC 10.
Methods
Group number
DM
DM-FairDD
DC
DC+FairDD Time Memory
Dataset
Time
Memory
Time Memory
Time
Memory
CelebA32 x 32
10.93
2293
2293
32.98
34.65
CelehA64 x 64
2
11.18
8179
12.20
8177
43.67
8525
47.07
8841
CelebA9G x 96
12.83
17975
14.62
17975
82.37
18855
86.88
19437
K Attribute missing in test dataset
Here, we investigate whether FairDD is agnostic to the attribute missing in the test dataset. We conduct our experiment by training the model on all PAs and testing on datasets that are missing one, two, or three PAs.
27

---

Table 15: Ablation study of missing group labels on C-MNIST and C-FMNIST under different IPC settings.
DM
Dalaset
IPC
Avc.
DEOA
Vanilla
C-MNIST (FG)
10
94.61
17.04
7.95
Missing
94.62
Missing Two
C-MNIST (PG)
10
94.63
16.79
7.36
Missing TTree
C-MNIST (FG)
10
94.64
11.98
6.63
Vanilla
C-MNIST (PG)
50
96.38
10.05
Missing One
U-MNIST (FG)
50
96.60
10.05
5.38
Missing
C.MNIST
96.59
Missing Three
C-MNIST (PG)
50
96.59
8.53
4.87
Vanilla
C-FMNIST (BG)
10
71.10
33.05
19.72
Missing
C-EMNIST (BG)
71.34
Missing Two
C-EMNIST (BG)
10
71.23
28.75
17.84
Missing Three
C-FMNIST (BG)
71.23
Vanilla
C-EMNIST (BG)
50
79.07
24.50
14,47
Missing One
FMNIST (BG)
50
79.31
23.60
13.72
Missing
C-EMNIST (BG)
50
79.24
Missing Three
C. FMNNSS (BG)
50
79.24
22.55
12.58
Table 16: Ablation on BR at IPC 50.
Methods
DM
DM-FairDD
BR
Dalasel
DEOM
DEO,
Acc.
DEOM DEO, Acc.
C-MNIST
0.85
99.54
70.13
76.24
10.13
5.20
96.62
91.68
(FG)
0.95
100.0
100.0
33.73
10.30
5.84
96.05
0.85
100.0
95.54
46.14
23.75
13.85
79.61
C-FMNIST
0.90
100.0
99.71
36.27
24.50
14.47
79.07
(BG)
0.95
99.79
0.85
71.75
50.11
46.99
16.44
6,58
59.12
CIFARIO-S
0.90
75.13
55.70
45.02
18.28
7.35
58.84
0.95
75.43
58.58
43.56
17.49
7.10
58.18
The missing attribute does not actually affect performance for the following reasons. First, although the color (PA) is missing in the test dataset, its TA still contributes to the model's ability make accurate classifications on the corresponding TA. Therefore, these missing attributes not considered outliers in terms of TA. Second, the absence of the color in the test dataset does not impact fairness performance because FairDD is designed to generate cirrbute-balanced synthetic datasets. Models trained on these attribute-baland attribute equally. Even though the test dataset misses some existing attributes in training datasets, the model trained on such distilled datasets could still present no bias to the remaining attributes in the test dataset.
distilled datasets are expected to treat each
L Ablation on biased ratio of original datasets
BR reflects the extent of unfairness in the original datasets and indicates the level of PA skew that the distillation process of S will encounter. We investigate the impact of BR values fairness performance by setting BR to {0.85, 0.90, 0.95) on C-MNIST (FG), C-FMNIST (BG), and CIFAR 10- S. The results at IPC 50 in Table T6 show that DM is sensitive the BR of original datasets, with its DEOM decreasing from 70.13 to 100.0 as BR increases from 0.85 to 0.95. A similar trend is observed in other datasets. Compared DM, FairDD maintains consistent fairness and accuracy levels across different biases. This is attributed to the synchronized matching, which explicitly aligns each PA A--wise subtarget, reducing sensitivity to group-specifh sample numbers. This shows FairDD's robustness to PA skew in the original datasets.
M Ablation study on group label noise and missing
Here, we evaluate the robustness of spurious group labels could provide more insights. randomly sample the entire dataset according predefined ratio. These samples randomly assigned to
28

---

group labels to simulate noise. To ensure a thorough evaluation, we set sample ratios at 10%, 15%, 20%, and 50%. As shown the table, when the ratio increases from 10% 20%, the D results range from 14.93% to 18.31 % with no significant performance variations observed. These results indicate that FairDD is robust to noisy group labels. However, as the ratio increases further to 50%, relatively significant performance variations become apparent. It can be understood that under a high noise ratio, the excessive true samples of majority attributes are assigned to minority labels. This causes the minority group center to shift far from its true center and thus be underrepresented.
Table 17: Ablation study on group label noise.
Methods
[Pc
DM-FairDD
DM-FairDD (10%) DM-FainDD (15%) DM-FairDD (20%) DM+FainDD (50%)
Dataset
Acc:
DEOM DEO, Auc, DEOg DEOA Acc. DEOM DEO, Aoc. DEOs DEO, Acc. DEOM DE0g Apc. DE00 DEO,
CMNIST (RC)
10 |37.95 1000
99.11 |04.88 13.42 6.77 104.34 16.54 7.81 94.44 17.90 8.61 104.22 18.31 9.50 189.56 66.19 25.97
We investigate the experiment when the labels are missing. To provide attribute-level pseudo labels, we choose an unsupervised clustering method DBSCAN. Specifically, we do not have any group labels and use DBSCAN to cluster the samples within batch. The clustering label regarded as the pseudo-group label. From Table, FairDD achieves 94.77% accuracy, and 12.38% DEOM and 6.80 6.80% A This demonstrates the potential of FairDD combined with an unsupervised approach when group labels are unavailable.
Table 18: Comparison of FairDD using prior vs. pseudo labels under different DDs on CMNIST-BG.
FairDD (ipcT0)
Method
Dataset
DEOM
DEO,
Prior label
FairDD + DM
CMNIST BG
96.86
13.42
6.77
Pseudo label
FairDD DM
CMNIST-RG
94.77
12.38
6.80
Prior label
FairDD DC
CMNNIS-BGG
90.84
20.66
9,34
Pscudo label
Fai:DD DC
CMNIST-BG
90.99
27.96
10.57
N Ablation study on balanced original dataset
We synthesized a fair version of CelebA, referred to as CelebA The target attribute is attractive (attractive and unattractive), and the protected attribute is gender (female and male). In the original dataset, the sample numbers for female-atraactive female-unane cemale-nnttractive, male-attractive, and male- unattractive groups imbalanced. To create a fair version, CelebAPair samples the number of instances based on the smallest group, ensuring equal representation across all four groups We tested the fairness performance of FairDD and DM at IPC= 10, as well as the performance of models trained on the full dataset. As shown in Table 19 vanilla DM achieves 14.33% DEO, and 8.77% DEOM- In comparison, the full dataset achieves 3.66% DEO, and 2.77% DEOM- While DM still exacerbates bias with a relatively small margin, this is primarily due to partial information loss introduced during the distillation process. FairDD produces fairer results, achieving 11.11% DEO and 6.68% DEOM
Table 19: Performance on balanced original dataset
Methods
JPC
Whole
DM
DM-FairDD
Dataset
Acc. DEOg DEO, Acc. DEOM DEO, Acc. DEO, DEO,
ColebA
10 16.33
3.66
2.77 663.31
14.33 8.77 (63.17 11.11 6.68
Ablation study on nuanced PA groups
We perform a fine-gran -raiggr PA division. For example, we consider gender and wearing-necktt as two correlated attributes and divide them into four groups: males with necktie, males without a necktie, females with necktie, and females without a necktie (CelebAgton). Similarly, we consider gender and paleskin and divide them into four groups (CelebAgtyp) Their target attribute is attractive. As shown in the Table 20 Fai outperforms vanilla DM in the accuracy and faimess performance
29

---

on these two experiments. The performance for necktie and gender is improved from 57.50% to 25.00% on DEOM and 52.79% 21.73% on DEO Accuracy is also improved from 63.25% to 67.98%. Similar results can be observed for gender and paleskin. Hence, FairDD can mitigate more fine-grained attribute bias, even when there is an intersection between attributes.
Table 20: Performance nuanced groups.
Methods
IPC
DM
DM-FairDD DEO, Acc. DEOM DEO,
Dataset
Acc.
DEOy
CelobAggen
52.79 (67 67.98 25.00 21.73 10 63.25 57.50
ColebAg@p
62.48 44.81 41.60 64.37 19.33
P Ablation study on imbalanced PA groups
To further study FairDD robustness under more biased scenarios, we keep the sample number of the majority group in each class invariant and allocate the sample size to the remaining 9 minority groups with increasing ratios, i.e., 1:2:3:4:5:6:7:8.9 We denote this variant CMNIST unbalance This could help create varying extents of underrepresented samples for different minority groups. Notably, the least-represented PA groups account for only about 1/500 of the entire dataset, which equates to just 12 samples out of 6000 in CMNIST unbolance- As shown in Table 21 FairDD achieves robust performance 16.33% DEOM and 9.01% A compared to 17.04% and 7.95% in the balanced PA groups. A similar steady behavior is observed in accuracy, which changes from 94.45% to 94.61 %. This illustrates the robustness of FairDD under different levels of dataset imbalance.
Table 21: Performance on imbalanced PA.
Methods
IPC
DM
DM-FairDD
Dataset
Acc. DEOM DEO, Acc. DEOM DEO
CMNIST
10 25.01 100.0
99.96
94,61
7.95 94.45 16.33 9,01
17.04
CMNIST
10 23.38 100.0 99.89
Exploration on Vision Transformer as backbone
Although the Vision Transformer (VIT) is powerful backbone network, to the best of my knowledge, current DDs, such as DM and DC, have not yet utilized ViT as the extraction network. We conducted experiments using I-layer, 2-layer, and 3-layer ViTs. As shown Table [2 vanilla DM at IPC=10 suffers performance degradation in classification, dropping from 25.01 % to 18.63%. Moreover, as the number of layers increases, the performance deteriorates more severely. This suggests that current DDs not directly compatible with ViTs. While FairDD still outperforms DM in both accuracy and fairness metrics, the observed improvement gain is smaller compared to results obtained on convolutional networks. Further research into leveraging ViTs for DM and FairDD is promising direction worth exploring.
Table 22: Exploration on ViT architecture.
Methods
JPC
DM
DM-FairDD
Datnset
Acc.
DEO
DEOA
Acc. DFOM DEOA
VITI
10
18.63
100.0
98.48
56.155
82.10 $6.72
98.99
VIT3
10
16.15
100.0
95.75
26.70
65.71
29.46
30

---

R Comparison with MTT
Unlike DMF, MTT uses a two-stage method condense the dataset. First, it stores the model trajectories, and then it uses these trajectories to guide the generation of the synthetic dataset. To provide comprehensive comparison, we compare FairDD with MTT, as shown Tables 23 and
Table 23: Fairness comparison on diverse IPCs.
Table 24: Accuracy comparison on di-
The best results are highlighted in bold.
verse IPCs.
Methods Datasct
IPC
Random
MTT
DM
DM-FairDD
Whole
Methods Datasct
IPC
Rundon
MTT
DM
DEOy DEO,
DEOs DEO,
DEOM DEO, DEOs DEO,
DEOM DEO,
Acc.
Acc.
Acc.
Acc.
Acc.
C-MNIST
10
100.0
98.72
25.70
14.86
100.0
99.96
17.04
7.95
C-MNIST
10
30.75
92.0022.01
94.61
(FG)
100
100.0
100.0
88.64
99.58
25.46
26.81
12.60
13.02
99.36 66.38
100.0
91.68
10,05
8.17
4.86
5.46
10.10 5.89
(FG)
100
50
47.38
67.41
94.0886.84
94.2977.04
96.79
96.58
97.71
167.9222.26
C-FMNIST
C-FNNIST
50
100.0
98.52
96.60
62.02
100.0
99.71
24.50
14.47
91.40
51.68
50
34.92
70.32
79.07
77.97
(BG)
(BG)
100
97.20 63.66
100.0 93.88
21.95
13.33
100
44.87
70.7449.30
From the results, FairDD outperforms MTT in both faimness and accuracy. Notably, MTT surpasses DM by large margin, which we attribute to two factors: 1) Unlike DMF, which is directly influenced by biased data, MTT aligns the model parameters to optimize the synthetic dataset, and this indirect alignment reduces the impact of bias in the data. 2) An accurate model typically conceals its inherent unfairness, as it can better classify each class despite underlying biases. For example, when Whole model achieves high accuracy the C-MNIST (FG) dataset, MTT inherits this accuracy and conceals its biases. However, when the model's s accuracy declines on the C-FMNIST (BG) dataset, MTT reveals its underlying unfairness Fig. [[a) contrast, FairDD directly addresses unfairness rather than relying on high accuracy to obscure biased behavior in Fig. 6(b)
(a) MTT visualization.
(b) FairDD visualization.
Figure 6: Visualization comparison on C-FMNIST (BG) between MTT and FairDD DM.
S More visualizations
We provide more visualizations at IPC 50 on different datasets Figures
g
9
and
31

---

3
4
5
5
(a) Visualization initialized dataset IPC 50 C-MNIST (FG). foreground each class
dominated by one color.
1 3
5
3
y
6
(b) Visualization of the condensed datasct IPC 50 in C-MNIST (FG) using Vanilla DM. The foreground of each class inherits the bias
3 3 3
3
3
3
3
3
3
3
y
4 4
4
<
7 7
8
8
r
9 (
8
8
8
9
9
(c) Visualization of the condensed dataset IPC 50 in C-MNIST (FG) using FairDD DM. The foreground each class mitigates such bias.
Figure 7: Visualization comparison C-MNIST (FG) between vanilla DM and FairDD DM.
32

---

3
5
1
(a) Visualization of the initialized dataset at IPC 50 in C-MNIST (BG). The background of each class dominated by color,
O
O
O
2
2
2
A
3
2
9
(b) Visualization of the condensed dataset at IPC = 50 in C-MNIST (BG) using Vanilla DM. The background of
each class inherits the bias.
O
2
1
A
X
2
2
5
7
7
7
7
(c) Visualization of the condensed dataset at IPC = 50 in C-MNIST (BG) using FairDD + DM. The background of each class mitigates such bias.
Figure 8: Visualization comparison on C-MNIST (BG) between vanilla DM and FairDD + DM.
33

---

(a) Visualization of the initialized dataset IPC = 50 in C-FMNIST (FG). The foreground of each class is dominated by color.
8
(b) Visualization the condensed dataset at IPC 50 in C-EMNIST (FG) using Vanilla DM. The foreground of each class inherits the bias.
@
(c) Visualization the condensed dataset at IPC 50 in C-EMNIST (FG) using FairDD + DM. The foreground of each class mitigates such bias.
Figure 9: Visualization comparison on C-FMNIST (FG) between vanilla DM and FairDD + DM.
34

---

(a) Visualization of the initialized dataset IPC 50 in C-FMNIST (BG). The background of each class is dominated by color.
(b) Visualization of the condensed dataset at IPC 50 in C-FMNIST (BG) using Vanilla DM. The background
of each class inherits the bias.
(c) Visualization of the condensed dataset at IPC = 50 in C-FMNIST (BG) using FairDD DM. The background of each class mitigates such bias.
Figure 10: Visualization comparison on C-FMNIST (BG) between vanilla DM and FairDD + DM.
35

---

(a) Visualization of the initialized dataset at IPC 50 in CIFAR10-S. The top five classes (rows) are dominated by the grayscale images, and color ones dominate the bottle five.
(b) Visualization the condensed dataset IPC in CIFARIO-S using Vanilla The foreground and background of each class inherit the bias.
(c) Visualization of the condensed dataset IPC 50 in CIFAR10-S using FairDD DM. The foreground and background of each class mitigate such bias.
Figure 11: Visualization comparison CIFARIO-S between vanilla DM and FairDD DM.
36

---

(a) Visualization of the initialized dataset at IPC = 10 in CelebA. The top row is dominated by the male, and the female dominates the bottom row.
(b) Visualization condensed dataset at IPC in CelebA using Vanilla DM. The synthetic dataset inherits the gender bias.
(c) Visualization of the condensed dataset at IPC 10 in CelebA using FairDD DM. The synthetic dataset mitigates the gender bias.
Figure 12: Visualization comparison on CelebA between vanilla DM and FairDD + DM.
37


Reviews:
[{"review_id": "bCGRBjPNcz", "reviewer": "Reviewer_GpUd", "summary": "This work argues that existing dataset distillation methodologies are biased towards majority protected attribute groups and fail to achieve unfairness towards minority protected attribute groups with theoretical evidences. To solve this problem, this work proposes the FairDD framework, which divides a given original dataset according to attributes and aligns each sub-dataset with a learnable small dataset. The authors conduct extensive experiments on a wide range of benchmark datasets and experimentally demonstrate that FairDD simultaneously improves both accuracy and fairness on an unbiased test dataset.", "questions": "Please refer to the weaknesses also.\n1. I am curious about the derivation of Eqs. 3 and 5.\n2. Is FairDD applicable to datasets where each instance has more than one attribute? I suspect it would be difficult to apply Attribute-aware partitioning naively in this situation.", "limitations": "See weaknesses and questions", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n1. To the best of my knowledge, this research is the first work to discuss the fairness in dataset distillation.\n\n2. I like the analysis of why vanilla DD fails in PA imbalance situations. It is easy for the reader to follow why we should explore dataset distillation methodologies for fairness.\n\n3. Through the extensive experimental results, FairDD has demonstrated high performance gains despite its very simple form.\n\n**Weaknesses**\n1. My major concern is the novelty of the proposed method. FairDD partitions the original dataset into several sub-datasets w.r.t PA to mitigate the domination of the majority group. However, partitioning requires that every instance of the original dataset has a labelled attribute. This may not be realistic and becomes a shortcoming that limits the scope of application of FairDD. In addition, the methodology of dividing the real dataset according to the attribute is a simple but strong baseline, but the contribution is marginal to be called a core methodology. Finally, Attribute-based partitioning has clear drawbacks that make it difficult to apply to bilevel optimization and the trajectory matching framework.\n\n2. Lack of sufficient comparison to a baseline that could be simple but effective. For example, a methodology that applies vanilla DD and uses fairness-aware learning to train the network in the test phase could be a baseline. I also believe that gradient matching for fairness-aware loss is a baseline. Finally, this research should also include whether bias also occurs in bilevel optimization or trajectory matching using fairness-aware learning. As this research is the first to study the possibility of dataset distillation in a new context, it is necessary to make detailed comparisons with various possible baselines.", "was_revised": true, "final_justification": "In my review, I raised questions about 1) the novelty of the proposed methodology, 2) additional comparative experiments under various baselines and experimental settings, and 3) the theoretical derivation. During the rebuttal period, the authors provided 1) further explanation of the paper’s contributions, 2) additional experimental results, and 3) the details of theoretical derivation.\n\nI still believe that the proposed method, which is just partitioning the dataset to fit PA, lacks sufficient novelty. However, I consider the following contributions to be substantial: 1) the paper is the first to address fairness in the context of dataset distillation, 2) the theoretical justification for the proposed method (Theorem 5.1), and 3) the demonstrated improvements in performance. Furthermore, the additional experimental results provided during the rebuttal are also appropriate. For these reasons, I recommend a final score of 4 (Borderline Accept).", "rebuttal": "We thank you for your efforts in reviewing our paper.\n\n**Q1: The contributions of the proposed method.**\n\nThank you for pointing out your concerns. We use formal evidence to illustrate that vanilla DD fails to mitigate the bias of the original dataset but amplifies the bias, and further provide an effective solution to mitigate this phenomenon in the distilled dataset. We are the first attempt to consider attribute fairness into DD, opening up a novel and underexplored direction that connects fairness and efficient data disrtillation\n\n**Insights for Subsequent Research:**\n\n1. Fairness-Aligned Objective: The distilled dataset should be constructed such that its expected representation is unbiased across all attribute groups, aligning with fairness principles at the objective level.\n\n2. Equal Group Contribution: Each group should contribute equally to the overall loss, encouraging balanced optimization and preventing group-specific bias during training.\n\n3. Distributional Coverage of TA: It is essential to ensure that the distilled dataset maintains adequate distributional coverage over the TA, supporting both fairness and classification accuracy.\n\n\n**Technical contribution:**\n\nWhile partitioning based on PA may appear straightforward, FairDD is built upon three key components that directly enhance fairness in dataset distillation:\n\n1. Bias-Free Alignment Objective:\nFairDD introduces a group-wise alignment objective to ensure that the expected representation of each group in the distilled data remains unbiased—an essential requirement for achieving group-level fairness. This principle is formally expressed in Equation (5).\n\n2. Equal Group Contribution to the Loss:\nThe framework enforces that each group contributes equally to the total loss during optimization. This encourages balanced gradient updates and equitable learning across all attribute groups.\n\n3. Theoretical Guarantee over Vanilla DD:\nWhereas vanilla dataset distillation focuses exclusively on optimizing classification accuracy for TA, FairDD incorporates a fairness-aware loss function. As proven in Theorem 5.2, this loss provides an upper bound on the vanilla DD loss, integrating fairness objectives without compromising overall performance.\n\n**Q2: Reliance on attribute labels.** \n\nWe investigate the case where such labels are unavailable. As detailed in Appendix L, we apply the unsupervised clustering algorithm DBSCAN to generate pseudo attribute labels, and show in Table 16 that FairDD maintains good performance even under this setting. This indicates that our method is compatible with label-free scenarios, at least to some degree.\n\n|| CMNIST-BG |ACC |DEO-M|DEO-A|\n| --- | --- | --- | --- | --- |\n|DC|Prior label| 90.84| 20.66| 9.94 |\n|DC|Pseudo label|90.99 | 27.96 |10.57|\n\nAs is common in many research fields, progress often begins with supervised approaches before transitioning to unsupervised solutions. For example, the representative method GroupDRO also needs the group label to achieve the fairness regularization. After that, there are many approaches without labels. The need for labels does not diminish their contribution to model fairness. We believe this is a necessary process that calls for community-wide efforts rather than individual solutions.\n\nWe hope these clarifications address your concerns.\n\n**Q3: More baseline**\n\nWe thank you for your comments. \n\n**For fairness-aware learning at IPC=10, we assign weights to each group based on the inverse of its sample proportion.**\n\nSince the attributes of the distilled data cannot be obtained using vanilla DD, we manually annotate the attribute labels of the distilled data. These labels are then used to train the network with a fairness-aware loss.\n\n| CMNIST-BG |ACC |DEO-M|DEO-A|\n| --- | --- | --- | --- |\n|DC  | 65.91| 100.0 | 73.60 |\n|DC+FairDD | 90.84| 20.66 |  9.94 |\n|DC+ Model fairness learning   | 67.10 | 99.50 |70.60|\n|DC +Fairness loss| 82.45 | 76.44 |42.05|\n\n| CIFAR10-S|ACC |DEO-M|DEO-A|\n| --- | --- | --- | --- |\n|DC  | 37.88| 42.23 | 27.35 |\n|DC+FairDD | 41.82| 22.08 | 8.22 |\n|DC + Model fairness learning   | 39.47 | 35.86 |22.04|\n|DC + Fairness loss| 40.10 | 27.20 |10.29|\n\nThe distilled data from some minority groups was lost due to biased distillation. Although we apply fairness-aware learning to train the network, the missing attributes cannot be well represented.\n\nWe also apply a fairness loss to optimize the distillation process of DC, resulting in a variant referred to as DC + Fairness loss.\n\nAs for MTT [1], we use fairness loss to train the teacher network, and then use the teacher trajectory to align student one.\n\nAs for bi-level optimization, we use RFAD [2], a bi-level optimization approach, for evaluation. We use the weighted fairness loss to optimize the data generation\n\n| CMNIST-BG |ACC |DEO-M|DEO-A|\n| --- | --- | --- | --- |\n|MTT| 10.92| 75.83| 32.35 |\n|MTT+Fairness loss|93.58 | 29.70 |12.40|\n\n| CIFAR10-S|ACC |DEO-M|DEO-A|\n| --- | --- | --- | --- |\n|MTT| 56.92| 37.21 | 26.85 |\n|MTT+Fairness loss| 59.52 | 24.03 |12.05|\n\n| CMNIST-BG |ACC |DEO-M|DEO-A|\n| --- | --- | --- | --- |\n|RFAD| 15.84| 100.0| 98.82 |\n|RFAD+Fairness loss|14.68 | 100.0 |99.16|\n\n| CIFAR10-S|ACC |DEO-M|DEO-A|\n| --- | --- | --- | --- |\n|RFAD| 26.25| 60.54 | 42.58 |\n|RFAD+Fairness loss| 28.56 | 55.46 |35.48|\n\nVanilla DD with fairness loss can mitigate the distillation bias to some extent. However, this approach does not guarantee that the alignment objective is unbiased across all attribute groups, nor does it ensure distributional coverage for TA.\n\n[1] Cazenavette, George, et al. \"Dataset distillation by matching training trajectories.\" In CVPR 2022.\n\n[2] Loo, Noel, et al. \"Efficient dataset distillation using random feature approximation.\" In NIPS 2022.\n\n\n**Q4:  Doubt about the derivation of Eqs. 3 and 5.**\n\n\\begin{aligned}\n&\\mathcal{L}(\\mathcal{S}; \\theta, \\mathcal{T}) := \\textstyle\n\\sum_{y \\in \\mathcal{Y}} \\mathcal{D} \\sum_{a_i \\in \\mathcal{A}} r_y^{a_i} \\cdot \\mathbb{E}\\_{x \\sim \\mathcal{T}_y^{a_i}} \\left[ \\phi(x; \\theta) \\right],\\mathbb{E}\\_{x \\sim \\mathcal{S}_y} \\left[ \\phi(x; \\theta) \\right] \\newline\n& \\frac{\\partial \\mathcal{L}}{\\partial {\\partial \\mathbb{E}[\\phi\\_{x \\sim \\mathcal{S}\\_y}(x; \\theta)]}}=\\textstyle\\frac{\\partial \\mathcal{D} (\\sum\\_{a_i \\in \\mathcal{A}}r\\_y^{a\\_{i}} \\mathbb{E}[\\phi\\_{x \\sim \\mathcal{T}\\_y^{a_i}}(x;\\theta)], \\mathbb{E}[\\phi\\_{x \\sim \\mathcal{S}\\_y}(x;\\theta)])}{\\partial \\mathbb{E}[\\phi\\_{x \\sim \\mathcal{S}\\_y}(x; \\theta)]} = 0\n\\end{aligned}\n\nFor distribution matching, $\\mathcal{D} = MSE$: \n\n$\n\\textstyle\\frac{\\partial ||\\sum\\_{a_i \\in \\mathcal{A}}r\\_y^{a\\_{i}} \\mathbb{E}[\\phi\\_{x \\sim \\mathcal{T}\\_y^{a_i}}(x;\\theta)] -  \\mathbb{E}[\\phi\\_{x \\sim \\mathcal{S}\\_y}(x;\\theta)])||^2}{\\partial \\mathbb{E}[\\phi\\_{x \\sim \\mathcal{S}\\_y}(x; \\theta)]} = 0\n$, \n\nwe have\n \n\\begin{align}\n\\mathbb{E}[\\phi\\_{x \\sim \\mathcal{S}\\_y}(x; \\theta)]=\\sum\\_{a\\_i \\in \\mathcal{A}} r\\_y^{a\\_{i}}\\mathbb{E}[\\phi\\_{x \\sim \\mathcal{T}\\_y^{a\\_i}}(x; \\theta)],\n\\end{align}\n\nFor gradient matching, $\\mathcal{D} = MAE$ or  $\\mathcal{D} = $Cosine distance:\n\n$\n\\textstyle\\frac{\\partial |\\sum\\_{a_i \\in \\mathcal{A}}r\\_y^{a\\_{i}} \\mathbb{E}[\\phi\\_{x \\sim \\mathcal{T}\\_y^{a_i}}(x;\\theta)] -  \\mathbb{E}[\\phi\\_{x \\sim \\mathcal{S}\\_y}(x;\\theta)])|}{\\partial \\mathbb{E}[\\phi\\_{x \\sim \\mathcal{S}\\_y}(x; \\theta)]} = 0\n$, \n\nwe have \n\n\\begin{align}\n\\mathbb{E}[\\phi\\_{x \\sim \\mathcal{S}\\_y}(x; \\theta)]= \\sum\\_{a\\_i \\in \\mathcal{A}} r\\_y^{a\\_{i}}\\mathbb{E}[\\phi\\_{x \\sim \\mathcal{T}\\_y^{a\\_i}}(x; \\theta)],\n\\end{align}\n\nWhen $\\mathcal{D} = $Cosine distance, \n$\n\\textstyle\\frac{\\partial |\\sum\\_{a_i \\in \\mathcal{A}}r\\_y^{a\\_{i}} \\mathbb{E}[\\phi\\_{x \\sim \\mathcal{T}\\_y^{a_i}}(x;\\theta)] -  \\mathbb{E}[\\phi\\_{x \\sim \\mathcal{S}\\_y}(x;\\theta)])|}{\\partial \\mathbb{E}[\\phi\\_{x \\sim \\mathcal{S}\\_y}(x; \\theta)]} = 0\n$, \n\nwe have\n \n\\begin{align}\n\\mathbb{E}[\\phi\\_{x \\sim \\mathcal{S}\\_y}(x; \\theta)]= \\textstyle \\frac{\\Vert \\mathbb{E}[\\phi_{x \\sim \\mathcal{S}y}(x; \\theta)] \\Vert_2}{\\Vert \\sum_{a_i \\in \\mathcal{A}} r_y^{a_{i}}\\mathbb{E}[\\phi_{x \\sim \\mathcal{T}_y^{a_i}}(x; \\theta)]\\Vert_2} \\sum\\_{a\\_i \\in \\mathcal{A}} r\\_y^{a\\_{i}}\\mathbb{E}[\\phi\\_{x \\sim \\mathcal{T}\\_y^{a\\_i}}(x; \\theta)],\n\\end{align}\nNote that $\\textstyle \\frac{\\Vert \\mathbb{E}[\\phi\\_{x \\sim \\mathcal{S}y}(x; \\theta)] \\Vert_2}{\\Vert \\sum\\_{a\\_i \\in \\mathcal{A}} r\\_y^{a\\_{i}}\\mathbb{E}[\\phi\\_{x \\sim \\mathcal{T}\\_y^{a\\_i}}(x; \\theta)]\\Vert_2}$ is a constant to the different groups. Therefore, we could derive vanilla DDs \n\n\\begin{align}\n\\mathbb{E}[\\phi\\_{x \\sim \\mathcal{S}\\_y}(x; \\theta)]=\\lambda\\sum\\_{a\\_i \\in \\mathcal{A}} r\\_y^{a\\_{i}}\\mathbb{E}[\\phi\\_{x \\sim \\mathcal{T}\\_y^{a\\_i}}(x; \\theta)], \\lambda \\enspace \\text{is a constant}.\n\\end{align}\n\nThe same analysis could be applied to FairDD.\n\n**Q5: Is FairDD applicable to datasets where each instance has more than one attribute?**\n\nThank you for your insightful comments.\n\nFairDD can naturally handle instances with multiple protected attributes. In such cases, we assign each sample to a fine-grained group, where all samples share the same combination of attribute values. For example, if the dataset contains two attributes A = {a₁, a₂} and B = {b₁, b₂}, this results in four distinct groups: {a₁, b₁}, {a₁, b₂}, {a₂, b₁}, and {a₂, b₂}. As shown in Equation (5), each group contributes equally to the generation of the distilled dataset.\n\nTo evaluate this capability, we conduct experiments on CelebA under two multi-attribute settings:\n\nGender + Wearing Necktie\n\nGender + Pale Skin\n\nThe results, shown in Table 18 (Appendix N), demonstrate that FairDD remains effective in preserving fairness while maintaining competitive accuracy under multi-attribute scenarios:\n\n|CelebA| Gender + Necktie |ACC |DEO-M|DEO-A|\n| --- | --- | --- | --- | --- |\n|DM || 63.25| 57.50| 52.79 |\n|DM +FairDD| |67.98 | 25.00 |21.73|\n\n|CelebA| Gender + Pale Skin |ACC |DEO-M|DEO-A|\n| --- | --- | --- | --- | --- |\n|DM || 62.48| 44.81| 41.60 |\n|DM +FairDD| |64.37 | 26.92 |19.33|"}, {"review_id": "wb7HIuGRfG", "reviewer": "Reviewer_yyqk", "summary": "This paper introduces FairDD, a fairness-aware dataset distillation framework designed to reduce bias towards protected attributes in synthetic datasets, without sacrificing target attribute accuracy. By aligning distilled data to protected attribute-wise distributions rather than majority-dominated global ones, FairDD improves fairness across multiple dataset distillation paradigms and shows strong performance that is justified theoretically.", "questions": "None", "limitations": "See Strengths and Weaknesses field.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: \n1. The paper addresses a critical yet underexplored issue, namely bias amplification in dataset distillation, and suggests FairDD, the first method to address this challenge.\n2. The authors provide formal proofs that FairDD mitigates protected attribute imbalance and ensures target attribute distributional coverage.\n3. Evaluation involves both synthetic and real-world datasets, showing consistent fairness improvements. Also, comprehensive ablations are reported in the appendix.\n\nWeaknesses\n1. The method requires known protected attribute labels, limiting its applicability to real-world scenarios where such labels are often unavailable and inferring pseudo-labels can be very challenging.\n\n2. While the technical content is strong, the writing could be clearer in dense theoretical sections (e.g., Sec. 4), which currently may hinder accessibility to a broader ML audience. Specifically:\n  a. (lines 125-126) The concepts of Gradient Matching and Distribution Matching have not been adequately explained neither here nor in the related works. The authors should elaborate more on how the gradient information and embedding distribution matching is formulated and achieved.\n  b. (line 130) Abbreviations IDC and DREAM are not defined and citations are missing.\n  c. (lines 129-130) How important is the choice of distance metric, and why are these particular metrics better suited for these methods?\n d. (lines 151-153) The fact that bias-aligned samples contribute more to the loss does not necessarily mean that the loss cannot be minimized for bias-conflicting samples. In typical classification tasks, previous work [1] has investigated the loss and gradient spaces with respect to bias-aligned and bias-conflicting samples to demonstrate how bias is introduced. Does this also apply to the Dataset Distillation task?\n e. Considering the previous comment, the authors should elaborate further on the two arguments (in bold) presented in lines 155–158. \n f. It is not clear whether the suggested method can generalize for multiple protected attributes.\ng. The authors should include an algorithm in Section 5 that outlines the proposed method to enhance the reproducibility of their work.\n\n3. While Worst Group Accuracy is a standard evaluation metric in the context of mitigating visual biases, the authors opt for Equalized Odds. It would be helpful if they could elaborate on the rationale behind this choice.\n\n[1] Sarridis, I., Koutlis, C., Papadopoulos, S., & Diou, C. (2024). BAdd: Bias Mitigation through Bias Addition. *arXiv preprint arXiv:2408.11439*.", "was_revised": true, "final_justification": "N/A", "rebuttal": "Thank you for your efforts in reviewing our paper.\n\n**Q1: Prior to protected attribute labels**\n\nAlthough our approach requires attribute labels, we want to clarify that, as is common in many research fields, progress often begins with supervised approaches before transitioning to unsupervised solutions. For example, the representative method GroupDRO [1] also needs the group label to achieve fairness. After that, there are many approaches without labels. The need for labels does not diminish their contribution to model fairness. We believe this is a necessary process that calls for community-wide efforts rather than individual solutions.\n\nWe also investigate the case where such labels are unavailable. As detailed in Appendix L, we apply the unsupervised clustering algorithm DBSCAN to generate pseudo attribute labels, and show in Table 16 that FairDD maintains good performance even under this setting. This indicates that our method is compatible with label-free scenarios, at least to some degree. We would like to leave the deep exploration for future work. \n\n|| CMNIST-BG |ACC |DEO-M|DEO-A|\n| --- | --- | --- | --- | --- |\n|DC|Prior label| 90.84| 20.66| 9.94 |\n|DC|Pseudo label|90.99 | 27.96 |10.57|\n\n[1] Sagawa, S., Koh, P. W., Hashimoto, T. B., & Liang, P. (2020). \"Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization.\" In ICLR 2020.\n\n**Q2: The detailed concepts of Gradient Matching and Distribution Matching**\n\nDistribution matching formulates the distilled process by alignment the embedding distribution between the original dataset and the distilled dataset. They minimize the MMD to update the distilled data.\n$\n\\mathcal{L}(\\mathcal{S}; \\theta, \\mathcal{T}) :=\\sum\\_{y \\in \\mathcal{Y}} ||\\mathbb{E}\\_{x \\sim \\mathcal{T}\\_y} \\left[ \\phi(x; \\theta) \\right]-\\mathbb{E}\\_{x \\sim \\mathcal{S}\\_y} \\left[ \\phi(x; \\theta) \\right]||^2 $\n\nThen, we can rewrite it into the attribute-level formulation: \n\n$\n\\mathcal{L}(\\mathcal{S}; \\theta, \\mathcal{T}) :=\\sum_{y \\in \\mathcal{Y}} ||\\sum_{a_i \\in \\mathcal{A}} r_y^{a_i} \\cdot \\mathbb{E}\\_{x \\sim \\mathcal{T}_y^{a_i}} \\left[ \\phi(x; \\theta) \\right] - \\mathbb{E}\\_{x \\sim \\mathcal{S}\\_y} \\left[ \\phi(x; \\theta) \\right]||^2 \\newline\n$\n\nIn gradient matching methods, the gradients of the classification loss on the original and distilled datasets are aligned using either the mean absolute error (MAE) or cosine distance.\n\n$\\mathcal{L}(\\mathcal{S}; \\theta, \\mathcal{T}) :=\\sum\\_{y \\in \\mathcal{Y}} |\\mathbb{E}\\_{x \\sim \\mathcal{T}\\_y} \\left[\\nabla\\_\\theta \\mathcal{L}\\^c(x; \\theta) \\right]-\\mathbb{E}\\_{x \\sim \\mathcal{S}\\_y} \\left[ \\nabla\\_\\theta \\mathcal{L}\\^c(x; \\theta) \\right]|$ or $\\mathcal{L}(\\mathcal{S}; \\theta, \\mathcal{T}) :=\\sum\\_{y \\in \\mathcal{Y}} \\text{Cosine-distance}(\\mathbb{E}\\_{x \\sim \\mathcal{T}\\_y} \\left[\\nabla\\_\\theta \\mathcal{L}\\^c(x; \\theta) \\right], \\mathbb{E}\\_{x \\sim \\mathcal{S}\\_y} \\left[ \\nabla\\_\\theta \\mathcal{L}\\^c(x; \\theta) \\right])$\n\nThen, we can rewrite it into the attribute-level formulation: \n$\\mathcal{L}(\\mathcal{S}; \\theta, \\mathcal{T}) :=\\sum\\_{y \\in \\mathcal{Y}} |\\sum_{a_i \\in \\mathcal{A}} r_y^{a_i} \\cdot \\mathbb{E}\\_{x \\sim \\mathcal{T}_y^{a_i}}\\left[\\nabla\\_\\theta \\mathcal{L}\\^c(x; \\theta) \\right]-\\mathbb{E}\\_{x \\sim \\mathcal{S}\\_y} \\left[ \\nabla\\_\\theta \\mathcal{L}\\^c(x; \\theta) \\right]|$ or\n$\\mathcal{L}(\\mathcal{S}; \\theta, \\mathcal{T}) :=\\sum\\_{y \\in \\mathcal{Y}} \\text{Cosine-distance}(\\sum\\_{a\\_i \\in \\mathcal{A}} r\\_y^{a\\_i} \\cdot \\mathbb{E}\\_{x \\sim \\mathcal{T}\\_y^{a_i}}\\left[\\nabla\\_\\theta \\mathcal{L}\\^c(x; \\theta) \\right], \\mathbb{E}\\_{x \\sim \\mathcal{S}\\_y} \\left[ \\nabla\\_\\theta \\mathcal{L}\\^c(x; \\theta) \\right])$\n\nWe will supplement the related illustration in the revised version.\n\n\n**Q3: Abbreviations IDC and DREAM are not defined and citations are missing**\n\nThank you for your detailed feedback. IDC and DREAM are two improved variants of DMF. IDC parameterizes the distilled data, while DREAM optimizes the initialization of the distilled data. However, both methods do not alter the optimization objective or the gradient flow. We will include additional illustrations and proper citations in the revised version.\n\n**Q4: How important is the choice of distance metric, and why are these particular metrics better suited for these methods?**\n\nWe use the same distance metric as in their paper to ensure consistency and fair comparison. The distance choice could be found in their own work. Our method is generalizable and can be applied with various distance metrics.\n\n**Q5: Does the previous work BAdd also apply to the Dataset Distillation task?**\n\nIn biased datasets, vanilla models rely on bias-aligned samples to reduce loss, but this prevents them from learning fair representations. BAdd solves this by explicitly adding bias features, allowing the model to keep the loss low on bias-aligned samples without encoding the bias itself. This frees the main representation to focus on fair, unbiased features. From my perspective, BAdd leverages the higher loss of bias-aligned samples, compared to bias-conflicting samples, to guide training more efficiently. This could potentially promote equal optimization across different groups. However, the fairness of the distilled dataset originates from\n\n1) no biased alignment objective between the original data and distilled data (alignment objective) in Equation 5; \n\n2)  each protected attribute should receive balanced optimization (agnostic to the sample number within attributes) during training. \n\nTherefore, Badd provides a potential solution for solution 2 but needs a deep exploration to solve 1.\n\n**Q6: Considering the previous comment, the authors should elaborate further on the two arguments (in bold) presented in lines 155–158.**\n\nThe first argument (in bold) is not related to gradient behavior. It illustrates that the alignment objective itself must be unbiased across all groups. Only when the learning objective is fair and the optimization process is balanced across different attributes, can the distilled dataset be fair.\n\nFor the second argument, we further elaborate that **minority groups often receive suboptimal optimization compared to their majority groups in vanilla models**. This imbalance limits the model’s ability to accurately represent its representations. This phenomenon is supported by studies such as Distributionally Robust Optimization (DRO) and Group DRO, which observe that:\n\n\"A model that learns this spurious correlation would be accurate on average on an i.i.d. test set but suffer high error on groups of data where the correlation does not hold.\"\n\n\n**Q7: It is not clear whether the suggested method can generalize for multiple protected attributes.** \n\nThank you for your insightful comments.\n\nFairDD can naturally handle instances with multiple protected attributes. In such cases, we assign each sample to a fine-grained group, where all samples share the same combination of attribute values. For example, if the dataset contains two attributes A = {a₁, a₂} and B = {b₁, b₂}, this results in four distinct groups: {a₁, b₁}, {a₁, b₂}, {a₂, b₁}, and {a₂, b₂}. As shown in Equation (5), each group contributes equally to the generation of the distilled dataset.\n\nWe conduct experiments on CelebA under two multi-attribute settings at IPC=10, Gender + Wearing Necktie and Gender + Pale Skin\n\nThe results, shown in Table 18 (Appendix N), demonstrate that FairDD remains effective in preserving fairness while maintaining competitive accuracy under multi-attribute scenarios:\n\n|CelebA| Gender + Necktie |ACC |DEO-M|DEO-A|\n| --- | --- | --- | --- | --- |\n|DM || 63.25| 57.50| 52.79 |\n|DM +FairDD| |67.98 | 25.00 |21.73|\n\n|CelebA| Gender + Pale Skin |ACC |DEO-M|DEO-A|\n| --- | --- | --- | --- | --- |\n|DM || 62.48| 44.81| 41.60 |\n|DM +FairDD| |64.37 | 26.92 |19.33|\n\n**Q8: The authors should include an algorithm in Section 5 that outlines the proposed method to enhance the reproducibility of their work.**\n\nWe will supplement it in the revised version.\n\n**Q9: While Worst Group Accuracy is a standard evaluation metric in the context of mitigating visual biases, the authors opt for Equalized Odds. It would be helpful if they could elaborate on the rationale behind this choice.**\n\n Worst Group Accuracy (WGA) is a valuable metric for evaluating fairness, as it captures the performance on the most disadvantaged group. However, it fails to provide fine-grained fairness evaluation across all groups. For example, WGA remains unchanged regardless of whether discrepancies among the remaining groups are small or large. Instead, Visual Fairness and metrics such as DEO-M and DEO-A reflect fine-grained discrepancies across multiple attributes. This allows us to measure how each group or attribute is treated in the model's predictions, providing a more localized and attribute-level fairness assessment. Therefore, we consider WGA a global fairness metric, while DEO-M and DEO-A serve as local, attribute-specific metrics. Since our goal is to generate balanced samples across all protected attributes in the distilled dataset, DEO-M and DEO-A are particularly well-suited for evaluating generation bias in each group."}, {"review_id": "qjO3B1RlKS", "reviewer": "Reviewer_cwZj", "summary": "The paper introduces a method to address fairness issues in the topic of dataset distillation (DD). Specifically, when distilling image instances in DD, the recognition performance for minor catetories / attributes drops, amplifying unfairness. Therefore, in this paper, by organizing protected attributes (PAs) separately and then processing & performing DD, the paper realizes a method called FairDD for the purpose of fairness treatment.", "questions": "- In the proposed FairDD, does treating fairness (protect PAs) generally tend to improve accuracy as well? By protecting minor yet important image feature representations, does FairDD reduce the visually similar features in the distillated image dataset, this means that it is acting as a training regularization? Moreover, if we consider other factors beyond fairness, DD may be further enhanced by the preserved and improved image features and enhanced overall performance?\n\n- The reviewer believes that [Nakamura et al., ECCV 2024] is closely related work. The research work uses a \"1p-frac dataset\" generated by fractal geometry to successfully pre-train a Vision Transformer (ViT) from a single synthetic sample. In the appendix, the present paper mentions that applying ViT did not work very well at this moment. The insights from their fractal-based approach for ViT pre-training help the next approach for using ViT as a backbone in the topic of FairDD / DD? Also, clarifying the relationship between DD and such that approaches, formula-driven supervised learning (FDSL), might further advance the DD topic.\n\n[Nakamura et al., ECCV 2024] R. Nakamura et al. “Scaling Backwards: Minimal Synthetic Pre-training?”, ECCV 2024.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "- S1. The FairDD paper tackles the problem of unfairness in DD. The conventional DD distillates a labeled image dataset into a smaller synthetic dataset for training, however, the distillated image dataset tends to eliminate image instances which have minor categories / attributes. It tends to focus on majority categories / attributes. In order to achieve fairer image recognition in this context, this work, FairDD, preserves the truly protected instances during the process of dataset distillation, this is defined as a FairDD in this paper.\n\n- S2. The proposed approaches and their established model used to improve both performance and fairness scores in the proposed FairDD are quite reasonable. For example, as shown in Figure 2, PA groups are split, and image instances can be separately distilled. By processing all PA groups simultaneously, FairDD avoids focusing only on major categories / attributes and ensures that minority categories / attributes are also represented at that time. Theoretical guarantees are further shown that the FairDD removes imbalance and improves fairness inside of the distillated datasets.\n\n- S3. In the experimental results against baseline / vanilla DD methods, the proposed FairDD successfully improves fairness metrics (DEO_M / DEO_A; lower is better) while also enhancing recognition accuracy (higher is better). The proposed method, FairDD, is evaluated comprehensively on representative datasets, namely C-MNIST, C-FMNIST, CIFAR10-S, CelebA, UTKFace, BFFHQ as shown in Tables 1, 2, and 3, and outperforms baselines on almost all metrics in the presented paper. Additional analyses include visualizations, scalability studies, and an ablation study examining the fair extractor and synthetic image initialization.\n\n- W1. Although extensive experiments in the experimental section at the main paper and appendices partially clarify FairDD’s merits and limitations, the paper will gain greater value by including a more comprehensive discussion. Specifically, a comprehensive description why FairDD works so well in terms of fairness scores and recognition performance. Here, this description should be appeared in the main paper rather than the appendices.", "was_revised": true, "final_justification": "The reviewer is grateful to the authors for great efforts and discussion in the rebuttal phase. In the rebuttal term, the reviewer has had a better understanding on the trial regarding the fairness issues on dataset distillation. This is enough to possess an accept-side paper rating. Hopefully the FairDD and their thoughtful knowledge can be widely shared in the research community. Moreover, the reviewer is looking forward to seeing the next step towards the improved FairDD in the near future.", "rebuttal": "Thanks for your effort in reviewing our paper.\n\n**Q1: Why FairDD works so well in terms of fairness scores and recognition performance.**\n\nThank you for your valuable comments. The superiority of FairDD originates from its ability to achieve unbiased generation with respect to protected attributes (PA) and to provide distributional coverage guarantees for target attributes (TA).\n\nSpecifically, for PA, FairDD employs a bias-free alignment objective, where the expectation of the distilled data is unbiased across all groups, as shown in Equation (5). To ensure fairness during optimization, each PA group contributes equally to the total loss, with gradients that are independent of group sample sizes. This design leads to balanced sample generation across all protected groups.\n\nRegarding TA, the loss used in vanilla DD is designed to align with TA-wise classification accuracy. In FairDD, as proven in Theorem 5.2, the loss function serves as an upper bound on the vanilla DD loss. This implies that minimizing the FairDD objective ensures distributional coverage across TA.\n\nAs a result, models trained on FairDD-distilled datasets achieve higher accuracy on minority PA groups, while maintaining stable performance on majority groups. Therefore, FairDD generally achieves a more effective trade-off between fairness and accuracy compared to traditional dataset distillation approaches. It is worth noting that although accuracy on majority groups may slightly decrease, dataset distillation is inherently a data-hungry task. In this context, the accuracy gain on minority groups typically outweighs the slight reduction on majority groups\n\n\n**Q2: In the proposed FairDD, does treating fairness (protect PAs) generally tend to improve accuracy as well?**\n\nYes, please refer to the above answers.     \n\n**Q3: The insights from their fractal-based approach for ViT pre-training help the next approach for using ViT as a backbone in the topic of FairDD / DD?**\n\nThank you for bringing this reference to our attention. This work also explores training models with limited data, sharing a similar objective with DD. We will include it in the Related Work section as a complementary approach.\n\nIn methods like DM and DC, the backbone or feature extractor is used to approximate the non-linear transformation from input to representation. However, learning on the backbone during distillation can limit the generalizability of the distilled data. This is why such methods typically network with random initialization or restrict the network training to just a few optimization steps when distilling the dataset.\n\nFrom my perspective, the challenge of applying dataset distillation to Transformers does not stem from the ViT pre-training itself, but rather from the nature of the self-attention mechanism. Unlike CNNs, which preserve local spatial structure, self-attention in Transformers mixes information across image patches from the very first layer. This early interleaving of patch-level features makes it more difficult for distilled data to encode coherent semantic cues.\n\n**Q4: Clarifying the relationship between DD and such that approaches, formula-driven supervised learning (FDSL), might further advance the DD topic.**\n\nFDSL and DD share a common objective: enabling effective model training using a limited amount of data. However, they approach this goal from different directions. FDSL starts with a small number of procedurally generated examples and expands the data manifold through structured variations (e.g., formula-driven perturbations) to expose the model to diverse representations. In contrast, DD begins with a large dataset and compresses it into a compact, information-rich synthetic set that mimics the training effect of the original data.\n\nFrom a technical perspective, these approaches could be orthogonal and potentially complementary. For example, one could first apply DD to compress a large real dataset into a small synthetic one, and then apply FDSL techniques to expand the utility of that distilled dataset. This combined strategy could further improve training efficiency and generalization of DD."}, {"review_id": "wai8sSyr1K", "reviewer": "Reviewer_KqGz", "summary": "This paper proposes a fair dataset distillation (DD) framework called FairDD, which matches synthetic datasets to protected attribute (PA)-wise groups of original datasets. The matching prevents collapsing into majority groups and balances generation more evenly across PA groups. Experiments show that FairDD significantly improves the fairness of vanilla DD methods.", "questions": "Please see the weaknesses above.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n* Improving fairness in data distillation is an important problem.\n* The paper is overall a smooth read.\n\nWeaknesses\n\n* The novelty of this work is questionable as there is a very similar work that has already been published at ICML 2024:\n**Cui et al., \"Mitigating bias in dataset distillation,\" ICML 2024**. The authors need to cite this work, which uses sample reweighting scheme utilizing kernel density estimation, and clearly explain and demonstrate why FairDD outperforms it. The above approach may be more advanced or at least comparable to the proposed alignment approach. Currently the baselines are just adding FairDD to existing DMF approaches, which is not sufficient.\n\n* This work is not the first work to point out that vanilla DDs fail to mitigate bias as claimed in the Conclusion. In addition to the ICML 2024 work above, here is an earlier work whose ArXiv version is cited in the paper: **Lu et al., \"Exploring the Impact of Dataset Bias on Dataset Distillation,\" CVPRW 2024**.\nThe authors should tone down their contributions.\n\n* How do you know the right ratio without looking at the final model's fairness? The paper emphasizes reflecting the sample ratio of each group, but does not consider the learnability of groups at all. For example, even if two groups A and B have a ratio of 2:1, but B is more challenging to learn, it may be better to weight them 1:3 instead of 1:2 in order to maximize fairness.\n\n* Another practical concern is whether the sensitive attributes are always available in image datasets. Unlike structured datasets, it is rarer for images to have this information. There needs to be a discussion what to do when this information is not available.\n\n* If sensitive attributes are available, there are actually more fairness measures (in addition to Visual Fairness) to consider including demographic parity and equal opportunity, which should also be supported in the paper.", "was_revised": true, "final_justification": "I carefully read the rebuttal and appreciate the clarifications on the comparison with related work, data-centric goals, sensitive attribute availability, and fairness measures along with new experiments. Assuming all these points will be reflected in the revision, I am willing to increase my score to 4.", "rebuttal": "**Q1: Difference from IMCL work**\n\nThank you for bringing this work [1] to our attention. While both studies aim to address bias, they differ significantly in both scope and technical approach.\n\n**Different Scope:**\n\nTheir work focuses on mitigating bias to improve classification accuracy on the TA without considering protected attributes (PA), staying within the traditional setting of DD. \n\nIn contrast, our work tackles a broader challenge, which ensures fairness with respect to PA while preserving TA accuracy. We use formal evidence to illustrate that vanilla DD fails to mitigate the bias of the original dataset but amplifies the bias, and further provide an effective solution to mitigate this phenomenon in the distilled dataset. We are the first attempt to consider attribute fairness into DD, opening up a novel and underexplored direction that connects fairness and efficient data disrtillation\n\n**Different insight:**\n\nOur paper provides three main insights into fairness distillation for subsequent research:\n\n1. Fairness-Aligned Objective: The distilled dataset should be constructed such that its expected representation is unbiased across all attribute groups, aligning with fairness principles at the data level.\n\n2. Equal Group Contribution: Each group should contribute equally to the overall loss, encouraging balanced optimization and preventing group-specific bias during training.\n\n3. Distributional Coverage of TA: It is essential to ensure that the distilled dataset maintains adequate distributional coverage over the target attribute (TA), supporting both fairness and classification accuracy.\n\n**Different Approach:**\n\nTheir method mitigates sample-wise bias by assigning higher weights to samples located in low-density regions of the original data distribution, primarily for improving TA classification. \n\nBy contrast, our method adopts a group-level alignment strategy to promote fairness across protected attribute groups. This achieves a balance between fairness and accuracy.\n\n[1] Cui, Justin, et al. \"Mitigating bias in dataset distillation.\" In ICML 2024.\n\n**Q2: Performance comparison**\n\nSince their work does not release their code, we follow the implementation details presented in their paper. We conduct experiments on CMNIST-FG and CIFAR10-S using DM at IPC=10.\n\n|CMNIST-FG|ACC |DEO-M|DEO-A|\n| --- | --- | --- | --- |\n|DM | 25.01| 100.0| 99.96 |\n|DM +FairDD|94.61 | 17.04 |7.95|\n|ICML|90.42 | 34.77 |18.46|\n\n|CIFAR10-S| ACC |DEO-M|DEO-A|\n| --- | --- | --- | --- |\n|DM | 37.88| 59.20| 39.31 |\n|DM +FairDD| 45.17 | 31.75 |8.73|\n|ICML| 41.42 |48.20|22.48|\n\nTheir method can mitigate the distillation bias to some extent. However, this approach does not guarantee that the alignment objective is unbiased across all attribute groups, nor does it ensure adequate distribution coverage. Instead, our methods have a fairness alignment objective to facilitate unbiased data distillation; In addition, we provide a theoretical proof to guarantee the distribution coverage for TA.\n\nWe summarize the three main advantages of FairDD over theirs:\n\n(1) Better preservation of target attribute representations:\nTheir approach assigns higher weights to samples in low-density regions of the data distribution. However, these samples may lie on the periphery of the data manifold and carry low information. As a result, the distilled dataset presents the patterns with low information and hinders the distillation of informative patterns. In contrast, our method explicitly aligns the centroids of each group (defined by protected and target attributes) between the distilled and original datasets. This ensures that each group in the distilled dataset preserves representative and informative patterns, thereby maintaining the semantic integrity of the original distribution.\n\n(2) Better support for protected attribute fairness:\nTheir weighting strategy can still underrepresent minority groups if those groups are dense in the data manifold. This results in biased generation against minority group attributes. In contrast, our method is agnostic to sample density and directly aligns groups across both PA. This ensures fair representation for all attribute groups, regardless of their density, and effectively mitigates PA-related bias.\n\n(3) Stronger theoretical foundations:\nTheir work primarily relies on empirical evaluation and does not provide a theoretical explanation for why distilled datasets inherit biases from the original data, or how their method mitigates these biases. In contrast, we offer a formal theoretical analysis that explains why dataset distillation naturally inherits bias from the source data. Furthermore, we provide provable guarantees on both target attribute accuracy and protected attribute balance.\n\nNonetheless, this does not undermine the significance of their method as a pioneering contribution toward bias mitigation in classification tasks. \n\nWe will supplement the related discussion in the revised version.\n\n**Q3: Comparison with CVPRW work**\n\nThank you for pointing this out. The referenced work [2] investigates the impact of bias on TA accuracy primarily from an empirical perspective. In contrast, our work is, to the best of our knowledge, the first to provide a theoretical explanation for why vanilla DD fails to mitigate bias. Moreover, we introduce attribute fairness into the DD framework and broaden the scope of dataset distillation to consider both performance and fairness. Our method is grounded in a theoretical foundation, offering formal guarantees for both TA accuracy and PA fairness. **We will revise the Conclusion to more accurately reflect these distinctions and acknowledge prior work accordingly:** \"`This is the first work to introduce attribute fairness into the field of dataset distillation and to systematically provide a theoretical analysis of why vanilla dataset distillation fails to mitigate attribute bias.`\"\n\n[2] Lu, Yao, et al. \"Exploring the impact of dataset bias on dataset distillation.\" In CVPRW 2024. \n\n**Q4: How do you know the right ratio without looking at the final model's fairness?**\n\nThank you for your insightful comments. We would like to clarify that our goal is to mitigate attribute-wise bias present in the original dataset in a manner that is agnostic to the downstream model. The idea that an attribute-balanced dataset can help reduce bias is a well-established concept in the deep learning community. Building on this foundation, our method achieves model fairness without relying on additional regularization techniques.\n\nFor your point about the learnability of each attribute, **this is fundamentally a model-centric problem, as it depends on the characteristics of the downstream model. In contrast, both DD and our proposed FairDD are data-centric tasks, agnostic to any specific model architecture.** While incorporating model-specific information into data generation may be a valid direction, **it aligns more closely with the domain of model-level fairness: how to train a fair model given a dataset. In contrast, FairDD addresses data-level fairness: how to generate a distilled dataset that is inherently unbiased, agnostic to the downstream model.** These two perspectives approach fairness from distinct yet complementary directions.\n\n**Q5: Another practical concern is whether the sensitive attributes are always available in image datasets.**\n\nAs is common in many research fields, progress often begins with supervised approaches before transitioning to unsupervised solutions. For example, the representative method GroupDRO [3] also needs the group label to achieve fairness. After that, there are many approaches without labels. The need for labels does not diminish their contribution to model fairness. We believe this is a necessary process that calls for community-wide efforts rather than individual solutions.\n\nWe investigate the case where such labels are unavailable. As detailed in Appendix L, we apply the unsupervised clustering algorithm DBSCAN to generate pseudo attribute labels, and show in Table 16 that FairDD maintains good performance even under this setting. This indicates that our method is compatible with label-free scenarios, at least to some degree.\n\n|| CMNIST-BG |ACC |DEO-M|DEO-A|\n| --- | --- | --- | --- | --- |\n|DC|Prior label| 90.84| 20.66| 9.94 |\n|DC|Pseudo label|90.99 | 27.96 |10.57|\n\n[3] Sagawa, S., Koh, P. W., Hashimoto, T. B., & Liang, P. (2020). \"Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization.\" In ICLR 2020.\n\n**Q6: Including demographic parity and equal opportunity, which should also be supported in the paper.**\n\nEqual Opportunity (EO) means that all groups should have the same true positive rate. DEO measures how much this condition is violated when there are multiple groups or attributes. DEO-M captures the worst-case difference in true positive rates between any two groups, while DEO-A computes the average difference across all groups.\n\nDemographic Parity (DP) requires that all groups receive positive predictions at the same rate, regardless of their true labels. However, in our work, we are specifically concerned with the fairness of true positive samples. Therefore, we adopt DEO-M and DEO-A as our evaluation metrics, both of which are widely used in prior research. While it is always possible to compute additional metrics for a more comprehensive evaluation, we argue that DEO-M and DEO-A already sufficiently evaluate the fairness in our setting."}]

Author rebuttal:

Meta-review: The paper introduces FairDD, a fairness-aware dataset distillation framework that partitions original data by protected attributes (PA) and aligns synthetic subsets accordingly. Reviewers agreed that the paper identifies and addresses an important and underexplored problem. They found the theoretical justification (particularly Theorem 5.1) and the comprehensive empirical evaluation across multiple datasets convincing. The results consistently demonstrate that FairDD improves fairness while also enhancing accuracy, which reviewers considered a strong point.

The reviewers raised some concerns about the novelty, applicability, and baselines. Despite these limitations, reviewers recognized the contributions of being the first to explicitly formalize fairness in dataset distillation, providing theoretical grounding, and showing strong empirical benefits. With clarifications and additional results provided in the rebuttal, the consensus is that the strengths outweigh the weaknesses.
The AC thus recommends acceptance.
Decision: Accept (poster)