Distilling Datasets Into Less Than One Image
Authors: Asaf Shul
Eliahu Horwitz
Yedid Hoshen
Venue: Accepted by TMLR ()

Abstract: Dataset distillation aims to compress a dataset into a much smaller one so that a model trained on the distilled dataset achieves high accuracy. Current methods frame this as maximizing the distilled classification accuracy for a budget of K distilled images-per-class, where K is a positive integer. In this paper, we push the boundaries of dataset distillation, compressing the dataset into less than an image-per-class. It is important to realize that the meaningful quantity is not the number of distilled images-per-class but the number of distilled pixels-per-dataset. We therefore, propose Poster Dataset Distillation (PoDD), a new approach that distills the entire original dataset into a single poster. The poster approach motivates new technical solutions for creating training images and learnable labels. Our method can achieve comparable or better performance with less than an image-per-class compared to existing methods that use one image-per-class. Specifically, our method establishes a new state-of-the-art performance on CIFAR-10, CIFAR-100, and CUB200 on the well established 1 IPC benchmark, while using as little as 0.3 images-per-class.

Published in Transactions on Machine Learning Research (03/2025)
Distilling Datasets Into Less Than One Image
Asaf Shul"
asdf.shaul@mailllnjj..cco i
The Hebrew University of Jerusalem
Eliahu Horwitz'
The Hebrew University of Jerusalem
Yedid Hoshen The Hebrew University of Jerusalem
Reviewed OpenReview: https: /ttpp:://ppereviaeu net/ permm/ id= www.s
Classic Dataset Distillation Image Plr-Class
Distillation (PoDD)
0.5 Imagen-Prr-ee Per-Class
Figure 1: Poster Dataset Distillation (PoDD): We propose PoDD, a new dataset distillation setting for a tiny, under image-per-class (IPC) budget. In this example, the standard method attains an accuracy of 35.5% on CIFAR-100 with approximately 100k pixels, PoDD achieves an accuracy of 35.7% with less than half the pixels (roughly 40k)
Abstract
Dataset distillation aims to compress a datasot into a much smaller one so that a model trained on the distilled dataset achieves high accuracy. Current methods frame this as maximizing the distilled classification accuracy for budget of K distilled images-per-class, where K is positive integer. In this paper, we push the boundaries of dataset distillation, compressing the datasct into less than an imagc-per-class. It is important to rcalize that the meaningful quantity is not the number of distilled images-per-class but the number of distilled pixels-per patess-er--attsst. We therefore, propose Poster Dataset Distillation (PoDD), a new approach that distills the entire original datasct into a single poster. The poster approach motivates new technical solutions for creating training images and learnable labels. Our method achieve comparable better performance with less than image-per per- class compared to existing methods that use one image-per-class Specifically, our method establishes a new state-of-the-art performance on CIFAR-10, CIFAR-100, and CUB200 on the well established IPC benchmark, while using as little as 0.3 images-per-class.
"Equal contribution
Project page: https://horittz.ai///

---

Published in Transactions on Machine Learning Research (03/2025)
Original Dataset All Images
Subset/Coreset K% images- Per-Class
Dataset Distillation K Images-Per-Class
Poster Dataset Distillation Ke 1 Image-Per-Cass
All Allimagos
-600M Poelss
R36 PFC.. =-12M Pixel
Class 2
Class
IPC IPC
=" 1M Pools
Kel IPC IPC --30 30K Ploals
Poo Clea
Figure 2: Dataset compression scale: We show increasingly more compressed methods from left to right. The original dataset contains all of the training dala and does not perform any compression. Coreset methods select subset of the original dataset, without modifying the images. Dataset distillation methods compress an entire dataset by synthesizing K E N+ images-per-class (IPC). Our method, Poster Dataset Distillation (PoDD) distills an entire dataset into single poster that achieves the same performance as 1 TPC while using as little as 0.3 TPC
1 Introduction
Deep-learning methods require large training datasets to achieve high accuracy. Dataset distillation (Wang et al. 2018) allows distilling large datasets into smaller ones so that training on the distilled dataset results high accuracy. Concretely, dataset distillation methods synthesize the images-per-class (IPC) that are most relevant for the classification task. Dataset distillation has been very successful, achieving high accuracy with as little single imagc-per-class.
In this paper, ask: "can go lower than one inage-per-olass?" Existing dataset distillation met are unable to do this as they synthesize one or more distinct images for each class. Assuming there are n classes, such methods would require distilling at least n images. On the other hand, using less than 1 IPC implies that several classes share the same image, which current methods do not allow. We therefore propose Poster Dataset Distillation (PoDD), which distills an entire dataset into single larger image, that we call poster. The benefit of the poster representation the ability to use patches that overlap between the classes, thus better utilizing redundancies of pixels across images. We set the size of the poster it has significantly fewer pixels than in n images, therefore enabling distillation with less than IPC. We find that correctly distilled poster is sufficient for training a model with high accuracy, outperforming current methods. See Fig. 2 for an overview of different dataset compression methods.
et al. 2009) where each image is of size x 32 pixcls. Current methods synthesize images independently and thus must use at least IPC (sce Fig. 10 (left)). Choosing IPC for CIFAR-100 entails using 100 images, each of size 32 x 32 pixels. In contrast, PoDD synthesizes single poster shared between all classes. During distillation,
To illustrate the idea of poster, consider CIFAR-100(Krizlerr
optimize all the pixels so that classifier trained on the resulting dataset achieves maximal accuracy. For example, to achieve 0.4 IPC, we represent the entire dataset as a single poster of size 202 x 202 pixels (see Fig. 10 (right)). This has about the same number of pixels as 40 images, each of size 32 x 32. The number of effective TPCs is therefore directly given by the size of the poster.
To distill poster, we first initialize all pixels with random values. We transform the poster into a dataset in differentiable way, by extracting overlapping patches, each with the same size image in the source
2

---

Published in Transactions on Machine Learning Research (03/2025)
dataset (e.g., 32 x 32 for CIFAR-100). During distillation, optimize this set of overlapping poster patches and propagate the (overlapping) accumulated gradients from the distillation algorithm back to the poster. The optimization objective is to synthesize a poster such that a classifier trained on the dataset extracted from it will reach high classification accuracy. This process requires a label for every patch, we therefore propose Poster Datasel Distillation Labeling (PoDDL), a method for poster labeling that supports both fixed and learned labels.
Since classes can now share pixels, their order within the poster matters it implics which classes share pixels with each other. It is thus important to find beneficial ordering of classes the poster. To this end, we propose Poster Class Ordering (PoCO), an algorithm that uses CLIP (Radford et al. 2021 text embeddings to order the classes semantically and efficiently.
Overall, in addition to stretching the limits of dataset compression, the overlapping patches in PoDD for- mulation lead to improved performance on existing IPC benchmarks. This holds true even when using less than 1 IPC, where PoDD can match or improve the performance that previous methods achieved on the well established 1 IPC benchmark. Indeed, sometimes PoDD can outperform competing methods using low as 0.3 IPC. PoDD sets new state-of-the-ar for 1 IPC on CIFAR-10, CIFAR-100, and CUB200.
To summarize, our main contributions are:
1. Proposing the poster dataset structure, which extends dataset distillation for tiny, less than IPC budgets.
2. Developing a method to perform PoDD that constitutes of class ordering algorithm (PoCO) and a labeling strategy (PoDDL).
3. Performing extensive experiments that demonstrate the effectiveness of PoDD with as low as 0.3 IPC and achieving new SoTA on the well established IPC benchmark.
2 Related works
Dataset distillation, introduced by Wang et al. (2018), aims to compress an entire datasct into a smaller, synthetic one. The goal is that methods trained on the distilled dalaset will achieve similar accuracy to a model trained on the original dataset. As highlighted by (Rebuffi et al.) [017] Castro et al.) 2018; Jubran et al. 2019), dataset distillation shares similaritics with coresct selection. Coresct selection identifies a representative subset of samples from the training set that can be used to train a model to the same accuracy. Unlike coreset selection, the generated synthetic samples of dataset distillation provide flexibility and improved performance through continuous gradient- based optimization techniques. Dataset distillation methods can be categorized into 4 main groups: Meta-Modes Matching (Wang et al., 2018 Nguyen
et a.l 2021: Loo et al.
2022 Zhou et. al.
2022 Feng et al.
2023) minimize the discrepancy between the
transferability of models trained on a distilled data and those trained on the original dataset. ii) Gradient
2022b Kim et al., 2022). proposed by Zhao et al. (202)), performs one-step distance matching between a network trained on the target dataset and the
Matching (Zhao et al.)
2020 Zhao & Bilen 2021 Lee et al.
same network trained on the distilled dataset. This avoids unrolling the inner loop of Meta-Model Matching methods. iii) Trajectory Matching (Cazenavette et al. 2022 Cui et al. 2023), proposed by Cazznavette et al. (2022), focuses on matching the training trajectories of models trained on the target distilled dataset and the original dataset. iv) Distribution Matching (Zhao & Bilen 2023 Wang al.) 2022 Lee et all 2022a
introduced by Zhao & Bilen (2023), solves a proxy task via single-level optimization, dircctly matching the distribution of the original dataset and the distilled dataset. See (Sachdeva & McAuley 2023) for an in-depth explanation and comparisons of the various distillation methods. Common to all these methods is the use of at least one IPC. Deng & Russakovsky (2022) observed that shared representations facilitate better distillation. In contrast to their approach, in this paper we introduce a new shared structure which we call poster, allowing distilling dataset into less than TPC.
3

---

Published in Transactions on Machine Learning Research (03/2025)
Distillation
Distillation algorithm
(b) Overlapping patches with soft labels
(a) Randomly Initialized poster
(d) Distilled
poster
Inference
Truck
Plane
Cat
Car
Boat
Frog
Horse
Dog
(e) Distilled
(f) Dataset from overlapping patches with soft labels
poster
Figure 3: PoDD overview: We propose PoDD, new dataset distillation setting for under 1 images-per class. We start by initializing a random poster (a), during distillation, optimize overlapping patches and soft labels (b-c). The final distilled poster has fewer pixels than the combined pixels of the individual images (d). During inference, we extract overlapping patches and soft labels from the distilled poster and use them to train a downstream model (e-f). PoDD achieves comparable or better accuracy to current methods while using as little as a third of the pixels
3 Preliminaries
Many methods tackle dataset distillation as a bi-level optimization problem. In this setup, the inner loop essentially involves training a model with weights on a distilled dataset Deyn- The outer loop optimizes the pixels the distilled dataset Dayn, so a model trained on Dayn has the maximal accuracy on the original dataset D. Let (D( denote the average value of the objective function, of model 0 on the datasot D. Formally the optimization problem can be described as follows:
arg min Lo (0*) Dayn
s.t.
====a arg
=rg min LD (0)
(1)
Outer Toop
Inner loop
We consider the case where the inner-loop optimization consists Tend SGD steps. The most common solution is backpropagation through time (BPTT) which unrolls the inner loop SGD optimization for Tend steps. It then uses computationally demanding backpropagation calculation to compute the gradient of the loss with respect to the distilled dataset Dayn;
arg min E Deyn
(LD (OTena))
8.8
Oct1 t 0, 11 VoLD (0:)
(2)
PgrPo
Inner loop unrolling
Outer loop
Backpropagating through Tend timesteps infeasible for large values of Tend: therefore many methods propose ways to reduce the computational costs. Here, we use RaT-BPTT (Feng al., 2023), which computes the inner loop through a random nunber of SGD steps Tend Random AT, AT +,.... T]. It
then approximates the gradient with respect to the distilled dataset by only backpropagating through the

---

Published in Transactions on Machine Learning Research (03/2025)
final AT << T steps. We chose RaT-BPTT because it achieves the top performance across many dataset distillation benchmarks.
4 PoDD: Poster Dataset Distillation
Our goal is to perform dataset distillation with less than IPC. Our main insight is that sharing pixels between classes can be effective, as this would make better use of redundant pixcls. Clearly, this requires more than one class to share an image. In this section, we propose Poster Dataset Distillation (PoDD) which provides the methods to realize this idea.
4.1 A shared poster representation
The key idea in this work is to distill an entire dataset into a single larger image that we call a poster. The poster can be of arbitrary height d, and width dw, leading to total of d₃ x d, pixels; posters with fewer pixels are said be more compact. Furthermore, we define a fixed procodure for converting a poster into distilled dataset. Our procedure extracts multiple overlapping patches each extracted patch is equal in size to the original images (see Fig, for an overview). We employ the same stride for both rows and columns and denote the total mumber of extracted patches by p. Consoquently, this yields a dataset of images, some of which share pixels. Without any overlap, optimizing the poster is equivalent to distilling each image separately.
fixed, user determined stride;
We initialize the poster with standard Gaussian noise and optimize its pixels end-to-end through both the dataset expansion described above and the inner-loop bi-level optimization described in 3 As result, the gradients from overlapping patches accumulated. The size of the distilled dataset is measured by the total number of pixels in the poster, allowing us to compare with previous approaches that used one more IPC. We provide pseudocode for PoDD in 2
4.2 PoCO: Poster Class Ordering
The poster representation relies on shared pixels between neighboring classes. To maximize the effectiveness of the shared pixels, it is therefore important establish good structure of neighboring classes. We hypothesize that poster would be more effective when pixels are shared between semantically related classes (e.g., Man, Woman, Boy vs. Plate, Tree, Bus).
Algorithm PoCO: Pseudocode for PoCO class order-
ing
(a) Initialized empty grid
Input: E: CLIP Text encoder.
Input: Oz, Oy: Class grid dimensions.
Input: C: Class list of the target dataset.
+ 08,Xay
(b) Zigzag traverse path
0p. + Cij
3: C 3
4: G Distance matrix using & embeddings
5: for i,j in oz, (Zigzag traverse) do
ngbr + {(i.j) neighboring classes in O} ct C-CCarg (iinggnnn
(c) Greedy
tiling
8:
Op +c
10: end for
Figure 4: PoCO tiling: (a) Initialize empty
11: return 0
grid O, (b) Traverse O in Zigzag order, (c)
Greedily tile O using CLIP text embeddings
5

---

Published in Transactions on Machine Learning Research (03/2025)
We propose method for approximating good initial class neighborhood structure. First, extract embedding for each class name using the CLIP (Radford et al. 2021 text-encoder &. Given the set of embeddings, we calculate the pairwise distance matrix between all classes. Using this pairwise class distance, we design greedy procedure for placing the classes on a roctangular grid x*=x0, where 0z = |C], denoting the spatial positions of classes. Let C be the set of classes and Cp be the set of already placed classes. We traverse the grid a Zigzag order (Chai et al. 2017). and at each step place a class as follows:
Σ (1-18t001) [[((TT]
(m) E(c) [[(m)]] -
O
min cECiC
ie [oo), j E [oy]
me( 12:0is
Intuitively, at each step, we place a remaining class that has the lowest distance from all its already-placed neighbors. We visualize this Zigzag traversal in Fig. 4 and summarize the PoCO algorithm in [ in Fig. we show an example PoCO tiling for CIFAR-100.
4.3 PoDDL: Poster Dataset Distillation Labeling
Having initialized the poster and the class order matrix O, now describe our labeling strategy. Previous approaches
more saages-per-class, hence, they can simply assign single label per image. However, using single label for the entire poster is not a good option, instead, we assign soft label (Sucholutsky & Schonlau 2021) vector to each overlapping patch. We therefore design a poster-oriented soft labeling strategy that supports both fixed and learned labels, see Fig, 5 for an overview.
Fixed labels. We upsample the class order matrix to the size the poster dy dw. For each overlapping patch, we extract its corresponding class label window. We compute majority class and use it as the one-hot label for the patch. In the of ties, we use soft label with equal probabilities for the majority classes.
Learned labels. As our method extracts an arbitrary number of overlapping patches, lcarning soft label for each patch would require more parameters than previous approaches. To keep the number of parameters constant, we learn a parameter tensor of the same size as previous works, and interpolate it to each overlapping window.
Concretely, learn label tensor Y of size Og Oy and spatially upscale it to the shape of the poster using nearest neighbor interpolation. The final size of Y is d, dhe n. For each overlapping patch, extract the corresponding label window and average pool it. To achieve valid label distribution, normalize the resulting vector. We use this vector as the learned soft label of the window. After each gradient step, we clip negative values Y to zero to avoid negative probabilities. Unlike the fixed labels, the learned labels are optimized alongside the distillation process.
Lı
(a) Poster
(b) Label array Y
(c) extracted window
(d) Soft label
Figure 5: PoDDL estraction: Each poster patch has a corresponding patch in the label array (a-b). We compute the poster patch label by extracting a patch along the channels of the label array (c). To obtain the final soft label for given poster patch, we pool and normalize the extracted label window, resulting in soft label vector (d). PoDDL supports both fixed and learned labels

---

Published in Transactions on Machine Learning Research (03/2025)
Algorithm 2 PoDD: Pseudocode using PoDDL learned labels
Input: Alg: Distillation algorithm.
Input: D: Dataset with classes C.
Input: da, dw: Poster size.
Input: p: Number of overlapping patches.
1: (000 1)dxxe
# Tnitialize poster from Gaussian
2: 0 PoCO(C)
# Initialize class order
3: Y + PoDDD(O,p)
# Initialize distilled labels array
4: for each distillation stop do
5: Dayn = {(p, 1) overlapping patches and labels from P and Y}
6: P.Y ADg(D, Dayn)
# Distill one step
7: end for
8: Return P, Y
5 Experiments
5.1 Experimental setting
Datasets. We evaluate PoDD on four datasets commonly used to benchmark dataset distillation methods:
i) CIFAR-10: 10 classes, 50k images of size 32 32 x (Krizhevsky et al.] 2009). ii) STFAR-100: 100 classos,
50k images of size 32 x 32 x (Krizhevsky al. 2009 ii CUB200: 200 classes, 6k images of size 32 32
(Welinder et al. 2010). iv) Ting-ImageNet: 200 classes, 100k images of size 64 64 3 (Le & Yang 2015
Distillation Method. For the dataset distillation algorithm, we use RaT-BPTT Feng et al. 2023), a recent. method that achieved SoTA performance on CIFAR-10, CIFAR-100, and CUB200 by a wide margin. In particular, RaT-BPTT's architecture uses three convolutional layers for 32 x 32 datasets and four layers for 64 x 64 datasets. Baselines. Our baselines can be divided into two groups: i) Inmer-Lopp: BPTT (Deng& & Russakovsky 2022), empirical feature kernel (FRePO) (Zhou al.) 2022]. , and reparameterized convex implicit gradient (RCIG) (Loo et al. 2023). and ii) Modified Objectives: gradient matching with
augmentations (DSA) (Zhao & Bilen 2021 distribution matching (DM) (Zhao & Bilen 2023). trajectory
matching (MTT) (Czznaavelle et al., 2022), flat trajectory distillation (FDT) (Du et al. 2023), and TESLA
(Cui et al. 2023). We use the results reported by the baselines.
Evaluation. Following the protocol of (Zhao & Bilen 2021 Deng & Russakovsky 2022). we evaluate the distilled poster using a set of 8 different randomly initialized models with the same ConvNet. (Gidaris & Komodakis 2018) architecture used by DSA, DM, MTT, RaT-BPTT, and others. The architecture includes convolutional layers of 128 filters with kernel size 3 x 3 followed by instance normalization [llyanov et al. 2016), ReLU (Nair& & Hinton 2010) activation, and an average pooling layer with kernel size 2 x 2 and stride 2. We report the mean and standard deviation across the 8 random models. We evaluate two TPC regimes:
Less than one IPC. We compute the results for IPCs within the range: K E (0.3,0.4, ...,,........ 1). Our criterion for success is if PoDD with K IPC performs comparably or better than RaT-BPTT with 1 IPC. To comply with previous baselines, we used PoDDL, which optimized the same number of label parameters as previous methods with 1 IPC.
One IPC. To properly compare PoDD with existing methods also evaluate it with the same total number of pixels used by our basclines (1 IPC). Since PoDD still uses overlapping patches, this evaluates the impact of compressing inter-class redundancies in the shared pixels, this is compared to the baselines which are unable to do so.
Implementation Details We use the same distillation hyper- parameters used by RaT-BPTT (Feng et al. 2023) except for the batch sizes. To fit the distillation into a single GPU (we use an NVIDIA A40), we use the maximal batch size can fit into memory for given dataset, (see exact breakdown below). Since the
7

---

Published in Transactions on Machine Learning Research (03/2025)
Table 1: Less than one Image-Per- Class: PoDD with less than one IPC (image-per-class) often out- performs stale-of-the-art methods with 1-IPC; in some cases with as little as 0.3 IPC. In (red), the relative performance drop compared to the 1 IPC results. In bold, the lowest IPC for which PoDD beats the current SoTA
Method
IPC +
CIFAR-10 t CIFAR-100 f
CUB200 f
T-ImageNet f
RaT-BPTT Feng et al. 2023
1.0
53.2
35.3
13.8
20.1
PoDD (Ours)
1.0
59.1 10.5
38.3
16.2
20.0.1..3
0.9
(8.55))))
(0,2%)))
15.2 (o.4%))
19.2:10.((25)
0.8
56.7 (49))
37.3
15. (5.69)
19.0=t0.2(3%)
0.7
54.6 (89%)
37. (3.3%)
35.0++0.((8%))
18. 18.6 (..9%)))((79)
PoDD (Ours)
0.6
50.6 (1.3())))))
36.6 36.611.3(69))
15.1 15.110.2(77)
18.8. (..2..)))))))
0.5
49.5 49.5=0.5(16%)
36.0 36.010.3(69))
(5.0:)0.3(8%)
18.7 188.::0.1(7%)
0.4
47. 1 (209))
35.7 (...
15.0
18.410.2(89))
0.3
42.3=0.36 ((285)))(28%)
34.7=0.2
(4.8++0.69996)
18.440.1
Full Dalaset
All
83.5
55.3+0.3
20.1
37.6+0.5
(No Distillation)
Images
optimization bi-level, distillation methods have two batch types, one for the distilled data which we denote by bss, and one for the original dataset which we denote by bs.
In addition to the K IPC parameter, we need to control the degree of patch overlap. In other words, given dataset with classes and after fixing K (i.e., once fix the size of the poster), we need to decide the number of overlapping patches to divide the poster into, As the number of classes and image resolution varies between the datasets, we use a different p for each one. Concretely, we use: i) CIFAR- 10: p = 96(16x0) patches, bsa = 96, bs = 5000, 4k epochs. ii) CIFAR-100: = 200 patches, bsa = 50, bs = 2000, 2k epochs. iii) CUB200: p = (6003))) patches, bsa = 200, bs = 3000, 8h epochs. iv) Tiny- Ned: = (00 patches, 2a. = 30, bs = 500, 500 epochs.
We use the learned labels variant of PoDDL for all of our experiments except for CIFAR-10 with K e 10.7, O, 8, 0.9, 1.0] IPC in which we the fixed labels variant (for which Icarned labels did not provide additional benefit). We use learning rate of 0.001 for CIFAR-10, CIFAR-100, and CUB200. For Tiny- ImageNet into a single GPU we use a much smaller batch size and learning rate of 0.0005.
5.2 Results
Less than one IPC. We now test our initial question: "can we go lower than one image-per-class"" Using PoDD, we show that across all four datasets, we can go much lower than IPC and for 3 out of the 4 datasets even maintain on-par performance to the SoTA basclinc. As hypothesized, using a poster that shares pixels between multiple classes allows us to reduce redundancies between classes in the distilled patches (See I. This effect is intensified when distilling datasets with large number of classes, e.g., for CIFAR-100 we can use 0.4 IPC and for CUB200 can as little 0.3 IPC and still outperform the baseline method.
One IPC. Having shown the feasibility of distilling a datasel into less than one IPC, we now quantitatively evaluate the benefit of the poster representation. To this end, we use the IPC setting which allows us to decouple the pixel count from the pixel sharing. Essentially, we are investigating whether the pixel-sharing in our poster can boost performance, even when the number of pixels matches our baseline. Our method outperforms the state-of-the-art for CIFAR-10, CIFAR-100, and CUB200, setting a new SoTA for IPC dataset distillation (See Tab. 2
5.3 Ablations
Class order ablation. We ablate the impact of the class ordering on the performance of PoDD on CIFAR- 10. We first compute the distillation performance after 250 distillation steps with 0.3 IPC for 5 random class orderings. The score of each ordering is the inverse of the sum of distances between all neighboring class pairs. The distance matrix is defined in the same way in PoCO i.e., using the embeddings of the CL.IP
8

---

Published in Transactions on Machine Learning Research (03/2025)
Table 2: One Image-Per-Clan Performance of PoDD under the imago-per-class (IPC) setting pared to SoTA dataset distillation methods across 4 datasets. PoDD sets new SoTA for CIFAR-10, CIFAR-100, and CUB200. On Tiny-lmageNet, PoDD achieves comparable results to the underlying distil- lation method it uses (RaT-BPTT). All models used ConyNet architecture with 3 convolution layers for 32 x 32 images and convolution layers for 64 x 64 images (one exception is DSA which uses AlexNet for CIFAR-10 and CTFAR-100 as seen its original paper)
Method
CIFAR-10 f
CIFAR-100
CUB200 t T-ImageNet
Average t
Inner Loop
BPTT Dong & Russakovsky 2022)
49. 1:..6
L0.3
FReP Zhou et al.
2022
45.6
26.3+0.1
16.9
RCIC Loo
2023
49.6
35.5
22.4
RaT-
53.2
35.3+0.4
13.8+003
20.1 10.3
30.6+0.4
Feng
DSA Zhao & Bilen
13.9
1.3
6.6
12.7::033
Modified
Objectives
2021
DMZhao
Zhao & Bilen
2023
26.0
11.4+0.3
1.6#0.1
3.9=0.2
10.8
MT Cazenavette et
2022
10.8
24.3 10.3
2.2
2..3
10.4
FTD
46.8
25.2+0.2
10.4
TESL Cui al.
48.5 2..88
24.8+0044
2023
PoDD (Ours)
59.1 10.5
38.2
16.2#t0.
20.0
33.4
Full Dataset
83.5.1.2
55.3
20.1
37.6.10.5
49.1
(No Distillation)
Number of
Test Accuracy 500 ep0chss
Patches (px x Pg)
55
Accurary
(0
45.15%
53
(4xx3)
47.28%
51
40 (10x4)
56.77%
49
60 (12x5)
54.14%
47
(6(16)))
56.73%
45
126(18x7)
55.55%
50
90
110
130
150
(60()0x8)
57.61%
Number of Patches
Figure 6: Number of patches ablation: Effect of the patch overlap on the accuracy (CIFAR-10, 1 IPC, (PC,500 500 distillation steps). Using 10 patches with no patch overlap results in the lowest accuracy. When increasing the number patches beyond 24, the results improve significantly. added titles to the axes
text encoder. We find that the class ordering can indeed impact the performance of the distilled poster, with a correlation coefficient of 0.76 between the score of the ordering and the accuracy the distilled poster achieves. This correlation motivates PoCO's search for good initial class ordering rather than using random one.
Patch number ablation. We ablate the role of the amount of overlap between patches of the poster (i.e., the number of patches for a given poster size and dataset). To study this, we use CIFAR-10 at IPC, we run PoDD for 500 steps multiple times, each with a progressively increasing number of patches. As can be seen in Fig. G using the same number of patches as the number of classes (i.c., no overlap between patches) this is exactly the RaT-BPTT baseline. When increasing the
results in the lowest score; this is expected number of patches, observe that beyond certain patch number threshold, the results improve drastically. This demonstrates the significance of the poster representation and the use of overlapping patches. Since the number of patches has a direct effect on the distillation time and the training time of downstream models, we use a small number of patches for the larger datasets and a larger number of patches for the smaller datasets.
More than 1-IPC ablation. Our evaluation focused on fewer IPC, a setting that has a performance gap compared to models trained the full dataset. We prioritized this low IPC setting it represents
9

---

Published in Transactions on Machine Learning Research (03/2025)
Woman
Whare
Dephand
Camel
Turtis
Lobster
Caterpilar
Ray
Gd
Courh
Dhat
Pate
Mouse
lins
Cruh
Raban
Squime
Bazy
Tank
Trectto
Tabl:
Bowl
House
Tiger
Splder
Kangaroo
Scal
Werm
Welf
Tenat
Mustroom
Poppy
Telaphone
Flathen
Skycerppr
Shrew
Sweet Poper
Wardrebe
Acuarium
Cockroach
Orange
Seetle
Bee
Raccoon
Possum
Porcucine
Rictup Truck
Telesbtin
Keyboord
Pear
Chimpanore
Figure 7: PoCO ordering: Output of PoCO for CLFAR-100, classes are separated into semantically meaningful classes (c.g., trees, humans, vehiclos). We colored semantically related clusters manually
the most compressed distilled dataset, aligning with the core vision of dataset distillation. To complete the analysis, we conducted additional experiment, evaluating PoDD) and RaT-BPTT on all common IPC settings as can be seen in Tab. G Overall, we can see that the poster becomes more beneficial as the number of IPC decreases. This makes sense, methods must make very efficient use of the distilled images for low IPC, while this is less critical for high IPC.
6 Discussion and future work
Beyond the exciting result of distilling dataset into less than one image, PoDD presents a setting and distillation representation that opens up new and intriguing research problems.
Class ordering algorithm. As in 5.3. the ordering the classes within a poster is strongly correlated with the performance of the distilled poster. We proposed PoCO, a greedy algorithm for choosing a class ordering. In Fig. 7 we show an example ordering for CIFAR-100, as can be seen, the classes are separated into semantically meaningful classes (e.g., trees, humans, vehicles). However, PoCO does not always yield a perfect ordering, c.g., the lcopard may not fit well in the top right corner. It might fit better next to the lion and the tiger (4 left and 2 down). Indeed, other ordering strategies may be better suited for the distillation task, e.g., a photometric-basee ordering that uses the color of the images or an ordering that uses the image semantics. However, computing such an ordering based on tens thousands, or even millions of images may be costly. In contrast, our label-based approach requires a single sample per class, and can thus scale to very large dataset. Further investigation of alternative ordering methods left for future work.
Patch augmentations In this work we use the extracted patches with no modifications. However, perform- ing spatial augmentations (e.g., scale, rotation) during the distillation process may be beneficial. Another option is to create cyclic poster where patches near the border of the poster are wrapped around.
7 Conclusion
In this work, we propose poster dataset distillation (PoDD), new dataset distillation setting for tiny, less than 1 image-per-class budgets. We develop method to perform PoDD and present a strategy for ordering the classes within the poster. We demonstrate the effectiveness of PoDD by achieving on-par SoTA performance on the well established IPC benchmark with as low as 0.3 IPC and by setting a new IPC SoTA performance for CIFAR-10, CIFAR-100, and CUB200.
8 Acknowledgments
This work was supported in part by the "Israel Science Foundation" (ISF), the "Council for Higher Educa- tion" (Vatat), the "Center for Interdisciplinary Data Science Research" (CIDR), the "Israeli Cyber Author- ity", and "KLA".
10

---

Published in Transactions on Machine Learning Research (03/2025)
References
Francisco M Castro, Manuel J Marin-Jiménez, Nicolás Guil, Cordelia Schmid, and Karteck Alahari. End- to-end incremental learning. In Proceedings of the European conference computer vision (BCCV), pp. 233 233-248, 2018.
George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A. Efros, and Jun-Yan Zhu. Dataset dis- tillation by matching training trajectories. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognátion (CVPR), pp. 4750-4759, 2022.
Xiuli Chai, Zhihua Gan, Yiran Chen, and Yushu Zhang. A visually secure image encryption scheme based compressive sensing. Signal Processing, 134:35-5) 51, 2017.
Justin Cui, Ruochen Wang, Si Si, and Cho-Jui Hsieh. Scaling up dataset distillation to imagenet- 1 with constant memory, In International Conference on Machine Learning, pp. 6565 6590. PMLR, 2023.
Zhiwei Deng and Olga Russakovsky. Remember the past: Distilling datasets into addressable memories for neural networks. In Proceedings of the Advances in Newral Informalion Processing Systems (NeurIPS), number 3, 2022.
Jiawei Du, Yidi Jiang, Vincent T. F. Tan, Joey Tianyi Zhou, and Haizhou Li. Minimizing the accumulated trajectory error to improve dataset distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Yunzhen Feng, Shanmukha Ramakrishna Vedantam, and Julia Kempe. Embarrassingly simple dataset distillation. In The Twelfth International Conference on Learning Representations, 2023.
Spyros Gidaris and Nikos Komodakis. Dynamic few-shot visual learning without forgetting. In Proceedings of the IEEE conference on computer vision and pottern recognition, pp. 4367-4375, 2018.
Ibrahim Jubran, Alaa Maalouf, and Dan Feldman. Introduction to corcsets: Accurate coresets. arXiv preprint ar xix::1190.0000 087007, 2019.
Jang-Hyun Kim, Jinuk Kim, Seong Joon Oh, Sangdoo Yun, Hwanjun Song, Joonhyun Jeong, Jung- Woo Ha, and Hyun Oh Song. Dataset condensation via efficient synthetic-data parameterization. In International Conference Machine Learning, pp. 11102 11118. PMLR, 2022.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.
Hae Beom Lee, Dong Bok Lee, and Sung Ju Hwang. Dataset condensation with latent space knowledge factorization and sharing. arXiv preprint ar arxiv:2208. 10494, 2022a.
Saehyung Lee, Sanghyuk Chun, Sangwon Jung, Sangdoo Yun, and Sungroh Yoon. Dataset condensation with contrastive signals. In International Conference on Machine Learning, pp. 12352 12364. PMLR, 2022b.
Noel Loo, Ramin Hasani, Alexander Amini, and Danicla Rus. Efficient dataset distillation using random feature approximation. Advances in Neural Information Processing Systems, 35:13877-1389) 2022.
Noel Loo, Ramin Hasani, Mathias Lechner, and Daniela Rus. Dataset distillation with convexified implicit gradients. Xix preprint ar arXim:332200655, 2023.
Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th international conference on machine learning (ICML-10), pp. 807 814, 2010.
Timothy Nguyen, Roman Novak, Lechao Xiao, and Jachoon Lee. Dataset distillation with infinitely wide convolutional networks. Advances in Neural Information Processing Systems, 34:5186-511 2021.
11

---

Published in Transactions on Machine Learning Research (03/2025)
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748 8763. PMLR, 2021.
Aivestro-Alvise Rebuffi, Alexander Kolesnikov, Goorg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
Noveen Sachdeva and Julian McAuley. Data distillation: A survey. orXiv preprint arXiv:2301.04272, 2023.
Πlia Sucholutsky and Matthias Schonlau. Soft-label dataset distillation and text dataset distillation. In 2021 International Joint Conference on Neural Networks (IJCNN) pp. 1-8. IEEE, 2021.
Dmitry Ulyanov, Andrea Vedaldì, and Victor Lempitsky. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022 2016.
Kai Wang, Bo Zhao, Xiangyu Peng, Zheng Zhu, Shuo Yang, Shuo Wang, Guan Huang, Hakan Bilen, Xinchao Wang, and Yang You. Cafe: Learning to condense datasct by aligning features. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pallern Recognition, pp. 12196 12205, 2022.
Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, and Alexei A. Efros. Dataset distillation. arXiv preprinl orXiv:1811.109h5 2018.
Peler Welinder, Steve Branson, Takeshi Mila, Catherine Wah, Plorian Schrolf, Serge Belongie, and Pietro Perona. Caltech-ucsd birds 200. 2010.
Bo Zhao and Hakan Bilen. Dataset condensation with differentiable siamese augmentation. In Proceedings of the International Conference on Machine Learning (ICML), pp. 12674-12685, 2021.
Bo Zhao and Hakan Bilen. Dataset condensation with distribution matching. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023.
Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. Dataset condensation with gradient matching. arXiv preprint 20v0 05929, 2020.
Yongchao Zhou, Ehsan Nezhadarya, and Jimmy Ba. Datasct distillation using neural feature regression. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2022.
12

---

Published in Transactions on Machine Learning Research (03/2025)
A
Appendix
0.9 IPC
0.8 IPC
0.7 IPC
IPC
0.5 IPC
0.4 IPC
0.3 IPC
Figure 8: Poster visualization: Here we provide the poster visualizat ion of all Less then IPC settings for CIFAR100. It can be seen that the overall structure of the poster is similar, while in larger IPC settings there are more details for cach general area.
Posters size: The poster size and label grid size used for distilling the posters in the paper.
Dataset
CIFAR-10
CIFAR-100
CUB200
T-Image
Or
10
20
20
Oy
10
10
d₂
160
320
640
1280
dy
64
320
320
640
Table 3: size hyperparameters
Uneven class distributions: We tested how docs PoDD handle class imbalance by removing some of the samples in CIFAR-10 so the class distribution will be uneven. The resulting amount of images from each class our experiment are: [5000, 4750, 4500, 4250, 4000, 3750, 3500, 3250, 3000, 2750]. We did not modify the test set. The resulted test accuracy compared R-T-BPTT, as seen Tab. We can clearly see that PoDD handles the uneven dataset better then RaT-BPTT, and that the learned labels also have positive effect.
13

---

Published in Transactions on Machine Learning Research (03/2025)
Method
RaT-BPTT
PoDD Learned Tabels
PoDD Fixed Labels
Accuracy (%)
46.8
56.8
55.7
Table 4: Uneven class distributions ablation: Test results of PoDD compared to RaT-BPTT uneven subset of CIFAR-10
Downstream tasks train time While may appear that the larger number of patches increases training cost, we demonstrate that does not do so practice. Indeed, our model may converge significantly fewer epochs than baseline models. We sometimes find that using fewer epochs increases accuracy due to less overfitting, We report our accuracy as function of training epoch below. Note that the baseline RaT-BPTT used set amount of 2000 epochs, which for CIFARIO IPC1 results in 20,000 steps, to be stepwise equal need to lower the number of epochs accordingly. Below there are tables that show the effect of lowering the number of epochs for each dataset. Tab. 5 shows the results for all IPCs settings reported in the main paper.
Method
IPC
CIFAR-10
CIFAR-100
CUB200
TinyImageNet
Adjusted
Full
Adjusted Full Runtime
Adjusted
Full
Adjusted
Full
RaT-BPTT PoDD
~
24
48
48
48
h
36 h
48 h Accuracy
72h
h
~ 72h
h
96 h
RaT-BPTT
1.0
53.2
35.3
13.8
20.1
PoDD
1.0
59.43
59.11
38.15
38.26
16.45
16.24
19.31
19.96
PoDD
0.9
58.27
58.32
37.18
37.16
15.58
14.99
18.63
19.15
PoDD
0.8
55.94
55.94
37.31
37.11
15.81
15.22
18.48
18.95
PoDD
0.7
54.39
54.48
37.04
36.82
15.61
14.91
18.38
18.51
PoDD
0.6
49.78
49.84
36.29
36.15
15.15
14.88
18.40
18.75
PoDD
0.5
49.49
49.45
36.04
36.06
15.18
14.71
18.03
18.49
PoDD
0.4
46.32
47.15
35.59
35.53
15.09
14.92
17.91
18.34
PoDD
0.3
42.57
42.13
34.59
34.42
14.75
14.44
17.77
18.13
Table 5: Downstream tasks train time accuracy comparison: Results and training times of PoDD on all reported datasets, with training steps adjusted to match RaT-BPTT. As can be seen, adjusting the number of training steps does not decrease the performance, allowing PoDD to train in the same time as RaT-BPTT while increased the increased performance.
IPC Method
RaT-BPTT
PoDD
53.2
59.1
69.4
71.0
50
75.3
75.3
Table 6: Larger IPC ablation: Test accuracy results of PoDD the CIFAR-10 dataset using common I-IP,, 10-IPC, 50-IPC settings compared to RaT-BPTT
14

---

Published in Transactions on Machine Learning Research (03/2025)
(a) 10 patches
(b) 24 patches
(c) 40 patches
(d) 60 patches
(e) 96 patches
(f) 126 patches
(g) 160 patches
Figure 9: Patch density visualization: Here we provide the poster visualization of CIFAR10 with different
patch numbers, as seen in 6 It shows the increasing patch overlap from no overlap at all where the patch number equals the number of classes (similar to regular distillation) up to 16 times more patches.
15

---

Published in Transactions on Machine Learning Research (03/2025)
50
test accuracy
45
40
35
distillation stop
25
15
Figure 10: Class order effect: Using CIFARIO as the base dataset to distill, we sampled 60 random class orders, and for each we conducted a distillation with the same amount of steps. We can see that the class ordering indeed has an effect on the test accuracy, with correlation to the ordering score.
Planes
Cars
Birds
Cats
Deers
Boats
Trucks
Horses
Dogs
Frogs
Figure 11: Global and Local Semantics: We train CIFAR-10 variant of PoDD with 10 IPC and separate per class poster. The local semantics well preserved, showing multiple modalities per class, e,g., different colors of cars, poses of animals, and locations of the planes. Moreover, some of the classes demonstrate global scale semantics, c.g., the planes have sky on the top and grass on the bottom
16

---

Published in Transactions on Machine Learning Research (03/2025)
Without soft labels
with soft labels
35
lequua
30
25
20
distill steps
5k
10k
15k
20k
25k
30k
Figure 12: Learnable soft labels ablation: We the same experiment the in the main paper with CIFAR100 and IPC L, but this time without the optimizing the soft labels alongside the image pixels. this resulted in a large negative impact on the test accuracy results throughout the distillation process
Car
Truck
Horses
Dog
Figure 13: Distilled poster semantics: We illustrate some of the semantics captured by CIFAR-10. IPC poster by sketching over the distilled poster. The poster is of dimension 5 x 2, with the top row containing: Truck, Plane, Bird, Deer, Cat, and the bottom row containing Car, Boat, Frog, Horse, Dog. We can see that the pixels shared between classes exhibit a smooth transition between colors
17


Reviews:
[{"review_id": "6F5cO9OeME", "reviewer": "Reviewer_9WSY", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n\n1. The paper is interesting. It proposes a method that distill one dataset into a poster. The proposed methods are novel and technically sound.\n\n2. The proposed method is simple and effective. It achieves impressive performance when compared to the state-of-the-art methods.\n\nWeakness:\n\n1. Some parts of this paper are hard to understand, such as Section 4.3. What do “fixed labels” and “learned labels” mean in this paper? The paper is suggested to provide more details for the PoDDL method. For example, what role does label array Y play? \n\n2. What is the practical significance of this work? I think it is enough to distill a dataset into a or several images for each class. Do we need to distill a dataset into less one image?\n\n3. There are some typos. For example, “we provide pseudocode for PoDD in 2”.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a method that can distill datasets into less than one image for each class. The main idea is to distill a dataset into a poster that contains all classes. There are two problems: 1) How to put the classes in this poster; 2) how to assign labels to this poster. To address the first problem, the paper proposes the poster class ordering (PoCO) method, which first extracts an embedding for each class name and places the classes on a rectangle grid using a greedy method. To address the second problem, the paper proposes the poster dataset distillation labeling (PoDDL), which assign labels by using a pooling operation.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The presentation of Section 4.3 is suggested to be revised properly to make it easier to understand.\n\nThe paper is suggested to discuss more about its practical significance.", "broader_impact_concerns": "na", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "adTchRriLe", "reviewer": "Reviewer_qYVG", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths:\n1. The paper proposes a novel design that enables dataset distillation into less than one image per class.\n2. The performance improves significantly, achieving strong results even for distillation into 0.3 IPC.\n\n\n### Weaknesses and Questions:\n1. The primary concern is the lack of comparisons with other relevant methods. The paper only compares approaches that optimize separate images for each class, while it overlooks several methods that use compressed bases [1, 2] for the distilled images. For example, [1] uses a smaller image basis to linearly span the distilled dataset and could effectively achieves less than 1 IPC within their framework. At that time, using integer IPCs was the standard setting and they only conducted experiments for that. However, with the current paper proposing new settings, a proper evaluation of these prior methods is necessary. Similar to PoDD, these methods leverage shared information across classes and images to reduce the parameters required for the distilled dataset. \n2. How do changes to the poster affect optimization speed? Does the proposed method take longer to optimize compared to baseline methods like RaT-BPTT? The paper should include runtime estimates for all settings.\n3. Use of \"Optimal\": The statement \"It is thus important to find the optimal ordering of classes on the poster\" is somewhat misleading, as PoCO is not demonstrated to generate the optimal ordering. Similarly, \"optimal\" is used in Section 5.3 and should be revised for accuracy.\n4. For the Fixed Labels setting, what would happen if soft labels were used in the first place, with the average probability after upsampling, instead of one-hot labels? How would this approach perform?\n5. In Algorithm 2, PoCO is shown to be used consistently. For PoDD with learned labels, is PoCO still used? If so, does it provide an initialization for the label tensor that is later optimized?\n6. The hyper-parameters $d_H, d_W, o_x, o_y$ are not specified in the paper.\n7. Figure 6 use $o_x, o_y$ for the number of patches. However, in Section 4.2, $o_x, o_y$ are defined as the label grid dimensions with $o_x \\dot o_y=|C|$.\n8. How does increasing the number of patches affect the visualization of the poster? Could the authors provide an analysis of how the information distribution changes on the poster as the number of patches increases? While the empirical performance presented in the paper is strong, more analysis of how information is stored when distilling into less than one image per class would enhance the paper's contribution.\n9. What network is used during training? Is it a CNN? How well does a poster trained on a CNN generalize to other architectures?\n\n---\n[1] Deng, Zhiwei, and Olga Russakovsky. \"Remember the past: Distilling datasets into addressable memories for neural networks.\" Advances in Neural Information Processing Systems 35 (2022): 34391-34404.\n\n[2] Kim, Jang-Hyun, et al. \"Dataset condensation via efficient synthetic-data parameterization.\" International Conference on Machine Learning. PMLR, 2022.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a novel distillation framework that enables dataset distillation into less than one image per class. The framework utilizes a large poster, which is transformed into patches to serve as the distilled dataset. A labeling method using CLIP embedding is proposed to designate the classes in the poster. Experimental results demonstrate the framework's effectiveness, achieving strong results even when distilling into 0.3 images per class.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Provide comparisons with prior work that utilize compressed parameterization of the distilled dataset.\n2. Include runtime estimates.\n3. Provide a cross-architecture analysis.", "broader_impact_concerns": "No need to discuss.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "F3PGNf5MyZ", "reviewer": "Reviewer_xPR6", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The paper is well written and well structured.\n\n2. The idea of distilling the entire dataset into a poster is interesting.\n\nWeaknesses:\n\n1. In Table 2, the SOTA works (for example DSA) seem to be using AlexNet as the network architecture. Is this consistent across both the proposal and the all the SOTA? Please add the details of the model architectures used.\n\n2. It is important to make clear the contributions of the paper. What is the benefit of using the proposed method versus some other SOTA work? It is essential for the reader to understand the benefits of using the poster. Do we have storage savings for the training set? Does it translate to improvement in the training time? To the best of my understanding, PoDD does not offer improvements in terms of the training time compared to SOTA as eventually the same number of images are used to train the model (we are using the various patches of the poster). By using the poster, do we save in terms of storage memory? Could you please clarify this? In Table 1 maybe add a column for each dataset that shows the benefits (maybe in storage?).\n\n3. The code is not included in the submission. Open-sourcing the code improves transparency.\n\n4. In Table 1, for using K = 0.9 there is a drop in accuracy from 1\\% till 6\\% depending on the dataset compared to using 1 IPC and for smaller K the drop is higher. Could you please elaborate more on that and share any insights for this? In connection to comment 2, if the paper highlights the savings then this creates a trade-off between savings and accuracy.\n\n5. The ablation study for the order of classes needs some more results. Please mention the accuracy of using PoCO and when the classes are ordered randomly. A table that includes this comparison would help the reader to undestand better PoCO.\n\n6. Please add the ablation study of using vs not using learned labels for the PoDD.\n\n7. From figure 3, we see that soft labels are used. Can you please report the experimental results when using soft labels and when not? \n\n8. What would be the results when using the PoDD with sharing representations and without sharing representations. in this way the reader can understand better the benefit of sharing pixels between different images. \n\nMinor comments:\n\n1. Please add the reference to Table 1 in the manuscript.\n\n2. In Table 1 and 2, the numbers reported after the plus and minus refer to the standard deviation over multiple runs/seeds? How many different seeds did you you use? Could you please add these clarifications?\n\n3. Please add the references in Table 2 for the SOTA works.\n\n4. Please mention what is the baseline that you refer to in section 5.2\n\n5. Please add the axes titles in Figure 6", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a new framework called Poster Dataset Distillation (PoDD). PoDD distills a dataset into a poster. In this way, we can have less than 1 image per class for the enitre dataset.  The approach is experimentally evalauted on CIFAR-10, CIFAR-100, and CUB200.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please address the weaknesses above.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: No further comments beyond those highlighted in the previous sections.
Decision: Accept (poster)