Title: Fetch and Forge: Efficient Dataset Condensation for Object Detection
Venue: NeurIPS 2024
Accepted: yes
Keywords: Dataset Condensation

Object Detection
Abstract: Dataset condensation (DC) is an emerging technique capable of creating compact synthetic datasets from large originals while maintaining considerable performance. It is crucial for accelerating network training and reducing data storage requirements. 
However, current research on DC mainly focuses on image classification, with less exploration of object detection.
This is primarily due to two challenges: (i) the multitasking nature of object detection complicates the condensation process, and (ii) Object detection datasets are characterized by large-scale and high-resolution data, which are difficult for existing DC methods to handle.
As a remedy, we propose DCOD, the first dataset condensation framework for object detection. It operates in two stages: Fetch and Forge, initially storing key localization and classification information into model parameters, and then reconstructing synthetic images via model inversion. 
For the complex of multiple objects in an image, we propose Foreground Background Decoupling to centrally update the foreground of multiple instances and Incremental PatchExpand to further enhance the diversity of foregrounds.
Extensive experiments on various detection datasets demonstrate the superiority of DCOD. Even at an extremely low compression rate of 1\%, we achieve 46.4\% and 24.7\% $\text{AP}_{50}$ on the VOC and COCO, respectively, significantly reducing detector training duration.

Fetch and Forge: Efficient Dataset Condensation for Object Detection

Abstract

Dataset condensation (DC) is an emerging technique capable of creating compact synthetic datasets from large originals while maintaining considerable performance. It is crucial for accelerating network training and reducing data storage requirements. However, current research on DC mainly focuses on image classification, with less exploration of object detection. This is primarily due to two challenges: (i) the multitasking nature of object detection complicates the condensation process, and (ii) Object detection datasets are characterized by large-scale and highresolution data, which are difficult for existing DC methods to handle. As a remedy, we propose DCOD, the first dataset condensation framework for object detection. It operates in two stages: Fetch and Forge, initially storing key localization and classification information into model parameters, and then reconstructing synthetic images via model inversion. For the complex of multiple objects in an image, we propose Foreground Background Decoupling to centrally update the foreground of multiple instances and Incremental PatchExpand to further enhance the diversity of foregrounds. Extensive experiments on various detection datasets demonstrate the superiority of DCOD. Even at an extremely low compression rate of 1%, we achieve 46.4% and 24.7% AP 50 on the VOC and COCO, respectively, significantly reducing detector training duration.

1 Introduction

In the past decade, the field of deep learning has witnessed the emergence of high-performance models, driven by the development of convolutional and Transformer networks [10, 24, 8, 5]. These models, while benefiting from large-scale datasets, also encounter significant challenges such as data storage and the demand for extensive training resources. Dataset condensation (or distillation) [27, 32, 2] has emerged in response, aiming to synthesize a small amount of data that approximates the training effectiveness of original datasets, thus offering hope to alleviate this dilemma.

Current dataset condensation (DC) research mainly focuses on image classification and is divided into two main frameworks: Meta-learning and Data-matching [11]. As shown in Figure 1 (a), the Meta-learning framework [27, 17, 34, 15] optimizes synthetic datasets by minimizing performance risks on the original dataset's validation set, using a bi-level optimization process. In contrast, Figure 1 (b) illustrates the Data-matching framework, which aligns gradients [32], feature distributions [31], or training trajectories [2] to simulate the original data's impact during different model training stages.

!Classification

ObjectDetection

S

T

T

MetaTest

Fetch

#(0)

(1)

(t)

S

Lcls

(a) Meta-learning Framework

Stage -I

Stage-II

θ

?)

Inversion

(0)

S

Lmatch

Grad

Dist

()

T

Forge

T

(Trajectoryi1

(b) Data-matching Framework

(c) Our DCOD Framework

ForwardPass

Synthetic Set

Trained on &

0(0) Initialized Parameters

S

←- ---- Backward Pass

sJ Original Set

(t)

Trained on T

Freeze Parameters

While both frameworks above make significant contributions to the field, they share common limitations. The complexity and computational expense of their bi-level optimization process restrict their application, typically confining them to smaller datasets (e.g., CIFAR10, CIFAR100, Tiny-ImageNet) and simpler network architectures (e.g., ConvNet-3, AlexNet, VGG-11, ResNet-18). This limitation is particularly pronounced in object detection tasks, which are inherently more complex than image classification. Specifically, i) the requirement in object detection for simultaneous localization and classification of multiple objects in an image considerably complicates the condensation process; and ii) the large-scale, high-resolution nature of detection dataset images, along with the complexity of detection network structures, poses significant challenges in extending existing DC methods.

In this study, we propose DCOD, the first dataset condensation framework for object detection. DCOD is distinct in its ability to flexibly generate foreground objects of varying sizes, shapes, and categories at any position within an image, as demonstrated in Figure 2. Additionally, DCOD streamlines the process by eliminating the complexities of traditional bi-level optimization, enhancing its compatibility with complex detection networks and large-scale, high-resolution data. DCOD operates in two stages: Fetch and Forge. As in Figure 1 (c), in the first stage Fetch, we train an object detector on the original dataset following a standard procedure, during which key information from the original data is stored within the detector. Then, in the second stage, we employ the model inversion to synthesize images from the trained detector. Specifically, we freeze the detector's weights and input initialization images and labels (including position, size, and category) randomly sampled from the original dataset to capture the instance distribution of real originals. During the inversion process, we propose the Foreground Background Decoupling ( F BD ) to enhance attention to the foreground areas through random erasure guided by a coarse mask and Incremental PatchExpand ( I PE ) expands a single image into multiple patches, each guided by different target labels, thus synthesizing a richer variety of instances. Finally, we utilize the detector's loss function as the guiding task loss for dataset condensation. To ensure the quality of the generated images, we implement two types of regularization: pixel-level alignment and feature-level alignment. Our method underwent rigorous evaluation on the MS COCO [13] and Pascal VOC [7, 6] datasets, achieving top-tier results that highlight its significant effectiveness. With some problems left open and a considerable improvement room existing, we hope this pilot study will attract more community interest.

The primary contributions of this paper can be summarized as follows:

We propose DCOD, the first dataset condensation framework for object detection datasets. Utilizing a two-stage process of Fetch and Forge, it can simultaneously condense crucial localization and classification information from the original dataset.

For the multi-object distribution characteristic of object detection datasets, we propose the Foreground Background Decoupling strategy and Incremental PatchExpand which notably boost the diversity of multi-instances, all within a constrained storage budget.

(b)Mutliple-objectsyntheticimages

Extensive experiments on various detection datasets underscore the superiority of DCOD. Remarkably, even at an extremely low compression rate of 1%, we achieve 46.4% and 24.5% AP50 on the VOC and COCO datasets respectively, significantly reducing the training duration for object detection.

2 Related work

Dataset Condensation. Dataset condensation (or distillation) aims to compress large-scale original data into a small amount of synthetic data, accelerating network training while maintaining comparable performance. Current research primarily focuses on the field of image classification and can be divided into two frameworks: meta-learning and data matching. In the meta-learning framework, methods such as DD [27], KIP [17], RFAD [15], and FRePo [34] update model parameters in the inner loop and synthetic data in the outer loop. In the data matching framework, techniques like DC [32], DSA [30], and IDC [9] utilize gradient matching, comparing neural network weight gradients from training on both real and synthetic data. Distribution matching approaches, exemplified by DM [31], CAFE [26], DataDAM [22], and IDM [33], align real and synthetic data distributions using Maximum Mean Discrepancy (MMD), often through a single optimization process but potentially limiting the performance. Trajectory matching, such as MTT [2], optimizes synthetic data by pre-calculating and storing training trajectories of expert networks from the real dataset. A critical downside of MTT is the significant storage requirement for these trajectories.

Recent research suggests compressing original data into models rather than synthetic data. DiM [25] minimizes the difference in predicted logits between real and generated images, using a generator to store original dataset information. SRe 2 L [29] indicates that key dataset information can be preserved in deep neural network training. Inspired by this, we argue that the localization and classification information of the detection data can be learned and preserved by the detector, allowing the key information to be recovered to reconstruct the synthetic image. Unlike SRe 2 L, we focus on object detection datasets with multiple instances in a single image. We propose a foreground-background decoupling strategy and incremental PatchExpand to enhance the updating of synthesized multiple instances.

Diverging from the traditional focus on image classification, our study pioneers dataset condensation in object detection. We introduce a streamlined approach that separates the traditional bi-level optimization into a two-stage Fetch and Forge process. This strategy effectively realizes dataset condensation in object detection tasks with increased efficiency.

High Training Overhead in Object Detection. Object detection methods are broadly divided into two categories: one-stage and two-stage detectors. One-stage detectors, such as YOLO series [19, 20], SSD [14], DSFD [12], are valued for their speed and simplicity. Conversely, two-stage detectors like Faster R-CNN [21] prioritize accuracy. Training object detectors, particularly on a single GPU, demands substantial time. Training a complex model like Faster R-CNN on datasets such as COCO can take several days or even weeks, depending on the configuration and hardware. This extensive training time underscores the pressing need for more efficient training methodologies in object

Stage-l:Fetch

Detector

Train

Real Training Images T

Stage-l: Forge

LoCk

LoC.

%

Ldet

Detector

Imgs

Suppressbackground

+

Targets:

Rreg

'Update

xyMhc

Enhanceforeground

Cls.

IncrementalPatchExpand

Fore-back Decopuling

SyntheticImages

OptimizationwithLoc.and Cls.

detection, highlighting the growing importance of dataset condensation advancements. To address this issue, we introduce the first dataset condensation framework for object detection.

3 Dataset Condensation for Object Detection

3.1 Preliminary

Dataset condensation reduces a large real dataset T into a smaller synthetic dataset ˜ S . We first extend the existing DC [27, 32] optimization framework to accommodate detection task. For object detection, each image is annotated with multiple bounding boxes and their associated class labels. The real dataset T , containing N labeled images, is given by T = { ( X i , A i ) } N i =1 , where A i = { ( x j , y j , w j , h j , c j ) } n i j =1 , with x j , y j , w j , h j representing the coordinates and sizes of the j -th bounding box, and c j being the class label. The synthetic dataset ˜ S , comprising M images where M ≪ N , is formulated as ˜ S = { ( ˜ X i , ˜ A i ) } i M =1 , where ˜ A i = { ( ˜ x j , ˜ y j , ˜ w j , ˜ h j , ˜ c j ) } ˜ n i j =1 represents the synthetic bounding boxes and their corresponding class labels. The optimization process for dataset condensation is twofold:

Model Parameters Update (Inner Loop): The model parameters θ are optimized over the synthetic dataset ˜ S to minimize the loss function L ˜ S ( θ ) :

Synthetic Dataset Update (Outer Loop): The synthetic dataset ˜ S is refined to minimize the loss on the real dataset T using the model parameters θ ˜ S ( ˜ S ) obtained from the inner loop:

However, this optimization process underlying this procedure involves a complex bi-level optimization scheme, which leads to high computational costs. The challenge intensifies when dealing with intricate model structures in the inner-loop or large synthetic datasets in the outer-loop. In the context of object detection, where high-resolution images and complex network structures are the norm, employing such bi-level optimization becomes even more impractical. This emphasizes the need for more streamlined and practical methods in dataset condensation, especially for tasks like object detection that involve high computational demands.

3.2 Fetch and Forge: DCOD Framework

Pioneering the application of dataset condensation to object detection, we develop the Dataset Condensation for Object Detection (DCOD) framework, which is a novel two-stage process depicted in Figure 3. This method diverges from the traditional bi-level optimization that compresses information directly into synthetic data. Instead, we optimize the model and synthetic data separately. Subsequent sections will delve into the optimization objectives for the Fetch and Forge stages.

Stage-I: Fetch. For object detection tasks, the real dataset is given by T = { ( X i , A i ) } N i =1 , with A i encompassing bounding boxes and class labels. By training an object detector ψ θ parameterized with θ on the original dataset T , we capture key information crucial for the tasks of localization and recognition within images. The optimization can be expressed as:

with L det representing the composite loss function that combines localization loss L loc and classification loss L cls :

minimizing L det adjusts θ to detect objects with higher precision.

Stage-II: Forge. In the forge stage, we borrow principles from model inversion [16, 28, 3], utilizing a well-trained detection model to guide the embedding of essential information back into the synthetic images. To achieve this, it is essential to recognize the unique aspects of detection tasks: an image typically contains multiple foreground objects with different positions, sizes, and shapes. Therefore, we propose Foreground Background Decoupling ( F BD ) and Incremental PatchExpand ( I PE ). Finally, we use the detection loss as the task loss to guide the update of the targets' corresponding areas with the foregrounds of various categories.

Foreground-Background Decoupling. We initialize the synthetic set ˜ S = { ( ˜ X i , ˜ A i ) } i M =1 by randomly selecting a subset of images and their corresponding targets from the original dataset, which includes position coordinates and category labels. The primary objective of object detection is to distinguish the foreground from the background and then classify the foreground objects. Therefore, we prioritize synthesizing the foreground. Additionally, while the background is treated as a single category in detection tasks, we adopt a background suppression strategy to avoid blending different contextual semantics (e.g., integrating sky or ocean information into a grassy background).

To this end, we propose Foreground-Background Decoupling ( F BD ) to separately handle the updating of the foreground and background areas, as follows:

We first separate the foreground and background regions using a binary mask based on the bounding box coordinates from the original labels. For the background ( x back), we apply a suppression strategy controlled by a hyperparameter α , which limits updates to preserve contextual information. For the foreground ( x fore), we employ a random erasure method R E , ensuring the model focuses more on refining foreground pixels during the inversion phase. By strategically applying suppression and enhancement, this approach prioritizes important visual information, optimizing the model's ability to generate high-fidelity synthesized images.

Incremental PatchExpand. Unlike classification datasets that contain only one object, images in detection datasets sometimes contain multiple instances of several different categories. Inspired by curriculum learning, we propose Incremental PatchExpand ( I PE ) to learn more instances during training. By adopting this approach, we increase the number of divisions, thereby introducing complexity into the synthetic data. We first standardize all images to a uniform size via padding. Subsequently, we employ a procedure, as defined by the function:

Algorithm 1 Dataset Condensation for Object Detection

Input:

Training set T

Required: Randomly initialized set of synthetic samples ˜ S , initialized detector ψ θ parameterized with θ , training iterations K, learning rate η s

1: Stage-I: Fetch

2: Train ψ θ on T : θ T = arg min θ L det ( ψ θ ( X ) , A )

3: Stage-II: Forge

4: for k = 0 , . . . , K -1 do

5: Apply F BD and I PE for synthetic set ˜ S = F BD ( I PE ( ˜ S ))

6: Compute task loss L det using Equ. 4

7: Compute R reg using Equ. 8, where compute R pixel and R feature using Equ. 9 and Equ. 11

8: Calculate L = L det ( ψθ T ( ˜ X ) , ˜ A ) + R reg

9: Update ˜ S ← ˜ S η s ∇ s L

10: end for

Output:

Condensed synthetic dataset ˜ S

which divides the images into k × k patches and each patch is then optimized for different targets, enhancing the diversity in object position, size, and shape. The incremental introduction of more patches allows the model to gradually adapt to various complexities and learn effectively across a broader range of scenarios.

Optimization with Pixel and Feature Regularization. Following the model inversion and decoupling framework [28, 29], the synthetic images ˜ X are optimized by solving the following minimization problem:

where L det denotes the detection loss and θ signifies the parameters of the model applied to ˜ X . Here, ˜ A represents the set of annotations for the targets present in ˜ X . The term R reg , a regularization term, is crucial for preserving the intrinsic qualities of the original dataset within the synthetic images and is defined as the sum of pixel and feature regularization:

The pixel-level regularization R pixel comprises two terms:

with R TV promoting spatial smoothness through the Total Variation regularization:

and R l 2 minimizing the squared Euclidean norm of pixel values. The coefficients α TV and α l 2 are hyperparameters that balance the Total Variation and L 2 terms, respectively.

Feature-level regularization R feature aims to align the feature statistics of the synthetic image ˜ X with the batch normalization (BN) layers of the pre-trained model, as follows:

where µ l ( ˜ X ) and σ 2 l ( ˜ X ) represent the mean and variance of the activations at the l -th BN layer of ˜ X , while µ BN l and σ BN l correspond to the statistics derived from the pre-trained model. This feature regularization is fundamental for the consistency of the feature distribution and the utility of ˜ X in training object detectors. At last, we summarize the complete two-stage process in Algorithm 1.

4 Experiment

4.1 Experiment Setup

The standard evaluation of dataset condensation consists of two steps: first, condensing the original training set into a smaller synthetic dataset; then, training an initialized model with this synthetic data and evaluating it on the original dataset's test set to assess the synthetic data's effectiveness in model training.

Datasets and Metrics. Our DCOD method is evaluated on Pascal VOC [7, 6] and MS COCO [13] benchmarks, with image resolution set to 512x512. For Pascal VOC, we merge the trainval sets of VOC2007 and VOC2012 into a single training set with 16,551 images, using the VOC2007 test set for evaluation. MS COCO comprises 80 categories with 118,287 training and 5,000 test images. Both datasets are assessed using standard COCO metrics: mAP (mean Average Precision), AP 50 (0.5 IoU threshold), AP 75 (0.75 IoU threshold), and size-specific AP s , AP m , AP l for performance evaluation. We initialize the network with pretrained backbone weights and train it 5 times on the distilled dataset. The evaluation is conducted on the original test dataset to obtain the ¯ x ± std .

Implementation Details. The initialization of the synthetic set involves random sampling from the original images, adhering to the specified compression ratio. The compression rate in classification tasks is typically set to around 1%. In our task, for VOC, we evaluate at compression rates of 0.5%, 1%, and 2%, while for COCO, we report at 0.25%, 0.5%, and 1%. During the standard image synthesis process, we set the image learning rate at 0.002. The weight for the task loss is set at 1, while R feature is assigned a weight of 0.1. The α TV and α l 2 is established at 1 and 0.001, respectively. The α in background suppression strategy is set to 0.7. All experiments were performed on a single V100 gpu.

4.2 Experimental Results

Counterpart Methods. As we are the first dataset condensation methods for object detection, we refer to the comparison settings of general DC in classification [27, 32] and choose three core-set selection methods: Random [18], K-center [23], and Herding [1, 4]. The Random method randomly selects real images from the original training set to form a subset. To adapt the K-center and Herding for object detection tasks, we implement a straightforward modification by using a pre-trained feature extractor (ResNet50) to process the features of entire images and employing the L2 norm to measure distances. Additionally, considering class balance, we also employ the Uniform, which is commonly used in practical applications within the field of object detection.

Comparison Results. Table 1 and 2 provide the comparative results of our dataset condensation method on the VOC and COCO datasets, underlining its superior performance at all compression rates. According to the official ultralytics/yolov3 implementation, the detector achieves an mAP of 46.5% and AP 50 of 76.4% on VOC when trained with the full dataset, and 35.6% mAP on COCO, which serves as an approximate upper limit of performance. On VOC, using a 2% compression rate, our method still reaches an AP 50 of 50.7%. On COCO, at a 1% compression rate, the mAP and AP 50 reach 12.1% and 24.7% respectively. Compared to random sampling, at the lowest compression rate of 0.5% on VOC, our method increases mAP and AP 50 by 9.3% and 22.1% respectively, and at COCO's lowest compression rate of 0.25%, our method improves mAP and AP 50 by 3.7% and 7.5% respectively. Moreover, while the Uniform method shows slight improvement over Random, it still falls short of the performance achieved by our approach. The Coreset method, focusing only on the overall image features and neglecting instance-level details, shows poor performance. Our method also demonstrates comprehensive advantages in multi-size object detection metrics like AP s , AP m , AP l , indicating that our DCOD significantly enhances the diversity of foreground positions, sizes, and shapes.

4.3 Ablation Study

Effectiveness of Each Component. Table 3 shows that using model Inversion from Forge as our baseline, our proposed augmentations I PE and F BD improve performance by 1.7% and 9.4% respectively. Combined, they yield an 11.5% overall improvement. Notably, the baseline alone underperforms compared to random sampling. We found that without these augmentations, the detection loss quickly converges in early iterations, missing key positional and classification details. Thus, incorporating both I PE and F BD is crucial to enhance the diversity and effectiveness of the synthetic images.

Cross-architecture Generalization. We verify the performance of condensed data on new architectures, and in Table 4, our approach is able to maintain good generalization at 1% compression. However, we observe a larger performance drop at lower compression rates. This is due to the fact that the one-stage detector and two-stage detector architectures differ significantly, preserving limited information when too few images are synthesized.

Sensitivity of Hyper-Parameters. Figure 4 shows how different α TV regularization weights affect our method's performance, using the VOC dataset with a 1% compression rate. The results indicate that an optimal regularization weight enhances performance, while no regularization or too much regularization reduces it. This happens because lack of regularization introduces excessive noise, and too much regularization causes excessive smoothing in the synthetic images, both of which degrade the quality of the images.

GT

Iter 1000

Iter 0

Iter 2000

Iter 3000

Iter 500

irdl

bird 0.78

oird0.89

060P

bird 0.91

bird 0.92

oirdl

bird 0.15

bird0.59

bird 0.65

bird 0.78

bird 0.85

sofa0.91

sofo0.94

doq0.90

person 0.94

dog0.90.

cor0.8cor0.81

00.86

persg0.0.88

person 092

tvmonitor 0.9

bottle 0.91

horse 0.83

horse_0.92

bicycle 0.88

person 0.91

bus 0.93

bus0.88

dog0.91

dog0.91

motorbia.A.o7

person0.85

sofo0.88sofo0.89

train 0.82

troin0.83

dog0.86

Performance upper bound As shown in Figure 7, we compare the performance variations of the Random baseline method and the DCOD method as the compression ratio increases. The performance of the full dataset, marked by a gray line, serves as the theoretical upper bound. When the ratio is below 5%, DCOD shows a significant advantage over the random method, while as the ratio exceeds 20%, the performance of both methods converge.

Discussion on Initialization We discuss the impact of initialization sampling on the performance of the synthetic dataset in this section. As shown in Table 5, the Random method, which is the primary choice for the experiments in this paper, is simple and performs well. Initialization strategies based on K-center and Herding lead to a decline in performance. This is these core-set methods fail to account for the complex factors in detection datasets, such as class distribution and object size distribution, resulting in significant differences between the synthetic and original datasets. Using noise as the initialization method, due to the absence of any prior information from the original dataset, causes a notable drop in the effectiveness of the synthetic dataset.

4.4 Visualization

We present a comparative analysis of synthetic images across various iterations, as shown in Figure 5. Notably, there is a progressive enhancement in the foreground scores with advancing iterations, indicative of the incremental integration of category-specific beneficial information into the images. Specifically, for sheep in the first row, the scores at iter500 and iter1000 start lower than the initial

50

80

Full:76.4

Full:46.5

70

40

60

50

30

50

mAP

40

20

30

20

10

Random

Random

DCOD (Ours)

10

DCOD (Ours)

FullDataset

Full Dataset

0

0

0

5

10

15

20

0

5

10

15

20

Compression Ratio (%)

Compression Ratio (%)

image but surpass initial scores after iter2000. This trend implies that early images are initially disrupted by noise from the update process, but as iterations progress, pixel, and feature regularization help maintain image clarity.

Additionally, we also showcase the visualization results of synthetic images containing multiple instances, as shown in Figure 6, demonstrating that DCOD effectively supports the generation of multiple instances required in detection tasks.

5 Conclusion, Limitations and Future Work

In this study, we introduce the first dataset condensation framework for object detection, DCOD. It is a two-stage framework where, in the first stage, Fetch stores information vital for detection from the original dataset into model parameters. In the second stage, Forge, the images are then reconstructed through model Inversion. We propose foreground background decoupling and incremental PatchExpand, to improve the efficiency and diversity of the synthetic images. Extensive experiments are conducted on two standard detection benchmarks, Pascal VOC and MS COCO, to demonstrate that DCOD can significantly maintain superior performance even at an extremely low compression rate.

We acknowledge the limitations of our work from two perspectives. First, due to the significant differences among object detectors, we did not extend our approach to more complex detectors like DETR, as this might require more specialized designs to accommodate their structures. Second, the performance across different architectures is still insufficient and remains an area that needs improvement.

Acknowledgments and Disclosure of Funding

This work was supported by National Natural Science Fund of China (62076184 , 62473286, 62306046) in part by Shanghai Natural Science Foundation (22ZR1466700).

References

Francisco M Castro, Manuel J Marín-Jiménez, Nicolás Guil, Cordelia Schmid, and Karteek Alahari. End-to-end incremental learning. In Proceedings of the European conference on computer vision (ECCV) , pages 233-248, 2018.

George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A Efros, and Jun-Yan Zhu. Dataset distillation by matching training trajectories. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4750-4759, 2022.

Akshay Chawla, Hongxu Yin, Pavlo Molchanov, and Jose Alvarez. Data-free knowledge distillation for object detection. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 3289-3298, 2021.

Yutian Chen, Max Welling, and Alex Smola. Super-samples from kernel herding. arXiv preprint arXiv:1203.3472 , 2012.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2020.

Mark Everingham, SM Ali Eslami, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes challenge: A retrospective. International journal of computer vision , 111:98-136, 2015.

Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision , 88:303-338, 2010.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770-778, 2016.

Jang-Hyun Kim, Jinuk Kim, Seong Joon Oh, Sangdoo Yun, Hwanjun Song, Joonhyun Jeong, Jung-Woo Ha, and Hyun Oh Song. Dataset condensation via efficient synthetic-data parameterization. In International Conference on Machine Learning , pages 11102-11118. PMLR, 2022.

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems , 25, 2012.

Shiye Lei and Dacheng Tao. A comprehensive survey to dataset distillation. arXiv preprint arXiv:2301.05603 , 2023.

Jian Li, Yabiao Wang, Changan Wang, Ying Tai, Jianjun Qian, Jian Yang, Chengjie Wang, Jilin Li, and Feiyue Huang. Dsfd: dual shot face detector. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5060-5069, 2019.

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13 , pages 740-755. Springer, 2014.

Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part I 14 , pages 21-37. Springer, 2016.

Noel Loo, Ramin Hasani, Alexander Amini, and Daniela Rus. Efficient dataset distillation using random feature approximation. Advances in Neural Information Processing Systems , 35:13877-13891, 2022.

Alexander Mordvintsev, Christopher Olah, and Mike Tyka. Inceptionism: Going deeper into neural networks. 2015.

Timothy Nguyen, Zhourong Chen, and Jaehoon Lee. Dataset meta-learning from kernel ridge-regression. In International Conference on Learning Representations , 2020.

Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition , pages 2001-2010, 2017.

Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 779-788, 2016.

Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767 , 2018.

Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems , 28, 2015.

Ahmad Sajedi, Samir Khaki, Ehsan Amjadian, Lucy Z Liu, Yuri A Lawryshyn, and Konstantinos N Plataniotis. Datadam: Efficient dataset distillation with attention matching. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 17097-17107, 2023.

Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. arXiv preprint arXiv:1708.00489 , 2017.

Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 , 2014.

Kai Wang, Jianyang Gu, Daquan Zhou, Zheng Zhu, Wei Jiang, and Yang You. Dim: Distilling dataset into generative model. arXiv preprint arXiv:2303.04707 , 2023.

Kai Wang, Bo Zhao, Xiangyu Peng, Zheng Zhu, Shuo Yang, Shuo Wang, Guan Huang, Hakan Bilen, Xinchao Wang, and Yang You. Cafe: Learning to condense dataset by aligning features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 12196-12205, 2022.

Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, and Alexei A Efros. Dataset distillation. arXiv preprint arXiv:1811.10959 , 2018.

Hongxu Yin, Pavlo Molchanov, Jose M Alvarez, Zhizhong Li, Arun Mallya, Derek Hoiem, Niraj K Jha, and Jan Kautz. Dreaming to distill: Data-free knowledge transfer via deepinversion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8715-8724, 2020.

Zeyuan Yin, Eric Xing, and Zhiqiang Shen. Squeeze, recover and relabel: Dataset condensation at imagenet scale from a new perspective. arXiv preprint arXiv:2306.13092 , 2023.

Bo Zhao and Hakan Bilen. Dataset condensation with differentiable siamese augmentation. In International Conference on Machine Learning , pages 12674-12685. PMLR, 2021.

Bo Zhao and Hakan Bilen. Dataset condensation with distribution matching. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 6514-6523, 2023.

Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. Dataset condensation with gradient matching. In International Conference on Learning Representations , 2020.

Ganlong Zhao, Guanbin Li, Yipeng Qin, and Yizhou Yu. Improved distribution matching for dataset condensation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7856-7865, 2023.

Yongchao Zhou, Ehsan Nezhadarya, and Jimmy Ba. Dataset distillation using neural feature regression. Advances in Neural Information Processing Systems , 35:9813-9827, 2022.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction clearly outline the primary contributions and scope of the paper.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We describe it in Section 5.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification:

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We describe it in Section 4.1.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: We commit to releasing the complete code later.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We describe it in Section 4.1.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: In all exp table we consider error bars.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We use a singlev100 gpu in section4.1.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification:

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: The potential positive societal impacts are described in Abstract and Introduction. In our view, this work has no potential negative social impact.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\n1) The paper introduces a dataset compression technique and applies it for the first time in object detection tasks.\n 2) For large-scale object detection tasks, the condensation of datasets can not only shorten model training time but also save a lot of computational resources.\n3) The method proposed in the paper is simple and effective, which can effectively solve the problems raised.\n\nSTRENGTHS:\nDCOD is distinct in its ability to flexibly generate foreground objects of varying sizes, shapes, and categories at any position within an image. Additionally, DCOD streamlines the process by eliminating the complexities of traditional bi-level optimization, enhancing \n its compatibility with complex detection networks and large-scale, high-resolution data.\n\nWEAKNESSES:\n1.Algorithm needs to be optimized.\n2.The description of the second stage of DCOC is not clear enough.\n\nQUESTIONS:\n1、In the experimental section, it is possible to add several more datasets for comparison of condensing methods.\n2、Why is the author's two-stage model framework more effective than a complex bidirectional compression optimization framework? Please discuss in detail.\n3、It cannot be seen from Figure 3 what the target in the annotation refers to.\n4、Formula 5's introduction is not specific enough.\n5、How is each patch optimized for different goals and increased complexity in the IPE module?\n6、Some annotations are missing in Tables 1 and 2, such as how the results of the entire dataset are obtained. What does \"±\" mean?\n7、Based solely on the experimental results, due to the lack of comparison with similar experiments, the experimental results lack some persuasiveness.\n8、How is the synthesized image updated under the guidance of loss?Suggest adding composite images with multiple instances in the visualization result graph.\n9、Lack of reference to Figure 5 in the visualization section.\n\nLIMITATIONS:\n1.Due to the significant differences among object detectors, you did not extend your approach to more complex detectors like\n DETR, as this might require more specialized designs to accommodate their structures. \n2. The performance across different architectures is still insufficient and remains an area that needs improvement.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-27"}

comments: SUMMARY:
The paper attempts to generalize dataset condensation to object detection. It proposes Fetch and Forge. Fetch: training a normal object detector on the original dataset as usual. Forge: the synthetic images are sampled from the original data and optimized by the detection loss. Experiments on VOC and COCO show the effectiveness of the method.

STRENGTHS:
1. The paper is well-motivated and easy to understand. 

2. The proposed methods are technically sound.

3. The experiments show promising results and there is much room for future.

WEAKNESSES:
1. Notation issue: see $\mathcal{I}_{PE}$ in Eqn. 6 and Step 5 of Alg.1.

2. Why is row 3 of Table 3 blank?

3. Implementation Details are not comprehensive. Is it the training epoch for IPE set to $M$/batchsize? And I would suggest not calling it a training epoch.

4. What would happen if we increase the condensation ratio, e.g., 5%, 10%, 20%, 50%?

5. Typos: it should be ''ratio'' in Table 3.

QUESTIONS:
see weaknesses above

LIMITATIONS:
see weaknesses above
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper looks at the task of dataset condensation for object detection. While there have been many works looking at dataset condensation for classification this is the first work to look at it for detection which is more challenging as each image can contain multiple objects of different categories. This is an important problem as it reduces the training requirements. The proposed method operated in two stages, first training a model on the full data. Followed by model inversion to get synthetic images which capture the most important classes and regions. The authors also propose new techniques to handle the multi object nature of images and show good results on VOC and COCO.\n\nSTRENGTHS:\nThe paper has the following strengths:\n1. The paper looks at an important and challenging problem of dataset condensation for object detection. I have seen many works look at data distillation for classification but this is more challenging and novel task to look at.\n2. The authors provide a clear way to propose the first method for this task. They remove the need for expensive 2 step approaches which include meta learning or feature/data matching and rather simplify the problem with the proposed fetch and forge approach.\n3. The authors show results on common datasets like VOC and COCO and while they cannot match the full data performance I think it might be a good first step.\n4. The authors do ablation of various components, hyperparams and generalization and also show qualitative results.\n\nWEAKNESSES:\nThe paper has the following weaknesses:\n\n1. In object detection and recognition multiple works have show that the context (background) plays an important role. [Noise or Signal: The Role of Image Backgrounds in Object Recognition]. The Fetch a Forge paper continuously suppresses background (L155-1156) and considers background not important. Is there backing of these claims and why would context (background) not help?\n2. Lacking discussion about the input initialization. What effect do different input initializations have. Is there a way to find an optimal initialization? I think the initialization part needs to be discussed in more detail.\n3. Section 3.1 seems less relevant given the paper is talking about object detection not classification. It should be modified to describe the setup in terms of detection data.\n4. Missing discussion of what happens with different initializations during the Fetch part. Do you get similar performance and visualization of the condensed data?\n5. Missing analysis of whether during condensation for object detection location is more important or classification while performing inversion to get the condensed synthetic samples.\n6. The authors should also report at what percent of condensed data they can meet the real number. That is important to get some idea of compression to number of images tradeoff and what the best performance this method can give.\n\nSmall clarifications:\n1. Its confusing why incremental patch expand is needed if the network is already trained during fetch step.\n2. Missing ablation regarding the Reg loss in Eq 7 and also within it the importance of pixel vs feature regularization.\nL112 For the complex of -> For the complex case/scenario\nL167 course -> coarse\n\nQUESTIONS:\nI think the paper is well written and in general easy to follow and sound. At the same time I have mentioned a few clarification and discussions in the weaknesses and would hope the authors answer some of them. Especially around background, initializations and upper bound with more images. Overall I feel the problem is important and the authors propose a first step towards solving it.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis manuscript introduces, as far as I’m aware, the first method to do object detection dataset condensation. The method accomplishes this by first training a detection model on the original dataset called the Fetch stage, then uses this trained model to synthesize a condensed dataset through model inversion and regularization, called the Forge stage. Two modules are introduced to aid in inversion: Forward Background Decoupling (FBD) and Incremental PatchExpand (IPE). FBD induces separate processing on background and foreground pixels; background pixels are down weighted and foreground pixels are randomly masked. IPE progressively divides images into smaller and smaller patches as training progresses and processes different targets in each of these patches. A total variation regularizer is used to ensure synthesized image smoothness and a batch norm feature regularizer is used to ensure the batch norm mean and variance statistics are similar between real and synthetic images. Results on generated datasets outperforms random subsamples of the larger real dataset.\n\nSTRENGTHS:\n*S1:* As far as I’m aware, this is the first work on dataset condensation for object detection and appears to be performant.\n\n*S2:* Aside from the missing definitions I’ve mentioned in the weaknesses section, this manuscript was clear and well written.\n\nWEAKNESSES:\n**Weaknesses:**\n\n*W1:* The distribution of classes varies quite a bit at small/medium/large object sizes for both VOC and MS-COCO. It’s possible that K-Center and/or Herding sampling strategies conform to this distribution, but if not, I expect that this would be a strong baseline and should be compared against.\n\n*W2:* In line 159, it states “the foreground is derived from the bounding box coordinates of the current targets”. This needs to be more precise. How exactly is it derived? Is a union taken over all bounding boxes of the current batch and those pixels used as the foreground?\n\n*W3:* In line 167, it states “inspired by course learning”. Is this meant to be curriculum learning? \n\n*W4:* “Expand” in Eq. 6 is never defined. Are image patches upsampled be the same size as the original image? \n\n*W5:* The rate of increase for $k$ in the IPE module is never provided. \n\n*W6:* Does $\\tilde{A}$ stay the same as the images sampled at initialization to run synthesis on? If so, how are these samples selected? This seems like it would drastically affect performance.\n\n*W7:* line 218: What is “the Uniform”? I have worked in object detection for years and this term isn’t something I recognize. A citation here is needed.\n\n*W8:* Specify how error bars were computed. Are these the standard deviation over multiple runs of dataset condensation?\n\n\n\n**Additional comments, not affecting my rating:**\n\n*C1:* The description of object detectors in line 99-102 isn’t quite correct. Fast models here are the YOLO series and SSD models. These are fast because they don’t include additional processing for small objects, e.g., an FPN. Faster R-CNN and RetinaNet are more accurate because they include an FPN. \n\n*C2:* On line 222: what open-source implementation is used? This should have a citation.\n\n*C3:* Line 230: “The Coreset method” I assume this is referring to K-Centers, but terminology should stay consistent.\n\n*C4:* It would be interesting to compare dataset condensations across more similar architectures. For instance, it should be easy to train a dataset condensed on RetinaNet or Faster-RCNN with FCOS, ATSS, or TOOD.\n\n*C5:* It would be interesting to see Figure 5 on multi-class images.\n\nQUESTIONS:\nThis manuscript is interesting and if the authors address the weaknesses I’ve listed, I’ll be happy to raise my score from a 4 to a 5 or above. The most involved item I’d like to see is W1. This should hopefully be a relatively straightforward calculation followed by evaluation on already trained models. W2-8 are items that need to be added to the manuscript to ensure the method can be reproducible given the manuscript text.\n\nAdditionally, I’ve provided additional comments on how to improve the manuscript at the end of the weaknesses section, but these have not factored into my score.\n\nLIMITATIONS:\nThe manuscript has addressed any concerns I’d have regarding limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-17"}