Training-Free Dataset Pruning for Instance Segmentation
Authors: Yalun Dai
Lingao Xiao
Ivor Tsang
Yang He
Venue: ICLR 2025 Poster (2025)
Keywords: dataset pruning
instance segmentation
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: A fast and training-free dataset pruning method for instance segmentation

Abstract: Existing dataset pruning techniques primarily focus on classification tasks, limiting their applicability to more complex and practical tasks like instance segmentation. Instance segmentation presents three key challenges: pixel-level annotations, instance area variations, and class imbalances, which significantly complicate dataset pruning efforts. Directly adapting existing classification-based pruning methods proves ineffective due to their reliance on time-consuming model training process. To address this, we propose a novel **T**raining-**F**ree **D**ataset **P**runing (**TFDP**) method for instance segmentation. Specifically, we leverage shape and class information from image annotations to design a Shape Complexity Score (SCS),  refining it into a Scale-Invariant (SI-SCS) and Class-Balanced (CB-SCS) versions to address instance area variations and class imbalances, all without requiring model training. We achieve state-of-the-art results on VOC 2012, Cityscapes, and MS COCO datasets, generalizing well across CNN and Transformer architectures. Remarkably, our approach accelerates the pruning process by an average of **1349$\times$** on COCO compared to the adapted baselines.

Published as a conference paper at ICLR 2025
TRAINING-FREE DATASET PRUNING FOR INSTANCE SEGMENTATION
Yalun Dai Lingao Xiao Ivor W. Tsang1.2.3 Yang He1.2,1+ CCARR, Agency for Science, Technology and Research, Singapore 2THPC, Agency for Science, Technology and Research, Singapore "Nanyang Technological University, *National University of Singapore daiy0018@e. ntu. edu. sq, xiao.lingao@u. nus edu [Ivor Tsang, He Yang))egaaff.... a-star edu.
ABSTRACT
Existing dataset pruning techniques primarily focus classification tasks, limit- ing their applicability to more complex and practical tasks like instance segmen- tation. Instance segmentation presents three key challenges: pixel-level annota- tions, instance area variations, and class imbalances, which significantly compli- cate dataset pruning efforts. Directly adapting existing classification-base prun- ing methods proves ineffective due to their reliance on time conecconu model training process. To address this, we propose a novel Training- Free Dataset Pruning (TFDP) method for instance segmentation. Specifically, we leverage shape and class information from image annotations to design a Shape Complexity Score (SCS), refining it into Scale- SCS) versions to address instance area variations and class imbalances, all with- out requiring model training. We achieve state-of-the-arr results on VOC 2012, Cityscapes, and COCO datasets, generalizing well across CNN and Transformer architectures. Remarkably, our approach accelerates the pruning process by average of 1349x on COCO compared to the adapted baselines. Source code is available at:
(SI-SCS) and Class Balanced (CB-
INTRODUCTION
Current dataset pruning methods (Coleman al. 2019 * Ours
EL2N
ccs (1CLR'23)
Toneva et al., 2019 Tan et al. 2023) focus on image clas- sification tasks, while neglecting more complex and prac- tical tasks such as instance segmentation. Classification is relatively simple, typically dealing with images contain- ing one primary object. In contrast, instance segmenta- tion faces greater challenges, handling real-world images with multiple objects of varying classes, areas, and posi- tions within a single image.
(JCLA
(NourIPS 211
55.0
40%
54.5
0.014 hn
Ours
50%
Entropy 21:16
53.5
40%
Accuracy
53.0
40%
50%
In this paper, address dataset pruning for instance seg- mentation and identify three unique challenges. 1) Pixel- level annotations. Unlike classification tasks where each image has single one-hot category label, instance seg- mentation tasks require labeling each pixel of an image, often with multiple different category labels present in a single image (Lin al. 2014). 2) Variable instance ar- eas. While images in classification tasks generally deal with images of consistent resolution, instance segmen- tation involves objects of varying areas within the same image. Fig. P and Appendix B demonstrates this diver- sity in multiple datasets, aligning with the observations in
CCS 20.29
52.0
50%
51.5
0.014
12:37
20.25
21.16
Time required for ranking samples (Hours)
Comparison of AP30 and runtime for sample ranking on COCO dataset: Our method vs. Entropy, EL2N, and CCS at 40% and 50% prun- ing rates, Our approach demonstrates superior efficiency and accuracy.
Figure 1:
"Corresponiing Author. This work was completed during Yalun Dai's internship at CFAR, STARR.

---

Published as a conference paper at ICLR 2025
(Lin et al. 2014). 3) Class imbalance. In contrast to classification tasks where the image count of each category is typically uniform, instance segmentation (as shown in Fig. 2b) inherently contains imbalanced object counts across classes. This imbalance stems from the natural image collection process, which often captures multiple objects of varying frequencies in real-world scenes (Oksuz)
et al. 2020). Addressing these challenges is crucial for advancing instance segmentation dataset pruning.
Additionally, existing dataset pruning methods face fundamental contradiction: they often require a time-
Instance Count (%)
8%
Reckumomma
Large
1- axii
consuming model training process to identify important
6%
samples despite aiming to reduce overall training time.
4%
This paradox stems from several fundamental issues in-
herent in existing methods: 1) These methods often ne-
2%
cessitate training on the entire dataset to calculate rel-
1%
10%
evance scores (Paul et al., 2021 Coleman et al.) 2019
Instance Area Image Area
(a) Distribution area instances
Pleiss et al. 2020 Toneva et al. 2019), negating the in- tended saving benefits. 2) Samples selected based on a single model's output often show limited generaliza- tion to models with different architectures (Yang al. 2023), further compromising efficiency gains. 3) In real- world scenarios, model training may be infeasible due to insufficient resources (Khouas et al. 22024), rendering ex- isting methods inapplicable to these scenarios.
700
Class Names
chair
12. boat
Count
bottle
neroplane
cow
tymomitor
16. motorbile
400
pottedplant
doog
18 train
bus
bicyclo
200
eat
diningtable
100
Class Index
To address the problems mentioned above, we propose a novel Training- Free Dataset Pruning (TFDP) pipeline for instance segmentation that does not require training on
(b) Distribution of number of Instances
Visualization of VOC 2012 dataset to show variable instance area (a) and class imbalance (b).
Figure 2:
any data in advance. Instance segmentation is sensitive to boundary regions (Tang et al. 202T Zhang et al. [202]] Li al. 2023): early in training, predicted masks primar- ily cover the object's Iss central area, while later stages refine the boundary pixels. Many studies have focused on making models pay more attention to boundaries to enhance final performance (Cheng et al. 2020 Wang al. 2022a Borse et al., 2021 Zhang et al. [021). Therefore, leveraging the rich shape information provided by pixel-level annotations of masks, we designed the Shape Complex- ity Score (SCS), which characterizes the importance of instance by calculating the complexity of each mask's s boundary. Furthermore, to address the inherent scale variability arising from vary- ing object sizes, we implemented Scale-Invariant for SCS (SI-SCS). allowing this score to fairly represent the complexity of mask boundaries regardless of size. Additionally, to solve the prob- lem of significant class imbalances in instance segmentation tasks (Oksuz et al., 2020), which can lead to noticeable class variability when simply summing instance-level scores to calculate image- level scores. We also design the
for SCS (CB-SCS). Specifically, we normalize instance importance scores within each class across images, ensuring each class contributes equally, regardless of its instance count. This design allows us to obtain a unified metric that enables fair comparisons across images, regardless of the number of objects. These effective designs not only address the challenges faced in instance segmentation but also enable the superiority our method in both time efficiency and performance (see Fig. I
As the first work on dataset pruning in instance segmentation and to fully validate the effectiveness of our method as illustrated in Fig. 3 we also adapt existing classifcation-ooriette dataset pruning methods to this task, implementing some baselines with strong performance. Specifically, instead of calculating the importance score for each image in the classification task, we assign an importance score for each pixel based on existing criteria. We then aggregate the scores within each object and subsequently across objects in an image derive an image-wise score. In our experiments, we con- duct extensive comparisons between our implemented baselines and our model-independerr TFDP, including performance comparisons, generalizability, and time consumption, to foster a foundational contribution to future work.
In summary, our contributions are as follows: 1) To the best of our knowledge, we are the first to introduce a training -fre dataset pruning frame- work for instance segmentation.
2

---

Published as a conference paper at ICLR 2025
2) We adapt existing classification-oriiiii methods to instance segmentation and establish strong baselines for comparison.
3) We propose novel bodellinddeeeneent importance criterion SCS based the shape information of masks to prune samples. Additionally, we implement Scale-Invariant and Maass-Balanced versions address the issues of scale variability and class imbalance.
4) Our method achieves the best results on mainstream instance segmentation datasets such as VOC 2012, Cityscapes, and MS COCO, without utilizing any model outputs. It also demonstrates enhanced generalizability across various architectures (including
and Transformer- based networks) while offering significantly faster and more practical pruning process.
2 RELATED WORK
2.1 DATASET COMPRESSION
Dataset Pruning, or Coreset Selection, reduces dataset size by selecting key samples based on
specific criteria. Herding (Welling 2009) and Moderate (Xia al. 2022) measure feature space distances, while Entropy (Coleman et al. 2019) and Cal (Margatina et al. 2021) focus on uncer-
tainty. EI.2N (Paul et al. 2021) uses gradient magnitudes to quantify importance. Some methods (Tang et al. 2023] Okanovic et al. 2023] Xu et al. [023]] Dolatabadi et al., 2022) improve train- ing efficiency via online selection. GradMatch (Killamsetty et al., 2001) and Craig (Mirzasoleiman
et al. 2020) minimize gradient differences between full and pruned datasets, while ACS (Huang ett al. 2023) accelerates quantization-awan training by selecting high-gradient samples. However, these methods primarily focus classification tasks NLP task (Nguyen & He (025) and have not explored other more complex computer vision tasks further. Additionally, they rely on models to calculate importance scores, which is time-consuming and results in limited generalizability.
Dataset Distillation an another direction to compress datasets, aiming to learn a synthetic dataset that recover the performance of the full dataset (Nguyen al., [202a bb] Zhou al.]
2022
Loo
et al.
2022; Zhao et al. 2021; | Jiang et al. 2022
) Lee et al. 2022 Loo et al.
2023
He el al.
2023
Zhao & Bilen 2023 Wa ang et 2022b Zhao et al. 2023] Cazenavette et al. 2022 Du al. 2023
Cui et al. 2023
et al. 2023 Tukan el al.] 2023 Shin el al.] [202]]
| Yin et al. [023] Yin & Shen
2023 Shao et al., 2024; Zhou al. 2024 He et al.
2024 Xiao & He 2024). However, since every
pixel requires a gradient update during the optimization process, the training cost is significantly higher than dataset pruning-
2.2 INSTANCE SEGMENTATION
Instance segmentation is a significant and challenging task in computer vision, as demands both instance-level and pixel-level predictions. The existing methods can be roughly summarized into
the following categories. Top-down methods (Li et al. 2017 He al. 2017 Liu et al. 2018b
Huang et al. 2019 Bolya et al., 2019 Chen et al. 2019b 2020: Zhang et al. 2020) address the problem by approaching it from the object detection perspective, i.e., first detecting objects and then segmenting them within the bounding boxes. Specifically, recent approaches to (Chen et al.) 2020 Zhang et al. 2020 Xie et al., (020)) build their methods on the anchor-free object detectors Tian al.) 2019), showing promising performance. 2) Bottom-up methods (Newell et al. 2017 De Brabandere et al., 2017; Liu et al.) 2017] Gao et al., 2209)) treat the task as label-and cluster issue, e.g., learning pixel embeddings and clustering them. 3) Direct methods (Wang et al. 2020a b) directly perform instance segmentation without relying on embedding learning or box detection. 4) transformer-based methods. More recently, QueryInst (Fang al. (021) and Mask2Former (Cheng et al. 2022) proposed decoding random queries into objects for end-to-end instance segmentation by extending DETR (Carion et al. 2020). Instance segmentation typically requires large training datasets, but training efficiency in this domain remains understudied.
3 METHOD
3.1 PROBLEM DEFINITION
We first define the task of dataset pruning for instance segmentation. We have a complete training
dataset D {(0, ) where D is the total number of images in the dataset. Here, 22 E X' is an
3

---

Published as a conference paper at ICLR 2025
(a) Previous Dataset Pruning on Classification
Avg
Average
Backbone Model
P
Perimeter
0.61
Classification Head
A
P
Perimeter Area
Model Training
Segmentation Head
Input Image
10 hrs
Image-wise Score
A
(b) Adaptation Instance Segmentation
0.760.00
0.02 0.01
0002
0,000000003
0.00
Avg
0.83
Cat 0.83
Dog
0.80 1.63
0.02
0.05
0.83
0.80
0.01
0.04
0.67
Model Training
Input Image
10
Pixel-wise Score
Instance-wise Score
Image-wise Score
(c) Our Traening-Free Dataset Pruning (TFDP) on Instance Segmentation
809
340 A 3290
0.11
Dog
Cat
O0o
0.11 + 0.05 0.16
No Model
175
0.05
Shape Analysis
3660
Input Image
< mins
Shape Metric
Instance-wise Score
Image-wise Score
steps
Figure 3: Comparison of different dataset pruning pipelines. (a) Pruning classification dataset re- quires model training. (b) Our adaptation previous methods on instance segmentation by training segemntation head and computing the importance score for each pixel. (c) The proposed method that is training-fres and model-independent.
input image, and yi E y' represents the ground truth for instance segmentation. Unlike classification tasks where the ground truth for image corresponds to single label (category), the ground truth for instance segmentation assigns a label (category) to each pixel of the image through masks. Specifically, each y/ contains set of labeled instances:
{GGgj mi,)
where G; is the number instances image i, cjjj class label of the j-th instance, and mi.j a binary mask that defines the spatial pixel boundaries of the instance.
The objective of dataset pruning is to reduce the size of D by selecting a subset & C that maintains or maximizes the performance of model trained on this reduced set compared to training on the entire dataset 2, thereby improving training efficiency and reducing training time. our approach, reduce the number of images (image-level) rather than instance annotations (instances-level) since the large volume of image data primarily impacts training time and storage requirements. While annotations for instance segmentation tasks include category labels, bounding boxes, and segmentation masks, their storage footprint remains significantly smaller compared to the images themselves. For instance, in the COCO dataset, images consume about 17 GB, whereas annotations occupy merely 0.7 GB.
3.2 ADAPTED BASELINES FOR INSTANCE SEGMENTATION
Existing mainstream dataset pruning methods for classification primarily rely on medel-derived log- its for each image to calculate scores. However, for instance segmentation, each image requires the model compute a logit for every pixel rather than for an entire image, making these classification- oriented methods unsuitable without modifications. To ensure a fair comparison, shown in Fig. 4 we implement some strong basclines: we adapt these above-mentioned methods to suit instance segmentation tasks without altering their criteria for sample selection.
In the context of instance segmentation, we employ the widely-used Mask R-CNN framework as example, and it can be directly applied other segmentation models in the same manner. The mask loss Mask R-CNN is computed using a pixel-wise binary cross-entropy loss between the predicted masks and the ground truth masks for each class. The mask loss Iam for each instance is
4

---

Published as a conference paper at ICLR 2025
Images
Boundary
Score (SC3)
(31-03)
SCS (CB-3C5)
Score
Siz - 1.70
6.11
0,52
Sij 1.50
5.6
0.55
$12
0.55
4 - 0.224005---111 AGIT BUS
2.10
0.32
0.66
Perimeter
Ry
0.84
Apco
FAU (b)
(c)
Figure 4: Overview of our proposed framework. We introduce the Shape Complexity Score (SCS), in which we leverage the Perimeter-to- Area ratio to represent the boundary complexity. Following this, we apply scale normalization and intra-class normalization to address the inherent scale vari- ability and class imbalance in instance segmentation tasks.
defined as:
w
T 1
H ΣΣ
(Wa. log(ūa.)) + (1 Va.b) log(1 -a,,))
where H and W are the height and width of the Rol mask respectively, No.b is the ground truth label pixel (a, b) for object, 0 for background), and Lo.b is the predicted logit of the pixel belonging to the object. This formulation allows for pixel-level precision in predicting masks, ensuring detailed and accurate segmentation outputs.
To adapt previous dataset pruning methods to instance segmentation tasks, we extend pixel-level im- portance scoring to instance-level and image-level scoring. Our model computes per-pixel logits for each predicted instance. We then apply established classification importance metrics (e.g., EL.2N) to these logits. The instance-level score is obtained by averaging the pixel scores within each in- stance. The image's 's importance score is derived from the sum of its instance scores. Formally, the importance score I for the image is calculated as follows:
G.
7=
Σ a.
80,b3
H
where G, is the number of instances in the image Hij is the number of pixels in the j-th instance mask in image i, and Sa,b.j is the importance score at pixel (a, b) in the j-th instance mask. For pixel- to-instance score aggregation, larger instances with more pixels. For instance-to-imet aggregation, we use sum since images with more instance masks contain more objects and thus more information. Related experiments can also be found in the Appendix D.I
choose average avoid area bias introduced by sum, which favors
3.3 TRAINING-FREE DATASET PRUNING FOR INSTANCE SEGMENTATION
While effective, our adapted baselines are time-consuming and lack cross-architecture generalizabil- ity. To overcome these issues, we propose novel Training-Free Dataset Pruning (TFDP) method for instance segmentation, shown in Fig. The algorithm is outlined in Appendix
3.3.1 SHAPE COMPLEXITY SCORE (SCS)
We propose the Shape Complexity Score (SCS, Fig. 4a), novel model-independent metric that leverages the shape information of instance masks to select challenging samples. For each image x; in our dataset, we have a set of G, instance masks [M,, Mr....... M. J. The SCS is calculated for each instance mask as follows:
Mask Preparation: For each Maj, we obtain a binary mask B.y:
(i) { if (a, b) E Mij otherwise
(4)
5

---

Published as a conference paper at ICLR 2025
2. Contour Extraction: We extract the set of contours Taj (Tr,,1 ..., from B and select the primary contour:
Th arg max Area(T) TET:
(5)
Perimeter Calculation: The perimeter P.g is calculated as:
H
P.. Perimeter
Σ
/ax=ak+11 -
(b: bk+1)?
(6)
where (11 b1 = (a1, bi) to close the contour.
4. Area Calculation: The area Az, of the instance is computed
Has (abbet1 ak+1bx)
1
Arj Area((T)))
2
where (ax, by) are the coordinates of the k-th point in the contour Th with Hs points.
5. Shape Complexity Score Computation: The SCS Si, for the j-th instance in the i-th image is defined as:
Pi..
St = Ar.
(8)
This perimeter-to-are ratio increases with the boundary intricacy of the instance's shape, providing measure of its complexity. To the best of our knowledge, the SCS is the first model-independent metric in the dataset pruning area that leverages the shape information of instances' masks to select challenging samples. This approach effectively alleviates the issue of limited generalizability caused by biases in selection based on specific model predictions and significantly reduces computational overhead.
3.3.2 SCALE-INVARIANT SCS (SI-SCS)
The Shape Complexity Score (SCS) exhibits clear bias towards scale, which substantially impacts instance segmentation tasks, as shown in Fig. Dh. Specifically, smaller-scale instance masks receive higher scores, even when their boundaries are the same. To address this, we propose the Scale- Invariant SCS (SI-SCS, Fig. 4b)
For polygon with perimeter P., and area As,. scaling by factor f results in:
P SPa,
Pij
(9)
A PA..
fAis
This decreases with increasing scale, biasing towards smaller instances.
To solve this, normalize SCS using circle (the shape with the minimum perimeter for given area) as a reference. Let S7 denote the SCS of a circle:
P.j
2
2
Si
2
(10)
Aij
nr
r
Ai.j
The SI-SCS is defined as:
St
P.d
(11)
5
2 Avj
For scaled polygon (factor f), we have:
Ptá xf
P xf
P.ó
()) 2/ (Ais x f2)
St
(12)
2
x
2//
This demonstrates that SI-SCS is invariinv, depending only on the boundary complexity and not the instance's scale.

---

Published as a conference paper at ICLR 2025
3.3.3 CLASS-BALANCED SCS (CB-SCS)
While Icvarannveei addresses intra-instance shape variability, inter-instance imbalance due to class imbalance remains a sigmificant challenge. Instance segmentation datasets frequently exhibit highly skewed distributions of class instances (Oksuz et al. 2020), as illustrated in Fig. 2 and Appendix B Naive aggregation of instance-level importance scores within an image can result in rankings disproportionately influenced by high-frequency classes, thereby diminishing the contribu- tions of less frequent classes.
To mitigate this class imbalance issue, we propose the Class-Baie Balanced SCS (CB-SCS, Fig. 4c) score to ensures fair representation of all classes, regardless of the number of instances in each class. For each class, we normalize individual instance scores by the total score of all instances in that class across different images. Our normalized scoring method balances class influence, preventing overrepresented classes from dominating while simultaneously preserving the impact of classes with limited examples. Formally, the score for the i-th image is,
S
S Ever Eyec, 5 where Sh is the SI-SCS of the j-th instance in image i, j) denotes the class of the j-th object in image i.
where c(i, c(i',
=((,))=((('
(13)
Image-level Score. Consequently, we can directly sum these normalized scores across all instances in i-th image to obtain balanced image-level score formally defined 1, (Fig. 41):
S
(14)
where G; denotes the number of instances in the image i.
4 EXPERIMENTS
4.1 EXPERIMENT SETTINGS
Datasets. To evaluate the proposed method TFDP, we conduct instance segmentation experiments on three mainstream datasets VOC 2012 (Everingham et al. 2010), Cityscapes (Cordts et al. 2016).
and MS COCO (Lin al. 2014).
Evaluation Metries.
For all datasets, we evaluate instance segmentation results using the standard COCO protocol and report the average precision metrics for both mask and box AP (averaged over IoU thresholds): mAP, AP AP7s. APs. APM. APL. Due to the page limits more details about datasets and evaluation metrics are provided in Appendix C.I and C.2 respectively.
Baseline Comparisons. Six baselines are used for comparison: 1) Random; 2) Entropy (Coleman ct al. 2019): 3) Forgetting (Toneva et al. 2019); 4) EL2N (Paul et al. 202T); 5) AUM (Pleiss et al. 2020 6) CCS (Zheng et al. 2023). For all methods (except Random), we make specific adaptations for the instance segmentation task, as described in Sec. 3.2 As is common practice (He et al. 2017] Wang et al., 2020b), all network backbones are pre-trained on the ImageNet-Ik classification set (Deng et al. 2009) and then fine-tuned on the instance segmentation dataset. For hyperparameters, follow the settings described in the original paper with details provided in the Appendix C.3
4.2 PRIMARY RESULTS
MS COCO (Tab. I. Tab. Ta shows the results on the COCO dataset. Our TFDP method outper- forms all baselines across all pruning rate settings. For example, when pruning 50% of samples, TFDP still achieves 33.4% APso. which is 1.7% and 2.2% higher than adapted Entropy and EL2N, respectively. Moreover, TFDP's performance with only 80% of the data already matches the perfor- mance with 100% of the data. Additionally, with just 50% of the data selected by TFDP method, consistently outperforms random pruning by 30%. Impressively, compared to other baselines that require model training, the selection time for the model-indeeendent TFDP is almost negligible. For

---

Published as a conference paper at ICLR 2025
mAP
AP5o
AP
Time
0%
20%
30%
40%
50%
0%
20%
30%
40%
50%
0%
20%
30%
40%
50%
Random
34.2
32.1
31.1
30.8
55.2
54.5
52.8
51.1
35.6
Forgetting
20.29 h
33.1
32.3
31.4
30,4
54.2
53.4
52.2
51.2
35.2
34.3
33.4
32.1
Eniropy
21.16
33.2
32.3
31.4
30.9
54.4
53.5
52.5
51.7
35.5
34.5
33.2
32.6
E1,2N
12.37 h
33.4
12.1
31.2
30.5
54.5
52.9
51.7
51.2
35.6
34.2
33.2
32.0
AUM
20.29 h
33.5
32.4
31.5
31.0
54.6
53.3
52.4
51.7
35.5
34.7
33.4
32.8
CCS
h
33.4
32.4
31.7
31.5
54.1
53.3
52.6
52.3
35.6
33.2
Ours
34,4
33.6
33.1
32.5
58.5
54,8
54.2
36.7
35.4
34,3
Dift.
7 1349x
+0.8
+1.5
+2.0
+1.7
+1,0
+2.0
+3.1
+2.4
+1.1
+1.3
+1.9
+1.6
(a) The mask AP (%) results compare different dataset pruning baselines on COCO.
mAP
APsa Nh
AP- tb
Time
0%
20%
30%
40%
50%
0%
20%
30%
40%
50%
0%
20%
30%
40%
50%
Random
37.7
37.0
35,3
34.0
33.8
58.3
57.6
56.1
53.8
54.3
41.1
40.
38.0
37.3
36.3
Forgetting
20.29
36.8
35.6
34.5
34.0
57.7
56.2
55.2
54.4
40.4
38.7
37.5
36.8
Entropy
21.16 h
36.7
35.8
34.7
34,3
57.6
56.7
55.8
55.2
40.0
39.2
37.8
37.4
EL2N
12.37 h
36.9
35.7
34.7
34,0
57.7
56.4
55.1
54.5
40.1
38.9
37.6
36.5
AUM
35.8
34.8
34.3
57.9
56.6
CCS
20.29 h
36.8
35.7
35.2
34.7
57.6
56.5
56.1
55.7
40.3
39.1
38.2
37.5
Ours
0.014 h
37.8
37.2
36.7
35,9
58.8
58.1
57.6
56.9
41.1
40.2
39,9
38.8
DHI.
7 x349x
+0.8
+1.9
+2.7
+2.1
+1.2
+2.0
+3.8
+2.6
+1.0
+2.2
+2.6
+2.5
(h) The bounding-hox (bb) AP (%) results compare different dataset pruning baselines on COCO.
Table 1: Results on COCO with backbone network Mask R-CNN. The pruning rate represents the percentage of data removed from the full training dataset during pruning. The performance on the full dataset is indicated by 0%. Time indicates the time consumption for sample ranking, with details provided in Sec. 4.3 D i f for the improvement in time represents the average improvement, excluding Random since it does not consume time. Di ff for the performance denotes the difference
between our TFDP method and random pruning, as all baselines are also our adapted methods.
Dataset
VOC
Cityscapes
Model
SOLO-v2
QueryInst
p
Time
Time
0%
20%
30%
40%
50%
40%
50%
40%
50%
RI2N
Entropy
Random
Forgetting
21.21 min
21.75 min
12:70 min
40.9
38.4
33.6
39.4
34.2
32.0
30.8
29.0
28.1
31.7
29.3
23.7
21.6
5.61
5.54
27.6
26.4
26.2
26.1
25.8
22.6
22.2
21.8
23.2
20.1
20.3
19.0
19.3
17.2
16.9
17.1
17.4
FI.2N
Random
Entropy
AUM
52.1
52.5
52.8
51.4
51.0
51.1
51.6
50.3
55.0
55.6
55.6
53.3
52.7
53.9
$4.0
52.8
AUM
21.21 min
35.2
31.0
26.3
19.2
5.54
25.3
24.5
21.2
18.4
CCS
53.0
52.1
55.0
53.5
CCS
21.21 min
38.8
35.4
34.3
30.8
5.54
25.4
24.1
19.9
17.0
Ours
0.12 min
40.3
38.6
36.2
33.4
0.0051 h
27.5
25.4
23.4
19.4
Ours
53.1
52.3
58.9
55.0
Diff.
164x
100x
+4.4
+2.5
Diff.
+1.7
+1.2
+2.6
+2.2
Table 2: The mask AP (%) results compare different dataset pruning baselines on VOC and Cityscapes. The pruning rate represents the percentage data removed from the full training dataset during prun- ing. The performance on the full dataset is indicated by 0%.
Table 3: The AP30 (%) sults in the generalization ability to different architec- tures on COCO dataset.
more results on the COCO dataset, please refer to the Appendix D.2 Bounding-Box (bb) AP Re- sults. Following previous work (He et al., (017), we also report the bounding-box object detection results on three datasets in Tab. 1b Our method TFDP significantly outperforms random pruning all pruning rate settings. Notably, by selecting only 50% of the data, our TFDP consistently sur- passes all baseline methods that select 70% of the data (30% pruning rate) in both mApbb and APs0 bb performance. Compared to the Mask AP results, the performance improvement bounding-box AP more pronounced.
YOC and Cityscapes (Tab. D) Performance results on VOC and Cityscapes are reported in Tab. 2 Our TFDP method demonstrates superior performance on VOC. For instance, with the pruning rate of 40% and 50%, Mask R-CNN trained on the pruned VOC achieves mask AP accuracies of 36.2% and 33.4%, surpassing random pruning by 7.2% and 9.7%, respectively. On Cityscapes, TFDP also consistently shows improvements. For example, at a pruning rate of 40%, TFDP exceeds the performance of random pruning 40% by 4.4%, and 30% by 1.3%. Additionally, TFDP
8

---

Published as a conference paper at ICLR 2025
SI
CB
VOC
Cityscapes
COCO
mAP
AP5o
30%
40%
50%
30%
40%
50%]
30%
40%
50%
0%
20%
30%
random
32.0
23.7
22.1
19.0
53.8
52.1 52.0
Ramdom
36.4
35.4
35.0
35.6
33.8
61.8
60.5
60.8
60.6
58.9
36.6
34,0
28.8
22.4
21.5
19.1
54.2
52.9
52.5
Entropy
34.7
35.6
34.5
34.3
61.3
62.6
61.4
60.3
37.8
35,4
32.4
22.2
228
17.9
54.2
53.3
52.1
EL2N
34.2
34.1
35.2
32.1
59.2
59.7
61.8
57.3
35,9
33.5
30.7
24.9
22.4
17.1
54.4
54.0
52.7
AUM
62.6
60.9
59.4
59.4
38.6
36.2
33.4
25.4
23,4
19.4
54.8
54,2
53.4
CCS
36.1
36.1
35.0
34.0
61.7
61.7
60.7
59.7
Ours
36.9
36.6
36.6
36.6
62.8
63.9
63,4
62,8
Table 4: Ablation study of two components SI-SCS (SI) and CB-SCS (CB). When nei- ther normalization is used (both marked as ''), it indicates that only SCS is applied.
Diff.
+1.5
+1.6
+1.0
+2.8
+2.3
+3.1
+2.8
+3.9
Table 5: The mask AP (%) on Cityscapes (pre- trained on COCO).
method consistently outperforms the adapted SOTA methods (e.g., EL.2N and CCS) VOC and Cityscapes at all pruning rate settings. For more results on the VOC and Cityscapes dataset, please refer the Appendix D.3
Generalization to Other CNN-based and Transformer- Models (Tab. 3). In Tab. 3
we
show an experiment assessing the effectiveness of images selected by Mask R-CNN when tested on instance segmentation networks with new architectures. We included other CNN-based network SOLO-v2 (Wang et 2020b), as well as Transformer- based networks QueryInst (Fang al. 2021 to test for generalizability. Appendix [.4]show shows more results of the generalization evaluation. Ad- ditionally, we conducted experiments on different backhones (e.g., ResNet- 101 and ResNeXt-101) as shown in Appendix D.5 further validating the scalability of our method.
Ablation Study (). Tab. Tab. demonnttates the effectiveness of our components on three datasets. We notice using them alone does not give satisfactory results, and applying both SI-SCS and CB- SCS delivers the best performance. Further analysis on the Scale-Inv design provided in Appendix D.6
4.3
MORE ANALYSIS
Robust Performance Across Training Paradigms (Tab. 5). Our TFDP is method that prepares an pruned dataset independent of any specific training process. To comprehensively evaluate its effectiveness, we use TFDP-pruned datasets in two scenarios: 1) Models pre-trained on ImageNet- 1k (Results shown Tab. T. Tab. 2 and Tab. 3). 2) Models pre-trained on COCO (He et al. 2017 Liu et al. 2018a) as shown in Tab. 5 In both cases, models are fine-tuned on either the full or TFDP-pruned dataset. Impressively, TFDP outperforms all baselines across different pruning rates in both cases. Even when pruning 50% of the data, it (APso 62.8%) surpasses the performance achieved with the full dataset (APs0 61.8%). This demonstrates TFDP's effectiveness as a universal preprocessing step regardless of the subsequent training strategy. More experimental settings and results for training scenarios 2) are provided in Appendix D.7
Performance Gain at High Pruning Rates (Fig. 5 To further validate our method, also conduct experiments under high pruning rates, as shown in Fig. [5] Compared to random pruning, our method consistently improves performance across pruning rates from 60% 90%.
Time Consumption Results (Tab. 6). We detail the time required to calculate importance scores for all samples in the dataset, including model training, score computation via model inference, and stratified selection. All time tests were conducted in the same environment, with details available in Appendix D.8] and the times reported are averages of three trials. As shown in Tab, 6 our method significantly reduces sample selection time, with greater time-saving effects on larger datasets, high- lighting its effectiveness in the big data era.
Visualization (Fig. 6). Fig. 6 shows that our method effectively distinguishes "hard" (top-ranked) and "easy" images (least-ranked). Top-ranked images contain either many intricate-shaped objects (subfig-a, b, c) instances of rare classes (subfig-d, e), while least-ranked images typically contain a single object with simple contour f, g. h, j. k). We use the top-1 image (subfig-d, e) from MS COCO to explain how scale variability and class imbalance issues addressed. First, despite having various scales, SI-SCS assigns all toasters similar scores, illustrating the effectiveness of our SCS (SI-SCS). Second, despite having relatively simple shapes, CB-SCS assigns

---

Published as a conference paper at ICLR 2025
AUMI
Dataset
Ours
EL.N
Entroy
Forgetting
CCS
T
722.49
1265.40 39.71 8
1272.31
1272.31
7.19
39.39
VOC
0.009
Reaureyyve
Total
1305.11
1272.31
1272.32
2.89
5.54
5.54
I
1824
429.29
432.33
Cityscupss
0.029
Total
182.4
3.01 h
5.61
5.54 h
5,54 h
11.35 b
20.13 h
20.29
20.29
410
COCO
50.4
3676.48
3696.69 s
0.99
60C
43%
Pruning Rato (%)
Total
50.4
12.37
16
20.29
20.29
Figure 5: Experiments of high pruning rates (from 60% 90%) on MS COCO dataset. Seg- mentation metrics include mAP, APso and the corresponding bounding-bol box version, mApbb and AP50
Table 6: Detailed time calculation. denotes the model training time. I is the score computation via model inference. S the stratified selection time, for CCS (Zheng et al. 2023
Score:
Images
Top Ranked
Masks
36122
0,00005
0.
0,06338
O 0300143
0.00412
0.00404
0.00403
0.00410
0.00410
0.00100
(a)
(b)
(c)
(d)
score: 0.00078
ecore: 0 0.00800
score: 0.00003
0.0000266
0.000027
score: 0.000028
Images
Least Ranked
0.0000026
Masks
5 5,0000
0.000028
0.00078
O 0.0000927
(i)
(k)
VOC 2012
Cityscapes
COCO
Figure 6: Visualization of the images are preserved. Scores of too small objects are omitted for better visualization.
images on different datasets. The original aspect ratios of
high score to this image due to the rarity of the class "toaster", showcasing our Class-Balancee Balanced SCS (CB-SCS). The distribution Fig. in the Appendix [Beeellls that the number of toasters the second-tor -o-d--ttosst least counted object. More visualizations, examples and analysis are in appendix[D9]
5 CONCLUSION AND FUTURE WORK
We introduce Training-Fre Dataset Pruning (TFDP) for instance segmentation, leveraging mask annotations and novel scoring methods to address scale variability and class imbalance. Our ap- proach significantly reduces pruning time while improving performance, outperforming adapted state-of-the-arr baselines and demonstrating strong cross-architecture generalization. In future work, plan to extend the training-free free concept to diverse datasets. This expansion will include video dataset pruning by incorporating temporal information and motion patterns, language dataset prun- ing through analysis of text structure, syntax, and semantic richness. Exploring the theoretical foun- dations of training-fre pruning methods is another interesting direction.
10

---

Published as a conference paper at ICLR 2025
6 ACKNOWLEDGMENT
This work was supported in part by A*STAR Career Development Fund (CDF) under C243512011, in part by A A*STAR Centre for Frontier Al Research (CFAR) and in part by CFAR Internship Award and Research Excellence (CIARE).
REFERENCES
Daniel Bolya, Chong Zhou, Fanyi Xiao, and Yong Jae Lee. Yolact: Real-time instance segmentation. In ICCV, 2019.
Shubhankar Borse, Ying Wang, Yizhe Zhang, and Fatih Porikli. Inverseform: A loss function for structured boundary-aware segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5901-599111 2021.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey 7agoruyko. End-to-end object detection with transformers. In European conference on computer vision, pp. 213-229. Springer, 2020.
George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei Efros, and Jun-Yan Zhu. Dataset distillation by matching training trajectories. In Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2022.
Hao Chen, Kunyang Sun, Zhi Tian, Chunhua Shen, Yongming Huang, and Youliang Yan. Blend- Mask: Top-down meets bottom-up for instance segmentation. In Proc. IEEE Conf. Comput. Vis. Pattern Recog,, 2020.
Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, Zheng Zhang, Dazhi Cheng, Chenchen Zhu, Tianheng Cheng, Qijie Zhao, Buyu Li, Xin Lu, Rui Zhu, Yue Wu, Jifeng Dai, Jingdong Wang, Jianping Shi, Wanli Ouyang, Chen Change Loy, and Dahua Lin. MMDetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv: / 0906/996677555 155, 2019a.
Xinlei Chen, Ross Girshick, Kaiming He, and Piotr Dollar. TensorMask: A foundation for dense object segmentation. Proc. Ini. Conf. Comput. Vis., 2019b.
Bowen Cheng. Ishan Misra, Alexander G. Schwing, Alexander Kirillov, and Rohit Girdhar. Masked- attention mask transformer for universal image segmentation. In CVPR, 2022.
Tianheng Cheng, Xinggang Wang, Lichao Huang, and Wenyu Liu. cnn. In Computer Vision-ECCO ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XIV 16, pp. 660-676. Springer, 2020.
mask
Cody Coleman, Christopher Yeh, Stephen Mussmann, Baharan Mirzasoleiman, Peter Bailis, Percy Liang, Jure Leskovec, and Matei Zaharia. Selection via proxy: Efficient data selection for deep learning. In Int. Conf. Learn. Represent., 2019.
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3213-3223, 2016.
Justin Cui, Ruochen Wang, Si Si, and Cho-Jui Hsieh. Scaling up dataset distillation to imagenet- with constant memory. Proc. Ini. Conf. Mach. Learn., pp. 6565-6590. PMLR, 2023.
Bert De Brabandere, Davy Neven, and Lue Van Gool. Semantic instance segmentation with discriminative loss function. arXiv: 02551, 2017.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In IEEE Conf. Comput. Vis. Pattern Recog., pp. 248-255, 2009.
Hadi M. Dolatabadi, Sarah M. Erfani, and Christopher Leckie. L-robustness and beyond: Unleash- ing efficient adversarial training. In Eur. Conf. Comput. Vis., 2022.
11

---

Published as a conference paper at ICLR 2025
Jiawei Du, Yidi Jiang, Vincent TF Tan, Joey Tianyi Zhou, and Haizhou Li. Minimizing the accu- mulated trajectory error to improve dataset distillation. In Proc. IEEE Conf. Comput. Vis. Pattern Recog.. 2023.
Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voe) challenge. International journal of computer vision, 88: 303-335 2010.
Yuxin Fang, Shusheng Yang, Xinggang Wang. Yu Li, Chen Fang, Ying Shan, Bin Feng, and Wenyu Liu. Instances as queries. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6910-6919, 9919, October 2021.
Naiyu Gao, Yanhu Shan, Yupei Wang, Xin Zhao, Yinan Yu, Ming Yang, and Kaiqi Huang. SSAP: Single-shot instance segmentation with affinity pyramid. In Proc. Int. Conf. Comput. Vis., 2019.
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pp. 2961-2969, 2017.
Yang He, Lingao Xiao, and Joey Tianyi Zhou. You only condense once: Two rules for pruning condensed datasets. Advances in Neural Information Processing Systems, 01339382-393992 2023.
Yang He, Lingao Xiao, Joey Tianyi Zhou, and Ivor Tsang. Multisize dataset condensation. In Proc.
Int. Conf. Learn. Represent,, 2024.
Xijie Huang, Zechun Liu, Shih- Yang Liu, and Kwang- Ting Cheng. Efficient quantization-awarc training with adaptive coreset selection. arXiv preprint arXiv:2306.07215 2023.
Zhaojin Huang, Lichao Huang, Yongchao Gong, Chang Huang, and Xinggang Wang. Mask scoring
R-CNN. In Proc. IEEE Conf. Comput. Vis. Puttern Recog., 2019.
Zixuan Jiang, Jiaqi Gu, Mingjie Liu, and David Z Pan. Delving into effective gradient matching for dataset condensation. arXiv preprint 20X0::2008.0311, 2022.
Aymen Rayane Khouas, Mohamed Reda Bouadjenek, Hakim Hacid, and Sunil Aryal. Training machine learning models at the edge: A survey. arXiv preprint arXiv:2403.02619, 2024.
Krishnateja Killamsetty, Sivasubramanian Durga, Ganesh Ramakrishnan, Abir De, and Rishabh Iyer. Grad-match: Gradient matching based data subset selection for efficient deep model training. In Proc. Int. Conf. Mach. Learn., pp. 5464 5474, 2021.
Sachyung Lee, Sanghyuk Chun, Sangwon Jung. Sangdoo Yun, and Sungroh Yoon. Dataset conden- sation with contrastive signals. In Proc. Int. Conf. Mach. Learn.. pp. 12352-12364, 2022.
Hao Li, Dingwen Zhang, Nian Liu, Lechao Cheng, Yalun Dai, Chao Zhang, Xinggang Wang, and Junwei Han. Boosting low-data instance segmentation by unsupervised pre-training with saliency prompt. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- fion, pp. 15485-15494 2023.
Yi Li, Haozhi Qi, Jifeng Dai, Xiangyang Ji, and Yichen Wei. Fully convolutional instance-awan semantic segmentation. In Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2017.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740-755. Springer, 2014.
Shu Liu, Jiaya Jia, Sanja Fidler, and Raquel Urtasun. Sequential grouping networks for instance segmentation. In Proc. Int. Conf. Comput. Vis., 2017.
Shu Liu, Lu Qi, Haifang Qin, Jianping Shi, and Jiyy Jia. Path aggregation network for instance segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8759-8768, 2018a.
Shu Liu, Lu Qi, Haifang Qin, Jianping Shi, and Jaya Jia. Path aggregation network for instance segmentation. In Proc. IEEE Conf. Comput. Vis. Pattern Recog, 2018b.
12

---

Published as a conference paper at ICLR 2025
Yanqing Liu, Jianyang Gu, Kai Wang, Zheng Zhu, Wei Jiang, and Yang You. DREAM: Efficient dataset distillation by representative matching, arXiv preprint arXiv:2302.14416, 2023.
Noel Loo, Ramin Hasani, Alexander Amini, and Daniela Rus. Efficient dataset distillation using random feature approximation. In Proc. Adv. Neural Inform. Process. Syst., 2022.
Noel Loo, Ramin Hasani, Mathias Lechner, and Daniela Rus. Dataset distillation with convexified implicit gradients. Proc. Int. Conf. Mach. Learn., 2023.
Katerina Margatina, Giorgos Vernikos, Loic Barrault, and Nikolaos Aletras. Active learning by acquiring contrastive examples. arXiv preprint arXiv:2109. 0076037644 2021.
Baharan Mirzasoleiman, Jeff Bilmes, and Jure Leskovec. Coresets for data-efficient training of machine learning models. In Proc. Int. Conf. Mach. Learn., pp. 6950-6960 2020.
Alejandro Newell, Zhiao Huang, and Jia Deng. Associative embedding: End-to-end learning for joint detection and grouping. In Proc. Adv. Neural Inform. Process. Syst., 2017.
Binh-Nguyen Nguyen and Yang He. Swift cross-dataset pruning: Enhancing fine-tuning efficiency in natural language understanding. arXiv preprint arXiv:2501.02432 2025.
Timothy Nguyen, Zhourong Chen, and Jaehoon Lee. Dataset meta-learning from kernel ridge- regression. In Proc. Int. Conf. Learn. Represent., 2021a.
Timothy Nguyen, Roman Novak, Lechao Xiao, and Jachoon Lee. Dataset distillation with infinitely wide convolutional networks. In Proc. Ady. Neural Inform. Process. Syst.. pp. 5186-5198, 2021b.
Patrik Okanovic, Roger Waleffe, Vasilis Mageirakos, Konstantinos E Nikolakakis, Amin Karbasi, Dionysis Kalogerias, Nezihe Merve Gürel, and Theodoros Rekatsinas. Repeated random sam- pling for minimizing the time-to-accuracy of learning. arXiv preprint arXiv:2305.18422) 2023.
Kemal Oksuz, Baris Can Cam, Sinan Kalkan, and Emre Akbas. Imbalance problems in object detection: A review. IEEE transactions on pattern analysis and machine intelligence, 43(10): 1338834155. 2020.
Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. Deep learning on a data diet: Finding important examples early in training. In Ady. Neural Inform. Process. Syst., volume 34, pp. 20596-20607, 2021.
Geoff Pleiss, Tianyi Zhang, Ethan Elenberg, and Kilian Q Weinherger. Identifying mislabeled data using the area under the margin ranking. In Adv. Neural Inform. Process. Syst., volume 33, pp. 17044-17056, 2020.
Shitong Shao, Zeyuan Yin, Muxin Zhou, Xindong Zhang, and Zhiqiang Shen. Generalized large- scale data condensation via various backbone and statistical matching. In Proc. IEEE Conf. Com- put. Vis. Pattern Recog., 2024.
Seungjae Shin, Heesun Bac, Donghyeok Shin, Weonyoung Joo, and Il-Chul Moon. Loss-curvature matching for dataset selection and condensation. In International Conference on Artificial Intel- ligence and Statistics, pp. 8606-8628. 2023.
Haoru Tan, Sitong Wu, Fei Du, Yukang Chen, Zhibin Wang, Fan Wang, and Xiaojuan Qi. Data pruning via wewring-one-sampl-oon In Adv. Neural Inform. Process. Syst., 2023.
Chufeng Tang, Hang Chen. Xiao Li. Jianmin Li, Zhaoxiang Zhang, and Xiaolin Hu. Look closer to segment better: Boundary patch refinement for instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 13926-13935, 2021.
Zhenyu Tang, Shaoting Zhang, and Xiaosong Wang. Exploring data redundancy in real-world image classification through data selection. arXiv preprint 20t.::23336.111133. 2023.
Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. FCOS: Fully convolutional one-stage object detection. In Proc. Int. Conf. Comput. Vis., 2019.
13

---

Published as a conference paper at ICLR 2025
Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J. Gordon. An empirical study of example forgetting during deep neural network learning. In Int. Conf. Learn. Represent. 2019.
Murad Tukan, Alaa Maalouf, and Margarita Osadchy. Dataset distillation meets provable subset selection. arXiv preprini arXiv:2307.00 000006 2023.
Chi Wang, Yunke Zhang, Miaomiao Cui, Peiran Ren, Yin Yang, Xuansong Xie. Xian-Sheng Hua, Hujun Bao, and Weiwei Xu. Active boundary loss for semantic segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 2397-2405, 2022a.
Kai Wang, Bo Zhao, Xiangyu Peng, Zheng Zhu, Shuo Yang. Shuo Wang, Guan Huang, Hakan Bilen, Xinchao Wang, and Yang You. Cafe: Learning to condense dataset by aligning features. In IEEE Conf. Comput. Vis. Pattern Recog. pp. 12196-12205, 2022b.
Xinlong Wang, Tao Kong, Chunhua Shen, Yuning Jiang, and Lei Li. SOLO: Segmenting objects by locations. In Proc. Eur. Conf. Comput. Vis., 2020a.
Xinlong Wang. Rufeng Zhang. Tao Kong, Lei Li, and Chunhua Shen. Solov2: Dynamic and fast instance segmentation. Proc. Advances Neural Information Processing Systems (NeurIPS), 2020b.
Max Welling. Herding dynamical weights to learn. In Proc. Int. Conf. Mach. Learn., pp. 1121-1128, 2009.
Xiaobo Xia, Jiale Liu, Jun Yu, Xu Shen, Bo Han, and Tongliang Liu. Moderate coreset: A uni- versal method of data selection for real-world data-efficient deep learning. In Int. Conf. Learn. Represent. 2022.
Lingao Xiao and Yang He. Are large-scale soft labels necessary for large-scale dataset distillation? In Proc. Adv. Neural Inform. Process. Syst., 2024.
Enze Xie, Peize Sun, Xiaoge Song, Wenhai Wang, Xuebo Liu, Ding Liang, Chunhua Shen, and Ping Luo. PolarMask: Single shot instance segmentation with polar representation. In Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2020.
Xilie Xu, Jingfeng Zhang, Feng Liu, Masashi Sugiyama, and Mohan Kankanhalli. Efficient adver- sarial contrastive learning via robustness-aware coreset selection. In Adv. Neural Inform. Process. Syst. 2023.
Shuo Yang. Zeke Xie, Hanyu Peng, Min Xu, Mingming Sun, and Ping Li. Dataset pruming: Reduc- ing training data by examining generalization influence. In The Eleventh International Confer- ence Learning Representations, 2023. URL https::/ http::s///ppereviiec net forum?id= 4wZ1AXD29TO
Zeyuan Yin and Zhiqiang Shen. 0rXiv:2311.1883. 2023.
Dataset distillation in large data era. arXiv preprint
Zeyuan Yin, Eric Xing, and Zhiqiang Shen. Squeeze, recover and relabel: Dataset condensation at imagenet scale from new perspective. In Proc. Adv. Neural Inform. Process. Syst., 2023.
Gang Zhang, Xin Lu, Jingru Tan, Jianmin Li, Zhaoxiang Zhang, Quanquan Li, and Xiaolin Hu. Re- finemask: Towards high-quality instance segmentation with fine-grained features. In Proceedings of the JEEE/CVF conference on computer vision and pattern recognition, pp. 6861-6869, 2021.
Rufeng Zhang, Zhi Tian, Chunhua Shen, Mingyu You, and Youliang Yan. Mask encoding for single shot instance segmentation. Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2020.
Bo Zhao and Hakan Bilen. Dataset condensation with distribution matching, In Proc. IEEE Winter Conf. Appl. Comput. Vis., pp. 6514-6523, 2023.
Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. Dataset condensation with gradient matching. In Proc. Int. Conf. Learn. Represent, 2021.
14

---

Published as a conference paper at ICLR 2025
Ganlong Zhao, Guanbin Li, Yipeng Qin, and Yizhou Yu. Improved distribution matching for dataset condensation. In Proc. IEEE Conf. Comput. Vis. Pattern Recog.. pp. 7856-7865, 2023.
Haizhong Zheng, Rui Liu, Fan Lai, and Atul Prakash. Coverage-centric coreset selection for high pruning rates. In The Eleventh International Conference Learning Representations, 2023.
URL
net forum? 1dk
Muxin Zhou, Zeyuan Yin, Shitong Shao, and Zhiqiang Shen. Self-supervised dataset distillation: A good compression is all you need. arXiv preprint arXiv:2404.07976, 2024.
Yongchao Zhou, Ehsan Nezhadarya, and Jimmy Ba. Dataset distillation using neural feature regres- sion. In Proc. Ady. Neural Inform. Process. Syst., 2022.
15

---

Published as a conference paper at ICLR 2025
A TRAINING-FREE DATASET PRUNING (TFDP) ALGORITHM
Algorithm Iillustrates the code implementation process of our TFDP
Algorithm Training- Free Dataset Pruning (TFDP)
Input: Complete training dataset D. Number of images to select K.
1: Initialize: 8 = 0,, set of selected samples
2: for i = 1 to do
3: for j 1 to G; do
4: Calculate P., and As,; for each mask M,,k in image
5: Compute SCS Stj
Defined in Equation Defined in Equation
6:
Scale-Invariant SCS by circle ratio S
7: end for
8:
end for
9:
for to D do
10: for k 1 to G; do
Compute 12: end for
SCS S for each instance mask maj in image
Calculate image score
Defined Equation
14: end for
Sort all images in based on I descending order
16: Select top K images based on sorted scores to form subset S
Output: Selected subset S {(T,, Mi ) Importance scores based on 1;
16

---

Published as a conference paper at ICLR 2025
B DATASETS ANALYSIS
We show statistical details of more datasets, including VOC, Cityscapes and COCO, As shown in Fig. [8] there are significant differences in object areas across the datasets, along with clear class imbalance.
8%
Small
8.0%
3.8%
Medium-small
Medium-large
6%
7.1% 7.15%
Large
5.7%
4%.
25-
3.8%
2.4%
0%
0.6%
0.1%
15%
10%
100%
Instance Area / /Image Area
(a) Distribution of VOC 2012 dataset.
Small
Madium-large
Large
5.9%
47%
4,9%
2%
1.9%
0.9%
6.1% 6.0%
Instance Area
bb) Distribution of Cityscapes dataset.
6%
Small
6.2%
Medium-small
5.8%
Medium-large
5.3%
Large
4.5%
3.7%
2%
2.4%
1.4%
0%
0.7%
0.0%
0.1%
10%
100%
/Image Arca
(c) Distribution MS COCO dataset.
Figure 7: Distribution of area of instances.
17

---

Published as a conference paper at ICLR 2025
Clas Names
2. chair
12. boat
bottle
aeroplane
4. car
14. sofa
5.
15. tvmonitor
sheep
7. pottedplant
17. horse
9. bus
19. bicycle
diningtable
Instance
12
Class Indix
(a) Distribution of VOC 2012 dataset.
27,45
Closs Na kames
matarcycle
2. person
6. truck
7.
4. rider
0. train
Instance Count
15000
Class Index
(b) Distribution Cityscapes dataset.
Closs Strees
person
21. potted plant
41. bus
61. bod
tokket
baseball
remate
dining
denddattt
sink
keyboard
now!
28. aicycie
sizt ebooed
snowbbord
zoffic light
25, dkis
45, disphntt
fribboc
Invtance Count
10. handbeg
30. veuue
52. dog
yo. neit qeeator
11. umbreto
31. horse
51. fors
71. mouse
12. tird
32, Bk
52. zebro
stop sign
bost
phone
wirplene
toeitérush
barana
clock
to
bear
kite
sulcume
cing
19, mosaroydle
39 spoon
59. train
toaster
20. bacopack
40. Surlboard
60. sandwich
50. hair drier
(c) Distribution of MS COCO datasct.
Figure 8: Distribution of number of instances.
18

---

Published as a conference paper at ICLR 2025
C
EXPERIMENT DETAILS
C.I DATASET DETAILS
Our experiments conducted three different datasets:
Pascal VOC 2012 features 2,913 images, split into 1,464 for training and 1,449 for valida- tion, with 6,929 segmentations available for instance segmentation task.
Cityscapes includes 9 object categories designed for instance-level semantic labeling. This dataset is more complex as each image may contain a significantly higher number of in- stances per class compared to VOC, most of which are quite small. It contains 2975 training images collected from 18 cities and 500 validation images from different cities.
MS COCO is popular instance segmentation dataset, which contains an 80-category label set with instance-level annotations. Following previous works (He et al. 20177 Wang et al.) 2020b), we use the COCO train 2017 (1I8K training images) for training, and the ablation study is carried out on the val 2017 7 (5K validation images).
C.2 EVALUATION METRICS
For all datasets, we evaluate instance segmentation results using the standard COCO protocol (Lin al. 2014). This protocol provides comprehensive set of metrics to assess the performance of object detection and instance segmentation models. We report the average precision (AP) metrics for both mask and bounding box predictions, which are averaged over multiple Intersection over Union (loU) thresholds. The key metrics we report are:
mAP (mean Average Precision): This is the primary metric, calculated by averaging AP across all object categories and loU thresholds (typically from 0.5 to 0.95 in steps of 0.05).
AP Average Precision at IoU threshold 0.5. This metric more lenient, considering detections correct if they have at least 50% overlap with the ground truth.
AP75: Average Precision at IoU threshold of 0.75. This is a stricter metric, requiring more accurate localization of objects.
APs, APM, APL: These metrics evaluate performance small, medium, and large objects respectively.
-APs: AP for small objects (area 32" pixels)
AAPm: AP for medium objects (32 area < 962 pixels)
APL: AP for large objects (area > 962 pixels)
These size-specific metrics help assess the model's performance across different scales of objects, which is crucial for understanding its effectiveness in various real-world scenarios.
C.3 EXPERIMENTS SETTINGS
Except for the generalization and scalability experiments, we use ResNet-50 as the backbone for Mask R-CNN and FPN to extract multi-scale features. All models and training hyperparameters were trained using the default hyperparameters as specified in MMDetection (Chen al. (019a)..
Adapted Baseline Settings. For EL2N, in alignment with the description of the model in the early training stage from the original paper, we use the model halfway through the total training duration to compute EL.2N scores. For CCS, we strictly follow the hyperparameters in the original paper, including the hard pruning rate /3 and the number of strata It is worth noting that our TFDP does not contain any hyperparameters, further demonstrating the practicality of our TFDP approach.
Primary Experiment Settings. For the VOC experiment, we used a single NVIDIA 3090 GPU. For the Cityscapes experiment, used two NVIDIA 3090 GPUs. For the COCO experiment, we used two NVIDIA A 1100 80G GPUs. Time Consumption Experiment Settings. To ensure a fair comparison. all time consumption experiments were conducted the same machine: PyTorch Ubuntu 20.04, with NVIDIA RTX 3090 GPUs and CUDA 11.3. We used two NVIDIA 3090 GPUs for training and one NVIDIA 3090 GPU for inference.
19

---

Published as a conference paper at ICLR 2025
D MORE EXPERIMENTS
D.1 COMPARISON BETWEEN SUMMATION AND AVERAGE
We compared two methods for aggregating object scores through pixels: summation and average. As shown in Tab. 7 the average-based method performs significantly better than the summation-based method. This is because the summation- based method tends to favor larger masks, i.c., instances with more pixels, causing smaller instances be overlooked.
AP13
AP
AP;
APa
AP,
Avg.
Suna
Delt
Avg.
Sum
Dift
Avg.
Sum
Dit:
Avg.
Sum
Dit:
Avg.
Sum
Dif.
Avg.
San
Did..
Forgoting
33.4
Enerpp
28.1
47.0
30.5
12.7
16.2
33.2
42.7
EL2N
254
50.3
+4.9
45.3
50.9
+5.6
23.9
33.2
+4.3
11.0
14.3
+3.3
30.1
32.1
+2.0
38.7
41.3
+2.6
AUM
26.7
29.9
+3.2
46.5
51.2
+4.6
29.7
33.9
+4.2
11.4
15.6
+4.2
90.3
32.9
+2.6
39.0
41.2
+2.2
CCS
28.4
30.9
+2.5
47.2
518
44.6
31.9
34.8
+38
12.1
16.7
+4.6
31.1
33.2
62.1
4013
42.
02.6
Table 7: Comparison between averaging (Avg.) and summation (Sum).
D.2
MS COCO RESULT ON APs, APM AND APL
Tab.
shows more detailed AP results on the COCO dataset for different object areas (small, medium, large). Our TFDP consistently achieves stable improvements.
APs
APs:
APL
0%
20%
30%
40%
50%
0%
20%
30%
40%
50%
20%
30%
40%
50%
Random
16.1
15.9
14.1
13.1
13.2
36.7
36.0
34.6
33.2
33.2
49.9
48.8
48.1
46.2
45.6
Forgetting
Entropy
15.5
14.9
14.3
14,4
35.9
35.0
34,1
33.9
48.9
46.7
45.1
43.8
EL2N
14.5
14.4
14.4
36.3
35.2
34.1
33.6
47.9
46.3
44.6
43.2
AUM
15.8
14.7
14.6
14.4
36.3
35.4
34.3
34.1
48.5
46.6
45.3
44.0
CCS
15.8
15.2
14.8
14.7
36.0
34.9
34.6
34.1
48.4
46.7
45.0
45.2
Ours
16.2
15.5
15.5
15,1
37.1
36.3
36.0
35.3
49.3
48.5
46.9
46.2
(a) The mask AP (%) results for different object areas (small, medium, large) compare different dataset pruning baselines COCO.
APg
APM
AP,
0%
20%
30%
40%%
50%
(KS)
20%
30%
40%
50%
0%
20%
30%
40%
50%
Rundom
21.9
21.8
19.6
18.4
18.4
40,9
40.2
38.4
37.1
36.8
48.9
47.8
46.4
44.1
44.3
Forgetting
20.8
20.7
20.1
20.1
40.1
39.9
38.4
37.8
47.5
45.3
43.7
42.4
Entropy
20.7
20.3
20.7
40.3
39.5
38.3
37.9
43.3
EL2N
21.1
20.8
20.2
19.8
40.7
39.7
38.4
37.9
47.1
45.1
43.5
41.8
AUM
22.0
20.7
20.4
20.2
40.8
40.0
38.7
38.4
45.0
CCS
21.6
20.8
20.6
20.0
40.4
39.5
38.9
38.2
47.4
45.6
44.3
43.9
Ours
22.4
21.1
21.4
20.6
41.1
40.4
40,5
39,5
48.3
47.7
46.0
45.3
Diff.
+0.6
+1.5
+3.0
+2.2
+0.9
+2.0
+3.4
+2.7
+0.5
+1.3
+1.9
+1.0
(b) The bbox AP (%) results for different object areas (small, medium, large) compare different dataset pruming baselines on COCO.
Table 8: More results COCO. The pruning rate p represents the percentage of data removed from the full training dataset during pruning. The performance on the full dataset is indicated by p 0%. Diff. denotes the difference between our method and random pruning, and the improvement in time is the average improvement.
D.3 CITYSCAPES RESULT MAP, AP50, AND AP75
Tab. shows more detailed AP results on the Cityscapes dataset for different IoU thresholds. Our TFDP consistently achieves stable improvements.
20

---

Published as a conference paper at ICLR 2025
mAP
APso
APys
0%
20%
30%
40%
50%
0%
20%
30%
40%
50%
0%
20%
30%
40%
50%
Forgetting
11.3
10.3
8.3
7.3
25.8
23.2
19.3
17.1
8.2
7.6
5.7
4,9
Entropy
8.4
7.0
26.4
23.8
20.7
17.2
EL2N
11.6
10.1
9.2
8.0
26.2
23.1
20.8
17.9
8.9
7.7
7.4
6.6
AUM
11.1
10.7
9.2
7.8
25.3
24.5
21.2
18.4
8.1
8.1
7.0
6.2
CCS
10.9
10.4
8.5
7.0
25.4
24.0
20.0
17.0
6.8
5.0
Dift
+1.2
+1.8
+1.6
+0.3
+1.4
+3.6
+4.1
+2.5
+0.8
+1.5
+1,1
+1.1
(a) The mask AP (%) results different thresholds (0.5 0.95, 50, 75) compare different dataset pruning baselines on Cityscapes.
m&ppt
APrs
0%
2016
30%
40%%
50%
(1%
20/6
30%
40%
50%
20%
30%
40%
50%
Random
15.7
14.5
12.1
10.5
9.3
33.0
31.7
27.2
24.2
21.7
12.3
11.2
8.7
7.7
6.7
Forgetting
13.1
9.7
31.0
29.3
26.4
22.0
11.4
6.6
Entropy
14,8
13.4
11.5
9.4
31.6
29.6
26.5
21.5
11.8
10.1
8.0
688
EL2N
32.0
AUM
14.3
13.2
12.0
10.3
31.0
29.9
26.5
23.8
11.1
10.0
9.0
7.3
29.7
21.6
Ours
15.3
14.1
12.6
10.0
32.4
31.2
28.5
23.5
11.3
10.1
9.1
6.6
Diff.
+0.7
+0.7
+4.0
+4.3
+1.8
+1.4
+1.4
40.1
(b) The bbox AP (%) results for different IoU thresholds (0.5 to 0.95, 50, 75) compare different dataset pruning baselines Cityscapes.
Table 9: More results on Cityscapes. The pruning rate represents the percentage of data removed from the full training dataset during pruning. The performance the full dataset is indicated by p 0%. Diff. denotes the difference between our method and random pruning, and the improvement in time is the average improvement.
D.4 CROSS-ARCHITECTURE GENERALIZATION EXPERIMENTS
Tab. 10 shows more detailed results on models with different architectures (such as SOLO-v2 and QueryInst) for different IoU thresholds, and TFDP consistently achieves stable improvements. Since the adapted baselines rely on model-specific pruning, the data selected using Mask R-CNN shows varying degrees degradation when applied other architectures. This especially evident in the Transformer based model QueryInst, where the data selected by CNN-based Mask R-CNN performs worse than random selection under many pruning rate settings.
SOLO~a2
QueryInst
50%
mAP
AP
AP15
APs
APM
APL
mAP
APs0
AP
APs
APM
AP
Random
31.3
51.1
32.6
11.6
34.2
48.4
33.3
52.8
35.6
15.0
35.4
51.8
Entropy
31.4
51.6
33,0
12.5
35.0
46.6
33.5
53.9
35.6
16.1
36.3
49.4
EL2N
30.7
50.3
32.2
11.5
34.6
45.3
32.8
52.7
35.0
15.7
36.0
48.4
AUM
31.0
51.0
32.3
11.5
34.7
45.6
33.9
54.0
36.4
15.8
36.8
49,9
CCS
31.8
52.1
33.2
12.1
35.5
47.7
33.3
53.5
35.6
15.6
35.9
49.8
Ours
32.2
52.3
34.1
12.3
35,5
48.6
34.5
55.0
36.9
15.8
37.5
51.9
Diff.
+0.9
+1.2
+1.5
+0.7
+1.3
+0.2
+1.2
+2.2
+1.3
+0.8
+2.1
+0.1
Table 10: More detailed results in the generalization ability to different architectures on COCO
dataset.
D.5 NETWORK SCALING EXPERIMENTS
Tab. shows results to verify TFDP's scalability on the COCO dataset for different backbones (such ResNet-101 and 1011). On backbones with more parameters and stronger per- formance, our TFDP can still further improve the model's performance, demonstrating the good scalability of our method.
21

---

Published as a conference paper at ICLR 2025
mAP
APso
APs0
Validation Network
20%
30%
40%
50%
20%
30%
40%
50%
20%
30%
40%
50%
Random
35.5
34,8
34,0
33.1
56.6
55.7
54,7
53.8
37,9
36, 9
36.3
35.0
ResNet-101
Ours
35.8
35.3
34,9
34.3
57.1
56.6
56.4
55.6
38.2
37.6
37.3
36.6
Diff.
+0.5
+1.2
+0.9
Random
36.7
36.2
35.4
34.3
58.4
58.0
56.8
55.4
39.2
38.4
37.8
36.3
ResNeXt-101
Ours
37.2
36.6
36.1
35.5
59.2
58.8
58.2
57.6
39.9
39.0
38.4
37.9
Diff.
+0.4
+1.2
+0.8
+2.2
+0.6
+0.6
(a) The mask AP (%) results for different IoU thresholds (0.5 0.95, 50, 75) of different backbones COCO.
mAP
APso
APs0
Validation
20%
30%
40%
50%
20%
30%
40%
50%
20%
30%
40%
50%
Network
Random
39.2
38.4
37.6
36.4
59.7
58.8
$7.9
56.8
42,8
41.9
40.8
39.6
ResNet-101
Ours
39.4
38.1
60.2
58.9
42.6
42.4
Diff.
+0.6
+1.0
+1.3
+1.7
+1.4
+1.8
+2.1
+0.7
+0.7
+1.6
+1.6
Random
40.8
40.3
39.1
38.0
61.6
61.4
59.9
58.7
44.7
44.2
42.8
41.5
ResNeXt-101
Ours
41.0
39.6
62.1
60.9
Diff.
+0.6
+0.7
+1.2
+1.6
+1.0
+0.7
+1.5
+2.2
+0.7
+0.8
+1.2
+1.9
(b) The bbox AP (%) results for different IoU thresholds (0.5 to 0.95, 50, 75) of different backbones COCO.
Table 11: The 3P30 (%) results in the scalability ability to the different backbones of Mask R-CNN on the COCO dataset.
D.6 ANALYSIS ON
DESIGN
To further validate the effect of SI design, which is applied mitigate the small-scale bias of the SCS, we compare the Average Precision scores at different scales (APs. APM APL) following the COCO standard metries before and after applying SI design in Tab. 12 Our results indicate that, prior to applying SI, the SCS scores were significantly biased towards smaller-scale images, result- ing in poor performance in APM and APL. Conversely, after applying the Invarian design, there was a noticeable improvement in AP, levels, without any decline in APs results.
30%
P
APs
APM
AP.
APs
APM
AP.
APs
APM
AP,
w/o S1
20.6
12.9
17.8
20.5
12.2
16.3
19.5
11.3
13.8
22.6
13.4
20.7
21.7
1
+2.0
+0.5
+2,9
+1.2
+1.9
+2.0
+0.7
+1.2
+2.1
Table 12: Ablation study of the SI-SCS. Comparison of mask AP for objects of different scales followed by COCO official metries (Lin et al. 2014).
D.7 MORE RESULTS ON THE SECOND TRAINING PARADIGM
As mentioned in He et al. 2017): ) A major difficulty with the Cityscapes dataset is the limited number of training samples available for certain categories, such as truck, bus and train, which only have around 200-500 examples each, making challenging to train models effectively. To partially remedy this issue, we follow He et al. (2177) and report the results using COCO pre-training to verify our method further. Specifically, we strictly follow He et al.] (2017) and use pre-trained COCO Mask R-CNN model to initialize categories in Cityscapes, while the rider category is randomly initialized.
In this fine-tuning setting, as shown in Tab. 13 our TFDP achieves more significant improvements. Specifically, our method consistently surpasses random pruning as well as other baselines. Notably, pruming is better than the full dataset, pruming some low-quality or noisy data that could negatively impact model performance. Further- more, in the fine-tuning setting, using the full dataset might lead to overfitting, thereby affecting the
at 50% pruning ratio. This may be due to TFDP
22

---

Published as a conference paper at ICLR 2025
final performance. These results demonstrate that our method is not only applicable but also more promising under the fine-tuning setting.
mAP
AP50
0%
20%
30%
40%
50%
20%
40%
Random
36.4
35.4
35.0
35.6
33.8
61.8
60.5
60.8
60.6
58.9
Entropy
34.7
35.6
34.5
34.3
61.3
62.6
61.4
60.3
EL.N
34.2
34.1
35.2
32.1
59.2
59.7
61.8
57.3
AUM
36.3
34.9
34.4
33.9
62.6
60.9
59.4
59.4
CCS
36.1
36.1
35.0
34.0
61.7
61.7
60.7
59.7
Ours DaI.
36,9
36,6
36,6
36,6
62.8
63.9
63.4
62.8
+1.5
+1.6
+1.0
+2.8
+2.3
+3.1
+2.8
+3,9
(a) The mask AP (%) results compare different dataset pruning baselines Cityscapes (pre- trainer COCO).
mAp
APMa
0%
20%
30%
40%
50%
0%
20%
30%
40%
50%
Random
40.9
39.6
39.6
40.2
39.5
66.3
64.9
64.9
65.2
64.0
Eatropy
39.5
41.0
39.2
39.4
63.9
67.3
66.1
65.4
38.1
39.3
35.2
37.2
62.7
65.1
61.8
AUM
36.3
40.8
39.8
39.0
62.6
65.8
64.2
64.0
CCS
41.0
41.0
40.1
38.7
66.0
66.0
64.8
64.3
Ours
42.1
41.1
41.4
42.0
67.4
68.7
67.5
67.9
Dil..
+2.5
+3.9
(b) The bbox AP (%) results compare different dataset pruning baselines Cityscapes (pre-trained on COCO).
Table 13: Fine-tuned results with COCO pre-trained Mask R-CNN on Cityscapes.
D.8 TIME CONSUMPTION DETAILS
To ensure fair comparison, all time consumption experiments were conducted same machine: PyTorch on Ubuntu 20.04, with NVIDIA RTX 3090 GPUs and CUDA 1 11.3. We used two NVIDIA 3090 GPUs for training and one NVIDIA 3090 GPU for inference.
The following section provides explanations of some time consumption details. EL2N (Paul et al. 2021 and Entropy (Coleman et al., 2019) measure the importance of samples by calculating the distance between output logits and the ground-truth one-hot labels, which includes the time for both model training and inference (Scoring). According to the original settings EL2N, it only requires model weights in the early training stage, hence the training time is shorter than Entropy. AUM (Peiiss et al.) 2020) and Forgetting (Toneva et al. 2019) assess the difficulty of samples by tracking changes in the logits across different epochs during training, thus the process includes model training and scoring through training logs. The time difference between the two is minimal, and we report them single method. For CCS, in addition to the time taken to obtain AUM scores, the time for Stratified selection should also be factored in. However, the proposed TFDP does not require model and only needs the time for scoring through pixel-level annotations, which significantly less than the time required by existing model-based sample selection methods.
D.9 MORE VISUALIZATIONS
To further explore the effectiveness of our method, we provide more visualization on VOC, Cityscapes, and COCO in Fig. Notably, SCS itself can successfully distinguish "hard" and "easy" images as shown in Fig 9 (a). We can visually tell the top-ranked images are more complex than the least-ranked images. By applying Invariann SCS the effect of scale has been eliminated. We can observe from the least-ranked images in Fig. (b) that the criterion changes from simply picking the largest objects easier shapes (e.g., circles). Lastly, by considering class balance, the criterion selects not only the complex shapes but also the images that can best cover the distribution of the dataset.
23

---

Published as a conference paper at ICLR 2025
2012
Cityscapes
MS COCO
Images
Top Ranked
Masks
Images
II X
Least Ranked
Maeks
(a) Visualization only
Images
Top Ranked
Masks
Images
STOP
Least Ranked
Macks
(b) Visualization of Instance Scale-Invariant SCS (SI-SCS)
0.2098
0.0157
Images
Top Ranked
Macks
6.00085
0,80138
0.00433
0,00412
8.00404
0.00499
0.00410
5.00410
LE5427
0.00100
10 csace
BICONA SONOO SHIM
score: 0.00078
score: 0.00080
score: 0.00003
0.0000026
0.000027
0.000028
Images
Leaist Ranked
0.0000026
Masks
0.000028
0.00078
0.00080
(c) Visualization SI-SSS
Figure More visualization of the proposed method.
24


Reviews:
[{"review_id": "3dQvdEvZu4", "reviewer": "Reviewer_SyrL", "summary": "The paper presents a novel Training-Free Dataset Pruning (TFDP) method designed for instance segmentation tasks. Unlike existing dataset pruning techniques that focus on image classification and require time-consuming model training, TFDP leverages shape and class information from image annotations to prune datasets without any model training. The authors introduce a Shape Complexity Score (SCS), which is further refined into Scale-Invariant (SI-SCS) and Class-Balanced (CB-SCS) versions to address the challenges of instance area variations and class imbalances. This approach is intuitive and appears effective, with promising results.", "questions": "Some thoughts and explorations:\n1. Impact of Annotation Quality: The masks in the tested datasets are manually annotated, implying potential inaccuracies, especially for smaller objects. Since SI-SCS may assign lower scores to smaller objects due to annotation errors, it could lead to a bias towards pruning these objects. Given that the mean average precision (mAP) metric is averaged over instances, pruning smaller objects could disproportionately affect the overall mAP score.\n2. Balancing Scale Diversity: While SI-SCS aims to make the score scale-invariant and CB-SCS balances class contributions, it would be interesting to explore whether incorporating additional mechanisms to balance the pruning of objects across different scales could further improve performance, particularly in scenarios where small objects are critical for overall accuracy.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- This approach is intuitive and appears effective\n- This is the first work to address dataset pruning for instance segmentation.\n- By eliminating the need for model training, TFDP aligns well with the goal of dataset pruning, which is to reduce resource consumption. And significantly accelerates the pruning process.\n- The paper is well-structured, with clear and concise explanations supported by intuitive figures.", "weaknesses": "- Given that instance segmentation tasks are often data-scarce due to the high cost of annotations, the immediate practical utility of dataset pruning in this domain may be limited.\n- As prior work on instance segmentation pruning is lacking, the authors’ adaptation of classification-oriented methods may make baseline comparisons less compelling.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "U7LwfUpTD3", "reviewer": "Reviewer_TLfw", "summary": "This paper introduces a training-free dataset pruning method specifically for instance segmentation for the first time. The method leverages object shape information to propose two scores, SI-SCS and CB-SCS, which are then used to select the most challenging samples from the dataset. The method outperforms training-based approaches across different network architectures and datasets.", "questions": "The main concern is the rationality of the proposed task definition. As shown in Figure 6, the selected images predominantly contain a high number of objects, which implies that the actual object pruning rate could be smaller than the corresponding image pruning rate. The comparison between averaging image scores and summation image scores in the paper also supports this point. From this perspective, the comparison of the proposed method with other baseline methods may be unfair. The authors are suggested to:\n1. Provide a detailed analysis of how the image-level pruning translates to instance-level pruning.\n2. Discuss the implications of this discrepancy on the results and comparisons.\n3. Conduct additional experiments by controlling the total number of object instances rather than images when comparing methods. This would help address the potential bias towards selecting images with many objects and provide a more equitable comparison.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The writing in the paper is clear and easy to understand.\nThe paper provides thorough experiments across multiple datasets and network architectures.", "weaknesses": "1. It is too simple to set the random sample selection strategy as the training-free baseline. Other intuitive selection methods should be compared as baselines to understand better why the proposed method is effective. For example, select images with the most objects or those containing rare categories.\n\n2. The minimum training unit for instance segmentation is the object, not the image. In the proposed dataset pruning task for instance segmentation, is it appropriate to define the pruning ratio based on the number of images? It is suggested that the authors report\nboth image-level and instance-level pruning ratios for each experiment, and discuss how this affects the interpretation of the results.\n\n3. Does the method show performance improvement over the baseline for all categories? It is suggested that the authors provide a category-wise performance breakdown, highlighting any categories where the proposed method underperforms the baselines, and analyzing potential reasons for these discrepancies.\n\n4. In addition to the random strategy, results at high pruning rates should also include other training-based adaptation methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UG1eagcVY0", "reviewer": "Reviewer_p9Ls", "summary": "The cropping method for classification tasks is ineffective in instance segmentation applications. The paper proposes a Training-Free Dataset Pruning (TFDP) method, which utilizes the shape and class information in image annotations to design a Shape Complexity Score (SCS) to address issues of instance region variation and class imbalance.", "questions": "1. Combining theoretical and experimental results, we have considerable doubts about the use of P/A ratio and other metrics as Shape Complexity Scores in the instance-level score calculation in the paper. \n\n2. In SCS, the CB-SCS actually balances the number of instance objects in each diagram, but for datasets like COCO, the number of instances in some categories is very small. Does this potentially cause bias in the data distribution? Additionally, is the consumption of computing resources relatively high?\n\n3. For the ablation experiments of SI and CB in SCS, it can be seen from Table 4 that the experimental results obtained by increasing SI or CB alone are not satisfactory. However, the authors did not provide relevant analysis in the paper. Why does increasing the module individually affect the performance of the original model? Why does the combination of the two result in better outcomes?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper proposes the Shape Complexity Score method around the simple P/A ratio concept. It introduces a relatively simple cropping method in the instance segmentation task. On the other hand, it significantly shortens the time for sample sorting in the cropping method. In addition, this method is more prominent in solving the instance segmentation task compared to previous methods. As mentioned above, the proposed method in this work has certain innovation, the paper framework is logically clear, and the integrity of the paper is relatively high.", "weaknesses": "1.It is suggested that the author add literature on different pruning methods in the instance segmentation task in the Introduction section or the Dataset Pruning section of the related work, or provide a relevant summary analysis.\n\n2.It is suggested that detailed explanations be provided for formulas (5) to (12) in the paper, such as what does T represent in formula (5)?\n\n3.The discussion and derivation of SI-SCS content in Section 3.3.2 of the text are not detailed enough, and it is recommended to add supplementary explanations.\n\n4.The experimental part of the paper is not sufficiently comprehensive. 1) Important details regarding the training and inference stages are not provided in the experimental setup; 2) No detailed explanation is given for the poorer results in Table 4 after using SI. In addition, the authors have not provided a detailed analysis of the experimental results for all figures and tables in the experimental section.\n\n5.The visualization results presented in Figures 6 and 9 can be intuitively observed to show that for some complex scenarios, when dealing with targets of more intricate shapes, or when addressing the local areas of the targets, the performance may be somewhat underwhelming.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)