Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Authors: MAYNARA DONATO DE SOUZA
Cleber Zanchettin
Venue: ICML 2025 poster (2025)
Keywords: Medical tasks
synthetic data
data-centric
Primary area: applications->health_medicine

Abstract: Data scarcity and quality issues remain significant barriers to developing robust predictive models in medical research. Traditional reliance on real-world data often leads to biased models with poor generalizability across diverse patient populations. Synthetic data generation has emerged as a promising solution, yet challenges related to these sample's representativeness and effective utilization persist. This paper introduces Profile2Gen, a novel data-centric framework designed to guide the generation and refinement of synthetic data, focusing on addressing hard-to-learn samples in regression tasks. We conducted approximately 18,000 experiments to validate its effectiveness across six medical datasets, utilizing seven state-of-the-art generative models. Results demonstrate that refined synthetic samples can reduce predictive errors and enhance model reliability. Additionally, we generalize the DataIQ framework to support regression tasks, enabling its application in broader contexts. Statistical analyses confirm that our approach achieves equal or superior performance compared to models trained exclusively on real data.

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Maynara Donato de Souza
Cleber Zanchettin'
Abstract
generative models, loss functions, or generation methods without adequately addressing how to handle synthetic data effectively. This gap highlights the need for data-centrie approaches that prioritize the quality of synthetic data itself.
Data scareity and quality issues remain signifi- cant barriers developing robust predictive mod- els in medical research. Traditional reliance on real-world data often leads to biased models with poor generalizability across diverse patient pop- ulations. Synthetic data generation has emerged as promising solution, yet challenges related to these samples' representativeness and effec- tive utilization persist. This paper introduces Profile2Gen, novel data-centric framework de- signed to guide the generation and refinement of synthetic data, focusing on addressing hard-to- learn samples in regression tasks. We conducted approximately 18,000 experiments to validate its effectiveness across six medical datasets, utilizing seven ofate-of-the-art generative models. Results demonstrate that refined synthetic samples reduce predictive errors and enhance model reli- ability. Additionally, we generalize the DatalQ framework to support regression tasks, enabling its application in broader contexts. Statistical anal- yses confirm that our approach achieves equal or superior performance compared to models trained exclusively on real data.
Hansen et al. (2023) proposed a data-centric method to gen- erate synthetic data for generative dlssification-fouusse setting. They showed that it is possible to improve the perfor- mance by profiling at the preprocessing and postprocessing stages. However, such approaches remain lacking in explo- ration concerning generative regression eggression-oocused tasks. In this context, we introduce Profile2Gen, a systematic frame- work designed to enhance the generation and refinement of synthetic data by incorporating profiling techniques, en- suring that hard-to-leam samples are considered from the generation process through to the synthetic samples.
Hard-to-learn samples affect model performance, either leading models to predict correctly with low confidence or misclassifying with high confidence. These samples are crit- ical and need to addressed, especially in medical research, which faces real data scarcity issues and requires represen- tative synthetic samples to enhance diagnostic precision, treatment effectiveness, and patient outcomes, contributing to more efficient and personalized treatment.
To validate the effectiveness of Profile2Gen, we conducted approximately 18,000 experiments using seven tfate-of-the- art generative models across six distinct medical tabular datasets. The main contributions of our work are summa- rized follows:
1. Introduction
Numerous studies have explored various methods to gener- ate synthetic data for augmenting datasets (Bandara et al., 2021), problems related to the representativeness and use- fulness of the synthetic data persist (Northcutt et al., 2021), I). Despite the central role synthetic data these approaches, much of the existing research focuses on enhancing model performance (Semenoglou al., 2023; Abeysinghe et al., 2023; Iglesias et al., 2023) - typically by improving the
We show that integrating Profile2Genn synthetic data can statistically significantly reduce predictive error in regression models.
We show that models trained through Profile2Gen per- form better and exhibit more consistent results, enhanc- ing their reliability for real-world deployment.
Our approach leads regression models trained only using synthetic data to achieve, in some cases, sta- tistically equal or superior performance compared to models trained using only real data.
'Centro de Informática, Universidade Federal de Permam- buco. Recife, Brazil. Correspondence Maynara Souza <mds3 @et..eeiii...bb...... Cleber Zanchettin <cz@cin.ufpe.br>.
Proceedings of the 4 International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
The Proposed approach can preserve the proportion, in most parts of the experiments, of the minority groups.

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Profiling real data stage
Synthetic data generation
Profiling synthetic data
Ambiguous
Easy
Easy
Mixed profiles
Ambiguous
Hard
Figure 1. The employed methodological process involves a multi-step approach to profilc and generate synthetic data. Initially, the training dataset profiled and categorized into three sample subgroups: easy, hard, and ambiguous. Each of these subgroups then used independently to train a generative model. The synthetic data generated from these subgroups is combined, and new profiling conducted. In the final stage, hard subset samples are identified and removed refine dataset, mitigating the risk of distribution poisoning and subsequent performance degradation.
We propose generalization of the DataIQ framework (Seedat et al., 2022) to support regression models. This extension addresses the original framework's s limita- tion, as was initially designed solely for classification tasks. This adaptation enables comprehensive experi- mentation within the regression context.
The iterative approach ensures that only high-quality sam- ples contribute to the final synthetic dataset. The 'ambigu- ous" or 'hard' samples are identified based on predictive uncertainty metrics, such as low confidence scores or high errors regression tasks (Seedat et al., 2022). These hard samples play a pivotal role in guiding refinement efforts.
A central component DatalQReg, our version of the orig- inal DatalQ framework (Seedat al., 2022) handle re- gression tasks (Northcutt et al., 2021). It enables systematic data quality profiling and provides actionable insights to guide the generative sample process. Profile2Gen combines data profiling with generative modeling to improve synthetic dataset quality iteratively through this integration.
2. Related Work
We engage with AI data-centric based on characterization, with an extended discussion of related work added to the Supplementary Materials. In real-world AI applications, data scarcity can limit model performance. While improv- ing models is common approach, enhancing the dataset itself can also lead to better outcomes. One strategy is data characterization, which involves identifying and addressing problematic samples. Tools like DatalQ (Northcutt et al., 2021), and CleanLab (Seedat et al., 2022) help to categorize data based on confidence and uncertainty. DataIQ labels samples as easy, hard, ambiguous, while CleanLab uses metrics such entropy. Another approach, the Area Under the Margin (AUM) ranking (Pleiss et al., 2020), focuses analyzing model confidence to identify difficult samples. As shown by Hansen et al. (2023), integrating these frame- works can enhance synthetic data generation, resulting high-quality synthetic data that improves performance.
3.1. Framework Workflow
Preprocessing: Given a dataset D, framework F, and a label flipping proportion T:
D' FlipLabels(DDr)
where FlipLabeess(D,r) randomly alters the labels of [D] samples. Considering the scoring function F (1) associated to F (See Appendix) - for each sample x such:
x E Hard -
Fscore (x) < thresholdy
3. Profile2Gen framework
where thresholdy corresponds to a user-defined value that establishes the acceptable margin for error. The performance of evaluated, according to the F1 score f as in Eq. The best-performing Fopt proceeds to the remaining steps.
The proposed framework is multi-stage process designed to guide the synthetic data generation, as illustrated in Figure We adapt Hansen et al. (2023) (Hansen et al., 2023), which focuses on classification tasks, to regression tasks.
2

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
data-centric preprocessing without additional postprocess- ing steps like those in Profile2Gen.
Profile2Gen (Proposed Method); Integrates both prepro- cessing and postprocessing steps.
r n_correct_hard flipped P n_correct_hard
. 2(p= rr)
n.
p +
4.2. Evaluation Tasks
Here, n flipped represents the number of samples with flipped labels, nard denotes the number of samples that were classified as hard by F, and n correct hard refers to the number of instances where the flipped samples classified as hard match the samples identified as hard by F.
Train on Synthetic, Test on Real (TSTR): Evaluate the contribution of synthetic samples on the model's training (Fekri et al., 2019). We compared the Root Mean Squared Error (RMSE) in each step, aiming for the lowest RMSE.
Augmentation: The evaluation ensures the validity of the synthetic data for expanding the training dataset.
According to Faps scoring function (See Appendix) and thresholdy, it profiles the D) in subsets. For each subset S E {Easy, Hard, Ambiguous):
Generating Synthetic Data:
Data quality: Based three criteria to evaluate synthetic data's 's quality: Fidelity (accuracy). Diversity, and General- ization (Alaa et al., 2022).
S sya GeneraateSyytheeicCCaaa.a,,,,
where M is the generative model trained on S.
4.3. Data-centric AI frameworks
Postprocessing: After combining synthetic subsets
To profile the data, we utilized CleanLab (Seedat et al., 2022) and our proposed DatalqReg, which is based on DatalQ (Northcutt et al., 2021). Further details in Section B.
Dayn = Us DS syn
Dinal = Dun''s HardSamples(Dgmm Fopt)
4.4. Datasets
discussion of the optimization process, including the sen- sitivity of r, is provided in Section E.
We used six public datasets available on the OpenML web- site. The datasets were selected based on availability and accessibility due to the well-documented scarcity of public, high-quality medical datasets for synthetic data research. The datasets include the Parkinson dataset (T'anns al., 2009) (Parkinson), which has primary aim to predict the motor and total UPDRS scores (Tsanas & Little, 2016); The urinary dataset (Czerniak & Zarzycki, 2003) aims to predict the diagnosis of two diseases of the urinary sys- tem (Czerniak, 2015). Cholesterol dataset (Detrano et al., 1989) concerning heart disease diagnosis (OpenML, 2024d), Body Fat dataset (fat) lists estimates of the percentage of body fat (OpenML, 2024c). Plasma Levels dataset (plasma) (Nierenberg et al., 1989) studies the plasma concentra- tions. (OpenML, 2024b). Diabetes dataset (Torgo, 2024) aims to predict the C-peptide level (OpenML, 2024a). The Supplementary Material (Section A) contains more details.
4. Methodology
We compared Profile2Gen with traditional generative tech- niques and a baseline preprocessing data-centric approach (PreProcess). The generated data was assessed through multiple evaluation tasks to determine the efficacy of each method. The source code is available GitHub
4.1. Data Generation Methods
Traditional Generative Techniques: Data is generated us- ing traditional methods, where generative models are trained without any profiling distribution- separation re- ferred to as Traditional in our analysis. More details about the baseline choice are available in Section N
PreProcess: The PreProcess technique serves as a baseline data-centric generative method. involves the following steps: The dataset undergoes a data-centric profiling phase using frameworks such Cleanlab or DatalQReg; Based the profiling results, the data is categorized into subsets (e.g., "easy' and 'hard'); Generative models are trained separately on each subset, taking advantage of the profile-informed grouping; The final dataset is created by combining the generated data from all subsets without further profiling or refinement. This baseline evaluates the direct impact of
4.5. Models
To generate synthetic data, we utilized seven stat-of-art in tabular synthetic data generation models: CTGAN (Xu et al., 2019a), TVAE (Xu et al.. 2019a), NFlow (Kobyzev et al., 2021), Bayesian Network (Young al., 2009), Ddpm (Ho et al., 2020), Tabformer (Padhi et al., 2021), and Great (Borisov et al., 2023). To assess the experiments, we adopted twelve tabular data predictors: XGBoost, Weighte- dEnsembleL2, Neural Network, CatBoost, NeuralNetPas- (AI, ExtraTreesMS., LightG RandomF
3

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Parkinson
Column
3.0
0.6
0.2
Traditional
PreFrocees
Profie23on
Traditional
ProProcess Profile2Gen
Traditionl
PreFrocess
Prottle2Gee
Tradiional
PreProcess
Profile6ean
bayasion_natwort
otgan
Figure 2. Violin plots for randomly selected column. The first violin represents the real distribution, followed by four groups of violins, each corresponding to a generative model and separated by horizontal lines. Violins share the same color palette within each group, with each violin representing specific technique, starting with Traditional. A caption for each group provided below the graph. The plot illustrates the decreasing fidelity of the distributions, moving from the real data to the final technique in each group. Figure 15 provides complementary details of this analysis.
NFlow
5. Results and Discussions
50
We utilized the Wasserstein distance (Panaretos Zemel, 2019b) as a metric for measuring the similarity between synthetic and real samples (see Figure for details). In addition, we analyzed the distribution of randomly se- lected column from each dataset at various stages. This analysis enabled us to compare distributions and assess data quality based key criteria such fidelity, diversity, and generalization (Alaa et al., 2022). These comparisons are visually illustrated Figure Due to the number ex- periments, we focused on commenting on highlights, and further details, including model-specific performance (see Section R) and datasets-variation impact (see Section S) provided in the supplementary materials. To assess compu- tational efficiency, we selected the largest dataset, Parkinson, with approximately 3,500 training samples. We applied the framework selection process (Cleanlab and DatalQ) along with profiling, using two thresholds and label replacement ratio. These experiments required 3,510 MB of memory and approximately minutes complete. More details about scalability are discussed in Section D.
40
Wasserstein Distance
20
10
0
50
Bayesian Network
40
30
10.
0
urinary
parkinson
cholesterol
diabetes
fat plasma
Profile2Gen
Traditional
Figure 3. Wasserstein Distance across Data Stages for Generative Models. The grey bars represent the Wasserstein distance between the synthetic data generated in the 'Traditional' stage and the real data. The light blue bars correspond to the 'PreProcess', and the dark blue bars indicate the 'ProfileGenn' stage. The y-axis displays Wasserstein distance values, while the x-axis corresponds to the evaluated datasets. Fig. 14 is complementary to this Figure.
5.1. TSTR Results
Considering that Profile2Gen's ' synthetic datasets about 10% smaller than the original (due to hard profile removal, see Figure 13), its overall performance can be considered strong. In some cases (Figure 19), TSTR even outperforms Real data, achieving up to 1.2% lower RMSE. Since match- ing or surpassing real data performance using only synthetic data is significant challenge (Hao al., 2024), we focus on comparing Profile2Gen with other approaches-partieee Traditional, widely used baseline.
Lig LightGBMXT, KNeighborsUnif, and KNeigh- borsDist. These models are all available within the Au- toGluon library (Erickson et al., 2020), a framework that automates the utilization of state-of-the ahe- models.

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Diabetes
ducing RMSE values compared to the Traditional and Pre- Process techniques (see Figure 5). In the Urinary, Parkinson, Plasma, and Fat datasets, models such as Bayesian Network, NFlow, and TVAE consistently showed reduced RMSE us- ing the Profile2Gen approach, with some models even sur- passing the baseline performance of real data, particularly in the Plasma dataset (see Figure 6). However, the CTGAN model often exhibited high variability, and in the Diabetes dataset, the PreProcess stage outperformed the Profile2Gen stage, indicating some inconsistency in the effectiveness of the synthetic data across different contexts (Fig. 23 - 26).
11-
Evalustor
Raeighoorsun
10
ExtraTreesMSE
weighoddnnsmmble L2
RMSE
9-
Lightcaa
CorBoost
Khegloossst
Lighac grge
8-
NeurathetFastal
Handantonwutse XGRoost
7
Traditional
ProProcess
ProfileGGen
The results suggest that the refined synthetic data enhances model performance, but its impact varies depending on the dataset and model used. However, despite showing clear benefits, it is not one-size-fits-al solution. The variability observed across different scenarios indicates the need for ongoing optimization and possibly even custom strategies for each model-dataset combination.
Figure 4. Line plots depicting RMSE across the approaches for the twelve models employed in the TSTR task. The plot illustrates the Diabetes dataset synthetically provided by the TVAE model. The plots represent the best-perfomning scenarios observed.
5.2. Augmentation Results
5.2.2. PROPORTIONS ANALYSIS
Profile2Gen consistently outperforms Traditional, reduc- ing RMSE by 4.8% (Figure 5) and achieving superior per- formance in most scenarios with less data. Profile2Gen produces high-quality synthetic data that enables better gen- eralization and lower RMSE. Figure 4 illustrates common trend: Traditional exhibits high RMSE, indicating poor per- formance: PreProcess improves upon this with some RMSE reduction; Profile2Gen achieves the lowest RMSE, demon- strating the best overall performance. This underscores the value of data profiling and refinement, showing that a smaller, well-curated dataset can be more effective than a larger, unrefined one. Additionally, Profile2Gen enhances generalization across models and datasets (Figures 20-22), making it particularly useful when real data is scarce.
Souza et al. (2023) (de Souza et al., 2023) assert that there is no established rule or recommendation regarding the pro- portion or quantity of synthetic data used in augmentation tasks. Our experiments align with this statement, as the results vary across models, datasets, and the amounts of data added. However, there are patterns in the performance. Adding 22.7% (0.227) of synthetic data maintains perfor- mance close to the baseline, indicating minimal impact on model accuracy. However, when the proportion of synthetic data increases 68.2%, many models begin to show no- ticeable deviations from the baseline, suggesting that this is critical point where performance can start to degrade. At 90.9% synthetic data, performance typically declines significantly, likely due to the model being overloaded with non non-gen genererlaliizble information, particularly in the 'Tradi- tional' and 'PreProcess' stages. Adding 45.4% synthetic data often keeps model performance stable, though some variations may occur depending on the model and stage (Figure 23- 26). Robust models like Bayesian Network and CTGAN generally handle this amount well, without significant increases in RMSE.
However, performance differences across approaches are minor in some cases, revealing trade-offs. For example, Pre- Process sometimes achieves lower RMSE than ProfileGGen, though the latter is more consistent across models and datasets. This suggests that preprocessing alone-without additional profiling refinement- may be preferable in spe- cific scenarios, depending on the dataset and model used.
We have a two-level evaluation: i) overall performance across experiments, grouped by dataset, model, augmenta- tion proportions, random seeds (333, 555, 666, 888), and evaluation models; and ii) the impact of varying synthetic data proportions for each dataset and model. Proportions were randomly selected between 0.1 and to avoid bias and ensure fair evaluation of the augmentation process.
5.3. The Trade-off Between Generalization and Diversity
The Wasserstein distance quantifies the effort required to transform one probability distribution into another (Panare- tos & Zemel, 2019a), measuring how closely synthetic data approximates real data. Lower values indicate better gener- alization, while higher values suggest greater diversity.
5.2. 1. GENERAL CONSIDERATIONS
Figure 3 shows the Wasserstein distance across different approaches. As data-centric processes progress, similarity to real data generally decreases. For the urinary dataset,
The results across various datasets demonstrate that Pro- file2Gin significantly improves model performance by re-
5

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Dataset: parkinson
Urinary dataset
10
Proportion 0.682
Proportion 0.909
8-
-4.8
6-
[-1.14
+0.83
RMSE
0.8
CTGAN
Random Seed
41
0.6
333
2-
0.4
0
ctgan
nflow
tvae
Dataset: fat
10
1.2
1.0
Bayesian Network
6-
-0.52
0.8
41
-0.69
-0.54
-1.34
RMSE
0.6
Random Seed 333
2
555
666
B88
bayesian_netwon network
ctgan
nflow
tvae
0.2
Real
Traditional
PrePracess
ProfileZGen
Real
Traditional
PreProcess
Proffle2Gen
Profile2Gen
PreProcess
Traditional
Figure 5. Average improvement per model and dataset using the TSTR protocol. Bars represent the average performance of genera- tive techniques across 12 predictors, grouped by model, with each plot corresponding different dataset. Green arrows indicate provement relative to traditional methods, while red arrows show no improvement. Improvement values are displayed alongside arrows with + or signs indicating the change direction. See the Supplementary Material for the full figure.
Figure 6. Augmentation performance on the urinary dataset across random seeds, ensuring unbiased results. Columns compare mod- cls at the same synthetic data proportion, while rows show RMSE ranges across varying seeds. Color tones align with the legend. Real performance servos as the baseline. The Profile2Gen stage shows more consistent behavior compared to other stages. Com- plete results are available in the supplementary materials.
synthetic data closely mirrors the real distribution, suggest- ing strong generalization but limited diversity. In contrast, datasets like Plasma and Parkinson's exhibit higher dis- tances, indicating more significant variability. Figure 2 rein- forces this, showing that Profile2Gen increases variability, enhancing diversity but potentially introducing noise that reduces fidelity. The shift from Traditional to Profile2Gen highlights the generalization- rinrreriiationddieersttty trade-off. Greater di- versity helps prevent overfitting but may reduce the synthetic data's ability capture the real dataset's complexity. For ex- ample, while the urinary dataset maintains consistent mean values across generative alization-its lack of variability may limit its usefulness for augmentation (Gong et al., 2019). contrast, Plasma and Parkinson's 's datasets show greater diversity, which en- hances robustness but may hinder generalization. Balanc- ing generalization and diversity is erucial for optimizing synthetic data. High generalization ensures fidelity to real data, making it ideal for TSTR tasks where data is scarce. Conversely, high diversity improves model robustness and supports edge-case exploration but may reduce effectiveness tasks requiring precise real-data representation. Tailoring
synthetic data for specific tasks requires a strategic model and approach selection to achieve an optimal balance. More discussions in Section K
5.4. Data Consistency, Robustness and Reliability
Using different random seeds during augmentation reduces reliance single sample and allows performance vari- ability assessment. We analyze this variability through box plots (Figure 6), where narrower boxes and shorter whiskers indicate greater consistency. Our results show that removing hard samples, as proposed by Profile2Gen, signif- icantly reduces variability and enhances consistency across models and datasets. For instance, the Bayesian Network model, which showed high variability with the Traditional approach the Urinary dataset, became more stable with Profile2Gen-a trend also observed in Parkinson's. Plasma, Fat, and Diabetes datasets. This suggests that Profile2Gen minimizes sensitivity to randomness, improving stability and robustness (Cateni et al., 2023).
lifeeel-iindicainng good gener-
Interestingly, while Profile2Gen enhances consistency,
6

---

Figure 7. Percentage of statistically significant results 0.05, Wilcoxon test) for different data generation approaches. The x- axis compares pipelines (e.g., Real vs. Traditional) under two evaluation protocols: TSTR and Augmentation. 'Real' refers to Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Diabete
TSTR
Augmentation
Mlan Poffoooz
odpm Profic2001
55
100
19
10
10"
80
-10
10
60
40
110
23 12
100
-5
25
1affurme Tindiooral
sablormen
Profhccess
0
Real Traditional
Real s Profrocesss
Real ProfitezGeen
Traattonal Preprocass
Tradittonal Profitezeen
PreProceess Profile2Gen
-33
-10
43
-20
-10
-6
46
-10
Target.
Tured
Figure 8. Histograms comparing the target value distributions of real and synthetic data for the Diabetes dataset. Synthetic data is generated using different models and techniques (Traditional, Preprocessing. Profile2Gen). This comparison emphasizes the ef- feet of each model and technique representing minority classes, particularly values in the distribution's 's tails.
the baseline performance using only real data.
also increases diversity (see Fig. 3). This diversity chal- lenges models to generalize better, reducing overfitting and improving performance on unseen data (Ramanujan et al., 2023). Thus, Profile2Gen balances consistency and diver- sity, ensuring robust and reliable results.
mance. Additional analyses are available in Section P.
5.5.1. TSTR RESULTS
Using the TSTR protocol, we evaluated 30 dataset-model combinations (e.g.. Bayesian Network, CTGAN). The Kruskal- -allis test found significant differences in 96% of cases, confirming substantial performance variations. Post- hoc Wilcoxon tests (Figure 7) further identified these differ- ences. In 91 % of pairwise comparisons, Real demonstrated significant differences, often achieving comparable or better performance than Profile2Gen (see Figure 19). However, given the high statistical significance across experiments and the fact that Profile2Gen achieves similar, better, or worse performance depending on the scenario, its overall performance is comparable to Real data. This suggests that Profile2Gen can serve
The iterative profiling process removes hard samples in suc- cessive iterations and progressively refines the augmented dataset by preserving core patterns and filtering out prob- lematic samples. Even across multiple random seeds, perfor- mance remained stable with minimal variability (see Section L), demonstrating that our approach maintains generaliza- tion without overfitting to specific data configurations. Cru- cially, our results show no significant performance degrada- tion across iterations, confirming that removing hard sam- ples prevents model collapse (Shumailov et al., 2024)-a common issue where generative models lose diversity. In- stead, our method balances refinement and variability, en- suring stability and robustness model performance.
viable alternative in situations where real data is scarce, particularly when model perfor- mance remains stable across synthetic and real datasets. Traditional methods significantly differed from Real data in 96% of cases, often producing lower-quality synthetic data with higher RMSE, highlighting their unsuitability. PrePro- cess showed intermediate performance, with RMSE values typically between Real and Profile2Gen. While it occa- sionally outperformed Profile2Gen, significant differences appeared in 69% of cases, underscoring Profile2Gen'ss 's su- periority. Therefore, while predictors trained on real data consistently achieve statistically significant performance, ProfileGGen offers the best performance potential. Its syn- thetic data is statistically more reliable than that generated
5.5. Statistical Significance
In these experiments, non-parametric statistical tests were applied due insufficient evidence of normality and small sample sizes. The Kruskal-Walln test assessed perfor- mance differences across four groups- Real, Traditional, PreProcess, and Profile2Gen-unm under two evaluation proto- cols: TSTR and Augmentation. For post-hoc analysis, the Wilcoxon test was employed, as commonly done in similar studies. All tests were performed at a 95% confidence level (a = 0.05). This section focuses on Profile2Gen perfor-
7

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
by other approaches, making it promising alternative for predictive modeling tasks.
Diabetes
12
5.5.2. AUGMENTATION RESULTS
Under the augmentation protocol, we evaluated over 90 dataset-prope pratteettroootiionnnee model combinations. The Kruskal- Wallis test found significant differences in 75% of the cases, indicating that at least two methods were performed dis- tinctly. Pairwise Wilcoxon tests (Figure 7) further pin- pointed these differences: Profile2Gen vs. Real: Statisti- cally significant differences were observed in 60% of ex- periments, with Profile2Gen often showing improved per- formance (see Section 5.2). Profile2Gen vs. Traditional and PreProcess: Profile2Gen demonstrated significant dif- ferences in 79% of cases, often outperforming these ap- Proaches.PreProeee vs. Traditional: The PreProcess tech- nique significantly improved over Traditional in 80% of ex- periments. However, its performance varied across datasets, suggesting that preprocessing alone cannot guarantee im- proved results.
grozt
tabfiornr
firel Frashtonal
Profectioo
Phofilecin
Figure 9, This graph shows the performance of generative mod- cls trained synthetic data (TSTR) under three preprocessing scenarios: Traditional, PreProcess, and Profile2Gen, with Real data as the benchmark. RMSE is used as the evaluation met- ric, and colors represent different models. Results indicate that Profile2Gen- prepppccceed models generally outperform others, in- cluding those trained on real data, with lower RMSE.
The PreProcess technique, which involves profiling data based on quality before generation, acts as an intermediary step that reduces noise, minimizes group contamination, and facilitates the identification of behavioral patterns. This approach enhances generative model performance, aligning with findings in (Maharana et al., 2022; Dupont et al., 2022). However, due to the inherent instability in generative model training (Goodfellow, 2016), the outputs remain susceptible to quality issues, underscoring the need for profiling tech- niques. Traditional methods, in contrast, failed to improve predictor performance in 80% of cases, often leading to increased RMSE (see Figures 6 and 7).
Former perform consistently well, with Profile2Gen yield- ing competitive results across settings. Overall, Profile2Gen enhances model performance by preserving minority groups and ensuring a more balanced distribution, often outper- forming other techniques and, in some cases, even real data.
5.7. Ablation on Hard Samples: Are They True Outliers?
5.6. About Fairness
We investigated whether the removed hard sam- ples-instances where the model had low confidence but predicted correctly or high confidence but misclassi- fied-werd statistical outliers. We selected the Cholesterol dataset for this ablation study and analyzed all generative models using the Z-score to determine whether the removed samples were statistical outliers. We initially applied a threshold of +3, following the Empirical Rule (68-95-99.7), which considers values beyond this range as anomalies, accounting for less than 0.3% of normal distribution (Tacq, 2010). However, since our dataset lacked strong evidence of normality, we extended the threshold to #4 to test a more relaxed outlier detection criterion. The results remained consistent: there was no statistical evidence that the removed samples were outliers, even with the extended thresholds. Thus, our findings suggest that the removal of hard samples is unlikely to be driven by extreme values, reinforcing the idea that these samples are distinct from traditional statistical outliers 2
In regression tasks, particularly in medical applications, cap- turing minority events-low-ffn low-rrqueeyy occurrences in the target distribution--s crucial. We aim to preserve these rare cases. By analyzing histograms of the target variable across different synthetic datasets, we assess whether Profile2Gen retains or distorts these events.
As shown in Figure 8, Profile2Gen generally maintains low- frequency events without excessive overrepresentation, like the Traditional method, which significantly amplifies event frequencies. While PreProcess performs similarly, it sometimes fails to retain minority classes, particularly with models like DDPM in the diabetes dataset. The full comparison is provided the appendices. Figure 9 demon- strates that models trained on Profile2Gen data achieve lower RMSE than those trained on Real or Traditional synthetic data, indicating improved data quality. However, when Profile2Gen fails to retain minority classes, RMSE increases, as seen in DDPM. Models like GREAT and Tab-
Thh source code is available on GitHub
8

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
6. Conclusion
References
This paper introduces ProfileGGen, data-centric framework designed to generate and refine synthetic data, specifically targeting challenges posed by hard-to-learn samples in re- gression tasks. The method ensures the creation of high- quality synthetic medical datasets by profiling data during both the preprocessing and postprocessing stages. Through extensive experiments across diverse medical datasets, we show that integrating refined synthetic data significantly reduces predictive errors and enhances model reliability. Additionally, extending the DatalQ framework to support regression tasks highlights the flexibility of data-centric ap- proaches in improving predietive modeling. Our findings emphasize the critical role of high-quality synthetic data in ensuring consistent and reliable model performance, bridg- ing the gap between synthetic data generation and its prac- tical application. While RMSE and Wasserstein's 's distance offer valuable insights into model performance and data sim- ilarity, incorporating additional evaluation metrics-such as fairness, privacy, and clinical cilliy--wull provide more comprehensive assessment of synthetic data quality.
Abeysinghe, A., Tohmuang, S., Davy, J. L., and Fard. M. Data augmentation on convolutional neural networks to classify mechanical noise. Applied Acoustics, 203: 109209, 2023.
Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M. Optuna: A next- next-generation hyperparameter optimization framework. In KDD (arXiv), 2019.
Alaa, A., Van Breugel, B., Saveliev, E. S., and van der Schaar, M. How faithful is your synthetic data? sample- level metrics for evaluating and auditing generative mod- els. In International Conference on Machine Learning, pp. 290-306. PMLR, 2022.
Bailly, A., Blanc, C.,, Élie Francis, Guillotin, T., Jamal, F,, Wakim, B., and Roy, P. Effects of dataset size and interac- tions on the prediction performance of logistic regression and deep learning models. Computer Methods and Programs in Biomedicine, 213:106504,, 2022. ISSN 0169- 2607. doi: http:///bbiogg///..ooooc..pppppp URL httass///ww..cceeneedreeeeeett. scteenee//rrtticeeee//////222222
Impact Statement
Bandara, K., Hewamalage, H., Liu, Y.-H., Kang. Y., and Bergmeir, C. Improving the accuracy of global forecast- ing models using time series data augmentation. Pattern Recognition, 120: 108 148, 2021. ISSN 0031-3203, doi: http://.o...........ooo.aaoooooo
This research introduces Profile2Gen, a novel data-centric framework that significantly enhances the quality and utility of synthetic data for regression tasks, particularly in the medical domain. By addressing the challenges posed by hard-to-leary samples, Profile2Gen empowers researchers and practitioners to:
Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., and Kasneci, G. Language models are realistic tabular data generators, 2023. URL /ttttss///axivv... org//bs/ 2210 06280.
1. Overcome data scareity in medical research, access to large, high-quality datasets is often limited due to privacy concerns and ethical restrictions. Profile2Gen offers viable solution by generating realistic synthetic data that can be used for model training, validation, and testing, thereby accelerating research progress.
Cateni, S., Colla, V., and Vannucci, M. Improving the stability of the variable selection with small datasets in classification and regression tasks. Neural Processing Letters, 55(5):5331-5355 2023.
Czerniak, J. Acute inflammations data set. OpenMI., ID: 1455, 2015, URL https: :/t/::///www /wwww openml. org/d/ 1455. Accessed: 2024-05-31.
2. Improve model robustness and generalization by re- fining synthetic data through iterative profiling; Pro- file2Gen ensures that models trained on this data are more robust and generalize better to real-world scenar- ios. This leads to more reliable and accurate predic- tions, which can significantly affect clinical decision- making.
Czerniak, J. and Zarzycki, H. Application of rough sets in the presumptive diagnosis of urinary system discases. In Artificial Intelligence and Security in Computing Systems, 2002 9th International Conference Proceedings, pp. 41-51. Kluwer Academic Publishers, 2003.
3. This work contributes to the growing field of data- centric AI, emphasizing the importance of data qual- ity and refinement in achieving optimal model perfor- mance. By providing a practical framework for syn- thetic data generation, Profile2Gen sets a precedent for future data-centric innovations.
de Souza, M. D., Junior, C. R. S., Quintino, J., San- tos, A. L., da Silva, F. Q. B., and Zanchettin, C. Exploring the impact of synthetic data on human activity recognition tasks. Procedia Computer Science, 222:656-665. 2023. ISSN 1877-0509. doi: https://do.oor/10.. http////...11/////..// 601.i 2003.es...22....2200

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
URL sci sttp:s////wwsssicceiiieeeeeeee com/
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion prob- abilistic models, 2020. URL URL.tttps://rxxii..oo
International Neural Network Society Workshop on Deep Learning Innovations and Applications (INNS DLIA 2023).
Iglesias, G., Talavera, E., González-Prieto, Á., Mozo, A., and Gómez-Canare S. Data augmentation techniques in time series domain: survey and taxonomy. Neural Computing and Applications, pp. 1-23, 2023.
Detrano, R., Janosi, A., Steinbrunn, W., Pfisterer, M., Schmid, J., Sandhu, S., Guppy, K., Lee, S., and Froelicher, V. International application of new probability algo- rithm for the diagnosis of coronary artery disease. Amer- ican Journal of Cardiology, 64:304-310, 1989. Inter- national Probability Analysis. Address: Robert Detrano, M.D. Cardiology 1 111-C V.A. Medical Center 5901 E. 7th Street Long Beach, CA 90028. Results in percent accu- racy (for 0.5 probability threshold): Data Name: CDF CADENZA. Hungarian: 77, 74; Long beach: 79, 77:
James, G., Witten, D., Hastie, T., Tibshirani, R., et al. introduction statistical learning, volume 112. Springer, 2013.
Kobyzev, I., Prince, S. J., and Brubaker, M. A. Nor- malizing flows: An introduction and review of cur- rent methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(11):3964-3977) Novem- ber 2021. ISSN 1939-3539. doi: 10. 10.1109/tpami.2020. 2992934. URL http:: wtt.:oooo...o......///.. TPAMI 2020 2992934.
Swiss: 81, 81. Approximately 77% correct classification accuracy with a logistic-tegreesion--deeeee discriminant function.
Dupont, E., Teh, Y. W., and Doucet, A. Generative models as distributions of functions, 2022. URL https:// arxiv. org/ abs
Maharana, K., Mondal, S., and Nemade, A review: Data pre- precescessing and data augmentation techniques. Global Transitions Proceedings, 3(1):91-99, 2022. ISSN 2666-
285X. doi:
Erickson, N., Muelerr, J., Shirkov, A., Zhang, H., Larroy, P., Li, M., and Smola, A. Autogluon-tabular: Robust and accurate automl for structured data, 2020.
com/ International Conference on Intelligent Engineering Approach(ICIEA-2023)
URL
cciemne/arrtiilee/p///2/6666
Fekri, M., Ghosh, A. M., and Grolinger, K. Generating energy data for machine learning with recurrent gener- ative adversarial networks. Energies, 13, 12 2019. doi: 10.3390/en13010133
Nierenberg, D., Stukel, T., Baron, J., Dain, B., and Green- berg, E. Determinants of plasma levels of beta-carotene and retinol. American Journal of Epidemiology, 130: 511-521, 1989.
Gong, Z., Zhong, P. and Hu, W. Diversity in machine learning. IEEE Access, 7:64323-64350, 2019. ISSN 2169-3536. doi: 10.1109/access.2011 2917620. URL http://dkx doi oro/o...g/1.1100 ACCESS 2019.222222222 2917620.
Northcutt, C., Jiang, L., and Chuang, I. Confident learning: Estimating uncertainty in dataset labels. Journal of Artifi- cial Intelligence Research, 70:1373-1411, 04 2021. doi: 1..63//air...111222.
Goodfellow, I. Deep Learning. MIT Press, 2016.
OpenML. Heart disease dataset, 2024a. URL https: / / www.ll org/d/212. Accessed: 2024-06-
Hansen, L., Seedat, N., van der Schaar, M., and Petro- vic, A. Reimagining synthetic tabular data generation through data-centric AI: A comprehensive benchmark. In seirth- Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URL https:://// wwtp///eeeeeeeeeeemeeeeee hwtpe/iwwwwweeeeeeeeeeeeeeeeeeee forum? id=dK1Rs1oCI:
28.
OpenML. Heart disease dataset, 2024b. URL https: www.oppeml.. org /25/l. Accessed: 2024-06-
28.
OpenML. Heart disease dataset, 2024c. URL https: / peppemm.orgg///.//.e.ee Accessed: 2024-06-
Hao, S., Han, W., Jiang, T., Li, Y., Wu, H., Zhong, C., Zhou, Z. and Tang, H. Synthetic data in ai: Chal- lenges, applications, and ethical implications, 2024. URL https: arxiv. org/ abs 2401..2 01629.
28.
OpenML. Heart disease dataset (id: 204), 2024d. URL https wwwoop..oooogoooooooee.eoooooooe wwtod/////////ooooo// Accessed: 2024- 06-28.
Heckerman, D. A tutorial on learning with bayesian networks, 2022. URL https: /tte:///arxiie /tt//aaxvvv..///////ee///aa org/ abs/ 2002. 00269.
Padhi, I., Schiff, Y., Melnyk, L., Rigotti, M., Mrouch, Y., Dognin, P., Ross, J., Nair, R., and Altman, E. Tabular
10

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
transformers for modeling multivariate time series, 2021. URL www.::ttt:////mmmm hnttts://arxiv...
2010. ISBN 978-0-08-0044894-7, doi: http:://di..rr// 10.10061177740000004444777711333 URL https:
abs/2011 01.11.0181 01843.
/ //ww:/cclencedrrecc.coo/////////
Panaretos, V. M. and Zemel, Y. Statistical aspects of wasserstein distances. Annual Review of Statistics and Its Application, (1))44054311) March 2019a. ISSN 2326- 104938.
Torgo, L. Regression datasets, 2024. URL http:
831X. doi: 10. 10..1466amuureesssttttttto-00000 URL
/ //www. www..ncc.o. wttpec...ppp//oorrppo... /ttp:/.....pp..//////oo.o...... Regorgg/ on/
http::/dx.doii org. 1146/
DataSets. htm.m.tmmmm................m.m.. Original source: Book Generalized Additive Models (p.304) by Hastie & Tibshirani,
annurev-statisticn www.ssttatttsstee -1tt
Chapman & Hall. Contact: Itorgo@ nec.upp..
Panaretos, V. M. and Zemel, Y. Statistical aspects of wasser- stein distances. Annual review of statistics and its appli- cation, 6(1):405-44 1,,,-4312200 2019b.
Tsanas, A. and Little, M. Parkinson's telemonitoring
dataset. OpenML, ID: 4531, 2016. URL https:
/ //www. www.ml.pemm.comm .rg. orgg///5311 Accessed: 2024-05-
Pleiss, G., Zhang. T. Elenberg, E., and Weinberger, K. Q. Identifying mislabeled data using the area under the margin ranking. In Larochelle, H., Ran- zato, M. Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 17044-17056. Curran Associates, Inc., 2020. URL https:: httpss:/proceediigg..com neurips. / 06100337222211188116668111188881 pdf.
31.
Tsanas, A., Little, M. A., McSharry, P. E., and Ramig, L. O. Accurate telemonitoring of parkinson's 's disease progres- sion by non-invasive speech tests. IEEE Transactions on Biomedical Engineering, 2009. to appear.
Xu, L., Skoularidou, M., Cuesta-Infante, A., and Veera- machaneni, K. Modeling tabular data using conditional gan. In Advances in Neural Information Processing Sys- tems, 2019a
PyTorch.
torch.nn.MSEL.sss 2024a
URL
Xu, L., Skoularidou, M., Cuesta-Infante, A., and Veera- machaneni, K. Modeling tabular data using conditional gan. Advances in neural information processing systems, 32, 2019b.
siteratteettorrc...m.MMsssss.... .M..oos......s.....o...o html. cessed: 2024-07-01.
Ac-
PyTorch.
torch.m..NLLosss. 2024b.
URL
stable// generated/torch. nn. NLLLOSS NL..cosss....LLs................ html.
/ttps://pytorchhcoo
Young. J., Graham, P., and Penny, R. Using bayesian net- works to create synthetic data, Journal of Official Statis- tics, 25(4):549, 2009.
Ac-
cessed: 2024-07-01.
Ramanujan, V., Nguyen, T., Oh, S., Schmidt, L., and Farhadi, A. On the connection between pre-training data diversity and fine-tuning robustness, 2023. URL httpps:/aare /arxivvvvvvvv////////.a///aaaav. org/ abs /2000 12532.
Seedat, N., Crabbé, J., Bica, I., and Schaar, M. Data-iq: Characterizing subgroups with heterogeneous outcomes in tabular data, 10 2022.
Semenoglou, A.-A., Spiliotis, E., and Assimakopoulos, V.
Data augmentation for univariate time series forecasting with neural networks, Pat. Rec., 134:109-132, 2023.
Shumailov, I., Shumaylov, Z., Zhao, Y., et al. Ai models collapse when trained on recursively generated data. Nature, 631:755-759, 2024. doi: 1031/ -41586-004-075566 URL https:::////........ httt:s://do...or//// 1038/s41586
07566-.
Tacq, J. The normal distribution and its applications. In Peterson, P., Baker, E., and McGaw, B. (eds.), International Encyclopedia of Education (Third Edition), pp. 467-473. Elsevier, Oxford, third edition edition,

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
This appendix provides additional details on the data-centric methods, experimental details, and findings.
A. Datasets
The Parkinson dataset (Tsanas et al., 2009) consists of various biomedical voice measurements from 42 individuals with early-stage Parkinson's disease. These participants were recruited for a six-month trial of a telemonitoring device designed for remote symptom progression monitoring. The recordings were automatically captured in the patients' homes. The dataset includes subject number, age, gender, time of recruitment date, motor Unified Parkinson's 's Disease Rating Scale (UPDRS), total UPDRS, and 16 biomedical voice measures. The primary aim of the dataset is to predict the motor and total UPDRS scores from the 16 voice measures. In this work, we focused solely the total UPDRS scores. The dataset is available on OpenMI. (Tsanas & Little, 2016).
The Urinary dataset (Czerniak & Zarzycki, 2003) aims to predict the diagnosis of two diseases of the urinary system: acute inflammation of the urinary bladder and acute nephritis. The dataset includes the following features: patient temperature, occurrence of nausea, lumbar pain, urine urgency (continuous need for urination). micturition pains, burning sensation of the urethra, itching, swelling of the urethral outlet, and the diagnosis (inflammation of the urinary bladder or nephritis of renal pelvis origin). The dataset is available on OpenML (Czerniak, 2015). We changed the target to the data column named 'V1', aiming to adapt for regression task.
The Cholesterol dataset (Detrano et al., 1989) concerning heart disease diagnosis comprises data collected from four different locations: the Cleveland Clinic Foundation, the Hungarian Institute of Cardiology, the V.A. Medical Center, the Long Beach, and the University Hospital, Zurich. Each of these databases contains the same format and attributes. However, there are 76 raw attributes in the datasets, and only 14 attributes are commonly used in most experiments, including in this work. The dataset can be found on the OpenML website (OpenML, 2024d). The dataset is used in regression cases, and it indicates that the 'chol' column is the target. Still, we believe that the data is mislabeled since it corresponds to targets for classification task. Due to that, we used the 'slope' our target.
The Body Fat dataset (fat) lists estimates of the percentage of body fat determined by underwater weighing and various body circumference measurements for 252 men. The data comprises information such as age, weight, height, and neck and chest measurements. It contains fifteen columns, including the target column, corresponding to the fatness measure. No more information is available about this dataset; the web pages that describe it are no longer available; only the OpenML is available (OpenMI.. 2024c).
The Plasma Levels dataset (plasma) (Nierenberg et al., 1989) is a dataset raised from a study with 315 subjects. The patients who had an elective surgical procedure over three years to biopsy or remove a lesion of the lung, colon, breast, skin, ovary, or uterus that was found to be non-cancerous. Plasma concentrations of the micronutrients varied widely from subject to subject. Age, sex, and alcohol consumption have an influence, for instance. The data comprises information about the age, Sex, Smoking status, Vitamin Use, Number of calories consumed per day, and Number of alcoholic drinks consumed per week. A total of fourteen pieces of information were collected. The data target used was the Betaplasma, which corresponds to the Plasma beta-carotene levels. The data are available in (OpenML, 2024b).
The Diabetes dataset (Torgo, 2024) concerns the study of the factors affecting patterns of insulin-dependent diabetes in children. The objective is to investigate the dependence of the level of serum C-peptide on various other factors in order to understand the patterns of residual insulin secretion. The data contains information about the age and insulin deficit and aims to predict the C-peptide level. is available on (OpenML, 2024a).
A.1. Dataset Selection
The datasets used in this work were selected based on availability and accessibility due to the well-documented scarcity of public, high-quality medical datasets for synthetic data research. While an ideal selection process would involve curating datasets specifically tailored to diverse medical tasks, we relied on the following practical criteria:
1. Public Availability: We prioritized publicly available datasets to ensure the reproducibility and transparency of our experiments.
2. Tabular Data Format: The datasets selected were in a tabular format, aligning with the requirements of our proposed method, Profile2Gen, which is designed for tabular regression tasks.
12

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
3. Medical Relevance: While not exhaustive, the chosen datasets reflect variety of medical domains, providing useful starting point for evaluating the effectiveness of synthetic data generation in this field. We acknowledge that the datasets selected may not fully capture the breadth of challenges encountered in real-world medical applications. However, this limitation underscores the need for broader efforts to create and share public medical datasets to advance research in this area.
B. Frameworks Used
Cleanlab (Northeutt al., 2021) is open-source tool designed improve the quality of machine learning datasets by identifying and correcting label errors. It leverages state-of-the-an algorithms to detect mislabeled data, noisy annotations, and outliers, which can significantly degrade model performance. Cleanlab integrates seamlessly with popular machine learning frameworks, making it easy for data scientists and machine learning engineers to enhance their datasets' integrity and achieve better model accuracy and reliability. It classifies the data as 'hard' or 'easy' based on the likelihood of label errors and the confidence of predictions. This classification helps understand which data points are more challenging for the model learn and can guide targeted data-cleaning efforts.
DatalQ (Seedat al., 2022) is framework that enhances data quality through various preprocessing and validation techniques. It focuses improving the accuracy and robustness of machine learning models by ensuring the datasets used for training and testing are clean, complete, and free of inconsistencies. DatalQ offers tools for handling missing values, detecting and correcting anomalies, and balancing datasets. DatalQ helps build more reliable and generalizable machine learning models by prioritizing data quality. It classifies the data as 'hard', 'easy', and 'ambiguous' according to the models' uncertainty. The examples with low data uncertainty that the model can correctly predict with high confidence are classified as 'easy', while the ambiguous examples are those with high data uncertainty. Hence, the model cannot predict with confidence, and the examples with low data uncertainty that the model cannot predict, i.e., predicted incorrectly yet with high confidence which have low confidence for the correct class, the 'hard' samples.
Each framework works based on a given threshold. This means the classification is based on the threshold's s confidence intervals.
C. DatalqReg
Our tool builds upon the foundational framework of DatalQ (Seedat et al., 2022), bringing suite of innovative improvements designed to elevate data quality and model performance. While DatalQ focuses enhancing data accuracy and robustness through preprocessing and validation techniques, our tool refines this approach to better address regression tasks and offer deeper insights into data quality.
The framework supports two modes of use: for models from Scikit-Learn and models from PyTorch, both employing classification metrics. We made adjustments to make more suitable for regression tasks. The changes began with the choice of the loss function. Originally, the framework used Negative Lok-Likclihood Loss (NLLLoss), common classification problems (Py (PyTToch, 2024b).
LNLL Σ y log(ū:)
We replaced this with Mean Squared Error (MSE), which is widely used in regression tasks (PyTorch, 2024a).
N
LMSE =
Σ
(us
-2))
N
where ys is the true target value and Je is the predicted value.
Other modifications include:
1. Targets: the original framework, targets were treated as 'long' for categorical data, but we adjusted them to 'float" to handle continuous targets.
2. on.epoch.end Modified to save predictions instead of ground truth, reflecting the focus on prediction accuracy rather than confidence levels.
13

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
3. gold_laaels.ppobbblliiie
probabilities: Adjusted predicted values instead of confidence levels, emphasizing the precision of predictions in regression tasks.
4. true probabilities: Renamed to true values, representing the actual predicted values (y).
5. confidence:For regression, the mean of predicted values is more appropriate than the mean of confidence levels used in classification.
6. aleatoric uncertainty: Adapted calculate the variance of predicted values, analogous capturing random uncertainty in numerical predictions in classification problems.
N
91 p2)
where μ N/ Ee ys is the mean of the predicted values ÿL-
7. variability: Similar to standard deviation, providing insights into the variability of prediction metrics.
8. Correctness:
Interpreted RMSE (Root Mean Squared Error), key metric for evaluating regression performance.
1
RMSE
Σ yys 0.)²
9. entropy: Although originally named entropy, use R-squared (R2) calculation, which assesses the model's fit to the data and is relevant for evaluating regression model adjustment.
(yi D1
R²
(y/
g)²
where +/4 Ei is the mean of the true target values.
10. mi (mutual information): In the context here, mutual information corresponds to the Pearson correlation coefficient, suitable for evaluating the strength and direction of linear relationships between predicted and true values.
Σ i= (Y: 9) (g: 5)
E (y: D)² V E/ (B. 5)
where and ÿ are the means of the true and predicted values, respectively.
These adjustments ensure that the framework effectively supports and evaluates regression models, aligning metrics and functionalities with the specific requirements of regression tasks. For better comparison with the original version, (Northcutt et al., 2021).
The objective of this work was not to introduce DatalQReg but rather to use it as tool for the proposed technique. However, the DatalQReg source code will be made available, and in future work, we will present experiments and results obtained using this tool.
D. Computational Cost
To assess computational efficiency, we selected the largest dataset, Parkinson's, which contains approximately 3,500 training samples. We applied the framework selection process (Cleanlab and DatalQ) along with profiling, using two thresholds and a label replacement ratio. These experiments required 3,510 MB of memory and took approximately 3 minutes to complete.
For larger datasets, it important to consider that memory usage will increase proportionally. as observed the profiling stage. However, should be noted that the process itself does not require GPU resources. Memory remains the main concern
14

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
at this stage, We utilized an Nvidia RTX4090 generate synthetic samples, particularly for Transformer and LLM-based models.
When working with datasets larger than 3,500 samples, memory consumption could be calculated based on this scaling factor, considering the amount of data processed. While simulating this for larger datasets may be difficult, we believe this scaling factor can provide a reasonable estimate. It is worth noting that finding large datasets, especially in healthcare, is challenging, and the idea of generating synthetic data is particularly relevant in scarcity scenarios where such large datasets are not readily available.
E. About the Optimization Process and Sensitivity
The choice of the hard sample threshold is based on the performance of the profiling framework. In Appendix H, we explain this selection process. Specifically, the threshold is determined by identifying the best-performing framework in terms of Fl score across different levels of label flipping. Threshold Selection Process: We tested six proportions of label flipping: O, 0.02, 0.08, 0.1, 0.20, and 0.25. For each proportion, we evaluated six different threshold values: 0.1, 0.125, 0.15, O. 175, 0.2, and 0.25, The framework that achieved the highest average Fl score across these conditions was selected as the best performer.
Why do we not treat the threshold as a key factor in the main paper? ? Once the best framework is identified, we use the corresponding threshold that yielded the highest Fl score to profile the data. This process occurs at both the preprocessing and postprocessing stages. Since we always use the best-performing framework, we consider threshold selection a non-critical hyperparameter, as it is inherently optimized within the profiling stage. Formally, let F be the set of evaluated frameworks and TT the set of tested thresholds. The goal of the process described in the paper find the optimal pair (J", (*) such that:
(,,(') arg max TE ([) JEF,IET
(4)
where E[FI(f;;) f)] represents the average Fl Fl-score across different proportions of label flipping. Sensitivity analysis typically involves testing the robustness of a fixed parameter, but in our case, t" is not arbitrary - it chosen as part of an optimization process that depends on f*. Since we have already found the optimal pair (f*, ,²*) for each dataset, testing variations in t" without re- re-evaluating f would undo the optimization performed and could lead to misleading conclusions. In other words, the threshold is already thoughtfully selected along with the framework, making an additional sensitivity analysis unnecessary.
Empirical validation of the threshold robustness: Given that sensitivity was concern across reviewers, we assessed how the number of hard-profiled samples changes when varying the threshold T. To ensure reproducibility, we a random seed and let NumPy randomly select T from the range (0.05, 0.6) with six distinct values. Figure 10 illustrates the observed behavior using the smallest and largest datasets from our experiments.
The threshold can be understood as a flexibility level how much confidence is required before a sample is no longer considered "hard." Lower thresholds allow greater flexibility, meaning the model tolerates lower confidence scores. Given that our predictor is good generalizer, we observed a few samples with low confidence. As expected, increasing the threshold leads to higher number of hard samples. However, the change follows smooth, almost linear trend rather than abrupt shifts.
Empirical validation of the influence of the flipping rate and threshold: We also assessed how the number of hard- profiled samples changes when varying the threshold or the flipping rate while keeping the other parameter fixed. To ensure reproducibility, we set a random seed. We let NumPy randomly select a fixed parameter value from the range (0.05, 0.6), while the variable parameter was chosen from the same range but with six distinct values. The Figure 11 illustrates the behavior observed.
The results show that dataset characteristics significantly influence sensitivity. For the Diabetes dataset, fixing the threshold and varying the noise level leads to considerable changes in the number of hard samples, which is expected, given that label flipping in small datasets intuitively affects the data distribution more than in larger ones. However, when fixing the flipping level and varying the threshold, the sensitivity is relatively smooth for both datasets. The curves do not exhibit abrupt changes, confirming that selecting the best threshold for the stable choice.
framework remains a reasonable and
These findings reinforce our original methodology: optimizing , " jointly with f ensures that the threshold adapts to
15

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Diabetes
18
Hard samples
16
14
12
10
0.15
0.20
0.25 0.30 0.35 0.40 0.45 0.50
Parkinson
2400
2200
2000
Hard samples
1800
1600
1400
1200
1000
0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 Threshold
Figure 10. Variation of the number of hard samples according to the threshold.
dataset characteristics without introducing unnecessary complexity or computational overhead.
F. The Generative Models
To generate synthetic data, we utilized four out of the five models employed by Hansen al. (2023) (Hansen et al., 2023): CTGAN (Xu al., 2019a), TVAE (Xu al., 2019a), NFlow (Kobyzev al., 2021), and Bayesian Network (Young et al., 2009). We decided to maintain these models due to their recognition in tabular data generation.
CTGAN (Conditional Tabular GAN) (Xu et al.. 2019a) is a GAN-based model tailored for generating synthetic tabular data, tackling specific challenges inherent tabular data, such as mixed data types, non-Gaussian distributions, multimodal distributions, sparse one-hot encoded vectors, and highly imbalanced categorical columns. CTGAN accommodates continuous and discrete columns by applying softmax to discrete variables and hyperbolic tangents to continuous variables in the generator's output. To handle con-Gaussian distributions in continuous columns and mitigate issues like the vanishing gradient problem, CTGAN utilizes mode-specife normalization. It employs kernel density estimation to determine the number of modes in continuous column and applies a variational Gaussian mixture model (VGM) to normalize values within each identified mode.
The TVAE (Tabular Variational Autoencoder) (Xu et al., 2019a) model extends the traditional variational autoencoder (VAE) framework to generate synthetic tabular data. Proposed by the same authors as the CTGAN model, TVAE employs similar preprocessing techniques and adjusts the VAE loss function suit tabular data generation. utilizes the evidence of
16

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Diabetes
threshold 9.255
Flipping level 0,255
18
16-
Hard samples
14_
12
10
0.3
D.5
0.1
0.2
0.3
0.4
0.5
Parkinson
Flipping level = 0.235
600
3000
500
Hard samples
2500
400
2000
300
1500
200
1000
100
500
0.1
0.3
0.4
0.5
0.1
0.2
0.3
0.4
0.5
Flipping level
Threshold
Figure 11. Variation of the number of hard samples according to the threshold and flipping rate.
lower-bound (ELBO) loss for training, processing the latent variable through fully connected layers with ReLU activations. Continuous variable outputs are generated using a hyperbolic tangent function and sampled from Gaussian distributions, while categorical variable outputs are generated using softmax functions.
Normalizing Flows (NFlow) (Kobyzev al., 2021) are generative models that produce tractable distributions, enabling efficient and exact sampling and density evaluation. This model transforms simple probability distribution into more complex one by applying sequence of invertible functions, allowing to capture complex data distributions efficiently. The transformation is designed to be computationally feasible and invertible, facilitating sampling and density estimation tasks.
The Bayesian Network (Young et al., 2009) a graphical model representing the joint probability distribution for a set of variables. Each node represents a random variable, and directed edges denote probabilistic dependencies between variables. fundamental rule of Bayesian networks is that arrows from one node cannot form cycles leading back the same node, Associated with each node is conditional probability distribution, specifying the probability of each node's value given the values of its parents in the graph. Bayesian networks can be employed to create multiple synthetic datasets released by an official statistics agency while maintaining the confidentiality of the real data. This allows external analysts to explore associations between attributes of interest and other variables.
We conducted the finetuning using Optuna (Akiba et al., 2019) to minimize the training error in the Urinary dataset, the intermediary length dataset. saved pickle file for each model containing the optimized parameters, which are then loaded and utilized during model training. shows it.
G. Data Preparation
The research utilized four medical datasets available on the OpenML website. The data formats included either pandas DataFrames or ARFF files. The preparation process involved excluding rows with missing values, renaming the target column to 'target', and splitting the dataset into training and test sets with 67%-33% split, a commonly used proportion in tabular dataset experiments. Additionally, we provided further details on our analyses:
17

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Model
Setting
generator_n laaprannaaaaeiciceee hidden:
generator_n_ume hidden: 50 generator nonlin: tanh
n.iter: 1000
generator .rropott:: 0.1 1888343533334
CTGAN
Anteriminnator_nllaeeee hidden: 3
discriminator _uintts hidden: 100
discriminator :nnoiiiiinatooooonnnn: leaky relu
discriminator
discriminator : 0.012025584400064902
Ir: 0.001
weight decay: 0.001
batch_size: 100
encoder.max.clle clusters: 1
n_iter: 400
Ir: 0.0001
decoderrn__ayers_bl hidden: 5
decay: 0.001
batch_size 64
TVAE
n.units.embeddding 300
decoder_n_ume hiideen 350
decoder.nonlin: relu
decoder dropout: 0.023351136745550774
encooler...lacrr.hhddee hidden: 3
ancoder_n_umits_hideen 100
encoder_nonlin: elu
encoder dropout: 0.17955055753247003
1r: 0.036553322910036633
batch size: 2918
DDPM
num_timesteps: 213 n iter: 8835
iterr: 3200
n layers hidden:
n_umits hidden: 94
batch size: 32
NFlow
dropout: 40446045911370377
batch norm: False
Ir: 0.001
linear ttwassffrnnttypee..... permutation
base_trannformmtyye quadratic-coupling
Bayesian
struct learning search method: tree search struct.leaminggssorre k2
_network
Table f. Parameter values for the different models
We examined low-probability events by plotting histograms to assess whether their distribution was preserved.
To perform statistical validation, we applied the Shapiro-Wii Wilk test (a 0.05) to check whether the datasets followed a normal distribution.
We conducted an exploratory data analysis (EDA) to investigate missing values and NaNs, ensuring data completeness and consistency.
18

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
We included standard statistical metrics such mean, standard deviation, and other relevant descriptive statistics.
H. Choosing the Framework
The main part of the paper describes that selecting the appropriate framework is prerequisite before profiling the data. This process involves flipping the labels in the training data by a proportion т, which means randomly altering the labels of some samples. F12 illustrates this procedure.
Original
Flipping rate 0.5
Figure Labels flipping. Given rate r. samples have their labels changed randomly. figure, each circle has color, and the color corresponds to its label. After the flipping, the strong pink upper circle is purple, the pastel pink is now green, the upper-left purple strong pink, and the green is pastel pink.
After the data labels are flipped, the frameworks Cleanlab and DataIQReg used to profile the data, focusing identifying hard samples. The framework that achieves the highest F1 score is selected. This process is applied to each dataset, meaning that each dataset may have different preferred framework, as each framework may be better suited to different distributions.
The Fl score calculation follows the method outlined (Hansen et al., 2023) and involves identifying hard data and matching it with the (n correct hard) data with flipped targets. The recall (r), precision (p), and F1 score (f) are calculated as follows:
n-correct_hard
n.correct.hard hard
2(p r) f
nippip
p
where n. hard corresponds to the number of sopped-targed samples according to т.
The frameworks profile the data based on specific thresholds (see their documentation for details). Our experiments tested six proportions for label flipping (0, 0.02, 0.08, 0.1, 0.25, 0.20) and six thresholds (0.1, 0.125, 0.15, 0.175, 0.2, 0.25) for both frameworks. The final decision was based on the average F1 score, with the framework showing the highest average score being chosen for data profiling. If both frameworks achieved the same average Fl score, one of them was selected randomly. The threshold used for profiling the data after the framework selection the one that achieved the highest Fl score.
Profiling the Synthetic data
As mentioned in the main paper, the framework selection process is also applied to synthetic datasets. Following this, data identified as 'hard' is removed from the synthetic dataset. The synthetic datasets produced by the 'Profile2Gen' stage contain less data than those from the 'Traditional" and 'PreProcess' stages. Figure 13 illustrates the differences in data length between the 'Original' and 'Profile2een' datasets. On average, the proportion of hard data across the datasets is approximately 10%.
J. On the Generalization and Diversity
Figure 14 displays the Wasserstein distance between synthetic samples and real samples across the stages. This pattern suggests that the stages introduce more variation in the synthetic data, potentially creating broader range of scenarios and examples.
The dataset Urinary highlights that the synthetic data for this dataset also has a high degree of consistency and uniformity, which indicates limited data diversity since the synthetic data closely replicates the real dataset without introducing variability.
19

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
4000
Type
Real
3500
Profile2Gen
3000
Dataset length
2500
2000
B. Netwok
1500
NFlow
CIGAN
1000
Netwok
TVAE
B. Netwok
B. Netwok
B. Netwok
B. Netwok
500
NFlow
TVAE
CTGAN
NFlow
TVAE
CTGAN
NFlow
TVAE
NFlow
TVAE
CTGAN
NFlow
TVAE
CTGAN
0
Urinary
Parkinson
Cholesterol
Diabetes
Fat
Plasma
Figure Comparison of Dataset Lengths: This figure compares the length of the original dataset with the dataset obtained in the
'Profile2Gen' stage. The x-axis represents the generative model and datasct, while the y-axis shows the number of samples.
TVAE
CTGAN
Technique
Traditional
wasserstein Distance
PreProcess
60
Profile2Gen
50
40
30
20
10
x
urinary
parkinson
cholesterol
diabetes
fat plasma
urinary
parkinson
cholesterol
diabetes
fat
plasma
Figure 14. Wasserstein Distance across Data Stages for the Generative Models. The grey bars represent the Wasserstein distance between the synthetic data generated the "Traditional" stage and the real data. The light blue bars correspond the 'PreProcess' stage, and the dark blue bars indicate the 'Profile2Gen' stage. The y-axis displays the Wasserstein distance values, while the x-axis corresponds to the CTGAN model
evaluated datasets. The red means insufficient data to
diabetes dataset.
However, datasets like plasma and Parkinson's, which show greater distances across models and stages from the real samples, may reflect higher data diversity. The lower similarity suggests that the stages introduce more variation in the synthetic data, potentially creating a broader range of scenarios and examples.
Considering a model level, the main paper displayed that in the Bayesian network model, the Wasserstein distance between the real and synthetic data remained low across almost the datasets, particularly in the "Traditional" stage. This trend
20

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Parkinson dataset RPDE Column
3.6
necl
Traditinaal
ProProcess
ProfledBeen
Traditional
PreProcess Prof le2Sen
Trasiticnaal
PreProcess Profie2Sonn
Traditional
PreProcess Profl lediiin
beyeslon_ r.ttwok
Plasma dataset SmokeStatus Column
2.5
1.9
0.5
Bacl
Traditional
PreProcess
Frofie23an
Iraditinal
Profracess
Profie20en
Tradtiional
Profracess
Profile2Geer
bayes onnetworr
Urinary dataset Calumn
125
375
330
300
325
250
Real
Trantiionic
ProProuess
Tradhiancl
ProProcess
Profie2Benn Traditional
PraProcess
ProfileGGen Traditiona
ProPracess
Profiee2Ser
Cholesterol dataset- Chal Column
hraPronses ProfleeCiaan
Trincttoonal
ProProcess Profits23een
Traditional
Pröfracess
Profleditan
Trandhional
ProPracess Profiiedeen
bayeslan_nntwean
Diabetes dataset Deficit Column
20
Real
Traditionall
PreFrocess
Profic23onn
Tracitional
PreProcees
Profle2Gen
Traditional
PreProcsss
ProfledDen
boyes on. network
nffoww
Tede
Fat dataset-
Weight Column
350
200
130
Real
Traditional
Prefhocess
c2Cn
Traditional
Profrocess
Profie2tsen
Traditioncl
Prefrocess
Profiedoo
Traditional
ProProcess Prolie2Seen
bayesoon, network
nnloww
tvoe
otgan
Figure Violin plots randomly chosen column. Each graph corresponds dataset. each dataset. distribution shown (first violin), along with violin groups delineated by a horizontal line, all using the same color palette. Each group represents generative model. caption for the groups provided below each graph.
21

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
underscores the model's ability to generate synthetic data closely resembling the real dataset when Traditional is applied. The model follows the general tendency the similarity decreases toward the 'Profile2Gen' stage - except the Parkinson dataset, where there is rise in distance was observed in the 'PreProcess' stage, followed by subsequent decrease in the 'Profile2een' stage, indicating possible recovery of data quality after initial divergence.
When considering the NElow model, the urinary dataset demonstrated the lowest Wasserstein distance values across all stages. Despite exhibiting an unusual behavior with higher dissimilarity in the PPrProcess'' stage compared to the 'Profile2Gen' stage, this dataset the most consistent in maintaining similarity to the real data. In contrast, the plasma dataset showed the highest distance values, indicating significant divergence from the real data.
In the case of the TVAE model, Figure 14, the Urinary dataset behaved differently from the previous models. This model is particularly distinctive since plasma, cholesterol, diabetes, and fat datasets particularly have the 'Traditional' stage showing higher distances than subsequent stages, where the PreProcess stage registered the lowest distance. However, the Parkinson and urinary datasets deviated from this pattern, with the 'Traditional" stage yielding the lowest distance, which increased progressively in the 'PreProcess' and 'Profile2Gen' stages.
The CTGAN model could not be applied to the diabetes dataset due to insufficient data, limiting its analysis. Among the datasets analyzed, the Urinary dataset showed the lowest distance across all stages, reinforcing its robustness and possible lack of diversity in the training samples. The Parkinson and plasma datasets exhibited the highest distance in the 'PreProcess' stage. As the general pattern, the cholesterol and fat datasets showed the lowest distances during the "Traditional" stage, with subsequent stages revealing nearly identical distance values.
Figure 15 indicates that Wasserstein distance metrics and reveals that the 'Profile2een'' stage across most models and datasets exhibits distribution shape less similar to the real data than the initial 'Traditional' stage. Despite the observed trend of increasing dissimilarity, the Urinary dataset maintained consistent mean values across all generative models and stages, closely mirroring the real data. For the plasma dataset, the mean value remained unchanged when using the TVAE and CTGAN models, though it initially differed from the real data. In contrast to other models, the mean began similarly to the real data but shifted as the processing stages advanced. However, the minimum and maximum values exhibited convergence by the' Profile2Gen' stage, reducing the overall dissimilarity. The Parkinson dataset presented a less consistent pattern, with means generally aligning with the real data but without a clear trend in the minimum and maximum values.
Therefore, as we advance in the stages, the data becomes more diverse and less faithful the real data. This may be more suitable for tasks such as augmentation. However, some datasets, such as urinary resistance, remain more similar to the real data and are more suitable for training tasks.
There is a trade-off off between generalization and diversity. Increasing diversity is positive point since, in training ML models, increasing the size of the dataset does not necessarily imply improving performance and data, and can cause the opposite effect if the data used for augmentation is 'more of the same' (Bailly et al., 2022). However, increasing generalization too much could make the data in dissimilar distribution to the point that we are introducing noise when mixing it with the real data. Consequently, an opposite effect obtained.
K. On the Trade-off Between Generalization and Diversity and Information Loss
Profile2Gen, which incorporates postprocessing, reduces Wasserstein's 's similarity between real and synthetic samples. This indicates that the generated samples are less similar to real data compared to other techniques. Here is the highlight: The similar samples, which have the same statistical characteristics, are a generalization of the real ones, while the diversity concerns about the samples that are not too similar but follow the distribution patterns (Alaa et al. (2021) (Fekri et al., 2019)). When analyzing the dissimilarity alongside the profile2gen samples and their distributions, we observe that the generated samples are not only different but also diverse while still following the patterns of the original distribution. The lower similarity indicates that Profile2Gen prioritizes more variable synthetic data, potentially creating a broader range of scenarios and examples. Consequently, postprocessing increases diversity at the cost of generalization since this set of samples is diminished in the synthetic dataset. Higher diversity is generally beneficial in tasks such as data augmentation, where increasing the dataset size does not necessarily improve model performance unless it introduces novel and meaningful variations. However, a lack of generalization may cause the synthetic data distribution to diverge too much from the real data, leading to a loss of fidelity and entering the previous trade-off discussed by Alaa et al. (2021), the diversity- trade-off.
22

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Even with the trade-off, the critical edge-case information is not lost. It means the low-probability events, which, despite being underrepresented and important in the technique, have synthetic representativity. Demonstrating that, overall, Profile2Gen enhances model performance by preserving minority groups, which correspond to events with low representa- tiveness in the dataset (see Section 5.6. However, the technique's effectiveness varies depending on the generative model adopted. In cases where the technique fails to capture and represent minority data properly, a performance drop is observed. To understand why Profile2Gen better preserves these minority classes, we hypothesize that separating data groups during preprocessing allows the generative model to learn distinct distributions more effectively. In the context of Generative Adversarial Networks (GANs), let Gg represent the generator with parameters 0, and D& the discriminator with parameters d. The standard GAN optimization objective is given by:
min max Ea (2) [log Do(x)) + Ex'~Go(s) [log(1 - D=(('))
where Preat is the real data distribution, and Go(2) generates synthetic samples from a latent noise distribution = ~ p(z).
By training separate generative models for different subgroups, we effectively decompose Preat (2) into conditional distribu- tions p(x]c), where e represents a specific subgroup. This modifies the objective function to:
min max Buwptale [log Do(x)] + ExtwGatzle) [log(1 - D2(x'))]
This forces the generator to learn the distribution of each subgroup separately, reducing early collapse and improving representation for minority groups. However, synthetic data inherently introduces noise, and excessive divergence from p(zlc) may result in synthetic distributions that fail to capture the underlying characteristics of minority classes. This is precisely why Profile2Gen proves more effective than PreProcessing (which lacks postprocessing). When analyzing the histograms Figure in the main paper, we observe that the PreProcessing technique alone overrepresents certain classes. However, postprocessing mitigates this overrepresentation.
We hypothesize that this correction occurs because overrepresented samples categorized hard samples during postprocessing. This reclassification aligns the synthetic data distribution more closely with the real data while preserving meaningful diversity. Since this adjustment is based on model confidence scores, it ensures that the selected synthetic samples remain representative of the original dataset. Consequently, postprocessing refines the synthetic data, ensuring it retains key structural properties while benefiting from the broader variations introduced by generative models.
L. Profiling Iteratively
This study extensively evaluated data augmentation methods by iterating over the provided data from the 'Profile2Gen' stage. The Diabetes dataset was selected as representative case due to its relatively smaller size, allowing for many experimental executions. This choice is instrumental in understanding the behavior of the models under various augmentation conditions, especially given the dataset's constraints.
We employed three generative models, tested them across four different proportions of added data, and varied the random seeds three times for each proportion ensure the robustness of our findings. Each dataset underwent multiple iterations, with the evaluation process thoroughly applied each step assess performance consistently.
Despite using four different augmentation proportions, only two of them completed the process. Due to the data removal in each iteration, there was insufficient data to augment for proportions of 50% or more. As shown in Figure 17 d, each model used a varying number of iterations, reflecting the different quantity of data removed for each model. Figure 18 illustrates the quantity of data removed per iteration. is important to note that the 'Profile2een'' stage also involved data removal, depicted in Figure 13.
Figure 17 illustrates the iterative process employed in this evaluation. It is crucial note that each of the twelve predictor models was utilized during this assessment. This approach enabled us to observe the performance variations and identify any potential indications of model collapse.
We conducted these experiments to investigate whether, similar to the findings in (Shumailov et al., 2024), the models exhibit a learning plateau and increased error when using each mode of profiled data or if the opposite occurs. This could indicate either model collapse or improved data refinement.
23

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Prafiling synthetic data
Ambiguous
Easy
Easy
Removing 'hard' data
Ambiguous
Hard
Evaluating
Ambiguols
Easy
Easy and Ambiguous
Easy and Ambiguous
Hard
Figure 16. Profiling Until Breaks: The profiling process for the synthetic data from the 'ProfileGeen' stage continues until an error occurs the framework used. In each iteration, synthctic data combined with real data, and augmentation evaluation process is conducted to assess performance.
Upon analyzing the results, it is evident that the model's performance demonstrated stability throughout the iterations, with no significant degradation in performance, The continuous and consistent performance across different random seeds and proportions of augmented data suggests that the models did not experience collapse. The robust nature of these models under varying conditions reinforces the reliability of the generative approaches applied, even when challenged by a smaller dataset like Diabetes.
the quantity of added data increased and the data used were more diverse.
Furthermore, the RMSE trends in the figures indicate a balanced and controlled behavior across iterations, signifying that the models maintained their predictive power without overfitting or underfitting, which could be symptoms collapse. The gradual and consistent changes in performance metrics across iterations and proportions confirm that the models adapted well to the augmented data. This further supports the argument that the models did not collapse under the experimental conditions tested.
The evidence suggests that the models were resilient and capable of handling the augmented data effectively without experiencing any collapse. This robustness a promising indication that by profiling the synthetic data, we are increasing the robustness and reliability the models, enhancing their applicability and value in predictive modeling.
M. TSTR
Figure 20 shows that for the Urinary dataset, when analyzing the performance of the Bayesian Network model, the 'Traditional' stage shows high RMSE value, indicating poor initial performance. In the 'PreProcess' stage, there is reduction RMSE, resulting in a performance improvement. Profile2Genly, the Prroill2Geen'' stage, the model exhibits the lowest RMSE, standing out with the best overall performance among all stages. Considering the generative model NFlow, the RMSE high the 'Traditional" stage, with reduction during the 'PreProcess' stage. In the 'Profile2Gen" stage, the model achieves the lowest RMSE among stages. Both the TVAE and CTGAN models started with high RMSE values in the 'Traditional' stage and suffered increase in 'PreProcess', but the 'ProfileGeen' remains the lowest RMSE. This pattern of the models is consistent across the Parkinson and Fat datasets.
Figure 21 shows that in the Cholesterol dataset, the performances obtained with the Bayesian Network model show a high RMSE value in the 'Traditional' stage, indicating poor initial performance, which progressively decreases in subsequent stages, with the PProfileGeen' stage achieving the best performance. However, this progressive error reduction does not occur for the other models; the "Traditional" stage still has the highest RMSE, while the "PreProcess' stage achieves the lowest
24

---

Breaking the Barrier of Hard Samples: A Data-Centra Approach to Synthetic Data for Medical Tasks
Dayesian Netwok
RACE
NFlow
11.0
10.0
BMSE
9.5
8.5
10.25
10.00
9.50
ROMSE
9.25
8.50
0.227
0.454
Proportion added
Figure Performance Twelve Models in Augmentation Tasks: Evaluation of the twelve models across each iteration of the profiling- hard-removal process during data augmentation.
RMSE, which slightly increases in this dataset's 'Profile2Gen'' stage. Although the increase compared to the 'PreProcess' stage, the 'Profile2een' stage still performs better than the "Traditional" stage, indicating that using the proposed data-centric Al approach remains the best method for generating synthetic data.
For the Diabetes dataset, considering the Bayesian Network model, the best performance is observed the 'PreProcess' stage, surpassing the real data's performance. In this model, the 'Traditional' stage shows the highest performance variation, but the 'rrofileGeen' stage ends with the highest average RMSE. For the NFlow model, the 'Traditional" stage has the lowest RMSE and the best overall performance, achieving an error lower than the real data. The other two stages also show emrors lower than the real data, but the 'Profile2Gen' stage performs better, with an average close to the 'Traditional' stage but with higher variability. For the TVAE model, the 'Traditional' stage has a performance inferior to the real data, while the 'PreProcess' and Profflc2Genn stages have a performance with RMSE lower than the real data, with the 'Profile2Gen' stage achieving the lowest RMSE.
25

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Model
Boyesian Network
NFllo
TVAE
3
2
Iteration
Figure 18. Number of hard data removed in each iteration.
Profile2Genly, Figure 22 shows that for the Plasma dataset, the Prrofil2Genn stage performs better than the real data only for the CTGAN model. For the other models, the overall performance of the PProfil2Genn' stage was equal to or lower than the "Traditional" stage, with the 'PreProcess' stage showing the lowest RMSE of all, though not lower than the real data.
Overall, the "Profil2Geen' stage presents the lowest RMSE compared to traditional techniques despite having less data (see supplementary material). Achieving the best performance compared to the baseline's performance training using only synthetic data is still challenge. This approach could not achieve it, but it presents evolution.
N. On the Baselines Choice
We appreciate your feedback regarding the justification for choosing baseline models. The baseline models we selected adhere to the following criteria:
1. They represent traditional techniques primarily used for generating synthetic data, allowing to demonstrate the impact of preprocessing and postprocessing phases.
2. We specifically chose a preprocessing technique to highlight that relying solely on it may not be sufficient for optimal performance.
The choice of baseline models (Traditional and PreProcess) was motivated by the need to compare our framework (Pro- file2Gen) with common approaches and a data-centric baseline approach. The Traditional method represents the generation of synthetic data without any type profiling. The PreProcess method represents a data-centric approach that uses profiling in the preprocessing step but not in the postprocessing. These comparisons allow us to isolate the impact of the different steps of Profile2Gen and demonstrate the effectiveness our approach. Specifically regarding the preprocessing step, we intentionally selected profiling framework optimized for each dataset. This framework is chosen through an optimization process to identify the most suitable preprocessing method for each specific dataset. We did not compare approach to traditional methods like simple feature selection, as we believe these methods do not adequately capture the complexity and improvements provided by our methodology. Using our profiling framework as a baseline, we demonstrated the added value of combining it with a postprocessing phase, which significantly enhances performance. However, we acknowledge that the main text should include a more detailed justification for the choice of baseline models. Due to space limitations, we plan to expand on this discussion the appendix, with an explicit mention in the main paper. This will allow us keep the main text focused the methodology and results while thoroughly explaining baseline model selection process.
O. Augmenting
We conducted an augmentation experiment using four proportions of synthetic data, with four different random seeds for each proportion, across six datasets and four generative models. Due to the high volume of experiments, the results for the Cholesterol dataset are not yet available but will be updated as soon as they are completed.
26

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Dataset: urinary
Dataset: parkinscn
7.59 7:13
3.4
1.39
Teditonal Prefracess Profiactaa
Resl
5.79
5:23
3.0
4.71 4.62
0.68
0.560.59
0.53
1.62
Soom
4.09
3.79
62-
0.54
Peal
baycalan,
1006
Mode
Model
Dittwee:: diabeles
10:09
10.
9.34
8.86
9.41 9.36
8.95
4.27
3.34
Scate
Real
nTow
Tell
bay helwerk
riow
3900
4.42 26,.77
4.51 0.54
2.02g.42 48
1.16 1112-19223
1.25
1:31
2,27
4. 1.004.033 024.0234.2
1.171.17
1.08
1.0
0.4
0.2
baycalmn
Cigan
bayosurnnceek
Otgan
Figure Average improvement per model and datasct using the TSTR protocol. Each represents the average performance of generative technique. evaluated across 12 predictors. Bar groups correspond to the generative models. while each plot represents different dataset. This plot is one version figure 5, including the real (TRTR) performance. Each bar represents RMSE according to the generative model. The number top of the bars represents the RMSE.
27

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Dataset: urinary
parkinson
Model: nflow
2.00
Evaluator
SandomForestMMs
LightGBMXT
CatBoost
1.50
LightGBM
LightGBMLarge
1.25
ExtraTreesMSE
1.00
NeuralNetTorch
0.75
WeightedEnsembl
XGBoost
0.50
KNeighborsDist
KNelghborsUnif
Real
Traditional
Profile2Gen
Real
Traditional
PreProcess
ProfileZGen
(NetFatllt
Model: tvae
2.00
Evaluvtoo
RandomForestMSE
1.75
LightGBMXT
CatBoost
LightGBM
1.25
LightGBMLarge
ExtraTreesMSE
1.00
NeuralNelTorch
0.75
WeightedEnsembl
0.50
e L2 XGBoost
KNeighborsDist
KNeighborsUnif
Real
Traditional
PreProcess
Profile2Gen
Reb
Traditional
PreProcess
Profile2Geh
NeuraiNetFastAI
Model: clgan
2.00
Evaluator
RandomForestMSE
1.75
LightGBMXT
1.50
CatBoost
LightGBM
LightGBMLarge
ExtraTreesMSE
1.00
NeurallNetTorch
0.75
WeightedEnsembl e L.22 XGBoost
0.50
KNeighborsDist KNeighborsUnif
0.25
Real
Traditional
PreProcess
Profile2Gen
Real
Traditional
PreProcess
Profile2Gen
NeuralNetFase Nettttt
Model: bayesian network Evaluntor
1.75
RandomForestMSs
LightGBMXT
1.50
CatBoost
1.25
LightGBM
LightGBMLarge
1.00
ExtraTreesMSE
NeuralNetTorch
WeightedEnsembl
0.50
e e.12 XGBoost
0.25
KNeighborsDist
Real
Traditional
PreProcess
Profile2Gen
KNeighborsUnif
NeuraiNetFastMl
Figure 20. Line plots depicting RMSE across the data stages for the twelve models employed in the TSTR task.
28

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Dataset: diabetes
cholesterol
Model: nflow
Evaluator
11
1.4
RandomForestMSE
LightGBMXT
CatBoost
10
LightGBM
LightGBMLarge
9
1.2
TxeesTre
NeuralNetTorch
1.1
WeightedEnsembl
L2 XGBoost
KNeighborsDist
1.0
KNeighborsUnif
NeuraiNetFastAl
Real
Traditional
PreProciss
ProfileGGen
Real
Traditional
PreProcess
Profile2Gen
Model:
11
Evaluator
RandomForestMSE
10
LightGBMXT
CatBoost
LightGBM
1.2
LightGBMLarge
ExtraTreesMSE
1.1
NeuraiNetTorch
WeightedEnsembl
e,L2 XGBoost
1.0
KNeighborsDist
KNeighborsUnif
Real
Traditional
PreProcess
Profile2Gén
Real
Traditional
PreProcess
Profile2Gen
NeuralNetFastAl
1.4
Model: ctgan
Evaluator
10
RandomForestMSE
LightGBMXT
CatBoost
9
1.2
LightGBM
LightGBMLarge
8
1.1
ExtraTreesMSE
NeuralNetTorch
WeightedEnsembl
7
1.0
122 XGBoost
Rebl
Traditional
PreProcess
ProfilezGen
KNeighborsDist
Real
Traditional
PreProcess
Profile2Gen
1.4
KNeighborsUmit
NeuralNetFastAl
Model: bayesian network
1.2
Evaluator
RandomForestMSE
LightGBMXT
1.1
CatBoost
LightGBM
1.0
LightGBMLarge
ExtraTreesMSE
NeuralNetTorch
Real
Traditional
PreProcess
Profile2Gen
WeightedEnsembl
e_12 XGBoost
KNeighborsDist
KNeighborsUnif
Neural
Figure 21. Line plots depicting RMSE across the data stages for the twelve models employed in the TSTR task.
We analyzed the results at various levels of granularity. Specifically, we evaluated the overall results across all experiments by dataset and model, including all augmentation proportions, random seeds, and evaluation models.
29

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Dataset: fat
Dataset: plasma
Model: nflow
Evaluator
RandomForestMSE
LightGBMXT
CatBoost
LightGBM
LightGBMLarge
ExtraTreesMSE
NeuralNetTorch
WelghtedEnsembl
4.0
e. e_12 XGBoost
KNeighbors Dist
3.5
KNeigh
Traditional
PreProcess
Profile2Gen
Real
Traditional
PreProcess
ProfileGGen
NeuralNetFasta
Model:
Evaluator
RandomForestMSE
5.5
5
LightGBMXT
CatBoost
LightGBM
LightGBMLarge
3
ExtraTreesMSE
2
NeuraiNetTorch
WeightedEnsembl
XGBoost
KNeighbors
Real
Traditional
ProfileGGen
Real
Traditional
PreProcess
Profile2Gen
KNeightorsUnii
6.0
NeuralNetFastAl
Model: ctgan
5.5
Evaluator
RandomForestMSE
LightGAMXT
5.0
CatBoost
LightGBM
4.5
LightGBMLarge
ExtraTreesMSE
NeuralNetTorch
WeightedEnsembl
L2 XGBoost
Real
Traditional
PreProcess
ProfileZGen
Traditional
PreProcess
Prafile2Gen
KNeighborsDist
7
6.0
KNeighborsUnif
NeuralNNetFastll
Model:
bayesian_network nnework
5.5
Evaluator
5
RandomForestMSE
4
LightGBMXT
CatBoost
3.
LightGBM
LightGBMLarge
2
4.0
ExeesTr
1
NeuralNetTorch
WeightedEnsembl
Real
Traditional
PreProcess
Profile2Gen
3.5
Real
Traditional
Profile2Gen
__ XGBoost
KNeighborsDist
KNeighborsUnif
NeuralNetFastAl
Figure 22. Line plots depicting RMSE the data stages for the twelve models employed in the TSTR task.
O.1. General
Figure 23 illustrates the results for the Urinary dataset. The 'Traditional' stage exhibited high RMSE values for the Bayesian Network model, indicating that traditional data generation methods negatively affected the model's performance. The 'PreProcess' stage significantly reduced RMSE, while the 'Profile2Gien" stage achieved the lowest RMSE values. For the
30

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Random
Random Seed
Random
= 333
333
333
Urinary
555
555
555
666
Amount = 0.227
Amount 0.682
Amount 0.909
1.2
....
RMSE
0.8
Bayesian Network
0.4
Real
Traditional
PrePiocans
ProfikzGGen
Reah
Traditional
PreProcedte
ProfiletGon
Real
Traditonall
PrePoccest
ProfietGee
Real
Traditional
PrePocessa
Proffle
1.2
1.0
RMSSE
0.6
0.4
0.2
Real
Traditionsi
PreProass
Profil220en
Redl
Traditional
ProProstat
ProfileGten
Real
Tradiional
PreProcess
ProfileZDen
Real
Traditonal
PreProoano
Profik220e
1.4
1.2
1.0
RMSE
TVVE
0.8
0.4
0.2
Real
Traditionall
PreProcess
Proffezteen
Roal
Traditional
PreProcent
Profiee0een
Real
Traditinal
PrePrccess
Profletttnn
Real
Traditonsi
ProProctat
Proffle20in
1.0
RMSE
0.6.
0.4
0.2
Real
Traditmnl
PreProsent
ProflezGen
Roal
Traditinnal
PreProceed
Profadtien
Redl
Tradittonal
Prerocaao
Pre
Real
Traditional
PreProcass
Profikc2Gen
Figure 23, Augmentation performance across random seeds.
31

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
NFlow model, the 'Traditional' stage also resulted in relatively high RMSE values, with improvements in the 'PreProcess' stage and further reductions in the 'Profile2Gen'' stage, though not as pronounced as in the Bayesian Network model. The TVAE and CTGAN models displayed high RMSE values in the 'Traditional' stage, improved in the 'PreProcess' stage, but with high variability. The 'rrffll2Gen' stage showed reduced RMSE, although CTGAN exhibited elevated variability.
The results for the Parkinson dataset (see Figure 24) mirrored those of the Urinary dataset. The Bayesian Network model demonstrated high RMSE values in the 'Traditional' stage, which decreased in the 'PreProcess' stage, and the 'Profile2Gen' stage achieved the lowest RMSE values. The NFlow model showed a similar pattern, with high RMSE values in the 'Traditional" stage and improvements in the 'PreProcess' stage, with minimal difference between the 'PreProcess' and 'Profile2Gen' stages. The TVAE and CTGAN models followed this patter, with high RMSE values in the 'Traditional' stage and improvements in the PreProcess' stage. The "Profile2Gen" stage saw further reductions in RMSE, though CTGAN showed high variability.
Figure 25 presents the results for the Plasma dataset. The Bayesian Network model exhibited a pattern consistent with previous results, with higher RMSE values in the 'Traditional' stage, reductions in the 'PreProcess' stage, and the lowest RMSE values in the ProffleeGenn' stage. For the NFlow model, the 'Traditional' stage had relatively high RMSE values, with improvements in the 'PreProcess' stage and similar reduction in the 'Profile2Gen' stage. The TVAE and CTGAN models showed high RMSE values in the 'Traditional' stage, improved in the 'PreProcess' stage, and further reduced in the 'Profile2Gen'' stage, although CTGAN maintained high variability. Notably, in this dataset, the average performance of the NFlow, TVAE, and CTGAN models in the 'ProfileGeen' stage exceeded the baseline (Real).
The results for the Fat dataset followed a similar pattern (see Figure 26). The Bayesian Network model exhibited high RMSE values in the 'Traditional" stage, followed by improvements in the 'PreProcess' stage. The 'Profile2Genn stage achieved the lowest RMSE values, highlighting the effectiveness of the proposed approach. The NFlow model also started with high RMSE values in the 'Traditional' stage, which were further improved in the 'PreProcess' stage. Compared to both stages, the ProffileGenn stage reduced RMSE, achieving performance similar to the baseline. The same occurred with the CTGAN model. For the TVAE model, the 'Traditional' stage began with high RMSE, which increased in the 'PreProcess' stage. However, the 'Profile2een' stage reduced RMSE, bringing the performance closer to the 'Traditional' stage.
Profile2Genly, Figure 27 shows that the Diabetes dataset, the Bayesian Network model, followed the same pattern observed previously, with high RMSE values in the 'Traditional' stage and improvements in the 'PreProcess' stage. The 'Profile2Gen' stage increased RMSE values, leaving the best performance for the 'PreProcess' stage, similar to the "Traditional" performance. The NFlow model repeated the behavior of the previous model. For the TVAE model, as in the other datasets, RMSE values started high in the 'Traditional' stage. The 'PreProcess' stage resulted in improvements, and the 'ProfileGen'' stage further reduced RMSE, leading to the best performance, surpassing the baseline.
O.2. Proportions
We also analyzed the impact of synthetic data proportions across different datasets and generative models, and the behavior of these models when varying the proportions and random seeds. Figures 27, 23, 24, 25, 26provide valuable insights into the optimal proportions of synthetic data to add.
Adding 22.7% (0.227) of synthetic data generally does not cause drastic changes in performance, remaining close to the baseline of real data. When 68.2% (0.682) of synthetic data added, it tends to be a critical point where many models show more significant deviation from the baseline, indicating that increasing this quantity can lead to more pronounced degradation in performance. The addition of 90.9% (0.909) synthetic data typically results in lower performance, suggesting that a high proportion of synthetic data may overload the model with non-generalizable information, especially in the 'Traditional' and 'PreProcess' stages. Adding 45.4 45.4% synthetic data, in many cases, maintains model performance relatively stable, with no significant drops compared to the real data baseline. However, this quantity may introduce variations depending on the model and processing stage. Models that are presented to be robust, such as Bayesian Network and CTGAN, tend to handle this quantity well without increasing RMSE.
Compared to 22.7%, adding 45.4% synthetic data may lead to a slight degradation in performance in some models and datasets, but it is generally better than adding 68.2% 90.9%. many cases, 45.4% appears to be balance point where the benefits of adding synthetic data can still be realized without overwhelming the models with information that could lead to inadequate generalization. The impact of 45,4% synthetic data is more controlled when the processing stage is "Profile2Gen," indicating that data from this stage helps mitigate any potential performance deterioration. This quantity
32

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
- 323
333
333
Parkinson
565
555
535
666
666
666
000
uma
Amount 0.227
Amount 0.454
Amount 0.682
Amount 0.909
3.0
2.5
Bayesian Network
2.0
Soore
1.5
0.5
Real
Traditonel
PreProcdd
Proficcten
Rral
Traditionll
PreProceno
Prefke2te
Real
Traditenar
PrePodess
ProfileDten
Real
Traditooal
Premocona
Profle
3.5
Spore
2.5
1.5
0.5
Resi
Traditional
PrePocces
Profle20on
Redi
Traditioall
PreProcaso
PrefecGGn
Real
Traditinal
PriPopees
Proficctian
Real
Traditional
PrePooooa
Proflkction
IVA6
2.0
1.5
Rindl
Trasffennl
PreProone
Profitztien
Real
Traditonal
ProPomol
Proficttion
Real
Trasmrons
ReProren
Profieziin
Real
Traditional
PrePesuee
Proficciom
2.5
CTGAN
1.0
Real
Tradtionl
PrePissenn
Poffezion
Real
Traditinnal
PeePootal
Proflczten
Real
Tratitonal
PaProetest
Prof
Reil
Traitionll
Profiectton
Figure 24. Augmentation performance across random seeds.
33

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Random Seed
Random Seed
Random
333
- 333
333
Plasma
566
555
535
666
535
666
888
888
Amount 0.227
Amount = 0.454
Amount 0.682
Amount 0.909
4.75
Bayasian Network
4,50
4.25
RMSE
4.00
3.75
3,90
3.25
Real
Traditooal
PrePractess
Profia20en
Real
Traditannl
PreProcesd
Prefiketen
Real
Traditional
Preprccess
ProlikaGien
Real
Tradittoral
PreProcesa
Profile2een
4.6
4.4
RMSE
4.2
NFlow
4.0
3.4
Real
Traditonal
PrePreessa
ProfieZGen
Real
Traditional
ProProcaas
Profie23en
Real
Traditional
PreProcess
ProfieeGen
Real
Traditional
PreProcesse
Profike0sm
4.8
4.6
4.2
TVAE
4.0
3.8
3.6
Real
Tredittoral
PreProseti
Profiez3en
Real
Traditiona
PreProctes
Profie200m
Traditional
PreProcets
ProfiectDen
Real
Traditional
PrePocasaa
Preflict0in
5.2
5.0
4.8
CTGAN
4.4
4.2
Real
Traditonnal
PeProsss
ProfeeZGen
Real
Traditional
PrePooce
Profecttin
Real
Traditonal
PrePrecces
Prefie20en
Real
Traditional
PrePoceedd
ProfiezZeen
Figure 25, Augmentation performance random seeds.
34

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Random Seed
233
355
333
Fat
533
666
888
888
888
Amount 0.227
Amount 0.454
Amount 0.682
Amount 0.909
t
Bayesian Network
RMSE
Real
Traditonal
PreProusst
Profile2iin
Real
Treditional
PreProcesd
Profik2ismn
Real
Traditonal
PreProcsan
Profile20en
Redl
Traaitonal
PrProcatts
ProfikeGeer
NFow
RMISE
Roall
Tradiional
PrePocasss
ProftetGian
Real
Traditanal
PrePecess
ProfiezSann
Real
Traditional
PreProoesd
Profie2Dam
Real
Traditioal
PrePovesi
ProflezGee
RMSE
TVAE
Real
Traditional
PreProcest
ProfitezCmm
Real
Traditional
ProProceuu
Profie25een
Real
Traditonal
PrePossao
Profilet6inn
Real
Treditional
ProProcess
Profikziiin
CTGAN
RMSE
Real
Traditioaal
PrePrecesa
Profiez0mm
Real
Traditioaal
PreProcesso
Profic22een
Tradiional
Prerocess
PreflleZeen
Real
Traditonal
PreProcano
Profie2Gee
Figure 26. Augmentation perfgrmance across random seeds.

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Random Seed
- 333
333
555
555
Diabetes
666
060
868
Amount 0.227
Amount 0.454
Amount 0.682
Amount
10.5
Bayesian Network
10.0
RMSE
9.5
9.0
8.5
8.0
7.5
Real
Traditonal
PreProcess
ProfileGen
Real
Traditronal
PrePracess
Profite2Gen
Real
Traditional
PreProcisss
Prafile2Gen
Real
Traditiona
PreProcess
ProfileZGen
12
11
NFlow
10
RMSE
Real
Traditional
PreProcasoo
ProfileZGen
Roal
Traditional
PreProcass
PrafileZGen
Real
Traditional
PreProcass
Proffle
Real
Traditional
PrePracess
ProfileGGe
10.5
10.0
RMSE
9.5
TVAE
9.0
8.0
7.5
Real
Traditional
PreProceut
ProfileZGen
Real
Traditioaa
PreProcess
ProffleGeen
Real
Traditional
PrePiocess
ProfilezGen
Real
Traditional
PreProcess
ProfleeGGen
Figure 27. Augmentation performance across random seeds.
may show more variations depending on the model's sensitivity to the new information introduced the' Traditional' and' PreProcess' stages. Figure 15 gives more insights.
36

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
0.3. Evaluation Models
Figures 31, 30, 29 and 28 demonstrate that models respond differently to synthetic data. For example, while LightGBM Large is often robust across various contexts, it exhibited moderate error and high variability in these experiments. This suggests that LightGBM Large may be less suited to certain types of synthetic data may require additional tuning for optimal performance.
The NeuralNet Fast model, particularly in the Plasma dataset, reached the highest RMSE values for some generative models, indicating that neural networks can struggle with synthetic data under certain conditions. This could be due to issues like overfitting or a high sensitivity to the quality of the generated data.
Similarly, the K-Neighbors model showed higher error rates in the Fat dataset, highlighting the potential limitations of instance-based learning algorithms when dealing with synthetic datasets. These datasets might lack the granularity necessary for effective nearest-neighbor searches, leading to suboptimal performance.
P. Statistical Significance
P.I. TSTR
The Wilcoxon signed-rank tests revealed significant differences between the Real data and the synthetic data generated by all models across most datasets (p < 0.05). This was expected, indicating that the synthetic data, while attempting to mimic the real data, does not perfectly replicate its distribution. However, the comparisons between the intermediate and final versions of the synthetic data (Preprocess and Profile2Gen) provided valuable insights. For instance, in the plasma dataset with the TVAE model, all comparisons between synthetic, PreProcess, and ProfileGeen were non-significant (p > 0.05), suggesting that the model generated statistically similar data across these stages. This contrasts with the urinary dataset, where the Bayesian Network produced synthetic data that was significantly different from the real data in all comparisons. These findings highlight the varying performance of different generative models and the importance evaluating the impact of refinement steps the synthetic data generation process. Further analysis, including effect size calculations and focusing on the specific metric used for comparison, will provide more complete understanding of these results.
Dataset: urinary, Model: bayesian_ network
Wilcoxon Results: Real vs Traditional: p-value 0.0029 (Significant)
Real vs PreProcess: p-value 0.0029 (Significant)
Real vs Profile2Gen: p-value = 0.0059 (Significant)
-Traditional vs PreProcess: p-value = -..0029 (Significant)
-Traditional vs Profile2Gen: p-value 0.0029 (Significant)
PreProcess vs Profile2Gen; p-value 0.0029 (Significant)
Dataset: urinary, Model: NFlow
Wilcoxon Results: Real vs Traditional: p-value 0.0029 (Significant)
Real vs PreProcess: p-value = 0.0029 (Significant)
Real vs Profile2Gen: p-value = 0.0029 (Significant)
Traditional vs PreProcess: p-value = 0.2051 (Not Significant
Traditional vs Profile2Gen: p-value 0.0059 (Significant)
PreProcess vs Profile2Gen: p-value. 0.0410 (Significant)
Dataset: urinary, Model: tvae
Wilcoxon Results: Real Traditional: p-value 0.0029 (Significant)
Real vs PreProcess: p-value = 0.0029 (Significant)
Real vs Profile2Gen: p-value = 0.0029 (Significant)
-Traditional vs PreProcess: p-vauue = 0.0293 (Significant)
Traditional vs Profile2Gen: p-value 0.0967 (Not Significant)
37

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
uniwyy
Figure 28, Performance of Augmeniiiio Methods at the Evaluator Level.

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Cotaset: diabetes
Figure 29, Performance of Augmentation Methods at the Evaluator Level.
39

---

Breaking the Barrier of Hard Samples: A Data- Data-Centric Approach to Synthetic Data for Medical Tasks
Detuset: fat
Figure 30. Performance of Augmentation Methods at the Evaluator Level.
40

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Ditasct: plaama
Figure 31. Performance of Augmentation Methods at the Evaluator Level.
41

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
PreProcess Profile2Gen: p-value 0.0146 (Significant)
Dataset: urinary, Model: ctgan Wilcoxon Results: Real vs Traditional: p-value = 0.00029 (Significant)
Real vs PreProcess: p-value 0.0059 (Significant)
Real vs Profile2Gen: p-value 0.0029 (Significant)
TTraditional vs PreProcess: p-value = 0.0029 (Significant)
Trraditional vs Profile2Gen: p-value = 0.0410 (Significant)
PreProcess vs Profile2Gen: p-value = 1.0000 (Not Significant)
Dataset: Parkinson, Model: moyesian.networt Wilcoxon Results: Real vs PreProcess: p-value = 0.0015 (Significant)
Real vs Profile2Gen; p-value = 0.0015 (Significant)
PreProcess vs Profile2Gen: p-value 0.0015 (Significant)
Dataset: Parkinson, Model: NFlow
Wilcoxon Results: - Real vs Traditional: p-value = 0.0029 (Significant)
Real vs PreProcess: p-value 0.0029 (Significant)
Real vs Profile2Gen: p-value 0.0029 (Significant)
-Traditional vs PreProcess: p-value 0.2051 (Not Significant
Traditional vs Profile2Gen: p-value = 0.0029 (Significant)
PreProcess vs Profile2Gen: p-value = 0.0029 (Significant)
Dataset: Parkinson, Model: tvae
Wilcoxon Results: Real vs Traditional: p-value 0.0029 (Significant)
Real vs PreProcess: p-value 0.0029 (Significant)
Real vs Profile2Gen: p-value = 0.0029 (Significant)
Traditional vs PreProcess: p-value 0.0029 (Significant)
-Traditional vs Profile2Gen: p-value 0.0029 (Significant)
PreProcess vs Profile2Gen: p-value = 0.6592 (Not Significant)
Dataset: Parkinson, Model: ctgan
Wilcoxon Results: Real vs Traditional: p-value 0.0029 (Significant)
Real vs PreProcess: p-value 0.0029 (Significant)
Real vs Profile2Gen: p-value = 0.0029 (Significant)
Traditional vs PreProcess: p-value 0.0029 (Significant)
Traditional vs Profile2Gen: p-value 0.0029 (Significant)
PreProcess vs Profile2Gen: p-value 0.2549 (Not Significant)
Dataset: cholesterol, Model: bayesian network
Wilcoxon Results: Real vs Traditional: p-value 0.0029 (Significant)
Real vs PreProcess: p-value 0.0029 (Significant)
Real vs Profile2Gen: p-value = 1.0000 (Not Significant)
Traditionnl vs PreProcess: p-value = 1.0000 (Not Significant
.Traditional vs Profile2Gen: p-value = 0.0029 (Significant)
PreProcess vs Profile2Gen: p-value 0.0029 (Significant)
42

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Dataset: cholesterol, Model: NElow
Wilcoxon Results: Real Traditional: p-value 0.0029 (Significant)
Real vs PreProcess: p-value 0.0059 (Significant)
Real vs Profile2Gen: p-value = 0.0029 (Significant)
TTraditional vs PreProcess: p-value = 0..029 (Significant)
Traditional vs Profile2Gen: p-value 0.6592 (Not Significant)
PreProcess vs Profile2Gen: p-value 0.0205 (Significant)
Dataset: cholesterol, Model: tvac
Wilcoxon Results: Real vs Traditional: p-value 0.0029 (Significant)
Real vs PreProcess: p-value = 0.0029 (Significant)
Real vs Profile2Gen; p-value = 0.0029 (Significant)
Traditional vs PreProcess: p-value 0.0029 (Significant)
Traditional Profile2Gen: p-value 0.4629 (Not Significant)
PreProcess vs Profile2Gen: p-value = 0.0029 (Significant)
Dataset: cholesterol, Model: etgan
Wilcoxon Results: Real vs Traditional: p-value = 0.0029 (Significant)
Real vs PreProcess: p-value 0.0029 (Significant)
Real vs Profile2Gen: p-value 0.0029 (Significant)
Trraditional PreProcess: p-value 0.3135 (Not Significant)
-Traditional vs Profile2Gen: p-value = 0.0410 (Significant)
PreProcess vs Profile2Gen: p-value = 0.4629 (Not Significant)
Dataset: diabetes, Model: bayesian.networt network
Wilcoxon Results: Real vs Traditional: p-value 0.0557 (Not Significant
Real vs PreProcess: p-value 0.0088 (Significant)
Real vs Profile2Gen: p-value 0.0029 (Significant)
TTraditional vs PreProcess: p-value = 0.0029 (Significant)
Traditional vs Profile2Gen: p-value = 1.0000 (Not Significant)
PreProcess vs Profile2Gen: p-value 0.0029 (Significant)
Dataset: diabetes, Model: NFlow Wilcoxon Results: Real Traditional; p-value 0.0029 (Significant)
Real vs PreProcess: p-value 0.2051 (Not Significant
Real vs Profile2Gen: p-value = 0.5537 (Not Significant)
-Traditional vs PreProcess: p-value = 0.0029 (Significant)
Trraditional vs Profile2Gen: p-value 0.0029 (Significant)
PreProcess Profile2Gen: p-value 1.0000 (Not Significant)
Dataset: diabetes, Model: tvae Wilcoxon Results: Real vs Traditional: p-value = 0.0205 (Significant)
Real vs PreProcess: p-value 0.0029 (Significant)
Real vs Profile2Gen: p-value = 0.0029 (Significant)
Traditional vs PreProcess: p-value 0.0029 (Significant)
43

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Traditional Profile2Gen: p-value 0.0029 (Significant)
PreProcess vs Profile2Gen: p-value = 0.0059 (Significant)
Dataset: fat, Model: bayesian network
Wilcoxon Results: Real Traditional: p-value 0..0029 (Significant)
Real vs PreProcess: p-value 0.0029 (Significant)
Real vs Profile2Gen: p-value = 0.0029 (Significant)
-Traditional vs PreProcess: p-value = 0.0029 (Significant)
Traditional vs Profile2Gen: p-value 0.0029 (Significant)
PreProcess vs Profile2Gen: p-value 1.0000 (Not Significant)
Dataset: fat, Model: NElow
Wilcoxon Results: Real vs Traditional: p-value = 0.0029 (Significant)
Real vs PreProcess: p-value 0.0029 (Significant)
Real vs Profile2Gen: p-value 0.0029 (Significant)
-Traditional vs PreProcess: p-value = 0.0410 (Significant)
Traditional vs Profile2Gen: p-value = 0.0029 (Significant)
PreProcess vs Profile2Gen: p-value = 0.0029 (Significant)
Dataset: fat, Model: tvae
Wilcoxon Results: Real vs Traditional: p-value = 0.0029 (Significant)
Real vs PreProcess: p-value 0.0029 (Significant)
Real vs Profile2Gen: p-value = 0.0029 (Significant)
Traditional vs PreProcess: p-value 0.0146 (Significant)
-Traditional vs Profile2Gen: p-value = 0.0029 (Significant)
PreProcess vs Profile2Gen: p-value = 0.0146 (Significant)
Dataset: fat, Model: etgan
Wilcoxon Results: Real Traditional: p-value 0.0029 (Significant)
Real vs PreProcess: p-value 0.0029 (Significant)
Real vs Profile2Gen; p-value = 0.0029 (Significant)
Traditional vs PreProcess: p-value 0.0293 (Significant)
Traditional vs Profile2Gen: p-value 0.6592 (Not Significant)
PreProcess vs Profile2Gen: p-value 0.0029 (Significant)
Dataset: plasma, Model: NFlow
Wilcoxon Results: Real vs Traditional: p-value 0.0029 (Significant)
Real vs PreProcess: p-value 0.0059 (Significant)
Real vs Profile2Gen: p-value = 0.0029 (Significant)
-Traditional vs PreProcess: p-value = 0.0029 (Significant)
Traditional vs Profile2Gen: p-value 1.0000 (Not Significant)
PrePPrcess vs Profile2Gen: p-value 0.0029 (Significant)
Dataset: plasma, Model: tvae
Wilcoxon Results: Real vs Traditional: p-value 0.0029 (Significant)
Real vs PreProcess: p-value 0.0029 (Significant)
44

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Real vs Profile2Gen: p-value 0.0029 (Significant)
Traditional vs PreProcess: p-value 1.0000 (Not Significant
Traditional vs Profile2Gen p-value = 1.0000 (Not Significant)
PreProcess vs Profile2Gen: p-value = 1.0000 (Not Significant)
Dataset: plasma, Model: etgan Wilcoxon Results:
Real vs Traditional: p-value 0.0029 (Significant)
Real vs PreProcess: p-value 0.0029 (Significant)
Reea vs Profile2Gen: p-value = 0.0410 (Significant)
-Traditional vs PreProcess: p-value = 0.2259 (Not Significant
Traditional vs Profile2Gen: p-value 0.0029 (Significant)
PreProcess vs Profile2Gen: p-value 1.0000 (Not Significant)
P.2. Augmentation
These results present post hoc tests (Wilcoxon tests) ) following Kruskal- Wallis tests, comparing 'Real' data to 'Traditional', 'PreProcess', and 'Profile2Gen' synthetic data generation methods across different datasets, models, and proportions. Our analysis of augmentation tasks revealed that the 'Profile2Gen' method effectively generated augmented data that closely resembled the 'Real' data, particularly at higher proportions. For instance, in the urinary dataset with the Bayesian Network, the 'Real vs. Profile2Gen' comparison was non-significant at proportion of 0.682, suggesting statistical similarity. This indicates that PProfileGee'' augments data in way that preserves the real data's underlying structure better than 'Traditional' or 'PreProcess' methods.
Dataset: Urinary, Model: Bayesian network, Proportion: 0.227 Post hoc Results: Real vs Traditional:p-valee value 0.0000 (Significant) Real vs PreProcess:p- 0.0000 (Significant) Real Profile2Gem:p-valuee 0.0379 (Significant) Traditional vs PreProcess:p-value 0.0000 (Significant) Traditional vs Proflle2Gin:p-vaauee 0.0000 (Significant) PreProcess vs Profile2Gen:p-value = 0.3006 (Not Significant)
Dataset: Urinary, Model: Beteok,, Proportion: 0.454 Post hoc Results: Real vs Tradiueonallp-value
0.0000 (Significant) Real vs PreProcess:p-value = 0.0000 (Significant) Real vs Profile2Gem:--value = 0,0010 (Significant)
Traditional vs PreProcess.p-value 0.0000 (Significant) Traditional vs ProflleGGenp-valuee = 0.0000 (Significant).
PreProcess vs Profile2Geenp-value = 0.0007 (Significant)
Dataset: Urinary, Model: Bayesian_network, Proportion: 0.682 Post hoc Results: Real vs Traditional:p-value-
0.0000 (Significant)- Real vs PreProcesspp-valee 0.0000 (Significant) Real ProflleGGen:p-vauue 1.0000 (Not Significant)- Traditional vs PreProcesssp-value = 0.0000 (Significant) Traditional vs Profile2Gen.p-vauue 0.0000
(Significant)- PreProcess vs Profile2Gen:p-value -0.0000 (Significant)
Dataset: Urinary, Model: Bayesian_network network, Proportion: 0.909 Post hoc Results: Real vs Traditionalp--valee =
0.0000 (Significant) Real vs PreProcess:p 0.0000 (Significant) Real vs Profile2Gen:p-value = 0.0004 (Significant) Traditional vs PreProcess.p-valuee 0.0000 (Significant)- Traditional vs Profile2Gen:p-valee 0.0000 (Significant) PreProcess vs Profile2Gen:--vauue 0.0000 (Significant)
Dataset: Urinary, Model: NFlow, Proportion: 0.227 Post hoc Results: Real vs Traditional:p-value 0.0000
(Significant)- Real vs PreProcess---xalue = 0.0000 (Significant)- Real vs Profile2Gemmp-value 1.0000 (Not Significant). Traditional vs PreProcess.p-value = 0. 1255 (Not Significant)- Traditional vs Profile2Gen:p-vllue 0.0000 (Significant)-
PreProcess vs Profile2Gen:--valuee = 0.0000 (Significant)
Dataset: Urinary, Model: NFlow, Proportion: 0.454 Post hoc Results: Real vs Praditionallp-value = 0.0000
(Significant)- Real vs PreProcess.-vvalee 0.0000 (Significant). Real Profilee2Grenp-valuee 1.0000 (Not Significant).
45

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Traditional vs PreProcessp--vluee 0.3649 (Not Significant) Traditional vs Profile22Gen:p-valuee 0.0000 (Significant)) PreProcess vs Proflle2Gen:--vauue 0.0000 (Significant)
Dataset: Urinary, Model: NFlow, Proportion: 0.682 Post hoc Results: Real vs Traditi = 0.0000
(Significant)- Real vs PreProcessp--value = 0.0000 (Significant)- - Real vs Profile2Gen:--valuee 1.0000 (Not Significant)
Traditional vs PreProcessp--value 0.0243 (Significant) Traditional vs Profile2Gin:--valuee 0.0000 (Significant).
PreProcess vs Profile2Gienp-valuee 0.0000 (Significant)
Dataset: Urinary, Model: NFlow, Proportion: 0.909 Post hoc Results: Real vs Traditional:--valuee 0.0000
(Significant)- Real vs PreProcess:p-vvluee 0.0000 (Significant)- - Real vs Profile2Gen:p-valuee 1.0000 (Not Significant) Traditional vs PreProcess:p--value = 1.0000 (Not Significant) Traditional vs Profile2Gienp--vatuee = 0.0000 (Significant)- PreProcess vs Profile2Gen:--value = 0.0000 (Significant)
Dataset: Urinary, Model: Tvae, Proportion: 0.227 Post hoc Results: Real vs Traditionall--value 0.0000 (Significant) Real vs PreProcess.--value 0.0000 (Significant) Real vs Profile2Gen:p-value 0.0000 (Significant) Traditional vs PreProcess.p-value = 0.0000 (Significant)- Traditional vs Profile2GGen:--valuee 0.0000 (Significant) PreProcess vs
Profile2Gen:p-aluue = 0.0001 (Significant)
Dataset: Urinary, Model: Tvae, Proportion: 0.454 Post hoc Results: Real vs Traditional:p-value = 0.0000 (Significant) Real vs PreProcess-p-value 0.0000 (Significant)- Real vs Proflle2Gen.--value 0.0000 (Significant) Traditional vs PreProcess.--value = 0.0000 (Significant). Traditional vs Profile2GGenp-vauue 0.0000 (Significant) PreProcess vs.
Profile2Gen:p-valuuee = 0.0000 (Significant)
Dataset: Urinary, Model: Tvae, Proportion: 0.682 Post hoc Results: - Real vs Traditional:p-value = 0.0000 (Significant) Real vs PreProcoss:-vvalue = 0.0000 (Significant) Real vs Profile2GGen---alueee = 1.0000 (Not Significant) Traditional vs PreProcess:p-valus = 0.0000 (Significant) Traditional vs Profile2Gen:-vvlue 0.0000 (Significant) PreProcess vs
Profile2Gennp-valuee 0.0000 (Significant)
Dataset: Urinary, Model: Tvae, Proportion: 0.909 Post hoc Results: Real vs :raditional:--valuee 0.0000 (Significant) Real vs PreProcess:p-value 0.0000 (Significant)- Real vs Profile2Gee-p--auuee = 0.1797 (Not Significant) Traditional vs PreProcess:p-value = 0.0000 (Significant) Traditional vs Profile2Gen:p-vauue = 0.0000 (Significant) - PreProcess vs
Profile2Gennp-valuee 0.0000 (Significant)
Dataset: Urinary, Model: Ctgan, Proportion: 0.227 Post hoc Results: Real vs Traditional:p 0.0000 (Signif- icant) Real vs PreProcess---value 0.1797 (Not Significant) Real vs Profile2Gem:p-value = 1.0000 (Not Significant) Traditional vs PreProcesspp-value 0.0000 (Significant) Traditional vs ProflleGGen:p-aleee 0.0000 (Significant).
PreProcess vs Profile2Gee:p-valuee = 0.1485 (Not Significant)
Dataset: Urinary, Model: Ctgan, Proportion: 0.454 Post hoc Results: Real vs Traditionallp-value 0,0000 (Signif-
icant) Real vs PreProcessip-value 1.0000 (Not Significant) Real vs Profile2Gen:p-value 1.0000 (Not Significant)
Traditional vs PreProcess:p-vauue 0.0000 (Significant) Traditional vs Profile2Gen:p-valee 0.0000 (Significant).
PreProcess vs Profile2Geenp-value = 0.1947 (Not Significant)
Dataset: Urinary, Model: Ctgan, Proportion: 0.682 Post hoc Results: Real vs Traditionalp--value 0.0000 (Signif- icant) Real vs PreProcess.p-valee 0.2524 (Not Significant) Real vs Profile2Gen:p value = 1.0000 (Not Significant) Traditional vs PreProcesspp-vabe value 0.0000 (Significant) Traditional vs ProfileGeen:pvallee 0.0000 (Significant)
PreProcess vs ProfleeGennp-value 0.0000 (Significant)
Dataset: Urinary, Model: Clgan, Proportion: 0.909 Post hoc Results: Real vs Traditional:--value 0.0000 (Signif-
icant) Real vs PreProcess:p-value = 0.4604 (Not Significant) Real vs Profile2Gen::pvalll value = 1.0000 (Not Significant)
Traditional vs PreProcess:p-vatur value 0.0000 (Significant) Traditional vs Profile2Gen.p-vall 0.0000 (Significant).
PreProcess vs Profile2Gee:p-value = 0.0000 (Significant)
46

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Dataset:
Parkinson, Model: Bayesian_netwone Proportion: 0.227 Post hoc Results: Real vs -
1.0000 (Not Significant)- Real vs PreProcesssp-valuee 0.0001 (Significant) Real vs Profile2Genn--valuee 1.0000 (Not
Significant) Traditional vs PreProcess:p-value = 0.9776 (Not Significant) Traditional Profile2Gen.p-valuee = 0.0000
(Significant)- PreProcess vs Profile2Gen:p-vaauee 0..132 (Not Significant)
Dataset: Parkinson, Model: beyeeian_netwood Proportion: 0.454 Post hoc Results: Real vs Traditional:p-value =
0.1254 (Not Significant) Real vs PreProcess:p-value 0,0000 (Significant) Real vs ProfileGGen.-vvluee = 1.0000 (Not Significant) Traditional vs PreProcess:p-value = 0.2528 (Not Significant)- Traditional vs Profile2Gien:pvvauee = 0.0000
(Significant)- PreProcess Profile2Gen:--valuee 0.0468 (Significant)
Dataset: Parkinson, Model: Bayesian network, Proportion: 0.682 Post hoc Results: Real vs Praditional:p-vatuur
= 0.0239 (Significant)- Real vs PreProcess-p-value = 0.0000 (Significant)- Real vs Profile2Gen:--vauue = 1.0000 (Not Significant) Traditional vs PreProcess:p-value = 0.0570 (Not Significant) Traditional vs Profile2Gen:--value = 0.0000
(Significant)- PreProcess vs Profile2Gen.--valuee = 0.0445 (Significant)
Dataset: Parkinson, Model: Bayesian network, Proportion: 0.909 Post hoc Results: Real vs Traditionallp-vauee
0.0043 (Significant) Real vs PreProcess:p-value = 0.0000 (Significant) Real vs ProfileGGen:-vvluee = 1.0000 (Not
Significant) Traditional vs PreProcess = 0.0691 (Not Significant) Traditional vs Proflle2Gen:p-vvlllle = 0.0000
(Significant)- PreProcess Profile2Gen:p-value 0.0354 (Significant)
Dataset: Parkinson, Model: NFlow, Proportion: 0.227 Post hoc Results: Real vs Traditionall--value 0.0000
(Significant)- Real vs PreProcess:p-valuee 0.0000 (Significant) Real vs Profile2Gen:p-value 0.0000 (Significant)- Traditional vs PreProcess:--value = 1.0000 (Not Significant)- Traditional vs Profile2Gen:p-value 0.0000 (Significant)-
PreProcess vs Profile2Gienp--value = 0.0193 (Significant)
Dataset: Parkinson, Model: NFlow, Proportion: 0.454 Post hoc Results: Real vs Traditionalp--vluee 0.0000
(Significant)) - Real vs PreProcessp--value 0.0000 (Significant) Real vs ProflleGGen.-vvluee 0.0000 (Significant)
Traditional vs PreProcesssp-value 1.0000 (Not Significant)- Traditional Profile2Gen:p-valuee 0.0000 (Significant)
PreProcess vs Profile2Gem:--valuee 0.0760 (Not Significant)
Dataset: Parkinson, Model: NFlow, Proportion: 0.682 Post hoc Results: Real vs Traditional:--value 0.0000
(Significant)- Real vs PreProcess:p-value 0,0000 (Significant) Real vs ProfileGGen:-vvluee = 0.0000 (Significant)- Traditional vs PreProcess:p-vvauue = 0.8619 (Not Significant) Traditional vs Profilee2Genp--vauue = 0.0000 (Significant)-
PreProcess vs Profile2Gem.--vluee = 0.8758 (Not Significant)
Dataset: Parkinson, Model: NFlow, Proportion: 0.909 Post hoc Results: Real vs Traditional:p-valuu 0.0000
(Significant) Real vs PreProcessp-vvauee 0.0000 (Significant) Real vs Profile2Gen:p-vauee 0.0000 (Significant)) Traditional vs PreProcess:--value 1.0000 (Not Significant)- Traditional vs Profile2GGen:p-valuee 0.0000 (Significant)-
PreProcess vs Profile2Gem:--valuee = 0.0000 (Significant)
Dataset: Parkinson, Model: Tvae, Proportion: 0.227 Post hoc Results: Real vs Traditional:p-valus 0.0004
(Significant)- Real vs PreProcess:p-value = 0.0000 (Significant)- - Real vs Profile2Gen:p-value = 0.0192 (Significant). Traditional vs PreProcess:p-value = 0.3665 (Not Significant)- Traditional vs Profile2Gen:p-value = 0.0000 (Significant)
PreProcess vs Profile2Gen.--value = 0.0006 (Significant)
Dataset: Parkinson, Model: Tvae, Proportion: 0.454 Post hoc Results: Real vs Traditional:.-valle 0.0000
(Significant) Real vs PreProcess:p-value 0.0000 (Significant) Real vs Profile2Geenpp-vllue 0.0005 (Significant) Traditional vs PreProcess:-vvauee = 1.0000 (Not Significant)- Traditional vs Profile2Gen:--vluee 0.0000 (Significant)
PreProcess vs Profile2Gem:--vauue = 0.0005 (Significant)
Dataset: Parkinson, Model: Tvae, Proportion: 0.682 Post hoc Results: - Real vs Travitional.--valee = 0.0000
(Significant)- Real vs PreProcesspp-value 0.0000 (Significant) - Real vs Profile2Gen:--vluuee = 0.0000 (Significant).
Traditional vs PreProcess:p-value = 1.0000 (Not Significant) Traditional vs Profile2Gee:p-valuee = 0.0000 (Significant)
PreProcess vs Profile2Gen.p-value = 0.0010 (Significant)
47

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Dataset: Parkinson, Model: Tvae, Proportion: 0.909 Post hoc Results: Real Traditional:p-value 0.0000
(Significant) Real vs PreProcess:p-valuee 0.0000 (Significant) Real vs Profile2GGenp--valuee 0.0001 (Significant)) Traditional vs PreProcess---value = 1.0000 (Not Significant) Traditional vs Profile2GGen:--value 0.0000 (Significant)-
PreProcess vs Profile2Gen:--vaauee = 0.0000 (Significant)
Dataset: Parkinson, Model: Ctgan, Proportion: 0.227 Post hoc Results: Real vs p = 0.0000
(Significant) Real vs PreProcess:p-valuee 0.0000 (Significant) - Real vs Profile2Geen--valuuee = 0.6071 1 (Not Significant). Traditional vs PreProcess:p-valine 1.0000 (Not Significant) Traditional vs Profile2Gienp-vvluue 0.0000 (Significant)
PreProcess vs Profile2Gen.--vauee 0.0002 (Significant)
Dataset: Parkinson, Model: Ctgan, Proportion: 0.454 Post hoc Results: Real p 0.0000
(Significant)- Real vs PreProcess:p-vallee 0.0000 (Significant)- Real vs Profile2Gee:p-vvaueee = 0.0124 (Significant) Traditional vs PreProcess:p-value = 0.4101 (Not Significant) Traditional vs Proflle2Gen:p-vauuee -..0000 (Significant)
PreProcess vs Profile2Gienp--vluee = 0.0001 (Significant)
Dataset: Parkinson, Model: Ctgan, Proportion: 0.682 Post hoc Results: Real vs Traditionallp-vauee = 0.0000
(Significant)- - Real vs PreProcessp--value 0.0000 (Significant) - Real vs ProflleGGen:--value = 0.0009 (Significant)
Traditional vs PreProcess-p-value = 0.11 (Not Significant) Traditional vs Profile2Geni--value 0.0000 (Significant)-
PreProcess vs Profile2Gen.-vvluee 0.0001 (Significant)
Dataset: Parkinson, Model: Ctgan, Proportion: 0.909 Post hoc Results: Real vs Traditional:p 0.0000
(Significant) Real vs PreProcess:p-value = 0.0000 (Significant)- Real vs Profile2Gen.--vauue = 0.0007 (Significant)
Traaditional vs PreProcesssp-value = 0.0383 (Significant) Traditional vs Profile2Gienp-vvluue = 0.0000 (Significant).
PreProcess vs Profile2Gen.--vluue = 0.0000 (Significant)
Dataset: Cholesterol, Model: Bayesian network, Proportion: 0.909 Post hoc Results: Real vs :raditionall--value 0.0036 (Significant)
Dataset:
Cholesterol, Model: NFlow, Proportion: 0.909 Post hoc Results: Real vs Tradee 0.0004 (Significant)
Dataset:
Cholesterol, Model: Tvae, Proportion: 0.909 Post hoc Results: Real vs -radie 0,0009 (Significant)
Dataset:
Cholesterol, Model: Ctgan, Proportion: 0.682 Post hoc Results: Real vs Traditionall--vlue 0.0000 (Significant)
Dataset:
Cholesterol, Model: Ctgan, Proportion: 0.909 Post hoc Results: Real vs Traditionallp-allue 0.0000
(Significant)
Dataset: Diabetes, Model: peeesi,,,,etwor,, Proportion: 0.227 Post hoc Results: Real vs Traditionallp-valuee
= 0.0000 (Significant)- - Real vs PreProcess-p-value 1.0000 (Not Significant) Real vs Profile2Gempp-value 0.0000
(Significant) Traditional vs PreProcess: = 0.0002 (Significant)- Traditional vs Profile2Genn--vluee = 1.0000 (Not
Significant)- PreProcess vs Profile2Gen:pvalll 0.0000 (Significant)
Dataset: Diabetes, Model: govesian_networke Proportion: 0.454 Post hoc Results: Real vs Traditional: value
0.4817 (Not Significant) Real vs PreProcess.p-valuee 0.0000 (Significant) Real vs Profile2Gen:--vllee 0.0343 (Significant) Traditional PreProcesssp-valuee = 0.0000 (Significant) Traditional Profile2GGe::-valuee 1.0000 (Not
Significant)- PreProcess Profile2Gen:p-value 0.0000 (Significant)
Dataset: Diabetes, Model: Bayesian network, Proportion: 0.682 Post hoc Results: Real vs Traditional:p-valuee
= 0.0000 (Significant)- - Real vs PreProcessp--value = 0.8360 (Not Significant) - Real vs Profile2Gen:p-vatee 0.0000 (Significant) Traditional vs PreProcess:p-valtu = 0.0000 (Significant) Traditional vs Profile2Gen:p-vauue = 1.0000 (Not
Significant) PreProcess vs Profile2Gen:p-vluee 0.0000 (Significant)
48

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Dataset: Diabetes, Model: _eteeo,,, Proportion: 0.909 Post hoc Results: Real vs -raditionallp-value
1.00000 (Not Significant) Real vs PreProcess-p-valuee 0.0000 (Significant) Real vs Profile2Geenp-vallee 0.0000
(Significant)- Traditional vs PreProcess.p-value = 0.0000 (Significant) Traditional vs Profile2Gen:--valee 0.0000
(Significant)- PreProcess vs Profile2Gen:p-valueee -.0000 (Significant)
Dataset: Diabetes, Model: NFlow, Proportion: 0.227 Post hoc Results: - Real vs Traditionalp--aalee = 0.0007
(Significant)- Real vs PreProcesspp-value 0.0001 (Significant)- Real vs Profile2Gen.--vluee = 0.0006 (Significant)
Traditional vs PreProcess:p-value 0.0000 (Significant) Traditional vs Profile2Gienp-vvauee = 1.0000 (Not Significant) PreProcess vs Profile2Genn--vauee 0.0000 (Significant)
Dataset: Diabetes, Model: NFlow, Proportion: 0.454 Post hoc Results: Real Traditional:p-value 0.0000
(Significant)- Real vs PreProcess---valuee 0.0000 (Significant)- Real Profile2Gennp-valuee 0.2337 (Not Significant)
Traditional vs PreProcess:p-value = 1.0000 (Not Significant)- Traditional vs Proflle2Gen:p-valuee = =0.0197 (Significant)-
PreProcess vs Profile2Gen:--valuee = 0.0000 (Significant)
Dataset: Diabetes, Model: NFlow, Proportion: 0.682 Post hoc Results: - Real vs T = 0.0000
(Significant) Real vs PreProcess:p-value = 0.5749 (Not Significant) Real vs ProfileGGenp-vvauue 1.0000 (Not Significant) Traditional vs PreProcessp-vvlue 0.4104 (Not Significant) Traditional vs Profle2GGen:-vvauee 0.0094 (Significant)-
PreProcess vs Profile2Gem:--vlue 1.0000 (Not Significant)
Dataset: Diabetes, Model: NElow, Proportion: 0.909 Post hoc Results: Real Traditional:p-value 0.0000
(Significant)- Real vs PreProcess:p-vvluue = 0.0010 (Significant)- Real vs Profile2Gennp-valuee 1.0000 (Not Significant)- Traditional vs PreProcess.--vauue = 1.0000 (Not Significant)- Traditional vs Profile2Gen:p-vauuee = 0.0000 (Significant)-
PreProcess vs Profile2Gien:-vvaluee = 0.0003 (Significant)
Dataset: Diabetes, Model: Tvae, Proportion: 0.454 Post hoc Results: Real vs Traditional:p-valuee 0.0000 (Significant) Real vs PreProcess:p-valuu 0.0000 (Significant)- Real vs Profile22Genp-vaauuee 0.0000 (Significant) Traditional vs PreProcess:p-value 0.0132 (Significant) Traditional Profile2Gem.p-value 0.0000 (Significant) PreProcess vs Profile2Gen:--valuee 0.0000 (Significant)
Dataset: Diabetes, Model: Tvae, Proportion: 0.682 Post hoc Results: Real vs Traditional:p-valee 0.0002 (Significant) Real vs PreProcess:p-vvluee 0. 1896 (Not Significant)- Real vs ProfileGGe..--vluee = 0.0000 (Significant) Traditional vs PreProcess:p-value = 0.0000 (Significant). Traditional vs Profile2GGn:p-value = 0.0000 (Significant) - PreProcess vs
Profle2GGe:p-vauue = 0.0021 (Significant)
Dataset: Diabetes, Model: Tvae, Proportion: 0.909 Post hoc Results: Real vs Traditional:p-value 0.0000 (Significant) Real vs PreProcess:ppvaluus 0.0000 (Significant) Real vs Profile2Gen:pvvatee 0.0000 (Significant) Traditional vs PreProcess:p-value 0.2398 (Not Significant) Traditional vs Profile2Gen:p-value 0.0000 (Significant) PreProcess vs
Profile2Gem:--vluee 0.0000 (Significant)
Dataset: Fat, Model: Bayesian network, Proportion: 0.227 Post hoc Results: Real vs Traditional:--value = 0.0000
(Significant)- Real vs PreProcess:p-valu = 0.0000 (Significant)- Real vs Profile2Gen:--value = 0.0000 (Significant)
Traditional vs PreProcess.p-valle = 0.0000 (Significant) Traditional vs Profile2Ge::-vvluee = 0.0000 (Significant).
PreProcess vs Profile2Gee:p-valuee = 0.7107 (Not Significant)
Dataset: Fat, Model: Bayeskan,network,
network, Proportion: 0.454 Post hoc Results: Real vs Traditional:p-vall value = 0.0000
(Significant) Real PreProcess.p-value 0.0000 (Significant)- Real vs Profile2Gen:p-valuee 0.0000 (Significant)
Traditional vs PreProcess:p-valuc 0.0000 (Significant) Traditional vs Profile2Gem:--vluee 0.0000 (Significant)
PreProcess vs Profile2Gen:p-vluue = 1.0000 (Not Significant)
Dataset: Fat, Model: Bayesian network, Proportion: 0.682 Post hoc Results: - Real vs Traditionaa:--valuee = 0.0000
(Significant) Real vs PreProcess:p-valun = 0.0000 (Significant)- - Real vs Profile2Gem:--vauue = 0.0000 (Significant)
Traditional vs TreProcess.p-valuu value 0.0000 (Significant) Traditional vs Profile2Gee.-vvluue 0.0000 (Significant)
PreProcess vs Profile2Gienp-valuee = 1.0000 (Not Significant)
49

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Dataset: Fat, Model: Gayesian_networks Proportion: 0.909 Post hoe Results: Real vs -raditional:p-value 0.0000
(Significant) Real PreProcess:p-valuu 0.0000 (Significant) Real vs Proflle2Gen:p-valuee 0.0000 (Significant)
Traditional vs PreProcess-p-valuee 0.0015 (Significant)- Traditional vs Profile2Geen--valuee 0.0000 (Significant)-
PreProcess vs ProfileGGen.-vvlueee = 0.0018 (Significant)
Dataset: Fat, Model: NFlow, Proportion: 0.227 Post hoc Results: - Real vs Traditionall--value = 0.0000 (Significant) Real vs PreProcess:p-valuee = 0.0000 (Significant) Real vs Profile2Gennp-valuee = 0.0243 (Significant) Traditional vs PreProcess:p-value = 0. 0.0000 (Significant) Traditional vs Profile2GGenp-value 0.0000 (Significant) PreProcess vs
Profile2Geni--value 0.0000 (Significant)
Dataset: Fat, Model: NFlow, Proportion: 0.454 Post hoc Results: Real :raditionalp--value 0.0000 (Significant) Real vs PreProcess:p-value 0.0000 (Significant)- Real Profile2Gen:p-vaueee = 1.0000 (Not Significant) Traditional vs PreProcess.p-value = 0.0002 (Significant). Traditional vs Profile2Gen:p-value 0.0000 (Significant) PreProcess vs
Profil22Gen--valuee = 0.0000 (Significant)
Dataset: Fat, Model: NFlow, Proportion: 0.682 Post hoc Results: Real vs Traditional:p-vauus = 0.0000 (Significant) Real vs PreProcess:p-value 0.0000 (Significant)- Real vs Profile2GGenp--vllue = 1.0000 (Not Significant) Traditional vs PreProcess:p-value = 0.0000 (Significant) Traditional vs Profile2GGenp-vaaue 0.0000 (Significant) PreProcess vs
Profile2Gen:p-value 0.0000 (Significant)
Dataset: Fat, Model: NFlow, Proportion: 0.909 Post hoc Results: Real Traditionall--value 0.0000 (Significant) Real vs PreProcess:--value = 0.0000 (Significant) Real vs Profile2Gen:p-valuee = 0.0015 (Significant) Traditional vs PreProcess.p-value = 0.0000 (Significant)- Traditional vs Profile2Gen.p-aluee 0.0000 (Significant) PreProcess vs
Profle22Genp-valuee = 0.0000 (Significant)
Dataset: Fat, Model: Tvae, Proportion: 0.227 Post hoc Results: Real vs Traditionall--vluee = 0.0000 (Significant) Real vs PreProcess.p-valuee 0.0000 (Significant) Real vs Profile2Geenp-valuee 0.0000 (Significant) Traditional vs PreProcess:p-value 0.0000 (Significant) Traditional Profile2Gen:p-value 0.0785 (Not Significant) PreProcess vs Profile2Gen---vauuee 0.0000 (Significant)
Dataset: Fat, Model: Tvac, Proportion: 0.454 Post hoc Results: - Real vs Traditional:--value = 0.0000 (Significant) Real vs PreProcess:p-value = 0.0000 (Significant)- Real vs Profile2Gee:p-valuee 0.0000 (Significant) Traditional vs PreProcess:p-valuue = 0.0000 (Significant) Traditional vs Profile2Gen:p-valuee = 021220 (Not Significant) PreProcess vs Profile2Gem:pvaluue = 0.0000 (Significant)
Dataset: Fat, Model: Tvae, Proportion: 0.682 Post hoc Results: Real vs Traditional: palvlue = 0.0000 (Significant) Real vs PreProcess.p-value 0.0000 (Significant) Real vs ProflleGGen.p-vluee 0.0000 (Significant) Traditional vs PreProcess:p-value 0.0000 (Significant). Traditional vs Profile2GGe:p-value 0.0011 (Significant). PreProcess vs
Profile2Gen:p-vlluee = 0.0000 (Significant)
Dataset: Fat, Model: Tvac, Proportion: 0.909 Post hoc Results: - Real vs Traditional:p-value = 0.0000 (Significant) Real vs PreProcess:p-value = 0.0000 (Significant)- Real vs Profile2Gen:--value 0.0000 (Significant) Traditional vs PreProcess:p-value = 0.0000 (Significant) Traditional vs Profile2Gen:--vauee = 1.0000 (Not Significant) PreProcess vs
Profile2Gemp-valuee 0.0000 (Significant)
Dataset: Fat, Model: Ctgan, Proportion: 0.227 Post hoc Results: Real vs Traditional:p = 0.0000 (Significant) Real vs PreProcess.p-value 0.0000 (Significant) Real vs Profile2Geenp-vaaueee 0.0000 (Significant) Traadiioonal vs PreProcess:p-value = L.0000 (Not Significant) Traditional vs Profile2Gen:--vluee 0.0007 (Significant) PreProcess vs
Profile2Geenp-valueee = 0.0000 (Significant)
Dataset: Fat, Model: Ctgan, Proportion: 0.454 Post hoc Results: - Real vs Traditionnal--vallee = 0.0000 (Significant) Real vs PreProcess:p-valuee = 0.0000 (Significant). Real vs Profile2Genp-valuue 0.0177 (Significant) Traditional vs PreProcess:p-value = 0.0634 (Not Significant) Traditional vs Profile2Gen:p-vauuee = 0.0000 (Significant) PreProcess vs
Profile2Gen:p-vluuee 0.0000 (Significant)
50

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
Dataset: Fat, Model: Ctgan, Proportion: 0.682 Post hoc Results: Real Traditionall--value 0.0000 (Significant)
Real vs PreProcess:p-value 0.0000 (Significant) Real Profile2Gienp-vaatue 0.0000 (Significant) Traditional vs PreProcess.p-value = 0.0049 (Significant)- Traditional vs Profile2Gen:--value = 0.0000 (Significant) PreProcess vs
Profile2Gen:-vaauue = 0.0000 (Significant)
Dataset: Fat, Model: Ctgan, Proportion: 0.909 Post hoc Results: - Real vs :raditeonall--value = 0.0000 (Significant) Real vs PreProcess:p-value = 0.0000 (Significant)- - Real vs Profile2GGen---vatue = 0.0000 (Significant) Traditional vs PreProces::p-vauus = 0.0197 (Significant) Traditional vs Profile2Gen.--vauue 0.0000 (Significant) PreProcess vs
Profile2Genp--aluee 0.0000 (Significant)
Dataset: Plasma, Model: Bayesian network, Proportion: 0.682 Post hoc Results: Real vs Traditionalp--vauee
1.0000 (Not Significant) Real vs PreProcess:p-value 0.0033 (Significant) Real vs Profile2Gen:p-valuee 0.0000 (Significant)) Traditional vs PreProcessop-value 0.1703 (Not Significant)- Traditional vs ProfileGGenp--vluee 0.0000 (Significant)- PreProcess vs Profile2GGen---vluue 0.0000 (Significant)
Dataset: Plasma, Model: Bayesian network, Proportion: 0.909 Post hoc Results: - Real vs Traditional: 0.1997
(Not Significant)- Real vs PreProcess:p-value = 0.0000 (Significant)- - Real vs Profile2Gen:p-valuee = 0.0000 (Significant)
Traditional vs PreProcess:ppvvllee value = 0.0000 (Significant)- - Traditional vs Proflle2Gen:--vauee = 0.0000 (Significant).
PreProcess vs Profile2Gee.--vauue = 0.0000 (Significant)
Dataset: Plasma, Model: NFlow, Proportion: 0.227 Post hoc Results: Real vs Traditional:p-value 1.0000 (Not
Significant)- Real vs PreProcesssp-valuee = 1.0000 (Not Significant)- Real vs ProflleGen:p-vvauee 0.0000 (Significant)- Traditional vs PreProcess:p-valuee 1.0000 (Not Significant) Traditional vs Profile2GGen:p-valuee 0.0000 (Significant)
PreProcess vs Profile2Gen:p-value = 0.0000 (Significant)
Dataset: Plasma, Model: NFlow, Proportion: 0.454 Post hoc Results: Real vs Traditional:p-vaalu 0. (Not
Sigmificant)- Real vs PreProcess:p-value = 1.0000 (Not Significant) Real vs Profile2Gen.--vauue = -.,0000 (Significant)-
Traditional vs PreProcess:p-value = 0. 1947 (Not Significant)- Traditional vs Profile2Gien:-vvauue = 0..000 (Significant)- -
PreProcess vs Profile2Gen:--valuee = 0.0000 (Significant)
Dataset: Plasma, Model: NFlow, Proportion: 0.909 Post hoc Results: Real vs Traditional:p-value 1.0000 (Not
Significant)- Real vs PreProcessi--valuee = 1.0000 (Not Significant) Real vs Profile2Gen:p-value 0..0000 (Significant).
Traditional vs PreProcessp--vllu- 1.0000 (Not Significant) Traditional vs Profile2Gen:p-value 0.0000 (Significant)-
PreProcess vs Profile2Gem.--vluue 0.0000 (Significant)
Q. Fairness
In this section, we present complementary figures to the main text related to the discussion of fairness.
R. Model-Specific Performance
In machine learning, there is no silver bullet for solving problems. This is especially true for generative models, whose diversity reflects different trade-offs and applications. Each model has unique characteristics that make it more suitable for specific scenarios, and our experiments highlight this variability. Our primary goal was to analyze whether, despite these differences, our approach remains effective. Variability is an inherent characteristic of generative models due to their distinct underlying mechanisms. CTGAN, for instance, relies on deep neural networks, which are highly sensitive to training conditions, hyperparameters, and data distribution. This sensitivity can lead to fluctuations in performance (Xu al., 2019b). In contrast, Bayesian Networks explicitly model dependencies between variables, making them less prone to such fluctuations (Heckerman, 2022). Our results confirm this pattern: lower RMSE values were observed in Bayesian Networks and CTGAN, particularly in TSTR, when trained with smaller synthetic datasets. This suggests that these models can generalize well to real data, as their generated distributions align closely with real distributions. However, the observed variation in performance across datasets indicates that factors such as dataset complexity, feature dependencies, and data impurity also play a role in model stability. Both models perform better with structured and cleaner data, as their ability to capture distributions improves in less noisy scenarios. Additionally, as detailed in Appendix D, we selected a
51

---

Breaking the Barrier of Hard Samples: A Data- Cata-Centric Approach to Synthetic Data for Medical Tasks
Paprosts
kEwyy Pratroom
untayy-ntow
entrry
134
Prepocel
1310
21000
1360
perddree
parirace hecroom
partrse Foheom
200
Figure 32, Histograms target values the various datasets, comparing the distribution of real data with synthetic data generated by different models ddpm, tabformer, and great) and techniques (Traditional, Preprocessing, Profile2Gen). This comparison highlights the impact of different models and techniques on the representation of minority classes (values in the tails of the distribution).
52

---

Breaking the Barrier of Hard Samples: A Data- Cala-Centric Approach to Synthetic Data for Medical Tasks
Ferd
Nao Prthocm
Tel. Tood
Tradhoed
piore: bayenles setver
Elomee beyelie stoott
Trak
pirmme Real
Tralined
r8m NeRoer
plime 1re Pre
auma Paal
3100 -1gar T15t334
pluma -ggan Prahooos
plama- -aa.--aa------- socar
Figure 33. Histograms of target values for the various datasets, comparing the distribution of real data with synthetic data generated by different models (ddpm, tabformer, and great) and techniques (Traditional, Preprocessing. Profile2Gen). This comparison highlights the impact of different models and techniques the representation of minority classes (values in the tails of the distribution).
53

---

Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks
medium-sized dataset for parameter tuning, considering dataset lengths and computational feasibility. We conducted an optimized parameter search using Optuna to minimize errors. However, this tuning not applied to every dataset due the large number of experiments and avoid overfitting. Therefore, the choice of hyperparameters may have also influenced model variability. Variability in performance is an expected outcome in machine learning models, where small parameter changes can lead to significant differences in results. Our study highlights both the most variable and the most stable models. Additionally, when evaluating model performance, we observed greater consistency in tree-based models such as Random Forest and Extra Trees, which split nodes based on impurity indices (James al., 2013). This characteristic may explain their robustness, particularly in the Profile2Gen experiments.
To corroborate our hypothesis that the nature of the model and dataset characteristics contribute variabilityy, conducted an additional experiment in which fine-tuning was performed on the same dataset used for training. This reduces external influences and allows us to analyze whether hyperparameter optimization is the main factor affecting performance. We randomly selected two datasets (Fat and Cholesterol) and evaluated CTGAN in two scenarios:
1. Fine-tuning performed on the same dataset used for training.
2. Fine-tuning performed on an intermediary dataset - the same one used in the main paper.
RMSE (Tuning on the same dataset)
Dataset
Approach
RMSE (Tuning on intermediary dataset)
2.801107
Fat
Profile2Gen
3.260037
1.227212
Cholesterol
Profie2Gen
1.234250
3.749975
Fat
Traditional
3.303178
1 1,032838
Cholesterol
Traditional
1.182572
4.167451
Fat
Preprocess
4.2000461
1951
Cholesterol
Preprocess
1.104390
Table shows that in Profile2Gen, variability was lower when the tuning was performed on the same dataset than using an intermediary dataset. However, for the other approaches, the variability increased, suggesting that although hyperparameter tuning has impact, it is not the primary factor driving performance fluctuations. Instead, our results indicate that the model's intrinsic nature and dataset complexity contribute more significantly to variability.
S. Dataset-variation analysis
In general, Profile2Gen outperforms traditional preprocessing methods. When analyzing the protocols separately, we observe that in the TSTR scenario, Profile2Gen achieves the best performance in approximately 80% of the cases. However, the datasets where preprocessing outperformed Profile2Gen the most were Cholesterol and Diabetes, which also happen to be the two smallest datasets. Conversely, in the Parkinson dataset-the largest one mne-Profile2Gen demonstrated significantly better performance than preprocessing, This suggests that Profile2Gen handles larger datasets more effectively than preprocessing in TSTR scenario. similar pattern was observed in the augmentation scenario.
54


Reviews:
[{"review_id": "py0dFIPKSy", "reviewer": "Reviewer_sXia", "summary": "This paper introduces Profile2Gen, a data-centric framework designed to enhance the generation and refinement of synthetic data for medical regression tasks. The key innovation lies in profiling and addressing hard-to-learn samples, which traditionally hinder model performance and generalization. The authors evaluate their approach across six medical datasets using seven state-of-the-art generative models and conduct experiments to validate its efficacy.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewers:\nWe want to thank you all for your very careful and thoughtful reviews. We were very encouraged by the numerous and significant strengths that you all identified in our study. Namely:\n* An innovative data-centric approach integrating data profiling and synthetic data generation targeting hard-to-learn samples in regression tasks. \n* Extensive empirical validation with approximately 18,000 rigorous experiments across 6 medical datasets, 7 SOTA generative models, and 12 predictive models.\n* We use a rigorous statistical approach in our comparisons. 4. Extending the widely recognized DataIQ framework from classification to regression tasks, enabling comprehensive profiling and data quality assessment in regression contexts.\n* Reduces model performance variability across random seeds and datasets, consistently outperforming traditional and baseline preprocessing methods.\n* Preserves minority distributions, crucial for accurately capturing rare and critical medical scenarios.\n* Clear potential for real-world deployment in data-scarce or privacy-sensitive medical applications.\n\nWe appreciate your thorough identification of potential steps to address the study’s weaknesses. Due to space constraints, we have grouped our responses by topic.\nThus, we hope to be given the chance to address those weaknesses in a revision. We eagerly await your answers.\n\nSincerely, --the authors\n\n* **Scalability Concerns e Computational Cost -- Reviwers sXia, BQrC (Q5), VmCi (Q3)**\n\nTo assess computational efficiency, we selected the largest dataset, Parkinson’s, which contains approximately 3,500 training samples. We applied the framework selection process (Cleanlab and DataIQ) along with profiling, using two thresholds and a label replacement ratio. These experiments required 3,510 MB of memory and took approximately 3 minutes. \n\nFor larger datasets, it is important to consider that memory usage will increase proportionally, as observed in the profiling stage. However, it should be noted that the process itself does not require GPU resources. Memory remains the main concern at this stage. For generating synthetic samples, particularly for Transformer and LLM-based models, we utilized an Nvidia RTX4090. \n \nWhen working with datasets larger than 3,500 samples, memory consumption could be calculated based on this scaling factor, considering the amount of data processed. While simulating this for larger datasets may be difficult, this scaling factor can provide a reasonable estimate. It is worth noting that finding large datasets, especially in healthcare, is challenging, and generating synthetic data is particularly relevant in scarcity scenarios where such large datasets are not readily available. \n \nTo ensure clarity and transparency, we could create a section in the supplementary materials to further detail the memory usage and computational considerations. In the main paper, we could then explicitly mention this section, where we discuss memory usage in more depth, offering readers more context on how the scaling might work for larger datasets. \n\nHard removal: Hard-to-learn samples (hard) can negatively impact model performance by either:\nIncreasing uncertainty in predictions. The model may predict correctly but with low confidence, leading to unreliable decision-making.\nReinforcing incorrect patterns. The model may misclassify these samples with high confidence, making it more prone to overfitting on noise rather than learning meaningful patterns.\n\n* **Hard removal -- Reviewer sXia** \nRemoving these samples is a well-established machine learning technique, often called dataset cleansing or data curation. Studies have shown that filtering out mislabeled or overly ambiguous samples can improve model generalization (https://arxiv.org/abs/1911.00068, https://arxiv.org/abs/2310.16981).\nFormally, let $ X = X_{\\text{easy}} \\cup X_{\\text{hard}} $ be the dataset, where $ X_{\\text{easy}} $ are well-confident samples, and $ X_{\\text{hard}} $ are ambiguous or mislabeled samples. The model’s expected loss can be decomposed as:\n\\begin{equation}\n\\mathbb{E}[\\mathcal{L}(X)] = \\mathbb{E}[\\mathcal{L}(X_{\\text{easy}})] + \\mathbb{E}[\\mathcal{L}(X_{\\text{hard}})].\n\\end{equation}\nSince $ X_{\\text{hard}} $ contributes disproportionately to loss without meaningful learning, its removal reduces noise and enhances generalization. Empirically, prior works on curriculum learning (https://ronan.collobert.com/pub/2009_curriculum_icml.pdf) } and label noise filtering (https://arxiv.org/pdf/1712.05055) support this approach, demonstrating that excluding ambiguous samples can lead to more robust models.\nTo avoid confusion, we will refine the introduction (where we commented about these samples) to emphasize that hard sample removal is not an arbitrary step but a widely used strategy for improving model robustness.\n\n\n\n**Please review the response provided to reviewers BQrC, fkDs, and VmCi, where we address your concern.**", "claims_and_evidence": "Generally well-supported. For instance, Profile2Gen reduces variability across random seeds and datasets. The authors provide statistical significance tests (Wilcoxon and Kruskal-Wallis) to validate this. This well-supports that ofile2Gen improves consistency and robustness in model performance.", "methods_and_evaluation_criteria": "The methods and evaluation criteria are ok. \n\nStrengths:\n1. Benchmarking on multiple datasets strengthens the validity of the claims.\n2. Rigorous statistical testing (Wilcoxon, Kruskal-Wallis) improves credibility.\n\nWeaknesses:\n3. Hyperparameter Sensitivity: The choice of hard sample thresholds is not deeply analyzed.\n4. Scalability Concerns: The computational cost of profiling large datasets is not discussed in detail.", "theoretical_claims": "No theory.", "experimental_designs_or_analyses": "Yes, the experiments are well-designed with:\n\n1. Multiple evaluation protocols (TSTR, augmentation)\n2. Diverse datasets (six medical datasets)\n3. Multiple generative models (CTGAN, TVAE, Bayesian Network, etc.)\n4. Twelve predictive models evaluated via AutoGluon\n\n\nConcerns:\n\nSynthetic Data Proportions: The results show that adding 68.2% synthetic data degrades performance, but the exact reason is not deeply analyzed.\nModel-Specific Performance: Some generative models (e.g., CTGAN) show high variability, but the authors do not discuss why.", "relation_to_broader_scientific_literature": "Synthetic data are generally very important in scientific discoveries.", "essential_references_not_discussed": "NA.", "other_strengths_and_weaknesses": "Strengths:\n1. Comprehensive experiments (18,000 trials).\n2. Fairness-aware synthetic data generation.\n3.Robust statistical evaluation (Wilcoxon test).\n4. Profile2Gen generalizes well across datasets.\n\nWeaknesses:\n1.  No deep theoretical justification for removing hard samples.\n2. Hyperparameter sensitivity (thresholds for hard sample removal).\n3. Scalability concerns for large datasets.", "other_comments_or_suggestions": "NA.", "questions_for_authors": "NA.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FrF3wG9aED", "reviewer": "Reviewer_BQrC", "summary": "The paper introduces Profile2Gen, a novel data-centric framework that generates and refines synthetic data specifically for regression tasks in medical applications. By profiling the original dataset into easy, ambiguous, and hard samples, the framework trains separate generative models and later refines the synthetic data through iterative postprocessing that removes hard samples. Extensive experiments across six public medical datasets—using seven state-of-the-art generative models and evaluating via metrics such as RMSE and Wasserstein distance—demonstrate that Profile2Gen can reduce predictive error and, in some cases, even outperform models trained solely on real data. The authors further extend the DataIQ framework to support regression tasks, making their approach broadly applicable in data-scarce scenarios.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "* **Generalization vs. Diversity Trade-off:** Profile2Gen, which incorporates post-processing, reduces Wasserstein's similarity between real and synthetic samples. This indicates that the generated samples are less similar to real data than other techniques. Here is the highlight: the similar samples, which have the same statistical characteristics, are a generalization of the real ones, while the diversity concerns about the samples that are not too similar but follow the distribution patterns ( Alaa et al., 2021,  https://arxiv.org/abs/2102.08921). When analyzing the dissimilarity alongside the profile2gen samples and their distributions, we observe that the generated samples are not only different but also diverse while still following the original distribution patterns. The lower similarity indicates that Profile2Gen prioritizes more variable synthetic data, potentially creating a broader range of scenarios and examples. As a consequence, post-processing increases diversity at the cost of generalization since this set of samples is diminished in the synthetic dataset. Higher diversity is generally beneficial in tasks such as data augmentation, where increasing the dataset size does not necessarily improve model performance unless it introduces novel and meaningful variations. However, a lack of generalization may cause the synthetic data distribution to diverge too much from the real data, leading to fidelity loss and getting into the previously discussed by Alaa et al. (2021) trade-off, the diversity-fidelity.\n* **Extensions for Other Tasks (Classification, Time-to-Event, etc.) -- Reviwers BQrC (Q2, Q6, Q7), fkDs:**\nWe appreciate the question about different tasks, and we emphasize that the main difficulty concerns the data-centric framework used to profile the data. We elaborate on our justification:\n\n* **High-dimensional non-tabular data:** Scaling to higher-dimensional or non-tabular data is indeed a relevant challenge. We conducted experiments using time-series data from wearable sensor devices - samples with shape (length, temporal window size, number of sensor axis) - specifically accelerometer data collected. At the same time, subjects performed daily activities such as brushing their teeth, jumping, running, and typing. However, our approach did not yield satisfactory results for this type of data. The main challenges were: 1. The CleanLab framework requires data to be in a tabular format, which is unsuitable for time-series data. 2 In time-series data, the entire temporal window contributes to label determination. Missing or misaligned steps could alter labels — for example, distinguishing running from jogging became problematic. Even when we added a datetime column to better structure the data, the approach still failed. 3. We tested the approach using DataIQTorch - from DataIQ - which allowed us to bypass the tabular conversion issue. However, obtaining confidence scores through the framework's methods was not straightforward, leading to inconsistencies.\nDeveloping a version of these frameworks specifically designed for time-series data could help address these challenges. However, despite several attempts, the effort required to adapt the existing framework for non-tabular data did not justify the results obtained. As a result, we decided not to pursue this direction further within the scope of our study.\n\n* **Classification tasks:**  A similar technique developed by Hansel et al. (2023, https://arxiv.org/pdf/2310.16981 ) addresses classification tasks. However, existing approaches still lack extensive exploration regarding generative regression-focused tasks. Our work aims to fill this gap by highlighting its limitations and the need for further research in this field, which also inspired us to adapt the method for regression tasks.\nTo achieve this, it was necessary to modify DataIQ to support regression, as we did in our approach. Additionally, performance metrics determined our framework selection entirely, whereas Hansel et al. (2023) incorporated additional aspects and limitations beyond purely metric-based decisions. \n\n* **Time-to-event and other tasks:**  We believe that the core of the framework —preprocessing, profiling samples, and following the remaining workflow — can be applied to time-to-event analysis and survival prediction. However, the current profiling frameworks (Cleanlab and DataIQ Reg) are not properly designed to handle survival data. Specifically, the concept of hard samples in survival analysis must account for censorship and the nature of time-to-event data, not just the confidence level of the models, which these frameworks do not consider. If a DataIQ Survival framework were developed to address these limitations, we strongly believe that Profile2Gen would be a suitable approach for synthetic data generation in survival analysis. \n\n**Please review the response provided to reviewers sXia and fkDs, where we address your concern.**", "claims_and_evidence": "The paper claims that Profile2Gen (1) statistically significantly reduces predictive errors in regression tasks; (2) enhances model reliability and generalization, sometimes achieving comparable or even better performance than real data; (3) preserves minority groups in the data distribution better than traditional methods.\nThese claims are supported by extensive experiments (approximately 18,000 runs) across multiple datasets and models, with clear quantitative evidence provided via RMSE improvements, Wasserstein distance analyses, and statistical significance tests (e.g., Kruskal-Wallis and Wilcoxon tests). While the evidence is comprehensive, the paper does note some cases where preprocessing alone sometimes outperforms Profile2Gen, indicating that the benefits may vary with dataset/model specifics.", "methods_and_evaluation_criteria": "The methodological approach is multi-staged: (1) Preprocessing: The original data is profiled (with label flipping used to gauge data quality) to identify easy, ambiguous, and hard samples. (2) Synthetic Data Generation: Generative models are independently trained on these subsets. (3) Postprocessing: The synthetic data is refined by removing hard samples based on a user-defined threshold.\nEvaluation is carried out using both the TSTR (Train on Synthetic, Test on Real) protocol and augmentation tasks. Key metrics include RMSE for predictive performance, Wasserstein distance for distribution similarity, and additional analyses of fairness and diversity in the synthetic data. Overall, the chosen methods and criteria are well-aligned to improve data quality for medical regression tasks.", "theoretical_claims": "The paper provides theoretical formulations to define “hard” samples via a scoring function and supports the design of the iterative refinement process with derived equations (e.g., Equation 1 for F1 score). Although the derivations appear reasonable, a deeper scrutiny of the proofs would be beneficial—especially to assess the sensitivity of the threshold parameters and their impact on model performance.", "experimental_designs_or_analyses": "The experimental design is robust, incorporating: (1) multiple medical datasets (e.g., Parkinson, Urinary, Cholesterol, Body Fat, Plasma, Diabetes) from OpenML; (2) A comprehensive comparison across seven generative models and twelve predictors; (3) Detailed analyses including RMSE plots, distribution comparisons via violin plots, and statistical tests.\nOne potential concern is the observed variability in performance across datasets—Profile2Gen sometimes lags behind simpler preprocessing approaches. This suggests that further ablation studies or sensitivity analyses (e.g., on the label flipping ratio and threshold T) could provide additional clarity.", "relation_to_broader_scientific_literature": "The paper situates itself well within the data-centric AI literature by building on recent frameworks such as DataIQ and CleanLab. It also connects with established methods in synthetic data generation (e.g., CTGAN, TVAE) and recent discussions on handling hard samples (e.g., AUM ranking). The discussion could be further enriched by comparing with the latest advances in synthetic data for medical domains.", "essential_references_not_discussed": "It might benefit from a discussion of recent advances in synthetic data generation using GAN-based methods in medical imaging or other non-tabular data domains. Including such references would provide broader context and help underline the unique contributions of Profile2Gen in a wider landscape of synthetic data research.", "other_strengths_and_weaknesses": "Strengths:\n- Novel integration of data profiling with synthetic data generation for regression tasks.\n- Extensive experimental evaluation across diverse datasets and models.\n- Practical relevance for overcoming data scarcity in sensitive domains like medicine.\n\nWeaknesses:\n- Performance gains are sometimes marginal or inconsistent, depending on the dataset and generative model.\n- The framework introduces additional complexity, which may affect reproducibility and requires careful parameter tuning.\n- More detailed ablation studies could clarify the influence of key hyperparameters (e.g., threshold T, label flipping ratio).", "other_comments_or_suggestions": "- Additional visualizations or error analyses could further illustrate the trade-offs between diversity and generalization in the synthetic data.\n- Discussing potential limitations in terms of computational cost and scalability would add value.\n- It would be helpful to provide guidelines for selecting optimal thresholds and label flipping ratios.", "questions_for_authors": "- How sensitive is Profile2Gen to the choice of the threshold T and the label flipping ratio? Have you explored adaptive thresholding methods that could adjust the threshold T dynamically based on dataset characteristics rather than relying on a fixed value?\n- Have you explored extensions of Profile2Gen for classification tasks? What modifications would be necessary for such an adaptation?\n- Given the variability in performance across different datasets, do you have insights into which characteristics of a dataset favor Profile2Gen over simpler preprocessing methods?\n- Can the proposed approach be scaled to higher-dimensional or non-tabular data, and if so, what challenges might arise?\n- Could you elaborate on the computational cost and scalability of the framework, especially when handling larger or more complex datasets?\n- How does error propagation from the early profiling stage affect the overall performance, and what measures can be taken to mitigate such issues?\n- Beyond regression tasks, would this framework apply to other downstream applications such as time-to-event analysis or survival prediction?\n- Could you provide further insights into the trade-off between generalization and diversity during the postprocessing stage, and how the method ensures that critical edge-case information is not lost?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cOCPZXp3yW", "reviewer": "Reviewer_fkDs", "summary": "The paper focuses on generating training data for regression models in the medical domain. The proposed approach is based on two existing methods, which the authors refer to as Traditional Generative Techniques and PreProcess methods. In the Traditional approach, the method does not consider the difficulty distribution of the data during training, while the PreProcess method first categorizes the training data by difficulty and then trains separate generative models on each difficulty group. The final data set is created by merging the generated data from all models. The approach proposed in this paper combines both methods, training the generative model while considering data categorization and further filtering difficult samples after data synthesis to obtain the final dataset. Experiments on six datasets show that the proposed method effectively improves the quality of the synthetic dataset and reduces prediction errors in regression models.\n\n### update after rebuttal:\nI have read the authors' rebuttal as well as the reviews from the other reviewers. I appreciate the additional details provided. However, my main concerns were not fully resolved. As such, I will keep my original score, which already reflects a positive assessment.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "* **Hyperparameter Sensitivity- Revwers sXia, BQrC (Q1), VmCi**\n\nThe choice of the hard sample threshold is based on the performance of the profiling framework. In Appendix F, we provide a detailed explanation of this selection process. Specifically, the threshold is determined by identifying the best-performing framework in terms of F1-score across different levels of label flipping.\n\nOnce the best framework is identified, we use the corresponding threshold that yielded the highest F1-score to profile the data. This process occurs at both the pre-processing and post-processing stages. Since we always use the best-performing framework, we consider threshold selection a non-critical hyperparameter, as it is inherently optimized within the profiling stage.\nFormally, let $ \\mathcal{F} $ be the set of evaluated frameworks and $ \\mathcal{T} $ the set of tested thresholds. The goal of the process described in the paper is to find the optimal pair $ (f^*, t^*) $ such that:\n\\begin{equation}\n(f^*, t^*) = \\arg\\max_{f \\in \\mathcal{F}, t \\in \\mathcal{T}} \\mathbb{E}[F1(f, t)]\n\\end{equation}\n\nwhere $ \\mathbb{E}[F1(f, t)] $ represents the average F1-score across different proportions of label flipping.\nSensitivity analysis typically involves testing the robustness of a fixed parameter, but in our case, $ t^* $ is not arbitrary — it is chosen as part of an optimization process that depends on $ f^* $. Since we have already found the optimal pair $ (f^*, t^*) $ for each dataset, testing variations in $ t^* $ without re-evaluating $ f^* $ would undo the optimization performed and could lead to misleading conclusions. In other words, the threshold is already thoughtfully selected along with the framework, making an additional sensitivity analysis unnecessary. \n\n**T analysis:** Given that sensitivity was a concern across reviewers, we assessed how the number of hard-profiled samples changes when varying the threshold \\textit{T}. \nTo ensure reproducibility, we set a random seed and let NumPy randomly select T from the range (0.05, 0.6) with six distinct values. The Figure in https://anonymous.4open.science/r/icml2025-F9C4/sensitivity_theshold.png illustrates the observed behavior using the smallest and largest datasets from our experiments.\nThe threshold can be understood as a flexibility level —how much confidence is required before a sample is no longer considered \"hard.\" Lower thresholds allow greater flexibility, meaning the model tolerates lower confidence scores. Given that our predictor is a good generalizer, we observed a few samples with low confidence. As expected, increasing the threshold leads to a higher number of hard samples. However, the change follows a smooth, almost linear trend rather than abrupt shifts.\nThis indicates that our threshold selection process is reasonable and stable. To enhance clarity in the main paper, we propose explicitly including the optimization equation and adding a section explaining why the chosen \\textit{T} is justified. This should benefit both the work and its readers.\n\n**Flipping analysis:** We also assessed how the number of hard-profiled samples changes when varying the threshold or the flipping rate while keeping the other parameter fixed. To ensure reproducibility, we set a random seed and let NumPy randomly select a fixed parameter value from the range (0.05, 0.6), while the variable parameter was chosen from the same range but with six distinct values. The Figure in: https://anonymous.4open.science/r/icml2025-F9C4/sensitivity_threshold_and_flipping.png illustrates the behavior observed. The results show that dataset characteristics significantly influence sensitivity. For the Diabetes dataset, fixing the threshold and varying the noise level leads to considerable changes in the number of hard samples, which is expected given that label flipping in small datasets intuitively affects the data distribution more than in larger ones. However, when fixing the flipping level and varying the threshold, the sensitivity is relatively smooth for both datasets. The curves do not exhibit abrupt changes, confirming that selecting the best threshold for the highest-performing framework remains a reasonable and stable choice. These findings reinforce our original methodology: optimizing $, T^*$  jointly with $f^*$ ensures that the threshold adapts to dataset characteristics without introducing unnecessary complexity or computational overhead.\n\n* **LMM adoption**\nWe considered using LLM-based methods during the data generation process and actually used GREAT (2023,  http://arxiv.org/abs/2210.06280), which exploits an auto-regressive generative LLM to sample synthetic and highly realistic tabular data. However, we did not highlight it separately; instead, we treated it alongside other models. This model was highlighted in Section 5.6, where it demonstrated good potential in replicating low-probability events, with its performance further improved when combined with Profile2Gen.", "claims_and_evidence": "The primary claim of this paper, that the proposed framework can effectively synthesize training data for regression tasks in the medical domain and improve model prediction accuracy and generalization ability, is supported by evidence. The experimental results on six datasets, using various models, demonstrate the validity of the claim.", "methods_and_evaluation_criteria": "I believe the proposed methods and evaluation criteria are reasonable for the problem. The authors performed large-scale testing of their method on multiple benchmarks from the medical domain, using common metrics. They also evaluated the synthetic data’s ability to substitute real data and the quality of the generated data, exploring several relevant aspects.", "theoretical_claims": "The paper does not present any theoretical claims or proofs.", "experimental_designs_or_analyses": "The experimental designs and analyses presented in the paper seem sound. The authors validated their proposed method on sufficient datasets and models, analyzing three key aspects: whether the synthetic datasets can replace real data, whether they can be combined with real datasets, and the quality of the generated datasets themselves.", "relation_to_broader_scientific_literature": "The key contribution of this paper lies in its domain-specific focus on the medical field, emphasizing data-centric approaches in contrast to existing large language model-based generative pipelines that focus on general domain data generation. This focus addresses a gap in the current research direction.", "essential_references_not_discussed": "I believe the authors should have discussed recent methods for synthetic data generation using large language models (LLMs) and compared them to their proposed method. Below are some relevant surveys/papers:\n\nSmolyak, D., et al. (2024). Large language models and synthetic health data: progress and prospects (JAMIA open 2024)\nLi, R., et al. (2023). Two directions for clinical data generation with large language models: data-to-label and label-to-data. (EMNLP 2023)\nKumichev, G., et al. (2024). MedSyn: LLM-based Synthetic Medical Text Generation Framework. arXiv preprint arXiv:2408.02056\nSeedat, N., et al. (2024). Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in low-data regimes (PMLR 2024)\nZhou, H., et al. (2024). A Survey of Large Language Models in Medicine: Progress, Application, and Challenge. arXiv:2311.05112", "other_strengths_and_weaknesses": "A major weakness of the paper is the innovation in its method. As mentioned in the paper, the proposed method is largely based on existing traditional and preprocess approaches. The primary innovation lies in the filtering of the generated data after synthesis, which may limit the contribution's significance.\n\nAnother weakness is the lack of comparison with recent methods for data generation or filtering using LLMs.", "other_comments_or_suggestions": "I believe the writing of the paper could be improved, particularly in Sections 3 and 4 where the logical flow and method descriptions could be clearer. For example, the authors should first introduce the traditional and preprocess methods, then explain their differences, and finally describe their proposed workflow to enhance the readability of the paper.", "questions_for_authors": "Why not consider using any LLM-based methods during the data generation process?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0i8ki7TVcF", "reviewer": "Reviewer_VmCi", "summary": "This paper introduces a novel approach to synthetic data generation, leveraging a combination of statistical modeling and generative techniques to produce high-fidelity, diverse datasets for machine learning applications. The proposed methodology is designed to enhance the realism and utility of synthetic data, thereby improving model performance in data-scarce or privacy-sensitive scenarios. Through extensive experimentation, the authors demonstrate the proposed framework’s advantages over existing data synthesis techniques.\n\n## update after rebuttal\n\nGiven these clarifications, most of my concerns have largely been resolved. As my initial score leaned towards acceptance, I decided to maintain it.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "* **Ablation Studies and Components- Reviewers BQrC, VmCi**\nWe did not conduct an ablation study because the preprocessing step we adopted already serves as a form of comparison itself. Specifically, we chose to use our proposed preprocessing method as a comparison to traditional preprocessing techniques, such as feature selection and data cleaning, which are commonly used in the field. In our methodology, we intentionally selected a preprocessing approach that includes a profiling framework, which is optimized for each dataset. This framework is selected through an optimization process to identify the best preprocessing method for each dataset. We did not compare it to traditional preprocessing techniques like simple feature selection because those methods do not effectively capture the complexity and improvements our approach brings. Using our profiling framework as a baseline, we demonstrated the added value of combining it with a post-processing phase, significantly enhancing performance. Therefore, the methodology itself, through the comparison between our full approach (preprocessing + post-processing) and the chosen preprocessing method, already performs the role of an ablation study. We could have used a traditional feature selection method as the preprocessing step for comparison. Still, we are confident that our complete methodology provides the most effective solution. We believe the direct comparison between our full approach and a traditional preprocessing method already validates the effectiveness of our proposed methodology.\n* **Fidelity Evaluation -- Reviewer VmCi**  \nWe understand the concern regarding the reliance on qualitative visual assessments and the fidelity of the synthesized data compared to the real world. However, we emphasize that our evaluation includes rigorous quantitative analyses in addition to visual representations, which we believe enhance result interpretation.\nRegarding evaluation metrics, we utilize the Root Mean Squared Error (RMSE) to assess the performance of predictive models and the Wasserstein distance to measure the similarity between real and synthetic data distributions. RMSE is a widely used metric for evaluating the accuracy of regression models, while the Wasserstein distance quantifies the representativeness of synthetic data. Additionally, we evaluate the quality of synthetic data based on three key criteria: Fidelity (accuracy), Diversity, and Generalizability.\nFurthermore, we follow a rigorous evaluation protocol that includes Train on Synthetic, Test on Real (TSTR), and data augmentation strategies. We also employ the Kruskal-Wallis and Wilcoxon tests to assess the statistical validity of our results, ensuring the robustness of our evaluation.\nRegarding the fidelity of the synthesized data compared to the real world, we used the Wasserstein distance, a robust statistical metric that computes the difference between distributions. This metric provides a meaningful and rigorous evaluation of the fidelity of synthetic data, enabling us to compare how well the synthesized data approximates real-world distributions.\nFurthermore, the evolution of statistical values, such as mean, standard deviation, and quantiles, further supports the validity of our findings. We believe you may be referring to tests like the Kolmogorov-Smirnov (KS) test to assess the fidelity of distributions. However, we chose to adopt a single metric—the Wasserstein distance—due to its statistical rigor and ability to handle many experiments and other aspects analyzed in our work.\nAdditionally, the TSTR metric is not solely designed to evaluate whether synthetic data is useful for training models. Instead, it assesses whether the synthetic data distribution significantly differs from the original one. A large discrepancy between distributions would suggest that the model trained on synthetic data might perform poorly when tested on real data. Thus, the use of these metrics provides valuable insights without the need to collect an overwhelming number of additional metrics.\nWasserstein distance is widely used in the literature for such evaluations, and we believe it is sufficient for this purpose. Furthermore, when evaluating synthetic data, it is essential to consider not only fidelity but also diversity and generalization. By combining various evaluation protocols, we can comprehensively assess the usability of the synthetic data.\n\n**Please review the response provided to reviewers sXia and fkDs, where we address your concern.**\n\nDear Reviewers VmCi, sXia,  fkDs, and BQrC. We understand you are not required to look beyond the comments, but we addressed all points. Due to space limits, we prioritized common responses. A full reply, including additional comments, is in this PDF: https://anonymous.4open.science/r/icml2025-F9C4/Answers.pdf. We’d be grateful if you could check it.", "claims_and_evidence": "The claims are generally supported by experimental results, though certain aspects warrant further substantiation:\n- While the paper asserts that the generated synthetic data maintains both realism and diversity, it does not employ quantitative metrics such as Frechet Inception Distance (FID) or Maximum Mean Discrepancy (MMD) to evaluate these attributes rigorously.\n- The choice of baseline models is not sufficiently justified. A more detailed discussion of the selection criteria for these baselines would strengthen the argument.", "methods_and_evaluation_criteria": "The paper employs standard evaluation methodologies for assessing synthetic data quality, yet some enhancements could improve the robustness of its findings:\n- A more extensive analysis of the effect of synthetic data on downstream machine learning models (e.g., classification, regression) would provide deeper insight into its practical applicability.", "theoretical_claims": "The paper does not introduce formal mathematical proofs, but the methodological framework appears sound. However, a theoretical discussion of the potential limitations—such as mode collapse in generative models or bias amplification in synthetic data—would strengthen the paper’s contribution.", "experimental_designs_or_analyses": "Yes, the experimental setup is reasonable, but has certain limitations:\n- The evaluation methodology relies heavily on qualitative visual assessments, which, while informative, should be supplemented with rigorous quantitative comparisons.\n- The paper does not provide a thorough hyperparameter sensitivity analysis, making it difficult to assess the reproducibility and robustness of the proposed method.", "relation_to_broader_scientific_literature": "The work advances the field of data synthesis by proposing a new generative framework that balances realism and diversity. However, additional context is necessary to situate it within the broader literature:\n- Recent **GAN-based and diffusion-based models** have addressed similar challenges in synthetic data generation, and a comparative discussion with these approaches would be beneficial.\n- The paper does not thoroughly address techniques used for **synthetic data validation and bias mitigation**, which are critical in practical applications.", "essential_references_not_discussed": "Yes, the paper would benefit from discussing:\n- State-of-the-art diffusion models and variational autoencoders (VAEs), which have demonstrated strong performance in synthetic data generation.\n- Data augmentation and privacy-preserving synthetic data generation strategies, which are highly relevant to this research domain.", "other_strengths_and_weaknesses": "- Strengths: The proposed approach is well-motivated, tackling an important challenge in machine learning by improving the quality of synthetic datasets.\n- Weaknesses: The lack of rigorous quantitative evaluation metrics and ablation studies limits the empirical contribution of the paper.", "other_comments_or_suggestions": "- Some mathematical derivations would benefit from improved clarity and notation.", "questions_for_authors": "1.\tHow does the fidelity of the synthesized data compare to real-world distributions under rigorous statistical evaluation?\n2.\tWould incorporating adversarial training or diffusion models further improve the quality and robustness of the generated data?\n3.\tCan the authors elaborate on the scalability of their approach, particularly in the context of large-scale datasets?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper proposes a new method for data synthesis, which guides the generation and refinement of synthetic data to address hard-to-learn samples. 

It gets mixed reviewing opinions from four reviewers. 

Reviewers sXia, fkDs, and VmCi hold a positive view, recommedning 'weak accept' after rebuttal. 

Reviewer BQrC holds a negative view, recommedning 'weak reject' after rebuttal. S/he is happy about the novelty of the proposed synthesis approach, extensive evaluation, the practical utility and has concerns about the marginal performance gains, insufficient ablation studies, etc. 

Overall, the AC thinks that Reviewer BQrC is a bit too critical about his/her assement. Hence, it is recommened to accept the paper.
Decision: Accept (poster)