Local Distribution-Based Adaptive Oversampling for Imbalanced Regression
Authors: Shayan Alahyari
Mike Domaratzki
Venue: Accepted by TMLR ()

Abstract: Imbalanced regression occurs when continuous target variables have skewed distributions, creating sparse regions that are difficult for machine learning models to predict accurately. This issue particularly affects neural networks, which often struggle with imbalanced data. While class imbalance in classification has been extensively studied, imbalanced regression remains relatively unexplored, with few effective solutions. Existing approaches often rely on arbitrary thresholds to categorize samples as rare or frequent, ignoring the continuous nature of target distributions. These methods can produce synthetic samples that fail to improve model performance and may discard valuable information through undersampling. To address these limitations, we propose LDAO (Local Distribution-based Adaptive Oversampling), a novel data-level approach that avoids categorizing individual samples as rare or frequent. Instead, LDAO learns the global distribution structure by decomposing the dataset into a mixture of local distributions, each preserving its statistical characteristics. LDAO then models and samples from each local distribution independently before merging them into a balanced training set. LDAO achieves a balanced representation across the entire target range while preserving the inherent statistical structure within each local distribution. In extensive evaluations on 45 imbalanced datasets, LDAO outperforms state-of-the-art oversampling methods on both frequent and rare target values, demonstrating its effectiveness for addressing the challenge of imbalanced regression.

Published in Transactions on Machine Learning Research (09/2025)
Local Distribution-Based Adaptive Oversampling for Imbalanced Regression
Shayan Alahyari Western University
calakya@wwo.com
Mike Domaratzki Western University
mdomarat &rwwo.co
Reviewed on OpenReview: https: // openreview. net/ forun? id- 6qYTR91Jam
Abstract
Imbalanced regression occurs when continuous target variables have skewed distributions, creating sparse regions that are difficult for machine lcarning models to predict accurately. This issue particularly affects neural networks, which often struggle with imbalanced data. While class imbalance in classification has been extensively studied, imbalanced regression remains relatively uncxplored, with few effective solutions. Existing approaches often rely on arbitrary thresholds to calegorize samples as rare or frequent, overlooking the continu- ous spectrum of target values. These methods can produce synthetic samples that fail to improve model performance and may discard valuable information through undersampling, To address these limitations, we propose LDAO (Local Distribution-bast Adaptive Over- sampling), novel data-level approach that avoids categorizing individual samples as rare or frequent. Instead, LDAO learns the global distribution structure by decomposing the dataset inlo a mixture of local distributions, each preserving its statistical characteristics. LDAO then models and samples from each local distribution independently before merg- ing them into balanced training set. LDAO achieves balanced representation across the entire target range while preserving the inherent statistical structure within each local distrriuttio. In extensive evaluations 45 imbalanced datasets, LDAO outperforms state- of-the-art oversampling methods both frequent and rare target values, demonstrating its effectiveness for addressing the challenge of imbalanced regression. Our code is available at hitps:/ http://ggthhhbu. htt.gh.tthooom///o http://ggh.....o/////aaaaaaaaaaa
1 Introduction
In classification tasks, imbalance occurs when some classes have far fewer samples than others, in both binary and multi-class settings (He & Garcia 2009 Guo et al. (017). The minority classes are overshadowed by the majority classes, causing traditional classifiers to focus on the more abundant classes and perform poorly in identifying minority classes (Chawla al. 2002). This leads inaccurate detection of minority classes since models struggle recognize patterns in sparsely sampled regions (Buda al. (018).. Critical applications like fraud detection, medical diagnostics, and fault detection particularly affected, rare but important classes might be missed (Johnson & Khoshgollaar 2019). Therefore, effective handling of imbalanced data is crucial for reliable performance across all classes, ensuring significant rare examples are properly detected [Liu et al. 2009).
While imbalance is widely studied in classification He Garcia 2009). also affects regression tasks that predict continuous values (Krawczyk) 2016). Imbalanced regression occurs when certain target ranges (typically rare or extreme cases) have significantly fewer samples than others (Branco et al. 2016 Torgo et al. 2013). Traditional regression models struggle accurately predict these rare values because they focus more frequent, well-represented ranges (Chawla et al. 2004 Branco et al. 2016). Accurately predicting

---

Published in Transactions on Machine Learning Research (09/2025)
Class Imbalance Challenge
Regression imbalance Challange
Dense Region.
sparce Region
Peotuin
Peatture 1
Valoe
Figure 1: Left: a long-tailed classification problem in which discrete class counts allow straightforward identification of minority (red) and majority (green) classes. Right: an imbalanced regression problem in which continuous target values create underrepresented regions (red) that cannot be detected by simple counting, Both settings can exhibit long-tailed distributions; here emphasize their difference in minority detection mechanisms.
these less frequent target ranges is challenging yet crucial for many real-world applications Toogo et al.
2013
Figure ! contrasts imbalanced classification with imbalanced regression. In classification, identifying minor- ity classes is straightforward because labels are discrete and can be counted. In regression, minority regions exist within continuous target distributions, often sparsely populated tails. While classification imbalance relates distinct labels, regression imbalance depends on distribution shape (skewed or multi-modal).. Clas- sification methods cannot be directly applied to regression since there is no simple way count labels in order to identify underrepresented values. This makes detecting and addressing sparse regions significant challenge in imbalanced regression.
Imbalanced regression impacts many critical real-world applications where accurately predicting rare but high-impact events in the tails of skewed distributions paramount. In electrochemical prognostics, lithium- ion battery saate-offheeathh data exhibit long-tailed degradation trajectories, with very few cells nearing end-of-life available for training, leading to signíficant underestimation of remaining useful life by standard models (Zhao et al.) 2025).. In automated pain assessment, continuous video-based pain intensity labels are overwhelmingly concentrated in low-pain frames, causing deep facial expression regression networks to underpredict severe pain episodes (Xiang et al. 2022). In carbon markets, sporadic extreme price surges in emissions trading schemes create highly imbalanced timo-scries segments that traditional regressors fail to capture, resulting in substantial forecasting errors during critical market movements {Yin & et. al., 2025). In muzzy-rule-based optimization, conventional soft computing methods disregard the uneven distribution of target values, undermining regression performance on rare but crucial value ranges (Artcaga-Jover et al., 2023). These examples represent only subset of numerous instances in which addressing imbalance in regression is essential.
Several solutions have been proposed to address imbalanced regression at both data and algorithmic levels. Some extend SMOTE from classification to regression by generating synthetic examples in the target distri- bution's 's tails, increasing representation of rare values (Torgo et al. 2013 Branco et al. 2017). Others apply cost-sensitive methods that penalize errors on underrepresenteed values more heavily, encouraging models to focus on these instances (Steininger al. 2021). Many data-level approaches rely on linear interpolation which is adopted from SMOTE, where a synthetic sample generated between minority sample x; and neighbor Xy:
Xnew = x, +Xx x:), X x~ U(0,1)
This is problematic because it assumes linear relationships between data points, which often fail to capture the complex, non-linear structure of the true data distribution. The addition of basic Gaussian noise is

---

Published in Transactions on Machine Learning Research (09/2025)
another common strategy, but this method is often insufficient for creating realistic samples that follow the intricate patterns of the true feature-target. relationship.
These synthetic samples often fail to accurately represent the true, complex relationship between the input features and the continuous target variable, which distorts the joint feature-targe distribution. Furthermore, the complex web of correlations between different features is frequently lost entirely because the generation process is often univariate or does not explicitly model these relationships. The generated data can also fail to reproduce the basic statistical profile of the real data, exhibiting incorrect statistical properties where the marginal distributions of the synthetic samples have incorrect moments, such as mean and variance, or shapes, such as skewness and kurtosis, compared to the original data.
We propose LDAO, a novel data-level oversampling approach designed to directly overcome these funda- mental limitations through adaptive, distribution-awarc sampling. Instead of imposing simplistic linear assumptions, LDAO operates on the joint feature-target space, first decomposing the data into a mixture of local distributions to capture its global structure. Within each discovered region, LDAO employs multi- variate Kernel Density Estimation (KDE) to build a rich, non-parametric model that explicitly learns the local covariance structure. By generating new samples from these localized models, LDAO ensures that syn- thetic data preserves the intricate tearure-t dependencies, inter-feature correlations, and the essential statistical properties of the original data. This method maintains the continuity of regression targets while providing a statistically sound and robust foundation for training regression models on imbalancod data.
2 Related Work
Adapting data-level oversampling strategies regression is more difficult because the target variable is con- tinuous (He & Ma 2013 Krawezyk 2016). Early approaches to imbalanced regression modified classification resampling by introducing relevance function <(v) to distinguish between rare and frequent target regions (Torgo& & Ribciro 2007). This function assigns higher scores to more extreme target values, while a user-
specified threshold determines which points are rare versus common (Torgo & Ribeiro 2007: Ribeiro 2011). Using this relevance function framework, Torgo et al. developed two main resampling methods for regression (Torgo et al. 2013 2015).
The first method applies random undersampling to remove samples with low p(y) scores (common target values), reducing their overrepresentation. The second method, SMOTER, generates new samples in regions with high @(y) scores (rare target values) by interpolating feature values of nearby rare samples and calculat- ing target values as weighted averages of these neighbors (Torgo et al. 2013 2015 While these approaches were pioneering in addressing regression imbalance and demonstrated that focusing on rare targets improves performance (Torgo et al. 2015), they depend on user-chosen rarity thresholds that may be arbitrary and fail to reflect the true structure of continuous target distributions (Branco et al., 2019
SMOGN (Synthetic Minority Over-sampling Technique for Regression with Gaussian Noise) extends SMOTER by combining interpolation with basse oversampling. For each rare instance, SMOGN generales either an interpolated synthetic point (as in SMOTER) or adds Gaussian noise to create per- turbed point (Branco et al. 2017). This hybrid approach provides more flexible oversampling of sparse regions and outperforms SMOTER (Branco et al.] 2017). SMOGN also applics undersampling to reduce overrepresented frequent examples, establishing it as a state-of-the-art resampling method for imbalanced regression (Branco et al. 2017).
More recently, Camacho et al. 2022) developed Geometric SMOTE (G-SMOTE) for regression, adapting geometrically enhanced SMOTE to continuous targets. G-SMOTE generates synthetic samples in minority regions by considering both feature-space interpolation and target distribution geometry, offering greater flexibility in sample placement (Camacho et al. 2022).
Stocksieker et al. 2023) combined weighted resampling and data augmentation to better represent covari- ate distributions and reduce overfitting. Their approach weights underrepresented regions while using noise addition or interpolation to cover wider data ranges. Camacho and Bacso introduced WSMOTER (Weight- ing SMOTE for Regression), replacing fixed thresholds with instance- based weighting to better highlight underrepresented targets and improve synthetic sample generation in sparse regions (Camacho & Bacao
3

---

Published in Transactions on Machine Learning Research (09/2025)
2024h. Stocksieker et al. (2024) later developed GOLIATH, a framework handling both noise and interpola- tion methods with hybrid generators and wild-bootstrap steps for continuous targets. Aleksic and García- Remesal proposed SUS (Sclective Under-Sampling), which selectively removes only those majority samples that provide little additional information, along with an iterative variant (SUSiter) that roduces dis- carded samples over time, showing strong results on high-dimensional datasets (Aleksic & Remeeal 2025
In addition to data-level strategies, algorithm-level solutions embed imbalance handling directly into the learning process. Cost-sensitive learning exemplifies this approach by modifying loss functions or instance weights to make models focus on rare targets and incur higher penaltics for crrors on these examples. This concept originated in classification by weighting classes inversely to their frequency and extends naturally to regression by emphasizing rare or extreme target values (Zhou & Liu 2010 Elkan 2001 Domingos 1999). By imposing higher costs on underrepresented outcomes, these approaches drive models to reduce errors where accuracy matters most.
DenseLoss is cost-sensitive approach designed specifically for imbalanced regression (Steininger et al. 2021). It uses DenseWeightt a density-based weighting strategy that assigns higher weights to samples with less frequent largel values. This directs the model to focus on rare or extreme cases where accuracy is most critical (Steininger et al. 2021). WenseWeight estimates the target distribution's probability density and gives larger weights to low-density targets, scaling the loss function during training to emphasize prediction accuracy on rare cases. Unlike oversampling methods, DenseLoss modifies the learning process without creating removing examples from the training set.
Yang ct al. 2021 proposed a framework for deep imbalanced regression that improves prediction performance for extreme larget values using label-distribution smoothing and adaptive sampling. Their work demonstrates how contimuous target imbalance can severely impact deep neural network performance. Ren et al. (2022) introduced Balanced MSE, which modifies standard mean-squared error to give higher weight to errors on rare targets. Validated across multiple computer vision tasks, their approach shows that directly addressing label imbalance outperforms standard MSE in skewed settings.
Deep inbalanced regression (DIR), particularly in vision applications such as age estimation and depth prediction, has recently seen a surge of methods that integrate representation learning and classification paradigms. Dong et al. (2025) propose geometric constraints, enveloping and homogeneity losses, within a surrogate-driven representation learning (SRL) framework to enforce uniform coverage and smooth transi- tions in the latent space, yielding significant improvements on several DIR benchmarks. Xiong & Yao (2024) reformulate regression as hierarchical classification, introducing hierarchical classification adjustment (HCA) and rango-preserving distillation (HCA-d) to balance quantization error and classifier accuracy across coarse- to-fine label partitions. Nejjar et al. (2024) shift from in-weight to in-context learning, demonstrating both theoretically and empirically that selecting localized prompts mitigates majority-region bias and enhances transformer-based DIR performance. Pu et al. (2025) adopt divide-and-conquer strategy by aggregating nearby labels into ordinal groups, leveraging group-aware contrastive lcarning, symmetric descending soft labeling, and a multi-experl regressor to preserve label continuity while addressing imbalance skewed distributions.
3 Motivation
Despite recent advances, imbalanced regression still faces significant challenges. Current methods often identify samples as rare or frequent and oversample the distribution based on that assumption. In many cases, these assumptions might be oversimplifications due to the complexity of continuous distributions and might not help the model generalize better or improve its prediction on rare samples. Also, many approaches try to oversample rare targets and undersample frequent targets, which misrepresents the dataset's underlying characteristics. Removing information from datasets and labeling certain ranges as rare or frequent is problematic, as rare samples can occur in any range within the global distribution, especially in multi-modal distributions. Often, distributions have latent characteristics which make it difficult to simply classify which samples are rare or frequent.

---

Published in Transactions on Machine Learning Research (205)
Original LDAO Synthetic SMOGN Synthetic
224
Figure 2: PCA projection of original (cyan), SMOGN synthetic (red), and LDAO synthetic (green) samples for the 2 75th-percentile tail, showing SMOGN's narrow interpolation ridge versus LDAO's natural disper- sion around the extreme values for the Boston dataset (Harrison & Rubinfeld 1978).
In addition, interpolation and noiso-basod oversampling techniques may generate synthetic samples that don't reflect actual data palterns, especially in extremely sparse regions (Branco al., 2019). Figure illustrates this effect: SMOGN synthetic samples (red) lie along a narrow interpolation ridge, while LDÃO samples (green) exhibit a broader, more natural dispersion around the extreme tail. Creating synthetic feature-target pairs that remain consistent with the original distribution is challenging, and most methods focus narrowly on extreme tails rather than the entire distribution. Many approaches rely heavily on relevance functions and thresholds that require careful parameter selection. Poor choices can lead to oversampling marginally rare areas while missing truly rare regions. This dependence on domain-specific luning limits broader applicability across different datasets. Real-world applications in healthcare, genomics, environmental science, finance, and engineering frequently involve imbalanced regression tasks, and need more robust approaches that tackle the problem of imbalanced regression as have seen in classification.
Adaptive oversampling techniques in classification, such as the method by Wang et al. (2020), which leverages local distribution information to generate synthetic minority instances, demonstrate the potential of these strategies. However, these methods rely on discrete class labels and fixed thresholds that do not translate well to regression tasks, where the target variable is continuous.
Our method, LDAO, addresses those limitations by preserving data structure across all target ranges. In- stead of using global thresholds or discarding frequent samples, LDAO augments each cluster of the target distribution independently, generating synthetic data that match local distribution patterns. Our goal in LDAO is to improve the model's prediction of rare samples while enhancing its overall generalization. LDAO aims to mitigate the trade-off between emphasizing rare samples and preserving overall generalization ability.
4 LDAO Method
We now describe LDAO, our local distribution-basee adaptive oversampling method that effectively addresses the challenges of imbalanced regression. IDAO decomposes the global distribution using k-means clustering. This clustering is applied simultaneously to both input features and target values, creating mixture of local distributions in a joint trature-taree space.
After clustering, we apply Gaussian kernel density estimation (KDE) within each cluster model the local distribution independently and generate synthetic data points that closely match the cluster's actual distribution. By drawing samples independently within each cluster, LDAO avoids creating unrealistic synthetic points. Finally, we merge these augmented clusters into single augmented dataset. The entire

---

Published in Transactions on Machine Learning Research (09/2025)
Original Imbolanced Distedbtion
Local Distributinn
Local Distibution
Demnty
Lucal Dist ributiinn
Clustering
Original Oversampled
Original
Original
Oversampled
Oversamplnd
Original Synthetic
Original Syzthetic
Original Synthetic
Merging Ovensompind Distrbbution 1
Augmented Distrtbution
Density
uarget value
Figure 3: LDAO process overview. First, the imbalanced dataset (lop) is decomposed into clusters. Then, each cluster is oversampled individually using kernel density estimation (middle). Lastly, these balanced clusters are combined into one dataset (bottom).
workflow summarized in Figure g data clustering, local density estimation for synthetic data generation, and merging to achieve an augmented overall distribution.
Given dataset DD {x,; y)) where x; E Rd represents features and y/s E represents the target variable, imbalance typically manifests regions with fewer observations, making these areas difficult to model accurately. LDAO models the joint distribution of features and targets. Each data point (x4, y.) is embedded in the combined featuro-target space as 2; (x,, y1) E Rd+! The overall joint distribution Px,y is approximated by a mixture model composed of localized distributions:
K
(x, 2
v)
where Ty denotes the proportion of data in cluster k, and Pk characterizes the local distribution of that cluster.
4.1 Clustering in the Joint Feature- Taager Space
We apply the standard k-means clustering algorithm to the dataset to identify natural clusters the joint larget space. Figure 4 illustrates clustering for sample data set, projected onto the first three principal components, and shows how points with similar features and target values grouped for more precise local density modelling, For illustration purposes only, we chose 3 the nunber of clusters. In this method, every data point z; is assigned to one of K clusters. The goal is to find the best set of cluster centroids {fLx ] and assignments {c) such that the total squared Euclidean distance between the data points and their assigned controids is minimized. Mathematically, we solve:
N MG, 11,
arg min (pL J
12
1=
This formulation ensures that points with similar characteristies are grouped together (Sugar & James 2003
Ikotun et al. 2023).

---

Published in Transactions on Machine Learning Research (09/2025)
Data Points
Cluster Centers
Cluster
C1 Centroid for cluster
Cluster
Centroid for cluster
Chuster
C3 Centroid for chuster
PC3
PC1
PC2
Figure 4: K-means clustering in the joint feature target space for the Boston dataset (Harrison & Rubinfeld, 1978), projected onto the first three principal components. For illustration only, points are projected onto the first three principal components; clustering itself is done in the full (d + 1)-dimensional space. Data points are grouped into three clusters (each indicated by a unique marker and color). Ellipsoidal density contours characterize the spread and oricntation of cach cluster. The centroids, marked with prominent gold symbols, represent the mean positions of the data points within their respective clusters.
4.2 Determining the Number of Clusters
The performance of k-means clustering largely depends the chosen number of clusters K. To choose optimal K, evaluate the Sun of Squared Errors (SSE), defined as:
K
SSE(K)
Iz
p4 11
1= DE
where D3 represents the set of data points in cluster k. As we increase K, the SSE typically decreases because each point closer to its cluster center.
To mitigate the pitfalls of the trade-off between oversimplification and excessive clustering, we use the elbow method (Syakur al. 2018; Nainggolan al.) 2019p. This method involves calculating the relative reduction in SSE when increasing the number of clusters. We define the relative improvement A(K) as:
A(K) SSE(K - 1) SSE(K) SSE(K-1)
2 Kmax.
The optimal number of clusters (K*) typically found where there a significant drop in A(K). One important note that the elbow point, adding more clusters does not considerably reduce the Sum of Squared Errors (SSE) and might cause overfitting by capturing noise rather than true underlying patterns [Syakur et al. 2018
Nainggolan al. 2019

---

Published in Transactions on Machine Learning Research (09/2025)
Cluster 1
Cluster 2
Cluster 3
PC
PC1
Figure 5: Clusters obtained via k-means on the feature target data are projected onto two principal com- ponents. For each cluster, kernel density estimation (KDE) is performed and the overlaid contour lines represent the density gradients of the data in the reduced space.
4.3 Kernel Density Estimation in Each Cluster
After the clustering step, we model the local distributions within each cluster separately using kernel density estimation (KDE) (Silverman 1986). illustrated in Figure
This approach preserves each cluster's local structure, maintaining accurate relationships between features and largels without mixing palterns across clusters. Let each cluster Dr contain ny data points in the joint feature-target space:
D3 (k) (k)
(R) (z) Z2 ..... Zn
where each data point is defined as zj (x)
(k)
Vj E Rd+1 We estimate the local density of cluster k
(k)
using KDE with a Gaussian kernel as follows:
1 n
fa(2)=
K 2, (M)),
(1)
n
where the Gaussian kernel K(-) is defined by:
K(u)= =
(2m) 1/2 exp
u - -uTH'u un:
u 14+1
Here, H is a (d+ 1) (d + 1) bandwidth (covariance) matrix that controls the shape and smoothness of the density estimate (Scott 2015). The bandwidth matrix H determines how closcly the estimated density follows the observed data points. A smaller delerminant |H| results in sharply peaked density estimates localized around data points, while a larger determinant, yields smoother densities.
Selecting an appropriate bandwidth matrix H is critical to KDE performance (Sheather & Jones 1991 If H is too small, KDE tends to overfit noise, resulting in unrealistic density estimates. Conversely, an excessively large [H] oversmooths important local features. Practical approaches such as cross-validan valsdavaii,,ion, plug-in estimators, generalized Silverman's rule for multivariate KDE can be employed independently for each cluster (Zhang et al., 2006). By fitting KDE independently to each cluster using carefully selected bandwidth matrices, create localized models that accurately represent each region's unique distribution.
8

---

Published in Transactions on Machine Learning Research (09/2025)
Original
Original Synthetic
PC3
PC3
PC1
PC1
PC2
PC2
Figure 6: Each cluster independently oversampled based on its own density estimate, ensuring that both sparse and dense areas adequately represented without altering their local characteristies; the same procedure is applied to the remaining clusters, with cluster 1 displayed here.
This ensures that synthotic data generated in the subsequent stops reflect genuine local patterns, effectively addressing imbalance without distorting critical data relationships.
4.4 Oversampling Clusters
Once the density of each cluster has been estimated, we independently oversample each cluster. Figure 0 illustrates this process. Each cluster k, originally containing ny points, expanded by generating synthetic data points until reaching a target size defined as:
n = [ag,,,
where the multiplier a is parameter of the algorithm that determines how much each cluster grows. We generate exactly n/g - synthetic points z" drawn from the local KDF-based density estimate fa The multiplier can be uniform (identical across clusters) adaptive (higher for clusters that are smaller less dense). Typically, smaller or sparser clusters receive a larger multiplier, ensuring balanced representation throughout the dataset.
Synthetic points z" are generated directly from the KDE-basod local distribution defined in Equation equa- tion II
z" z²~ fa(z).
For KDE with a Gaussian kernel using bandwidth matrix H, synthetic samples are generated by randomly selecting an existing cluster data point and adding a Gaussian perturbation scaled by the Cholesky decom- position the bandwidth matrix:
(K) //2 where j Uni.orm(1,.... nx e e (00,
z
Here, 1/2 is the matrix square root (typically via Cholesky decomposition) of the bandwidth covariance matrix H. This procedure accurately and efficiently samples synthetic points from the KDE estimate, reflecting the local structure and covariance of each Stuster(Silvennn 1986; Scott. 2015
4.5 Merging Augmented Clusters
After independently oversampling each cluster, the resulting augmented clusters are merged into a single balanced dataset, as illustrated in Figure 7 Formally, each augmented cluster is defined as:
D = DR U Di,
9

---

Published in Transactions on Machine Learning Research (09/2025)
Data Points
(onginal)
C uuste (orignna))
Custe foriginall
uster Eynnhetiic)
C Custtr lsyntheticl (synthetic)
PC3
PC2
Figure 7: Visualization of original (filled markers) and synthetic (outlined markers) points in each cluster after local oversampling. The merged dataset is augmented while preserving all original data.
where Dj represents the synthetic samples generated for cluster k. The final augmented dataset D is constructed by merging all augmented clusters:
D
U Dr. A=1
The size of each cluster after augmentation is explicitly given by:
|Dx| = n/g = [aynk].
Hence, the total size of the merged dataset becomes:
D
(nk + (nk n2)))
By carefully selecting the multipliers 0g; we precisely control the number of synthetic points contributed by each cluster. This yields an augmented representation across the entire range of frequent and rare target values. The complete LDAO procedure is presented in Algorithm [ which outlines all four steps of our proposed method.
5 Evaluation methodology
We evaluated LDAO with current state-of-the-art imbalanced regression approaches using diverse collection of benchmark datasets. This section details our experimental design, including dataset selection, bascline methods, implementation specifics, hyperparameter tuning, and validation procedures.
10

---

Published in Transactions on Machine Learning Research (09/2025)
Algorithm Local distribution based adaplive oversampling (LDAO)
1: Input: data {(T,, 1o)): Kmins Kmaxi factors }
Output: augmented set D
3: Embed 2; < (ri, ys)
4: for K = min to Kmax do
Cluster {2+} compute SSE(K) end for
Select K via elbow; recluster into {Dx) k²
8: for k: == 1 to K* do
9: Fit KDE f& on Dx
10: Sample [au [Da]] [Da] points from fa to form Dr
11: end for
K
12:
D
k=1 13: return D
Table 1: Dataset characteristics showing number of instances, features, and size category.
Dataset
Instances Features
Dataset
Instances Features Size
DEBUTANIZER
2394
Large
A2
198
11
Small
DEE
365
6 Small
A3
198
11
Small
DIABETES
43
Small
Small
ELE-1
Small
ABALONE
4177
Large
ELE-2
1056
Medium
ACCELERATION
1732
14
Medium
FORESTFIRES
517
12 Medium
AIRFOILD
Medium
FRIEDMAN
1200
Medium
ANALCAT
450
Small
FUEL
1764
37 Medium
AUTOMPG6
392
5
Small
HEAT
7400
11 Large
AUTOMPG8
392
Small
HOUSE
22784
16 Large
AVAILABLE
POWER
1802
Medium
KDD
316
18 Small
BASEBALL
337
16
Small
LASER
993
4 Medium
BOSTON
506
Medium
LUNGCANCER
442
24 Small
CALIFORNIA
20610
Large
MACHINECPU
209
Small
COCOMO
60
56
Smalll
1030
8 Medium
COMPACTI
8192
Large
META
528
65 Medium
MORTGAGE
1049
Medium
MAXIMAL TORQUE
1802
32 Medium
PLASTIC
1650
2 Medium
CPU
8192
12 Large
POLE
14998
26 Large
TRIAZINES
186
60 Small
QUAKE
Large
WANKARA
1609
Medium
SENSORY
576
11 Medium
WINE QUALITY
1143
12 Medium
STOCK
950
Medium
WIZMIR
1461
Medium
TREASURY
Medium
5.1 Datasets
We evaluated method using 45 datascts from three sources: the Keel repository (Alcalá-Fdez et al. 2011 the collection at httt::://pabraaooo.tttu..oo///
http:/ttttb..otttattttttt---taaa [Bancco et al. 2019 and the repository
IR
https://githie http:///waaaooooeaaaaaaaaaaaaaae
(Avelino et al. 2024
These datasets span multiple domains and vary in size, dimensionality, and degree of imbalance, making them standard benchmarks for the rigorous evaluation imbalanced regression methods and enabling fair comparisons with existing approaches. Table shows the number of instances and features for each dataset. For further analysis in the results section, we categorized the datasets based on their number of instances. Specifically, datasets with fewer than 500 instances labeled as 'Small', those with between 500 and 1999 instances as 'Medium', and those with more than 1999 instances as 'Large". This categorization facilitates detailed evaluation of our method's performance relative to datasct size.
5.2 Metrics
We evaluate LDAO against state-of-the-ars approaches using multiple metrics that measure performance both frequent and rare target values.
11

---

Published in Transactions on Machine Learning Research (09/2025)
5.2.1 Root Mean Square Error (RMSE)
Root Mean Square Error (RMSE) measures the overall prediction accuracy by calculating the square rool of the average squared difference between predicted values and actual observations:
RMSE
yt -;)
(2)
where yi represents the true target value and ØJ the predicted value for the i-th instance. RMSE provides a general assessment model performance but can be dominated by errors in densely populated regions.
5.2.2 Squared Error-Relevance Area (SERA)
This metric provides a flexible way to evaluate models under non-uniform domain preferences. Let c()): [0, 1] be a relevance function that assigns higher scores to more important (for example, rare or extreme) target values. Then for any relevance threshold let
D' = [(x,, vi) ((vi) 2 1},
and define
SER Σ
(0: (i)-
(3)
SERA then integrates this quantity over all I E [0,
SERA = [ SER, dt =
L''
Σ (FLA))E
(0: - 9i)² 2 dt.
(4)
SERA weights prediction errors by ((y)), emphasizing performance extreme values while still considering accuracy across the entire domain. This makes it well-suited for imbalanced regression, where predicting rare values accurately is crucial (Ribeiro & Moniz 2020
5.2.3 Mean Absolute Error (MAE)
Mean Absolute Error (MAE) is an alternative error metric that evaluates the overall prediction performance by averaging the absolute differences between the predicted and actual values:
MAE
94
Osl-
n
MAE less sensitive to outliers compared RMSE and provides a straightforward interpretation of the average prediction error.
5.3 Machine Learning Algorithms
Following the approach of (Steininger el al. 2021 we evaluated all methods using Multi-Layer Perceptron (MLP) with three hidden layers (10 neurons each) and ReLU activations. The output layer uses linear activation for regression. We trained models for 1000 epochs using Adam optimizer with carly stopping to prevent overfitting. We compared LDAO against four approaches: Baseline (no resampling, using the original imbalanced data), SMOGN (an extension of SMOTER that incorporates Gaussian noise during oversampling), G-SMOTE (Geometric SMOTE adapted for regression tasks, using geometric interpolation), and DenseLoss (a cost-sensitive approach that weights errors by target density). These methods represent the current state-of- state-of-the-art the-art method for handling imbalanced regression.
12

---

Published in Transactions on Machine Learning Research (09/2025)
Table Hyperparameter scarch spaces for each compared method
Method
Hyperparameter
Range
Description
LDAO
(candidate clusters)
[2. 6]
Number of clusters tested (optimal selected using elbow method)
Multiplier per cluster
[1.0, 3.0]
Oversampling factor
Bandwidth per cluster
(0.1, 2.0]
KDE smoothing parameter
SMOGN
k sampling method rel thres
(3. 5, 7, 9)
Number of neighbors
(extreme, balance)
Sampling approach
[0.0; 1.0]
Relevance threshold
G-SMOTE
rare truncation factor
[0.05, 0.25]
Quantile for rarity
[0.0, 0.9]
Geometric truncation
deformation factor
[0.0, 0.9)
Geometric deformation
k k_neigbbors oversampling
[2 5]
Number of neighhors
lactor
[1.0, 5.0)
Amount of oversampling
DenseLoss
alpha
[0.0, 2.0]
Density weighting factor
5.4 Implementation Resources
For our evaluation metrics, we utilized the SERA implementation from the Bebbalanceddeerunngggeeeee Python package [wu et al. 2022). The SMOGN method was implemented using the package developed by Kunz (Kunz) 2020). We implemented the DenseLoss and G-SMOTE methods based on their original papers, carcfully following the authors' descriptions and guidelines ensure faithful reproduction of their approaches. To optimize the hyperparameters for each method, employed the Opluna framework (Akiba et al. 2019 Optuna leverages Bayesian optimization using the Tree-structured Parzen Estimator (TPE) sampler, which efficiently balances exploration and exploitation during the search process. By tuning hyperparameters separately for each dataset, our approach accommodates the unique statistical properties and distribution characteristics inherent to each dataset, ensuring optimal performance for all methods.
5.5 Experimental Framework
We employed 5 runs of 5-fold voos-vaaldattion for all experiments. This outer fold cross-validation divided each dataset into five equal portions, with each fold using four portions (80% of data) for training and one portion (20% of data) as the test set. Each data portion served as a test set exactly once across the five folds in each run. For hyperparameter tuning within each fold, we further divided the training data into sub-training (80%) and validation (20%) sets. We utilized Bayesian optimization with 15 trials to efficiently search the parameter space, as it generally finds beiter hyperparameter values than a grid search while requiring fewer evaluations.
Table presents the hyperparameters and search ranges for each method. LDAO's parameters include the oversampling multiplier and KDE bandwidth. SMOGN uses neighborhood size, sampling approach, and relevance threshold. G-SMOTE involves quantile for rarity, truncation factor, deformation factor, number of neighbors, and oversampling factor. DenseLoss works with the density weighting parameter.
6 Results
We present the results of comparing LDAO with state-of-the-an oversampling methods across various perfor- mance dimensions. In Figure 8 conduct pairwise comparisons for each evaluation metric (RMSE, SERA, MAE) to demonstrale how LDAO performs relative to each competing method individually. RMSE and MAF evaluate overall model performance across the entire target distribution, while SERA places greater emphasis on errors associated with rare samples.
For each dataset, a winner is determined based on the outcomes over 25 folds, meaning that each dataset contributes one win, and there are 45 wins total. Specifically, within each dataset, the method that prevails in the majority of the 25 folds is declared the winner. This procedure repeated against each competing method. Next, perform the Wilcoxon Signed-Rand Rank Test to compare LDAO's performance vector with that of the competing mot Tn this test, we compute the differences between the 25-fold loss
13

---

Published in Transactions on Machine Learning Research /202)
LS40 Trai Win
LO40
42
38
MAE
27
SERA
30
10
15
33
RMSE
39
RMSE
24
7
14
LDAG
SMOGN
LDAD
BASELINE
(a) LDAO vS. SMOGN
(b) LDAO vs. Baseline
UMD
UM4 Rear Wia
Fcans
37
27
MAE
39
SERA
35
24
10
32
18
13
36
LDAD
G-SMOTE
LDAD
DENSELOSS
(c) LDAO vs. G-SMOTE
(d) LDAO vs. DenseLoss
Figure 8: Pairwise comparisons of LDAO with SMOGN, Baseline, G-SMOTE, and DenseLoss across RMSE, SERA, and MAE. Light-colored bars indicate total wins, while darker segments represent statistically sig- nificant wins (a = 0.05).
metric values of LDAO and those of the competing method to construct a difference vector. The Wilcoxon test then examines whether these differences are statistically significant, using an alpha level of 0.05, which indicates a 5% risk of incorrectly concluding that a difference exists when there is none (Wilcoxon 1945).
In Figure S the lighter color represents the total number wins for each method, while the darker portion indicates the number of wins that are statistically significant. The figure illustrates that LDAO outperforms all competing methods across all evaluated metrics. Although SMOGN exhibits lower performance in terms of RMSE and MAE, its SERA measurement shows significantly superior performance in more tests, aligning with its emphasis on rare samples; nevertheless, SMOGN remains inferior to LDAO overall. The other methods display more consistent performance across the different metrics.
Table 3: Mean rank and counts of best (rank 1) and worst (rank 5) across 45 datascts. The lowest mean ranks are highlighted in bold black, the highest rank 1 counts in bold green, and the highest rank 5 counts in bold red.
RMSE
SERA
MAE
Method
Mean
R1
Mean
R5
LDAO
2.151
27
4
2.491
25
2.001
29
2
BASELINE
2.817
1
3.106
2
2.843
1
DENSELOSS
3.118
3.353
G-SMOTE
3.286
SMOGN
3.749
22
2.998
10
3.936
28
Total
44
42
44
14

---

Published in Transactions on Machine Learning Research (09/2025)
RMSE
MAE
SERA
2133
12.
11.20
8.56
5000
4634,70-4
11.2%
11
8-
4500
4138.11
4332:54
10
Matric
7
4000
30.3%
1500
7,00
7.24
5.36
5.40
3000
3.5%
0.8%
2500
2113/07
23.3%
73.8%
54.2%
(a) RMSE- MeeSSn-Mediann
(b) MAE Median
(c) SERA-MErn
RMSE
MAE
SERA (Logu)
5500
5438.61 61
4500
4297.27
11.95
11.93
27.1%
4000
458.9%
11.90-
5000
4846.96
3583,94
11:85
Mebric
4500
4278.34
4325.
13.3%
3500
3094.80
3148.65- 115.8%
11,80
11.79
49.9%
3000
17.1%
4000
2500
3535,
2120,44
25:00
W1
4%
2
2000
5%
11:55
11:60
v32.2%
(d) RMSE ReMSE-Mean
(e) MAE
Mean
(I) SERA- Mean
Figure 9: Comparison of methods across median and mean values of RMSE, MAE, and SERA.
In Figure 9 we computed both the median and the mean of various metrics across all datasets for each method. In this analysis, the methods were evaluated collectively rather than in pairwise comparisons. The blue circle represents the baseline benchmark, and the percentages, whether negative or positive, indicate how each oversampling technique performs relative to this baseline.
LDAO again demonstrates robust performance and surpasses its peer methods even when all techniques are assessed logether. In each figure, the top three methods are highlighted, including the baseline. The only circumstance in which peer method, SMOGN, outperformed LDAO occurred when calculating the mean SERA value; in that specific case, SMOGN achieved slightly better results, although both methods signif- icantly outperformed the others. For the SERA mean computation across datasets, a base-10 10 logarithmic scale was employed to accommodate the high loss values observed on rare samples in some datasets, thereby enhancing clarity.
Table 3 presents the mean rank of each method for RMSE, SERA, and MAE across all 45 datasets. A Rank is the position given to method when methods are sorted by performance for given dataset and metric. Rank 1 for the lowest error (best) and Rank 5 for the highest error (worst). For each dataset and metric, a method's rank was first averaged over 25 cross-validation folds; these per-datasel ranks were then averaged across all datasets to produce the Mean column. The RI column indicates the number of datasets in which a method achicved the best (lowest) rank, and the R5 column indicates the number of datasets in which it achieved the worst (highest) rank. In cases where two or more methods shared virtually identical best or worst values, no count was assigned, so some column totals may sum to less than 45.
As shown in Table 3 LDAO outperforms the other methods, achieving the lowest mean ranks and the highest RI counts across RMSE, SERA, and MAE, while SMOGN exhibits the highest R5 counts for RMSE and MAF, indicating the weakest performance on those metrics. Baseline, DenseLoss, and G-SMOTE show intermediate performance, with G-SMOTE also recording a high R5 count in SERA.
Figure 10 offers an alternative view by displaying the mean ranking of each method as a function of dataset size. The rankings separate the data sets into small, medium and large based on the classifications previously given in table [ This evaluation aims to determine how oversampling methods perform across different dataset sizes, as some methods might be more effective on smaller datasets while others excel with larger ones.
15

---

Published in Transactions on Machine Learning Research (09/2025)
Large
3.88
3.67
4.0
3.69
3.40
434.9%
3.50
33.9%
35
A20.2%
3.5
3.0
2,80
2.92
1.6.10 1%
3.25
2.98
3.0
A14.7%
5,00
2.5
A1.5%
2.75
2.74
45.8%
2.0
1.92
2.50
2.41
1.30
Y33.2%
v12.0%
131.%
RMSE Small
(b) RMSE-MMeding
RMSE Large
Small
Medium
MAE
4.5
4.0
4,01
4.0
3,92
4.0
3.05
3.5
437.6%
49..6%
355
3.21
429.4%
3.0
2.77
2.91
220.8%
3.
7788%
3.0
2.5
74.9%
1.79
2.5
2.27
1.78
v38.5%
v16.9%
Y40.2%
(d) MAE-Smi
(e) MAE-Meding
(f) MAE Large
SERA Small
SERA Medium
SERA
3.4
3.30
3.25
3.75
3:65
3.14
45.7%
3.2
3.11
47.4%
350
112.2%
40.4%
3.0
2.99
3.03
42.0%
325
3.08
3.25
v1.2%
3.00
Y5-4%%
2.8
2.75
2.37
2.67
2.50
2.52
2.6
250
V26.0%
v13.4%
T23.1%
422.4%
(k) SERA Small
(h) SERA Medium
(i) SERA Large
Figure 10: Rank-visualizations for cach metric and datasct-size category. Letters denote methods: L-LDAO,
B-Baseline D=DENSELOOSS, G-G-SMOTE, S-SMOGN.
LDAO consistently outperformed competing methods, achieving the lowest mean ranking across all metrics and dataset sizes. The only case in which a competing method performed slightly better was for the SERA metric on large datasets, where SMOGN outperformed LDAO; however, LDAO remained superior for both small and medium datasets. In this figure, the blue circle represents the baseline model benchmark, providing reference for the performance of the various methods.
Table 4: Performance comparison across column are highlighted in bold dark green.
COIL-20, and Grocery. The best values in each
QSAR- SEAR-TID-1622
COIL-20
Grocery
Method
(466 samples, 1024 features)
(1440 samples, 1024 features)
(22500 samples, 870 features)
RMSE
lo (SERA)
MAE
RMSE
(SERA)
MAE
RMSE
log (SERA)
MAE
LDAO
0.92
0.86
0.64
1.68
0.63
0.96
655.53
8.37
495.74
Baseline
1.00
0.90
0.71
1.83
0.70
1.09
1479.03
8.93
1292.71
DenseLoss
1.03
0.86
0.73
1.81
0.70
1.10
1476.75
8.93
1290.42
G-SMOTE 0.98
0.84
0.69
1.81
0.69
1.03
1241.74
8.82
1037.22 24
SMOGN
0.98
0.84
0.69
1.83
0.69
1.08
843.49
8.41
672.88
We also evaluated LDAO on high-dimensional tabular datasets (QSAR-TID-1062) COIL-20) and on high- dimensional time-series data (Grocery with an LSTM). Table 4 shows that LDAO again achieves the lowest
16

---

Published in Transactions on Machine Learning Research (09/2025)
Table 5: Stratified performance comparison of LDAO against Bascline, DenseLoss, GSMOTE, and SMOGN across five larget bins (B1 sparsest to B5 densest). Bold green entries indicate wins for LDAO.
Comparison
B1 (Sparsest)
B2
B3
B4
B5 (Densest)
LDAO
Other
LDAO
Other
LDAO
Other
LDAO
Other
LDAO
Other
BASELINE
28
17
26
19
32
13
35
10
35
10
vs DENSELOSS
26
19
27
18
31
14
36
9
39
6
vs GSMOTE
24
21
31
14
37
36
34
vs SMOGN
24
21
26
19
33
12
RMSE, SERA, and MAE. In addition, following Steininger et al. (0021) we divide the target range into five equal-width bins, assign each sample to its bin, count and rank them by sample count (Bin sparsest, Bin 5 is densest), then evaluate performance within each bin to capture method behavior from rare frequent target regions. As shown in Table 5 LDAO achieves the most wins in bins B3-B5 and maintains leading accuracy in the sparsest bins B1 B2, despite SMOGN and G-SMOTE oulperforming it in 21 of 45 comparisons within B1-B2 This confirms that our method not only enhances predictions on rare samples but also significantly improves predictive accuracy in densely sampled regions, yielding robust performance across the entire target distribution.
7 Discussion
An interesting observation is the strong performance of the baseline model relative to the oversampling methods. This result suggests that oversampling techniques may underperform without meticulous fine- tuning informed by domain knowledge and underscores the inherent complexity of the imbalanced regression problem compared to classification. In many continuous oversampling approaches, samples are categorized as either rare or frequent; although such a biased assumption can sometimes assist the learning process, it may also lead to diminishod performance compared data-driven nature. Despile having a few parameters adjust, LDAO does not require extensive domain expertise, making it readily applicable to wide range of imbalanced datasets.
the basclinc. A key attribute of LDAO is its completely
Oversampling approaches have the benefit of increasing representation in sparse regions. By generating syn- thetic samples, these methods can help models learn better from rare target values and improve performance where data is limited. The process involves creating additional data points in underrepresented areas, al- lowing machine learning models to develop a more complete understanding of the entire target value range. For regression tasks with imbalanced distributions, these techniques provide a practical solution improve prediction accuracy across the entire spectrum of outcomes.
Specifically for LDAO, the approach leverages local clustering and density estimation to generate synthetic samples that more closely match the underlying data structure. LDAO aims to enhance model performance in regions where simpler approaches might struggle due to data scarcity. The method's focus on local density awareness makes it particularly useful for complex datasots with varying degrecs of sparsity. LDAO especially examine sparsity in the joint distribution of uras (X) and target. values (y), providing a more detailed and thorough understanding of where data is truly scarce in the multidimensional space.
Despite these advantages, oversampling methods encounter several limitations. They risk overfitting when the synthetic points are loo similar to existing samples and may introduce noise if they fail to accurately capture the underlying distribution. Moreover, these techniques typically require additional parameter tuning, such as selecting appropriate rarity thresholds and determining suitable oversampling factors. In LDAO's case, additional considerations include choosing the clustering parameters, specifying the number of synthetic samples, and configuring the density estimation settings. The reliance on k-means partitioning, which assumes roughly spherical clusters, can misassign regions with complex or irregular shapes, while per-cluster KDE bandwidth selection introduces parameter sensitivity and extra computational overhead compared to simpler oversampling schemes. Without proper calibration, synthetic samples might overfit local patterns
17

---

Published in Transactions on Machine Learning Research (09/2025)
or miss critical variations, thereby diminishing the overall effectiveness of the approach when the underlying data characteristios or sparsity patterns are not well captured by the chosen method.
8 Conclusion
In this work, we proposed LDAO, local bestrbbution-based adaptive oversampling method specifically designed to address the challenges of imbalanced regression. By modeling data in a joint feature-target space and generaling synthetic samples independently within identified clusters, LDAO effectively preserves the original dataset's statistical structure. Through comprchensive evaluation on a wide variety of datasets, we show that LDAO achieves strong and consistent predictive accuracy, frequently surpassing leading data-level and algorithm-lever approaches, particularly where data are sparse.
Oversampling methods, in general, have proven effective at handling imbalance by improving model repre- sentation in sparse areas, yet each method has its unique strengths and applications. LDAO contributes to this field by eliminating the need for predefined rarity thresholds and undersampling, thereby preserving valuable information and offering adaptive, data-driven augmentation. As shown our experiments, these attributes enable LDAO to maintain overall predictive accuracy while enhancing performance in challenging, underrepresented regions. Future work should continue to refine adaptive density-based sampling methods, particularly for datasets with complex, multimodal distributions.
References
Takuya Akiba, Shuji Sano, Takeru Yanase, Tetsuya Ohta, and Masanori Koyama. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD International Con- ference on Knowledge Discovery and Data Mining, pp. 2623-2631, 2019.
Jesús Alcalá-Fdez, Alberto Fernandez, Julián Luengo, Joaquín Derrac, Salvador Gareía, Luis Sánchez, and Francisco Herrera. Keel data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework. Journal of Multiple- Valued Logic and Soft Computing, 17(2 3):255 287, 2011.
Jelena Aleksic and Miguel Gareia-Remesal. A selective under-sampling (sus) method for imbalanced regres- sion. Journal of Artificial Intelligence Research, 82:111-136, 2025.
Miguel Arteaga-Jover, Miguel-Jesús Gacto, María Galende, José Acclá--Fdez, and Rafael Alcalá. Enhancing soft computing techniques to actively address imbalanced regression problems. Expert Systems With Applications, 234:121011, 2023.
Jusci G. Avelino, George D. C. Cavalcanti, and Renato M. O. Cruz. Resampling strategies for imbalanced regression: survey and empirical analysis. Artificial Intelligence Review, 57:Article 82, 2024.
Paula Branco, Luis Torgo, and Rita P. Ribeiro. A survey of predictive modeling under imbalanced distribu- tions. ACM Computing Surveys, 49(2)cetticle 31, 2016.
Paula Branco, Luis Torgo, and Rita P. Ribeiro. Smogn: A pre-processing approach for imbalanced regression. In Proceedings of Machine Learning Research: LIDTA, volune 74, pp. 36 50, 2017.
Paula Branco, Luis Torgo, and Rita P. Ribciro. Pro-processing approaches for imbalanced distributions in regression. Neurocomputing, 343:76 99, 2019.
Mateusz Buda, Atsuto Maki, and Macicj A. Mazurowski. A systematic study of the class imbalance problem in convolutional neural networks. Neural Networks, 106:249 259, 2018.
Chumphonk Bunkhumpornpat, Krerk Sinapiromsaran, and Chumpolphon Lursinsap. Dbsmote: Density- based synthetic minority over-sampling technique. Applied Intelligence, 36(3):664 684, 2012.
Luis Camacho and Fernando Bacao. Wsmoter: A novel approach for imbalanced regression. Applied Intelli-
gence, $4:8789-8700 2024.
18

---

Published in Transactions on Machine Learning Research (09/2025)
Luis Camacho, Georgios Douzas, and Fernando Bacao. Geometric smote for regression. Erpert Systems with Applications, 193:1 183:1, 2022.
Nitesh V. Chawla, Kevin W. Bowyer, Lawrence O. Hall, and William Philip Kegelmeyer. Smote: Synthetic
minority over-sampling technique. Journal of Artificial Intelligence Research, 16:321 357, 2002.
Nitesh V. Chawla, Nathalic Japkowicz, and Aleksander Kolez. Editorial: Special issue on learning from imbalanced data sets. ACM SIGKDD Eaplorations Newsleller, 6(1):1-6, 6, 2004.
Pedro Domingos. Metacost: A general method for making classifiers cost-sensitive. In Proceedings of the 5th ACM SIGKDD International Conference Knowledge Discovery and Data Mining (KDD), pp. 155 164, 1999.
Zijian Dong, Yilei Wu, Chongyao Chen, Yingtian Zou, Yichi Zhang, and Juan Helen Zhou. Improve rep- resentation for imbalanced regression through geometric constraints. In Proceedings of the IEFE/CVF Conference on Computer Vision and Pattern Recognition. Computer Vision Foundation, 2025.
Charles Elkan. The foundations of cost-sensitive learning. In Proceedings of the 17th International Joint Conference Artificial Inlelligence (IJCAT), pp. 973 978, 2001.
Haixiang Guo, Vijing Lá, Jennifer Shang, Mingyun Gu, Yuanyue Huang, and Bing Gong. Learning from class-imbalanced data: Review of methods and applications. Expert Systems with Applications, 73:220 239, 2017.
Hui Han, Wen- Yuan Wang, and Bing-Huan Mao. Borderline-smote: A new over-sampling method in imbal- anced data sets learning. In Proceedings of ICTC 2005, pp. 878 887. Springer, 2005.
David Harrison and Daniel L. Rubinfeld. Hedonic prices and the demand for clean air. Journal of Environ- mental Economics and Management, 5(1):81-102, 1978.
Haibo He and Edwardo A. Garcia. Learning from imbalanced data. IEEE Transactions on Knowledge and Dala Engineering, 21(9):1263 10s1223312888 16841284, 2009.
Haibo He and Yunqian Ma. Imbalanced Learning: Foundations, Algorithmss, and Applications. John Wiley & Sons, 2013.
Haibo Hc, Yang Bai, Edwardo A. Garcia, and Shutao Li. Adasyn: Adaptive synthetic sampling approach for imbalanced learning. In Proceedings of the International Joinl Conference on Neural Nelworks, pp. 1322 1328, 2008.
Terrance R. Hoens and Nitesh V. Chawla. Imbalancod datasots: From sampling to classifiers. In Imbalanced Learning: Foundations, Algorithms, and Applications, pp. 43-59. Wiley, 2013.
Abiodun Moturrayo Ikotun, Absalom Emmanuel Ezugwu, Laith Abualigah, Bassam Abuhaija, and Heming Jia. K-means clustering algorithms: A comprehensive review, variants analysis, and advances in the era of big data. Information Sciences, 622:178-21 210, 2023. doi: 10.1016/j.ns.22222111.33
Justin M. Johnson and Taghi M. Khoshgoftaar. Survey on deep learning with class imbalance. Journal of Big Dala, 6(1):1 54, 2019.
Bartosz Krawezyk. Learning from imbalanced data: Open challenges and future directions. Progress in Artificial Intelligence, 5(4):221-23) 2016.
Niklas Kunz. Smogn: Synthotic minority over-sampling technique for regression with gaussian noise. PyPI, Version 0.1.2, 2020.
Xiao- Yong Liu, Junhua Wu, and Zui-Hua Zhou. Exploratory undersampling for class-imbalance learning. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cyberneties). 39(2):533-5a) -5-53,995,,,, 2009.
19

---

Published in Transactions on Machine Learning Research (09/2025)
Nuno Moniz, Luis Torgo, and Carlos Soares. Smoteboost for regression: Improving the prediction of ex- treme values. In Proceedings of the 5th International Conference on Dota Science and Advanced Analylics (DSAA), pp. 127-136, 2018.
Rahmat Nainggolan, Ritha Perangin-Angin, Enda Simarmata, and Achmad Tarigan. Improved the perfor- mance of the K-means cluster using the sum of squared error optimized by the clbow method. In Journal of Physics: Conference Series, volume 1361, pp. 012015, 2019. doi: 17788//7716666/////
(11122-6566136111 / /1/110015.
Ismail Nejjar, Faez Ahmed, and Olga Fink. Im-context: In-context learning for imbalanced regression tasks. arviv preprint ar Xiv:224 1 18202, 2024. doi: (0.48550/arXiv.24066 18202.
Ruizhi Pu, Gezheng Xu, Raiyi Fang, Bing- Kun Bao, Charles Ling, and Boyu Wang. Leveraging group clas- sification with descending soft labeling for deep imbalanced regression. Tn Proceedings of the Thirky-Nmtth AAAI Conference on Artificial Intellignnce Association for the Advancement of Artificial Intelligence, 2025.
Mengye Ren, Weifeng Luo, and Raquel Urtasun. Balanced MSE for imbalanced visual regression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 118-427, 2022.
Rita P. Ribeiro and Nuno Moniz. Imbalanced regression and extreme value prediction. Machine Learning. (0099100)1800-1888 2020.
Rita Patrícia Almeida Ribeiro. Utility- Based Regression. PhD thesis, Faculty of Sciences, University of Porto, Porto, 2011.
David W. Scott. Multicariate Density Estimation: Theory, Practice, and Visualization. Wiley Series in Probability and Statistics. John Wilcy & Sons, Hoboken, New Jersey, 2nd edition, 2015.
Simon J. Sheather and Michael C. Jones. A reliable data-based bandwidth selection method for kernel density estimation. Journal of the Royal Statistical Society. Series B (Methodologicca)) 5383)6683-690, 1991.
Bernard W. Silverman. Density Estimation for Statistics and Data Analysis. Monographs on Statistics and Applied Probability. Chapman and Hall, London, 1986.
Markus Steininger, Kevin Kobs, Pascal Davidson, Andreas Krause, and Andreas Hotho. Density- based weighting for imbalanced regression. Machine Learning, 110(8))2177-2210, 2021.
Steffen Stocksieker, Didier Pommeret, and Anne Charpentier. Data augmentation for imbalanced regression. In Proceedings of the 26th International Conference on Artificial Intelligence and Statistics (AISTATS), 2023.
Steffen Stocksicker, Didicr Pommeret, and Anne Charpentier. Generalizod oversampling for lcarning from imbalanced datasets and associated theory: Application in regression. Transactions on Machine Learning Research, 6, 2024.
Catherine A. Sugar and Gareth M. James. Finding the number of clusters in a dataset: An information- theoretic approach. Journal of the American Statistical Association, 98(463):750-700 2003.
Muhammad Syakur, Budi Khotimah, Elly Rochman, and Bakhtiar Satoto. Integration K-means clustering method and elbow method for identification of the best customer profile cluster. In I0P Conference Series: Materials Science and Engineering, volume 336, pp. 012017, 2018. doi: 10.1088/ 11108817577-99x /336/ /
Luis Torgo and Rita P. Ribeiro. Utility-basee based regression. In Proceedings of PKDD 2007, pp. 597 604. Springer, 2007.
Luis Torgo, Rita P. Ribeiro, José Pedro da Costa, and Susmita Pal. Smote for regression. In Intelligent Data Engineering and Automated Learning (IDEAL 2013). Lecture Notes in Computer Science, volume 8206,
pp. 378 387, 2013.
20

---

Published in Transactions on Machine Learning Research (09/2025)
Luis Torgo, Paula Branco, Rita P. Ribeiro, and Bernhard Pfahringer. Resampling strategies for regression. Expert Systems, 32(3):465-476, 2015.
Xiaoxiang Wang, Jianfeng Xu, Ting Zeng, and Le Jing. Local distribution-based adaptive minority over- sampling for imbalanced data classification. Newrocomputingg 2020. doi: (.000//..eeccom20022 202..00.20000000
Frank Wilcoxon. Individual comparisons by ranking methods. Biometries Bulletin, 1(6):80 83, 1945.
Wenzhuo Wu, Niklas Kunz, and Paula Branco. Inbalaaceelennnnngggeessoon a python package tackle the imbalanced regression problem. In Joint European Conference on Mochine Learning and Knowledge Discovery in Databases, pp. 645-648, 2022.
Xiang Xiang, Feng Wang. Yuwen Tan, and Alan L. Yuille. Imbalanced regression for intensity series of pain expression from videos by regularizing spatio-temporal face nets. Pallern Recognation Lellers, 163: 152 152-158, 2022.
Haipeng Xiong and Angela Yao. Deep imbalanced regression via hierarchical classification adjustment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Computer Vision Foundation, 2024.
Jing Yang, Li Xie, Qi Yu, Xiao He, and Jie Liu. Delving into deep imbalanced regression. In Proceedings of the 38th International Conference on Machine Learning (ICML), pp. 8437 8447, 2021.
Hong Yin and al. Segment imbalance regression for carbon emissions trading price forecasting. Applied Energy, 377:124357, 2025.
Xin Zhang, Maxwell L. King, and Robert J. Hyndman. A bayesian approach to bandwidth selection for multivariate kernel density estimation. Computational Statistics g Data Analysis, 50(11):3009-333) 2006. 2006.
Zhibin Zhao, Bingehen Liu, Fujin Wang, Shiyu Zheng, Qiuyu Yu, Zhi Zhai, and Xuefeng Chen. Exploration of imbalanced regression in state-of-health estimation of li-ion batteries. Journal of Energy Slorage, 105: 114542, 2025.
Zhi- Hua Zhou and Xiao Yong Liu. On multi-class cost-sensitive learning. Computational Intelligence, 26(3): 232-257, 2010.
Appendix A: Comparison of Statistical Properties of Synthetic Data
Table 6: Distributional statistics, including mean, standard deviation, skewness, and kurtosis, for each feature and the target variable, comparing the original sparse region (samples for the 2 75th-percentile tail) to LDAO and SMOGN synthetic samples in the Boston dataset. Values closest to the original distribution are highlighted in bold green.
Mean LDAO
Skewness LDAO
Kurtosis LDAO
Feature
Orig
SMOGN
Orig
LDAO
SMOGN
Orig
SMOGN
Orig
SMOGN
CRIM
-0.314
-0.258
0.024
0.268
0.430
0.104
4.144
2.707
4.403
18.445
6.118
18.642
ZN
0.568
0.452
0.941
1.329
1.307
1.328
1.087
0.992
1.045
-0.111
-0.495
-0.687
INDUS
-0.678
0.803
1.415
1.337
0.672
0.940
CHAS
0.205
0.102
0.449
1.291
1.238
1.151
2.321
2.576
2.341
3.388
4.954
3.682
NOX
-0.061
-0.246
0.190
0.688
0,911
0.286
-1.768
-1.244
1.192
4.101
1.197
-0.281
RM
0.210
-0.298
0.756
1.253
1,383
0.412
-1.839
-1.238
-0.519
1.849
-0,223
-0.505
AGE
.0.474
-0.555
0.237
1.026
1.144
0.385
0.038
-0.092
1.290
-1.342
-1.330
0.012
DIS
0.219
0.093
0.576
L.733
1,259
0.710
0.517
0.583
0.972
0,213
0.278
-0.240
RAD
-0.385
-0.322
0.126
0.655
0.791
0.421
2,466
1.803
3.155
5.133
2.089
8.205
TAX
PTRATIO
-0.707
-0.485
0.066
1.004
0.926
0.197
-0.333
-0.304
3.150
-0.762
-0,220
8.632
0.164
0.122
0.541
0.912
0,964
0.228
-1.499
-1.177
-1.791
0.182
-0.281
1.420
LSTAT
-0.912
-0.597
-1.123
0.425
0.553
0.240
1.732
1,251
0.864
4.540
1.125
-0.154
target
34.638
28.409
45.990
8.017
1.620
13.949
0.808
-0.140
1.304
-0.611
-0.986
0.298
21

---

Published in Transactions on Machine Learning Research (09/2025)
Table [ compares the first four moments of each feature and the target between the original data and the synthetic samples generated by LDAO and SMOGN. Values closest to the original distribution are highlighted in green.
LDAO matches the original means for 13 out of 14 entries, while SMOGN's means deviate in 13 out of 14 entries. The standard deviations produced by LDAO have an average absolute deviation of 22.1% from the original values (2 out of 14 within =5%), indicating moderate variance prescrvation, whereas SMOGN underestimates variability in several features. In terms of skewness, LDAO retains the asymmelry in 10 out of 14 distributions, whereas SMOGN's samples produce the closest skewness in only 4 cases. For kurtosis, LDAO reproduces the tail behavior in 12 out 14 cases, while SMOGN froquently yiclds inflated or deflated values because its Gaussian- injection fails to capture higher-order moments.
These results directly address the gap of incorrect statistical properties: by preserving means, variances, skewness and kurtosis, LDAO avoids the distortions introduced by linear interpolation. Maintaining accurate marginal statistics is prerequisite for faithful joint feature- target distributions and for conserving inter- feature correlations, setting the stage for improved overall data fidelity.
Appendix B: Feature-wise Distribution Comparison: Original vs. LDAO vs. SMOGN for Sparse Region
Figure compares the distribution of each feature and target between the original minority data (cyan, samples above the 75th-percentile), LDAO synthetic data (green) and SMOGN synthetic data (red). Each subplot shows both histogram bars and smooth density curves to highlight how closely each method matches the real data. LDAO closely follows the shape and spread of the original minority samples in nearly every feature. It captures multimodal patterns, skewed tails and the full range of values seen in the real data. In contrast, SMOGN tends to produce narrow, overly regular distributions. Its samples often cluster around central regions and fail to reproduce the true variability of features.
Overall, this analysis shows that LDAO preserves the true characteristics of rarc, high-value samples without flattening narrowing their distributions. SMOGN's linear-interpolation approach, by comparison, smooths away important irregularities and underestimates the natural spread of the minority data.
Appendix C: Kernel Density Analysis of Target Distribution Preservation
Figure 12 presents density for the target variable, comparing the original minority samples (cyan), LDAO synthetic data (green) and SMOGN synthetic data (red). LDAO closely follows the original density curve across the full target range. It captures the bimodal shape and maintains the correct spread of values, producing synthetic samples that vary naturally and cover all regions seen in the real data. SMOGN, by contrast, concentrates samples in a few intervals and misses other regions entirely. Its synthetic density fails to match the real peaks and troughs of the minority distribution. These results show that LDAO more faithfully preserves the true target distribution, which is crucial for generating reliable synthetic data in imbalanced regression tasks.
Appendix D: Additional Related Work
Farly approaches to imbalanced data focused on data-level methods, particularly in classification, and data- level strategies appear in nearly one-third of imbalanced classification papers, though most were designed for discrete classes (Guo et al. 2017). These methods rebalance datascts by cither adding minority samples or removing majority samples (Chawla et al. 2002 He et al.) 2008). SMOTE exemplifies this approach by generating synthetic minority samples through interpolation in feature space (Chawla et al., 22002). It works by selecting minority sample, finding its k-nearest minority neighbors, and creating new points by randomly interpolating between their feature values. This technique helps distribute minority samples more evenly throughout the data space.
22

---

Published in Transactions on Machine Learning Research (225)
(a) CRIM
(b) ZN
(c) INDUS
(d) CHAS
(e) NOX
(f) RM
(g) AGE
(h) DIS
(1) RAD
1h0
Pitamic
(i) TAX
(k) PTRATIO
B
(m) LSTAT
(n) target
Figure 11: Each subplot shows the distribution comparison between the original sparse region (cyan, samples for the > 75th-percentile tail), LDAO synthetic data (green), and SMOGN synthetic data (red) for the respective feature in the Boston dataset.
23

---

Published in Transactions on Machine Learning Research (09/2025)
Original Ninority KDE LOAd Synthetic KGC
Original Minertty KDG
SHOON Synthetic KGC
004
624
600
64
Target Value
Target Value
Target Density: Original vs. LDAO
(b) Target Density: Original vs. SMOGN
Figure 12: Kernel density estimation analysis comparing target distribution preservation between original sparse region (cyan, samples for the 2 75th-percentile tail) and synthetic data generated by LDAO (green) and SMOGN (red) in the Boston dataset.
Several SMOTE variations have emerged address specific challenges. Borderline-MMOTE focuses on minority samples near class boundaries (Han al.] 2005), while DBSMOTE leverages local density infor- mation to generate synthetic points in regions where minority classes concentrated (Bunkhumporapat
et al. 2012). These methods aim to improve minority class coverage and enhance classifier performance imbalanced datascts.
Additional regression oversampling approaches include simple random oversampling (replicating high-ø ex- amples) and pure noise-based oversampling (adding small random perturbations to rare examples) (Branco ct al. 2019). Another method, WERCS (Weighted Relevanco- Combination Strategy), brobabilisticall selects instances for oversampling or undersampling based on relevance scores: rare instances have higher chances of duplication while common instances typically undersampled unless kept for diversity. This unified approach blends techniques to avoid hard threshold cutoffs (Branco et al. 2019).
Ensemble learning has been explored for imbalanced regression challenges. While boosting and bagging improve model generalization by combining multiple learners, they don't 't directly address skewed distributions (Hoens& Chawla 2013). Therefore, researchers typically combine ensemble methods with specific inbalance- countering techniques. One effective approach applies resampling within each ensemble iteration. Resampled Bagging, for instance, uses balanced subsets weighted samples in each bootstrap replicate, preserving diversity across learners while reducing bias toward majority targets (Branco et al. 2019
2018 extended boosting for rare targets through SMOTEBoost for regression, which generates synthetic samples in extreme target ranges and then uses boosting to emphasize these difficult-to-predii points This combination of oversampling and ensemble methods proves espocially valuable when models need to accurately capture lail values or outliers that standard regression approaches frequently miss.
Moniz et al.
Evaluation metrics for imbalanced regression have evolved alongside data-level, algorithmic-level, and ensemble-based solutions. Traditional metrics like MSE MAE are often dominated by errors frequent cases, prompting the development of specialized metrics that better assess performance on rare targets. The SERA metric (Squared Error Relevance Area) exemplifies this approach by incorporating relevance into evaluation, weighting errors according to the rareness of target values (Ribeiro & Moniz, 2020). Ribeiro and Moniz argue that SERA offers more informative model comparisons in imbalanced domains by highlighting performance on extreme values relative to common ones (Ribeiro Moniz, 22020). While SERA itself is an evaluation metric rather than a training method, its development reflects the field's increasing focus on accurately assessing how well models handle rare events.
24


Reviews:
[{"review_id": "fCFqy4b58p", "reviewer": "Reviewer_aaGN", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n1. This paper addresses a practically important but relatively underexplored problem: imbalanced regression.\n\n2. The proposed method is simple, intuitive, and shows good performance across 45 regression datasets.\n\n\nWeaknesses\n1. All datasets used in the experiments are of relatively low dimensionality (mostly < 20 features, none exceeding 65). This raises concerns about the method's applicability to real-world, high-dimensional settings.\n\n2. Several claims made against previous work are either imprecise or misleading. For example: The paper criticizes earlier studies for \"oversimplifying the complexity of continuous distributions\" but fails to state clearly on what specific aspects of the complexity are being ignored. It also states that previous methods \"distort the nature of the dataset\", while the whole principle of imbalance handling is inherently a kind of distortion to better suit learning objectives. These statement could mislead readers unfamiliar with the literature.\n\n3. While the paper criticizes previous work for being oversimplified, it essentially follows the same high-level idea — dividing the space into dense and sparse regions, and oversampling the sparse ones. This framing is not fundamentally different from what has been done before. What’s more confusing is that the method also oversamples in dense regions. This is counterintuitive — in class imbalance problems, we usually avoid oversampling the majority class. Although the rate is low, it’s not clear why any oversampling should be done there at all to tackle the imbalanced problem. The paper doesn’t provide a convincing explanation.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses the problem of imbalanced regression by proposing a simple yet intuitive oversampling method based on joint feature-output clustering and kernel density ratio estimation. The central idea is to identify dense and sparse regions in the joint input-output space, then apply region-specific oversampling, with more samples generated in sparse areas and fewer in dense ones. The contribution of the paper is\n1. Proposes a region-aware oversampling method for imbalanced regression based on joint feature-output clustering.\n2. Estimates kernel density ratios to guide adaptive oversampling in dense vs. sparse regions.\n3. Demonstrates effectiveness across 45 low-dimensional regression datasets with consistent improvements.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Discuss whether the method can be applied to high-dimensional cases. If possible, provide related test to show scalability.\n2. Please be specific when criticizing existing works. What exactly is being oversimplified? What part of the \"nature\" is being distorted? \n3. Please explain why you are oversampling in dense regions. In class-imbalanced learning, we usually avoid oversampling the majority class — is your setting different, or is this a design compromise?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "y8xnORQV3y", "reviewer": "Reviewer_LLbv", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The paper introduces an intuitive approach that eliminates the need for arbitrary global thresholds. \n2. The comprehensive experimental evaluation covers 45 datasets.\n3. The method preserves local statistical structure within each cluster by modeling them independently.\n\nWeaknesses:\n\n1.  The literature review overlooks several recent and highly relevant contributions from 2024–2025, [1-6], making it difficult to contextualize LDAO within the modern landscape of imbalanced regression research. No comparison to these recent approaches is made.\n\n2.  The evaluation does not analyze performance separately for rare versus frequent target regions. Reporting aggregate metrics such as RMSE and MAE without stratified breakdowns makes it unclear whether improvements benefit rare regions or are driven by majority cases.\n\n3. The introduction requires substantial reworking as it fails to establish clear research gaps beyond vague statements like \"current solutions remain limited.\" The motivational examples in wind forecasting, healthcare, and other domains lack proper citations and context, weakening the paper's foundation.\n\n4. The paper provides no theoretical foundation, lacking analysis of convergence properties, optimality conditions, or theoretical guarantees about when and why LDAO should work effectively.\n\n5. The method is demonstrated only on tabular datasets. There is no discussion or exploration of how LDAO might extend to structured data types such as images, time series, or text—modalities where imbalanced regression is also relevant.\n\n6. The method assumes k-means clustering is appropriate for the joint feature-target space without theoretical or empirical justification. This assumption may fail in high-dimensional settings, yet no sensitivity analysis or ablation study assesses the impact of this design choice.\n\n7. Figure 1's contrast between classification and regression imbalance oversimplifies both domains and ignores the substantial overlap. Long-tailed classification problems share similar distributional challenges with regression imbalance\n\n[1] Dong et al. (2025): \"Improve Representation for Imbalanced Regression through Geometric Constraints\" (CVPR 2025)\n\n[2]  Xiong & Yao (2024): \"Deep Imbalanced Regression via Hierarchical Classification Adjustment\" (CVPR 2024)\n\n[3] Nejjar et al. (2024): \"IM-Context: In-Context Learning for Imbalanced Regression Tasks\" (TMLR)\n\n[4] Luo et al. (2024): \"Revive re-weighting in imbalanced learning by density ratio estimation\" (NeurIPS)\n\n[5] Stocksieker et al. (2024): \"Generalized Oversampling for Learning from Imbalanced datasets\" (TMLR)\n\n[6] Pu et al. (2025): \"Leveraging Group Classification with Descending Soft Labeling for Deep Imbalanced Regression\" (AAAI)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces LDAO (Local Distribution-based Adaptive Oversampling), a novel method for imbalanced regression that targets the challenge of predicting rare continuous target values. Unlike traditional approaches that rely on global thresholds, LDAO performs local modeling by clustering the joint feature–target space using k-means. Within each cluster, kernel density estimation is applied to generate synthetic samples that reflect local data distributions. The method is extensively evaluated on 45 datasets, demonstrating superior performance compared to baselines such as SMOGN, G-SMOTE, and DenseLoss across RMSE, MAE, and SERA metrics.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see weaknesses", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "JCsIqtysFP", "reviewer": "Reviewer_EjBV", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n* The paper is generally well written; the structure is clear, the procedure of the proposed method is clearly explained. \n* Experiments show that the proposed method outperforms existing methods in many settings.\n\n## Weakness\n* The related works takes up almost two pages, which is 12 percent of the main text. This lengthy part decrease the readability, although this could be useful for studying imbalanced regression problem.\n* The limit of the proposed method is ambiguous, although the possibility of the overfitting in density estimation is discussed in Section 6.  For example, it is difficult to understand what limitations of existing methods the proposed method is expected to solve, and under what conditions it fails to improve performance.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work proposes a new data-level method termed LDAO, for regression tasks in imbalanced data. The method is based on estimating the density in the space of feature-response pairs using a mixture of Gaussian distributions based on the K-means clustering, and then resampling from there. The performance of the proposed method is quantitatively compared with other data-level approaches and baseline, using 45 benchmark dataset. The result show that the proposed method outperforms existing methods in a wide range of tasks.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* The related works sections is a bit like a review. I suggest that the only the works closely related to the proposed method be left in the main text, and the detailed reviews be moved to the appendix as \"further related works\".\n* Please add more detailed discussion of the limitations of the proposed method. For example, density estimation is generally difficult in high-dimensional spaces, but does the proposed method work in such a situation? Also, in the experiment with 45 types of data, the highest score was not achieved in all settings (and this is quite natural and not a drawback), but in what settings was the method inferior to other methods?", "broader_impact_concerns": "Data imbalance is an important topic in statistics. However I do not think that this paper requires adding a Broader Impact Statement as it does not involve verification using data that has as significant social impact.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)