Entropy-Calibrated Label Distribution Learning
Authors: Yunan Lu
Bowen Xue
Xiuyi Jia
Lei Yang
Venue: NeurIPS 2025 poster ()
Keywords: label distribution learning
learning with ambiguity
entropy calibration
Primary area: general_machine_learning

Abstract: Label Distribution Learning (LDL) has emerged as a powerful framework for estimating complete conditional label distributions, providing crucial reliability for risk-sensitive decision-making tasks. While existing LDL algorithms exhibit competent performance under the conventional LDL performance evaluation methods, two key limitations remain: (1) current algorithms systematically underperform on the samples with low-entropy label distributions, which can be particularly valuable for decision making, and (2) the conventional performance evaluation methods are inherently biased due to the numerical imbalance of samples. In this paper, through empirical and theoretical analyses, we find that excessive cohesion between anchor vectors contributes significantly to the observed entropy bias phenomenon in LDL algorithms. Accordingly, we propose an inter-anchor angular regularization term that mitigates cohesion among anchor vectors by penalizing over-small angles. Besides, to alleviate the numerical imbalance of high-entropy samples in test set, we propose an entropy-calibrated aggregation strategy that obtains the overall model performance by evaluating performance on the low-entropy and high-entropy subsets of the overall test set separately. Finally, we conduct extensive experiments on various real-world datasets to demonstrate the effectiveness of our proposal.

Entropy-Calibrated Label Distribution Learning
Yunan Lu Department of Computing
Bowen Xue
Department of Computing
The Hong Kong Polytechnic University Hong Kong, China yunan 1u@polyu. edu.
The Hong Kong Polytechnic University Hong Kong, China bowen. www.enxxeecoonect.poo polyu. hk
Xiuyi Jia
Lei Yang
School of Computer Science and Engineering Nanjing University of Science and Technology Nanjing, China
Department of Computing
The Hong Kong Polytechnic University Hong Kong, China
j jiaxynjjsst.ert edw.:uuttteuuu..........o.
ray www.aaaalluuuuuuu.....
Abstract
Label Distribution Learning (LDL) has emerged powerful framework for estimating complete conditional label distributions, providing crucial reliability for risk-sensitive
tasks. While existing LDL algorithms exhibit com- petent performance under the conventional LDL performance evaluation methods, two key limitations remain: (1) current algorithms systematically underperform on the samples with low-entropy label distributions, which can be particularly valuable for decision making, and (2) the conventional performance evaluation methods are inherently biased due to the numerical imbalance of samples. In this paper, through empirical and theoretical analyses, we find that excessive cohesion between anchor vectors contributes significantly to the observed entropy bias phenomenon in LDL algorithms. Accordingly, we propose an inter-anchor angular regularization term that mitigates cohesion among anchor vectors by penalizing over-small angles. Be- sides, to alleviate the numerical imbalance of high-entropy samples test set, we propose entropy caarbreee aggregation strategy that obtains the overall model performance by evaluating performance on the low-entropy and high-entropy sub- sets of the overall test set separately. Finally, conduct extensive experiments various real-world datasets to demonstrate the effectiveness of our proposal.
Introduction
Accurately estimating the entire conditional distribution of labels (a.k.a. the label distribution) according to a set of feature variables, beyond merely the mean mode of the distribution, is receiving increasing attention both in field of statistics and machine learning 4. 28), as information about the entire distribution crucial in scenarios that sensitive risk, extremes, or uncertainty. To achieve this goal, researchers have developed various kinds of techniques, such as model calibration [27] [29] and mixture density neural network. These techniques aim to estimate the true label distributions using the training samples that are labeled only with the mean or mode of the underlying true label distribution, which are beneficial in practical tasks where the true label distributions are hardly available. However, there remain a large number of real-world scenarios where the true label distributions readily available. For example, in rating prediction tasks crowd-sourced learning, the labeling results for a given sample usually can be normalized as the
"Corresponding Author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Emotion6
Music-Mood
KL Prediction Error
2.0
Entreny Region
High-Fnthogy Region
0.06
KL. (L-Based Prediction Error
Lew-Entropy Regron
High-tnirups Region
samples:
aimpler
0.05
0.5
143
of samipks Prodiction srree 6.091
0.06
Prediction
Preduction
Prediction ertor 0.219
1.5
0.04
0.4
0.04
1.0
0.03
0.3
0.02
0.2
0.01
Cverral prodiction erroe by current ovaluation: 0.620
0.00
Overall prediction acror curvent ovalus
0.00
0.0
0.0
1.25
1.75
2.0
Entropy
Entropy
Figure I: Distributions of the prediction error and the sample frequency w.r.t. the label distribution entropy Emotion6 [25] and Music Mood [13] datasets. Prediction error measured by Kullback- Leibler (KL) divergence. Prediction error and sample density are denoted by blue and orange histograms, respectively. Each subfigure is partitioned into low-entropy and high-entropy regions by vertical dashed line. The horizontal yellow dashed line in each region denotes the average prediction error of the test samples within that region. The horizontal green dashed line across the two regions denotes the overall prediction error of the test samples according to conventional evaluation methods.
proportion of participants who give each label or rating [2]: drug efficacy prediction tasks where drug efficacy is quantified by the concentrations of drug in the blood, the blood-drug concentration different time points can be easily summarized as a drug concentration distribution by kernel density estimation [14]. The problem of learning samples with true label distributions is called Label Distribution Learning (LDL) []] Compared to the cases without true label distributions, LDL is capable of predicting the entire conditional distribution of labels more accurately, as it is directly supervised by the true label distributions.
Although existing LDL algorithms demonstrate strong performance under the conventional perfor- mance evaluation method, two critical issues emerge when delving into the distribution prediction errors over test samples (visualized in Figure
First, current algorithms exhibit satisfactory prediction performance on high-entropy samples (i.e., the samples with high-entropy label distribution) yet markedly underperform low-entropy samples (i.e., the samples with low-entropy label distribution). which can be demonstrated by the blue histograms in Figure [ However, from perspective, low-entropy samples demand greater attention than high-entropy samples, as the former convey less uncertainty for practical decision-making.
Second, conventional performance evaluation methods quantify the overall model perfor- mance by the arithmetic mean aggregation of performance metrics across all test samples. However, this approach inherently favors the numerical dominant samples. Since the high-entropy samples tends to outnumber the low-entropy samples in real-world tasks (as il- lustrated by the orange histograms in Figure D.. such aggregation typically fail to adequately capture the model performance low-entropy samples.
Therefore, in this paper, we aim to address the entropy bias in current LDL algorithms and conven- tional model performance evaluation methods. Specifically, in terms of the LDL algorithm, we first analyze the generation mechanism of entropy bias from both empirical and theoretical perspectives, and consequently propose an assumption that the underperformance of LDL models low-entropy samples is significantly driven by the cohesion of anchor vector Based on these analyses, we propose IAR (i.e., an Inter-anchor Angular Regularization term) to penalize the anchor vectors with over-small angles. In terms of the performance evaluation method, we propose ECA (i.e., an Entropy-Calibrate Calibrated Aggregation strategy) to calculate the overall model performance. Following the
Typicaally, the output of LDL models can be expressed softmax([(w... v) M))M, 1). where denotes the feature vector of a sample. In the scenarios of deep learning, v is typically obtained by passing the raw feature vector æ through feature extraction network. In the scenarios of non-den deep learning, is usually set directly to the raw feature vector Then, the set of vectors (wm) J.M constitutes the anchor vectors of the LDL model.

---

divide-and-come principle, ECA partitions the test set into low-entropy and high-entropy subsets based on threshold. The average model performance is then computed separately for each subset. Finally, ECA evaluates the overall performance on the whole test set by the expected value of the average subset performance w.r.t. the threshold distribution. Empirically, extensive experiments on real-world datasets demonstrate that our proposal is effective in improving low-entropy sample predictions while maintaining satisfactory performance on high-entropy samples.
2 Related Work
Current research LDL primarily focuses on two directions: loss function engineering and task- specific customization. The research on loss function enginerring mainly focuses on exploring either label correlations or sample correlations within label distributions. For example, LDLLC [S] constructs distance matrix from training label distributions preserve label correlations during model learning. An algorithm based on optimal transport formulates the label correlation mining process as metric learning problem, employing optimal transport distances to capture geometric relationships in the label space [37]. An algorithm based on local sample correlation introduces local label correlation hypothesis, constructing sample-specifts correlation vectors as additional features 38 Differing from 38). an algorithm based on local low-rank label correlation 8 employs low-rank structures on local samples to discover label correlations. LCLR (26) simultaneously learns both global and local label correlations through low-rank approximation and clustering techniques. An algorithm based on label distribution manifold adopts a data-driven approach to leverage global and local correlations, learning the manifold structure of label distributions to constrain model outputs [30]- An algorithm based on fuzzy label correlation utilizes fuzzy membership-induueed label correlation and joint fuzzy clustering and label correlation to capture multiple local label correlations [31]. TLRLDL [12) introduced an auxiliary multi-label learning process within the LDL framework, focusing on capturing low-rank label correlation within this auxiliary multi-label learning component rather than the LDL itself. Two algorithms based on label rankings propose to regularize the learning process by the label ranking correlation underlying the label distributions 66 77. Beyond fundamental loss function engineering, significant research efforts have been devoted to developing specialized LDL algorithms tailored for particular task requirements. For example, noise-robust LDL algorithms have been proposed to mitigate the adverse effects of inaccurate label distribution supervision during model training 3 11 21 91. LDL algorithms based simple labels (e.g., binary labels [15 18 20 33 10), temarr labels [17]. or label rankings [16, [9]) have been proposed to
address the availability of label distribution supervision. LDL algorithms based on matrix completion
have been proposed to learning the label distributions with missing values 332 35 36).
3 Methodology
This section presents our approach to addressing the entropy bias problem in label distribution learning. We begin by introducing the commonly-used mathematical notation and providing a problem formulation for LDL in Section 3.1 Building upon this foundation, Section 3.2 systematically investigates the underlying mechanisms responsible for the entropy bias. Finally. Section 3.3 elaborates on the technical details of our proposed loss function.
3.1 Problem Formulation
Let z and y denote the feature vector and the corresponding label distribution of sample, respectively. Each element yo in y is called label description degree, and satisties that Σ 3/m 1 and ym 2 0, where M is the number of labels. The training set of LDL is denoted by {I vn,)),,)))). ) LDL algorithms aim to learn a multivariate function ff x + y based on the training set { vn,) 1N
3.2 Mechanisms of Entropy Bias in Label Distribution Learning Algorithms
Prior work by 19 preliminarily demonstrated that most baseline LDL algorithms exhibit a propensity
for uniform label distributions. Inspired by this discovery, we present more systematic visualization and analysis of the output entropy distribution across several recently proposed I.D. algorithms [6] 7 30 31). As shown in Figure 2 the prediction entropy (blue histogram) predominantly concentrates at
3

---

Emotion6
Normalized Entropy
Emotion6
Prediction
1.0
2.5
Dunaty 3
Ground-Truth
2.0
Ground-Tyutt
0.6
20
Density
1.5
1.0
0.4
0.5
0.2
1.0
2.0
081
Entropy
Prediction
Botweeo-Anchor Angle
(a)
(b)
(c)
Figure 2: Entropy distributions of the prediction and the ground-truth label distributions. (a): The first subfigure depicts the entropy distributions of predicted versus ground-truth label distributions on Emotions dataset. (b): The second subfigure presents their joint entropy distribution with sample- level visualization (gray points) and population density (kernel density plot). (c): The third subfigure depicts the distribution of the angles between anchors.
high entropy values, i.e., the prediction is over-uniform, whereas the ground-truth distribution (orange histogram) spans a broader range. This bias further corroborated in the joint distribution, where sample density concentrates in the lower-right triangular region (semi-transparent red), confirming that the prediction entropy exceeds the ground-truth on most samples. Meanwhile, we visualize the distribution of the angles between the anchor vectors of the LDL models. It can be seen that the inter- anchor angle predominantly concentrates at low values, i.e., the anchors are cohesive. Intuitively, the cohesion of anchors and the over-uniforme wver-uniformity of predictions typically occur in conjunction with each other. As illustrated in Figure 3[a] strong correlation exists between smaller inter-anchor angles and reduced entropy values. Specifically,
visualization demonstrates inverse relationship where narrower angular separation between anchors corresponds to lower entropy measures on average. This inverse relationship further suggests that the anchors with narrow angular separations are difficult to represent low-entropy samples. As illustrated in Figure 36 the space (indicated by the dark brown region) that can well-represent low-entropy samples remarkably constrained when using anchors with narrow angular separation. In contrast, the anchors with wider angular separation exhibit substantially more expansive solution spaces for low-entropy sample representation. Furthermore, we propose Theorem 3. rigorously verify the above idea, and the proof is provided in Appendix A.
Theorem 3.1. Let mwm M denote a group of cohesive anchor vectors, æ denote the feature vector of a sample, and y softmax([ww,, (m)mm:))mm (=)=)) denote the corresponding output. Without loss of generality, we assume that the anchors are all unit vectors. Then Equation (T) holds if vi # (w), < n, where L(wi, w)) denotes the angle between anchors wi and w.
MXt 1
MX X logg(A*)+ X= X X log(A))
H(y) 2 1o - XF
H(y) denotes the entropy of the label distribution y. T =minm Z(wm, x)) is the minimum angle between x and anchors, Z. = EM exp((w m: m)) denotes the normalization factor, and ||=||| denotes the L2 norm of the
where Xt = Z cxp(cos(T +||==1),,, 2 Z-1 exp(cos(7")
feature vector x. Equation achieves equality when MX - 1 = k(> - At) and k is positive integer.
Theorem 3.1 establishes the entropy range within which anchors with certain degree (7) of cohesion can effectively represent samples. The derivative of the right-hand side of Equation (T) w.r.t. 7 can be expressed as:
X X 1) (Xi 19)2
At A
At
dAt
dxf
log
=1 xt)x sin(sin(TT)))) 79) < 0 (2) dr
dr
is obvious that (A*-X*) X 1) 2 0. Besides, we have 1= log ( according to the Taylor series expansion of the logarithmic function log(u) at = 1. Therefore, the derivative of the right-hand side of Equation (1) w.r.t. T is non-positive. other words, decreasing т causes the lower bound of H((y) to increase. Motivated by the above observations and analysis,

---

1.10
1.0
1.0
0.70
0.65
0.5
Enuzopy
0.60
1.00
0.55
0.90
41 Anchors
Anchors
-0.5
0.50
Anchors
0.45
0.85
B0
100"
120"
-1.0
11.0 0.0
Jverage Angle
-0.5
(a)
(b)
Figure 3: Relationship between the prediction entropy and the anchor angle. (a): The first subfigure shows the average prediction entropy (line) and the corresponding standard deviation (shadow) on different inter-anchor angles under one million random trivials. (b): The last two subfigures denote the entropy distribution under two 2D anchor vectors. The black directed lines denote the anchor vectors; the line between the zero point and point on the unit circle represents a sample point whose color denotes the entropy of the predicted label distribution under the current anchor vectors.
argue that the underperformance of LDL models on low-entropy samples is significantly driven by the cohesion of anchors, which can be formalized as Assumption 3.2
Assumption 3.2. Given two groups of anchor vectors AA = (wm Veyj (wij <m Ja and
Aa = (wm Veæj (ww; w < T2 M.M 22m=l where Z(w;, wj) denotes the angle between the anchors w; and wj. the prediction performance of the anchor vectors Ai low-entropy samples tends to be
inferior to that of the anchor vectors A2 if TI < T2.
3.3 Inter-Anchor Angular Regularization
According to Assumption 3.2 we propose an inter-anchor angular regularization term (IAR) to penalize the small angles between anchors. The most straightforward formula for calculating the angle between anchors w; and wj is [ww,w,)) gradient calculations. Therefore, we minimize the cosine similarity between anchors w; and wj. which is equivalent to maximizing their angular separation due to the monotonicity of the cosine function within the [0, т interval. Then the loss function can be formalized Equation 3):
arccos(cos(ur (wos(w;; wj)), which is not conducive to
N Df
L
CK #B
Σ
(wi, w,}
(3)
w |||||||||| "
(K.)EB
where a is a trade-off hyperparameter, B is defined as ((1,j) : : M), #B denotes the cardinality of IB w; denotes the anchor vectors, and DKI. (f(x,) v,.) denotes the KL divergence between the ground-truth label distribution commonly utilized to encourage the model output be closer the ground-truth label distributions. The second term in Equation (3) is the inter-anchor angular regularization term, which penalize the small inter-anchor angle by minimizing the cosine similarity between anchors.
and the model output f(x,, of the sample C.. which
Next, we establish two mathematical properties of IAR to provide more comprehensive understand- ing of its behavior in practical deployment. First, in order facilitate optimization algorithms, give the partial derivative of IAR w.r.t. anchor vectors:
cos cos(w;, Wi wj) = ||sl|| Wj |
cos (w;; wزx)
wi ||=م||||
ows
Besides, we derive the value range of IAR. It is evident that IAR reaches its maximum value of 1 when all anchor vectors aligned in the same direction. However, the lower bound of IAR cannot reach -1 due to the geometric constraint that multiple vectors cannot be mutually antiparallel in all pairwise combinations. Therefore, give the lower bound of IAR in Proposition 3.3
5

---

Proposition 3.3. Given any group of non-zero anchor vectors {wm Jm= Equation M 2 2.
5
holds for any
(wi w3) w will|w Wارl
Σ
Wm
M
1 M
(5)
#I
M(M 1)
wm
(EJ))
3.3 clearly demonstrates that the lower bound of IAR converges to 0 (i.e., the anchor vectors asymptotically approach mutual orthogonality) as the number of anchor vectors grows.
Proposition
Performance Evaluation
In this section, we illustrate the proposed ECA (Entropy-Calibnn Calibrated Aggregation) strategy address the entropy bias in conventional model performance evaluation methods. Following the divide-and- conquer principle, the main idea underlying ECA is to evaluate the model performance separately on the low-entropy and high-entropy subsets of test set. Let (xx)), y,,) denote the performance for the test sample (æn, y, E(C) denote the average performance for the Conventional performance evaluation method compute the overall performance as:
test samples {(I yn) nnee
Enecio ((S(z)) yn.) + Eneembo E(s(xn)) yn)
E(C) - 1 #C
1
(((((n))))) REC
(6)
#Cin + #Cnigh
where C = = Clow U Chigh is the index set of test samples, Clow and Cnigh denote the index sets of low-entropy test samples and high-entropy test samples, respectively. It can be seen that the number of high-entropy samples greatly exceeds the number of low-entropy samples (which is common in most practical situations), the conventional evauation method is heavily biased in favor of the high-entropy samples, and thus fails to adequately capture the model's 's performance on predicting low-entropy samples. To address this limitation, we can directly modify Equation (6)
as Equation (7):
2 (Clow) (CCnigh)) = Enecin. E(f(En). Un) + EnEchigh E(f(xn), yw)
1
#Clow
#Chigh
Equation 7 first partitions the test into low-entropy and high-entropy subsets, then computes the performance scores separately for each subset, and finally takes the average of the performance scores of the two subsets. Besides, we utilize a threshold determine Clow and Cligh, i.e., Co {n E C H(y)) < kt,, Gligh = {n E C H(v)) 2 s.. The threshold x can be user-defined according to the specific task. For the sake of simplicity, we in this paper assume uniform threshold k p(k), where p(k) = Unif(N| minnec H(y,), maxnec H(y,)). Finally, we propose the evaluation method:
T
- Bn)ppN)) [E(C)()) + E(C
1
1 &
Σ
(CrO")) + E (CHIE)
(0() p(x).
(8)
2
2T
1=1
where T is the number of Monte Carlo samples utilized to approximate the intractable expectation.
5
Experiments
5.1 Experimental Configurations
Datasets. To ensure broad coverage of data complexity and practical scenarios, we select
datasets including Jaffe (H 0.96+4.03) [22], BU-3DFE (H 0.95_..01) [34], Movie (H
0.88+0.06) M. Music Mood (H: 0.944..03)
13 Natural Scene (H 0.47+0 Emotion6
where H denotes the normalized entropy. More details are provided in Appendix B.1. Based on the entropy. datasets can categorized into the high-entropy group (from Jaffe Music Mood) and the low-entropy group (from Natural Scene to M2B). Based on the task, the datasets cover emotion recognition (Jaffe and BU-3DFE), sentiment analysis (Music Mood, Emotion6, and Art Painting), scene recognition (Natural Scene), and rating prediction (Movie and M2B).
(H 0.64 255, Art Painting (H 0.72.0.13) 233, and M2B (H 0.41+a.12)

---

Table 1: Performance on High-Entropy Datasets (Jaffe, BU-3DFE, Movie, Music Mood).
KL (J)
Cosine (1)
ECA
LEA
HEA
ECA
LEA
HEA
Jaffe
AAR
0.041 t0,00
0.053 2001
0.044 1:000
0.962
0.951
0. 0.959 20003
LDM
0.114
* 00.159-0.029
0.0066
* -0.903=e.11
00.550=..22
0.939
DPA
0.015
* 00.097
0.0025
4..04
* -0.938
=..0ss
+0.919
* 0.934
FCC
0.091
10.11920.0a
n.0
00.928=0.011
0.906=2.033
0.924=
LRR
0.048 +0,00
* 0.063 20.08
0.043 20000
0.954 +00004
0..90 #0.077
0.961+0000
Ridge
0.02
* 03.
20001
* 0.916=00022
0.876
0.939
BU-3DFE
IAR
0.054
0.069
+0000
0.051 +0002
0.948 #00022
0.936
+0003
0.94940002
LDM
0.119 40.002
0.199
0.006
0.888 20.022
0.822
0.945
DPA
0.057 +00000
* +0.072
0.051 00.002
* 0.945
0.932 100001
0.949
0.004
10000
FCC
0.008
* +0.07220.00
0.055
* 0.9415=0.0s
0.93/1
* 0.9-16=0.00
LRR
0.057=#0.0
+0.075..0.00
0.049 +00000
0.9455.0.002
0.929
0.951
Ridge
0.110
* +0.171
0.055
0.895
0.837 +0.002
0.945
Movie
TAR
0.259 10,055
10.055
0.437 10.119
0. 0.097 1:0002
0.85210025
0.747 10064 10064
0.935 10002
LDM
0..26610032
0.508
0. 0.110.10.004
0.805
0..710
0..70=00002
DPA
0.222210056
* 0.44662..12
0. 00103310.000
* 0.8/19=20.025
0.7-42
0..322
FCC
* 0.284420088
* 10.48120.12
* 0.134420000
* 00.841
200026
* 0.7730=0.06t
* -0.916=0.00
LRR
0.262 +0.054
* 0..422 10.122
0.090
0.0000
* 0.850 +-0.244
0.743 10.057
0.940 +-0002
Ridge
0.338 10.048
0..99..0.14
1..00s
0.807
0.654
0.928
Music Mood
IAR
0.136
0.188
1000s
0.082
0.905
0.875
0.93310004
LDM
0.154
* 0.215
00.094
* -0.991
*
0.8542n.019
* 00.923=0.005
DPA
0.146
* 0..201
0..0822.mm
-0.898_0.0m
0.865_mmm4
00.930=0.00
FCC
0.166
* 0..205=0.023
0.099
* -0.890=a010
0.860=a.14
-0.920
LRR
0. 0.148 20,044
* .0.197
0.089
* 0.900 +0000s
0.867 10014
0.932 #0006
Ridge
0.150
0..202:2.0224
2.009
0.895
0.863
0.928
Evaluation Measures. Considering the suggestion proposed in [1 and the page limit, we employ both KL. divergence and cosine similarity as evaluation metries for individual sample. The better performance is represented by the higher value of KL. divergence (f) or the lower value of cosine similarity (4). For the overall performance assessment, we implement three aggregation approaches. The first one is our proposed ECA method (presented in Section 4) where T' is set to 10. The second one LEA (Low Entropy Aggregation), which only computes the average performance across low-entropy test samples. The third one is HEA (High Entropy Aggregation), which only computes the average performance across high-entropy test samples. The entropy threshold for sample partition is defined as the arithmetic mean of the maximum and minimum entropy values within the test set.
5.2 Comparison Algorithms and Experimental Procedure
Comparison Algorithms. We employ four recently proposed LDL algorithms for comparative study, including LDM [30), DPA (6). FCC BII, and LRR [7]. All hyperparameters for these com- parison algorithms are tuned within the ranges recommended by their respective publications. For our proposed method, the hyperparameter We employ L-BFGS to minimize the loss function of our method. Furthermore, to ensure fair comparison, we set the trade-off parameters of the T-2 regularization comparison algorithms as 0, consistent with the implementation of all comparison algorithms. To compare our proposed
optimized within the range of [1, 10, 20,..., 100).
7

---

Table 2: Performance on Low-Entropy Datasets (Natural Scene, Emotion6, Art Painting, M2B).
KL (4)
Cosine (1)
ECA
LEA
HEA
ECA
LEA
HEAA
Natural
Scene
AAR
0.736 10.03
0.8922a0m
0.655 +0003
0. 0.760 10.009
0.756+a010
0. 0.746 10010
LDM
1.2237
* 1.900
0.7866=0.08
* 0..567 =0:002
0.3888
* 00.675
DPA
4.004
* 00.907
00,031
0..699...n54
* 0.750=a..0s
00.511a.mm
* 0..733
FCC
1.049 +0.080
1..057 #00054
00.9994..00
0.698 0.009
0.715
-0.670 0.010
LRR
00.709+00 +0.018
10.90620mm
0..11 20002
O. 0.768 000
0.753 +0o1
0.761 100004
Ridge
0.980
* 1.400
0.715
* 0.663=a.0us
0.566
0.710=0.00
Emotion6
IAR
0.689 +0.054
0.810 +amss
0.490 4000
0.693 +0001
0.667+0015 +0o1s
0. 0.741 40011
LDM
0.801 +0.022
1.078 0.0028
0.5.4 #0.021
+0.639 =0.00s
0.534
0.710
DPA
0.709 10.054
* 0..88 00.116
0.003
0.51
20.019
* +0.684
0.661 20.016
* 0
FCC
0.717 10005
* +0..17 000035
0..22 100019
* 200019
0.661
20002 20.012
* 0.726
LRR
0.675 +0.0s
0..815 +0006
t0.12
0.699
0.661 +0006
0.749
Ridge
0.711
* +0.900
0.4466
0.677 +0.00
0.620
0.733
Art Painting
AAR
0.650 10.128
0.777
0.498 1000
0.709 10016
0.6871000
0. 0.740 10018
LDM
0.869
1.007 20,273
0.572..0253
00.657 20.061
0.5744
0.7231..00
DPA
0.906
* 0.965
10.19
* 0.6615
0.6557
0.691 10.025
FCC
* 181
* 1.193
*
0.970
* 00.50
* 0.624
* 0.648
LRR
0.646 10.128
* 0.807 1:0.120
0.462
20:40
0.713 2002
0.671
10.005
0.755 1001s
Ridge
0. 10.14
0.923
0..06
0.677
0.617
0.734
IAR
0.744
0.9061004
0.321
0.713 10016
0.608400t
0.83810013
LDM
0.95622056
* 1.150=a.028
0.801=2.066 n00064
* 0..91 20.032
0.5162n.06
* 00.635
DPA
1.022
1. 1.2522_....
108
00.667 =0.026
0.570
+0.773_0017
FCC
1.081
* #1.11
20.05
* 0..611
0.564
-0.759
LRR
0. 773 10.089
* 0..44 20042
0.358 +0.018
* 0. 704 10019
0.601 +0007
* -0.824 10010
Ridge
0.816
1.011
0.377
0.696
0.596
0.814
IAR term and L2 regularization term, we introduce ridge regression for LDL, which minimizes N Em DKL (xx,))l,, + M EM= |///l and a is selected from {10-3, 1102....... 103). Be- sides, we normalize the feature data to improve the convergence stability of all comparison algorithms.
I
Experimental Procedure. Given dataset with label distributions, we first randomly divide the dataset into two subsets (30% used as the test set and 70% is used as the training set). Further, we train an LDL model on the training set and apply the model to predict the label distribution of the test samples. Then, we evaluate the performance of the LDL model by comparing the ground-trute and the predicted label distributions. Finally, we repeat the above process ten times under randomly different dataset partitions and statistically summarize the results of the ten random experiments.
5.3 Discussion on Experimental Results
As shown in Tables Sablesaand [] the results are interpreted as follows: each cell entry (e.g.. 0.10444011) indicates the mean performance (#t standard deviation): the symbol "*" denotes the cases where our proposed IAR statistically superior to the corresponding algorithm under paired two-tailed 1-test with < 0.05, while "o" denotes the significant inferiority of IAR. Absence of annotations implies no statistically significant difference. Boldface and italics highlight the best and second best performance within each comparison group. The experimental results show that IAR performs outstandingly on the
8

---

(K1
(Coe)
(K1
HEA
(K3)
0.00
0.3
0.3
0.70
0.72
1.0
0.75
0.07
000
0.62
0.13
0.5
0.05
014
0.00
20
60 20
100
60
68
00
100
0.04
20
60
100
1Strength of 1AR0
1SRrength of [AR]
(Steength of FARS
(a) Jaffe (H 0.96
(b) Art Painting (H :.:2: 0.72
(c) M2B (H 0..1
Figure 4: Prediction performance varying The performance quantified by KL divergence (KL) and cosine similarity (Cos) is represented by blue and red, respectively. The performance evaluated by ECA, LEA, and HEA is represented by the solid lines, dashed lines, and dotted lines, respectively.
(XL)
[Cos]
(KL)
(Cos)
(KLI
(Cos)
6.100
1.
0.277
1.0
6.100
08
04
6.975
0.
0.050
02
0.6
0.0
0.523
02
0.000
80 00
00 00
(a) Jaffe (H 0.9620.03)
(b) Art Painting (H 0.72
(c) M2B (H 0.41
Figure 5: Prediction performance of ablation algorithms evaluated by ECA. Each subfigure is bisected by a vertical dashed line, with the left and right sides representing the performance measured by KL divergence (KL) and cosine similarity (Cos), respectively.
high-entropy datasets (Table while it performs sub-optimally on the low-entropy datasets (Table In terms of the high-entropy datasets, IAR demonstrates statistically significant superiority all competitors under both ECA and LEA evaluation method. When evaluated by HEA, IAR either achieves top performance shows no statistically significant difference from the top-performing competitor. The sole exception occurs on the Movie dataset, where IAR is significantly outperformed by LRR. In terms of the low-entropy datasets, while IAR still significantly outperforms all competitors on low-entropy samples, it sacrifices prediction performance for high-entropy samples compared to LRR on Natural Scene, Emotion6, and Art Painting. This aligns with our expectations. Since the model trained by high-entropy samples is more likely to output the over- wer--queezed anchor vectors, which can effectively avoided by adding IAR term. On the contrary, in order to fit the low-entropy label distributions, the model trained by low-entropy samples is not prone to output the over-squeezed anchor vectors, and thus the model cannot benefit significantly from IAR term. Nonetheless, IAR consistently performs best on low-entropy samples, which suggests that IAR is able to improve the prediction performance for low-entropy samples to varying degrees on various real-world datasets. More experiments demonstrating the effectiveness of IAR can be found in Appendix B.2.
5.4 Further Analysis
Hyperparameter Sensitivity. Figure presents the impact of hyperparameter on the KL. metric under the BCA evaluation method, demonstrating the performance sensitivity our method w.r.t. the hyperparameter variations. Experimental results demonstrate that moderately increasing the IAR weight (e.g., a E [1, 40)) consistently enhances model performance. However, excessive values (e.g., 40) should be applied with caution, excessive may induce severe underfitting accompanied by significant performance degradation on both low-entropy and high- gigheentroyy samples. Furthermore, for the datasets that most samples possess the label distribution with low entropy (c.g.. "M2B" dataset), we can safely employ relatively large a values for better model performance.
Ablation Study. We construct three ablation variants for comprehensive analysis. The first one is the LDLIAR with zeroed out, i.c., simple softmax regression, which is abbreviated as "SR".

---

The second one is an LDL algorithm with entropy regression, which is abbreviated as "ER", whose loss function for each sample is defined by (f y,,) + (fH(f((z,) Hly,)). The third LDL. algorithm, which is abbreviated as "FW"" whose loss function
one is an entropy-weighte for each sample is defined by exp(->- (((-))-(())))) (xI (r.) ll To address entropy bias, "ER" incorporates an additional loss term that penalizes the entropy discrepancy between the predictions and the ground-truth. while "EW" assigns higher weights to the low-entropy samples to prioritize their contribution during model training. "ER" and "EW" implement two straightforward approaches for addressing entropy bias. Their hy hyperparameter is selected from {10- 10002 ..... 10 The experimental results are shown in Figure [ which demonstrates that proposed IAR achieves significant improvements compared to other ablation algorithms.
Limitations and Conclusion
Limitations. First, our proposed inter-anchor angular regularization (IAR) term is not directly compatible with tree-based LDL algorithms,, as their learning process do not involve anchor vectors. Second, Theorem 3.1 assumes output layer with non -egnatiaeve gradients (e.g., softmax normaliza- tion). Consequently, the theoretical guarantees do not hold for the output layers with activation or normalization functions that violate the non-negativity.
Conclusion. In this work, we systematically investigate the limitations of existing algorithms and evaluation methods on label distribution learning in handling low-entropy samples. In terms of the algorithm, we reveal, from both empirical and theoretical perspectives, that excessive cohesion of anchor vectors is essential cause of the underperformance on low-entropy samples. According to this assumption, we propose an inter-anchor angular regularization (IAR) term to explicitly penalizes over-squeezed angular separations between anchor vectors, and derive several mathematical properties that can be beneficial for practical use. In terms of the performance evaluation method, we introduce an entropy-cale -aatopy-cllbbrtted aggregation (ECA) method to avoid the imbalance between low-entropy and high-entropy samples. Finally, extensive experimental results verify the validity of our proposal.
Acknowledgements
This work was partially supported by the National Natural Science Foundation of China (62176123, 62476130), the Natural Science Foundation of Jiangsu Province (BK20242045), the Innovation and Technology Fund (GHP/079/225Z, ITS/034/23FP), and the UGC/GRF (No. 15211024, 15215421).
References
[1] Xin Geng. Label distribution learning. IEEE Transactions on Knowledge and Data Engineering, 28(7):1734-1748 2016.
[2] Xin Geng and Peng Hou. Pre-release prediction of crowd opinion movies by label distribution learning. In International Joint Conference on Artificial Intelligence, pages 3511 1-351,,,, 2015.
[3] Liang He, Yunan Lu, Weiwei Li, and Xiuyi Jia, Generative calibration of inaccurate annotation for label distribution learning. In AAAI Conference on Artificial Intelligence, pages 12394- 12401, 2024.
[4] Torsten Hothorn, Thomas Kneib, and Peter Bühlmann. Conditional transformation models. Journal of the Royal Statistical Society Series B: Statistical Methodology, 76(1):3-27, 2013..
[5] Xiuyi Jia, Weiwei Li, Junyu Liu, and Yu Zhang. Label distribution learning by exploiting label correlations. In AAAI Conference on Artificial Intelligence, pages 3310-3317, 2018.
[6] Xiuyi Jia, Tian Qin, Yunan Lu, and Weiwei Li. Adaptive weighted ranking-orienter label distri- bution learning. IEEE Transactions on Neural Networks and Learning Systems, 35(8): 1302- 11316, 2024.
[7] Xiuyi Jia, Xiaoxia Shen, Weiwei Li, Yunan Lu, and Jihua Zhu. Label distribution learning by maintaining label ranking relation. IEFE Transactions on Knowledge and Data Engineering, 1 2023.
10

---

[8] Xiuyi Jia, Xiang Zheng, Weiwei Li, Changqing Zhang, and Zechao Li. Facial emotion dis- tribution learning by exploiting low-rank label correlations locally. In IEEE Conference on Computer Vision and Pattern Recognition, pages 9841-9850, 2019.
[9] Zhiqiang Kou, Jing Wang, Yuheng Jia, and Xin Geng. Inaccurate label distribution learning. IEEE Transactions on Circuits and Systems for Video Technology, 34(10):0237--1024 2024.
[10] Zhiqiang Kou, Jing Wang, Yuheng Jia, and Xin Geng. Progressive label enhancement. Pattern Recognition, 160: 160:11110 11722. 2025.
[11] Zhiqiang Kou, Jing Wang. Yuheng Jia, Biao Liu, and Xin Geng. Instance-dependent inaccurate label distribution learning. IEEE Transactions on Neural Networks and Learning Systems, 14377 2023.
[12] Zhiqiang Kou, Jing Wang. Jiawei Tang, Yuheng Jia, Boyu Shi, and Xin Geng. Exploiting multi-label correlation in label distribution learning. In International Joint Conference Artificial Intelligence, pages 4326-4334, 2024.
[13] Harin Lee, Frank Hoeger, Mare Schoenwiesner, Minsu Park, and Nori Jacoby. Cross-cultura mood perception in pop songs and its alignment with mood detection algorithms. In Interna- tional Society for Music Information Retrieval Conference, 2021.
[14] Haiqing Li, Wei Zhang, Ying Chen, Yumeng Guo, Guo-Zheng Li. and Xiaoxin Zhu. A novel multi-target regression framework for time-series prediction of drug efficacy. Scientific Reports, 7(1):40652, 2017.
[15] Yunan Lu, Liang He, Fan Min, Weiwei Li, and Xiuyi Jia. Generative label enhancement with Gaussian mixture and partial ranking. AAAI Conference on Artificial Intelligence, pages 8975-8988 2023.
[16] Yunan Lu and Xiuyi Jia. Predicting label distribution from multi-label ranking. In Advances in Neural Information Processing Systems, pages 36931-36943, 2022.
[17] Yunan Lu and Xiuyi Jia. Predicting label distribution from ternary labels. In Advances in Neural Information Processing Systems, pages 70431-70452, 2024.
[18] Yunan Lu, Weiwei Li, and Xiuyi Jia. Label enhancement via joint implicit representation clustering. In International Joint Conference on Artificial Intelligence, pages 4019 4027, 2023.
[19] Yunan Lu, Weiwei Li, Huaxiong Li, and Xiuyi Jia. Predicting label distribution from tie- allowed multi-label ranking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(12):15664-15377 15379, 2023.
[20] Yunan Lu, Weiwei Li, Huaxiong Li, and Xiuyi Jia. ment. Machine Learning, 112:4693-4721, 2023.
generative label enhance-
[21] Yunan Lu, Weiwei Li, Dun Liu, Huaxiong Li, and Xiuyi Jia. Adaptive-grained label distribution learning. In AAAI Conference on Artificial Intelligence, pages 19161-19169, 2025.
[22] Michael Lyons, Shigeru Akamatsu, Miyuki Kamachi, and Jiro Gyoba. Coding facial expressions with gabor wavelets. In IEEE International Conference on Automatic Face and Gesture Recognition, pages 200-205, 1998.
[23] Jana Machajdik and Allan Hanbury. Affective image classification using features inspired by psychology and art theory. In ACM International Conference on Multimedia, pages 83-92, 2010.
[24] Tam V. Nguyen, Si Liu, Bingbing Ni. Jun Tan, Yong Rui, and Shuicheng Yan. Sense beauty via face, dressing, and/or voice. In ACM International Conference on Multimedia, pages 239-248. 2012.
[25] Kuan-Chune Peng, Tsuhan Chen, Amir Sadovnik, and Andrew Gallagher. A mixed bag of emotions: Model, predict, and transfer emotion distributions. In IEEE Conference on Computer Vision and Pattern Recognition, pages 860-868, 2015.

---

[26] Tingting Ren, Xiuyi Jia, Weiwei Li, and Shu Zhao. Label distribution learning with label corre- lations via low-rank approximation. In International Joint Conference on Artificial Intelligence. pages 2325-3331,
[27] Hao Song, Tom Diethe, Meelis Kull, and Peter Flach. Distribution calibration for regression. In International Conference on Machine Learning, pages 5897-5906, 2019.
[28] Anton Frederik Thielmann, René-Marcel Kruse, Thomas Kneib, and Benjamin Säfken. Neural additive models for location scale and shape: A framework for interpretable neural regression beyond the mean. In International Conference on Artificial Intelligence and Statistics, pages 1783-1791, 2024.
[29] Juozas Vaicenavicius, David Widmann, Carl Andersson, Fredrik Lindsten, Jacob Roll, and Thomas Schön. Evaluating model calibration in classification. In International Conference on Artificial Intelligence and Statisties, pages 3459-34 -367,, 2019.
[30] Jing Wang and Xin Geng. Label distribution learning by exploiting label distribution manifold. IEEE Transactions on Neural Networks and Learning Systems, (4(2):839-852. 2023.
[31] Jing Wang, Zhiqiang Kou, Yuheng Jia, Jianhui Lv. and Xin Geng. Label distribution learning by exploiting fuzzy label correlation. IEEE Transactions on Neural Networks and Learning Systems, pages 1- 2024.
[32] Miao Xu and Zui-Hua Zhou. Incomplete label distribution learning. In International Joint Conference on Artificial Intelligence, pages 3175-3181, 2017.
[33] Ning Xu, Yun-Peng Liu, and Xin Geng. Label enhancement for label distribution learning. IEEE Transactions on Knowledge and Data Engineering, 33(4):1632-166) 1643, 2021.
[34] Lijun Yin, Xiaozhou Wei, Yi Sun, Jun Wang, and Matthew Rosato. A 3D facial expression database for facial behavior research. In International Conference on Automatic Face and Gesture Recognition, pages 21 1-16,,,, 2006.
[35] Xue-Qiang Zeng, Su-Fen Chen, Run Xiang, Guozheng Li, and Xuefeng Fu. Incomplete label distribution learning based supervised neighborhood information. International Journal of Machine Learning and Cybernetics, 11:11,1 2020.
[36] Xue-Qiang Zeng, Su-Fen Chen, Run Xiang. Shui-Xiu Wu, and Zhong- Ying Wan. Filling missing values by local reconstruction for incomplete label distribution learning. International Journal of Wireless and Mobile Computing, 16(4):314-321, 2019.
[37] Peng Zhao and Zhi-Hua Zhou. Label distribution learning by optimal transport. In AAAI Conference on Artificial Intelligenee pages
2018.
[38] Xiang Zheng, Xiuyi Jia, and Weiwei Li. Label distribution learning by exploiting sample correlations locally. In AAAI Conference Artificial Intelligence, pages 4556 4563, 2018.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims made in the abstract and introduction have clarified both the theoretical and methodological contributions of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of the work have been discussed in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
13

---

Justification:
The assumptions of the theoretical result have been formally described in
Theorem 3.1
The proof of Theorem 3.1 and Theorem 3.3 placed in supplementary
materials.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: In Section 5.2 we have provided the details of the comparison algorithms, including the hyperparameter configurations and the method of dataset partitioning. For proposed IAR, we have also provided the hyperparameter tunning range and the optimization algorithm.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
14

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification: In terms of the code, the implementation of our proposed IAR is to simple to provide additional code files describe it. Nevertheless will still make code public after the paper is published. terms of the data, open access to the datasets involved in this paper requires license from the corresponding creator, and this paper is not authorized to distribute them.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
guwdes/ddee//ee/ Code/ubmeeeeeaee wwabbbmmsssiooPoooooo ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: The data splits, hyperparameters, hyperparameter tunning method, and opti- mizers have provided in Section 5.2
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The mean value, standard derivation, and the statistical significance of the prediction performance is shown in Tables I and
Guidelines:
The answer NA means that the paper does not include experiments.
15

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eπror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [No]
Justification: Computer resources have negligible effect both the experimental results and the main claims of this paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental rums well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httpesstttt://::tt http:///wwrrp..coooaaaaacccccaaa
Answer: [Yes]
Justification:
The research conducted in this paper does not violate the NeurIPS Code of
Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [NA]
16

---

Justification:
There is societal impact of the work performed since our work aims to advance the field of machine learning.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We have cited all original papers that produced the datasets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
17

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the palaaets.paperssttt paperssii thcode. com con/datases package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
18

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMS as any important, original, non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy hatt:p//eearrpppcc///ooeeeeeeee/ //t/////ooeeeeeccc////////..1. for what should or should not be described.
19


Reviews:
[{"review_id": "dN2KkTiJ8U", "reviewer": "Reviewer_3R2w", "summary": "This paper finds that excessive cohesion between anchor vectors contributes significantly to the observed entropy bias phenomenon in LDL algorithms. It accordingly proposes an inter-anchor angular regularization term that mitigates cohesion among anchor vectors by penalizing over-small angles. To alleviate the numerical imbalance of high-entropy samples in the test set, it proposes an entropy-calibrated aggregation strategy that obtains the overall model performance by evaluating performance on the low-entropy and high-entropy subsets of the overall test set separately. Experiments demonstrate the effectiveness of the proposed method.  The main contributions of this paper are:\n\n- It analyzes the generation mechanism of entropy bias from both empirical and theoretical perspectives, and consequently proposes an assumption that the underperformance of LDL models on low-entropy samples is significantly driven by the cohesion of anchor vectors.\n\n- It proposes IAR (i.e., an Inter-anchor Angular Regularization term) to penalize the anchor vectors with over-small angles.\n\n- It proposes ECA (i.e., an Entropy-Calibrated Aggregation strategy) to calculate the overall model performance.", "questions": "1. How about the performances of different algorithms on other LDL metrics (e.g., Chebyshev distance, Intersection similarity, etc.)?\n\n2. On some high-entropy datasets, the performance gap between IAR and other algorithms is not significant, while on low-entropy datasets, IAR shows a noticeable performance gap compared to some SOTA algorithms. The authors should further analyze the underlying reasons for these phenomena.\n\n3. The impact of the hyperparameter $\\alpha$ on model performance varies across different datasets. The authors should further analyze the underlying reasons for the hyperparameter $\\alpha$ on model performance varies across different datasets, and provide practical recommendations for its application.", "limitations": "The authors could further discuss the potential societal impact of the proposed method.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n- The problem studied in this paper is interesting and valuable.\n- The paper is well-organized, which is easy to follow.\n- The theoretical work improves the value of the paper.\n\n\n\n**Weakness**\n\n- The paper primarily uses KL divergence and cosine similarity as evaluation metrics. Although these two metrics can measure the differences between the model outputs and the true label distribution from different perspectives, they may not fully reflect the model's performance in real-world applications. For instance, in tasks that require accurate prediction confidence or uncertainty estimation, relying solely on KL divergence and cosine similarity may be insufficient to thoroughly assess the model’s reliability and effectiveness.\n\n- Although the paper demonstrates the stability of the results through multiple randomized experiments and reports the mean and standard deviation, the analysis of the statistical significance of performance differences across different datasets and algorithms is not sufficiently thorough. For example, on some high-entropy datasets, the performance gap between IAR and other algorithms is not significant, while on low-entropy datasets, IAR shows a noticeable performance gap compared to some SOTA algorithms. These issues are not adequately explored or explained. The authors should further analyze the underlying reasons for these phenomena.\n- The impact of the hyperparameter $\\alpha$ on model performance varies across different datasets. The authors should further analyze the underlying reasons for this variation and provide practical recommendations for its application.", "was_revised": true, "final_justification": "After reading the reviews' comments and the authors' responses, I would like to give a positive rating for this paper.", "rebuttal": "Dear Reviewer 3R2w,\n\nThank you for your constructive comments on our paper. We sincerely appreciate the time and effort that you have dedicated to evaluating our work. We have carefully considered all the comments and have revised the paper accordingly. Below, we provide a point-by-point response to your suggestions.\n\n***Question 1: How about the performances of different algorithms on other LDL metrics (e.g., Chebyshev distance, Intersection similarity, etc.)?***\n\nDue to space limitations, we have only shown the performance of the algorithm using KL divergence and cosine similarity measures. Nevertheless, we have shown the performance of the algorithm using Chebyshev distance and intersection similarity in the table below. Due to the character limit for the rebuttal, we only present results under the ECA aggregation strategy for these two metrics, omitting detailed comparisons between low- and high-entropy subsets. \n\n|  Algorithm | ECA (Chebyshev Distance)         | ECA (Intersection Similarity)     | Dataset           |\n|:------------|:--------------------|:--------------------|:------------------|\n| IAR      | (1) 0.082$\\pm$0.005 | (1) 0.894$\\pm$0.005 | Jaffe            |\n| LDM      | (4) 0.099$\\pm$0.012 | (4) 0.881$\\pm$0.012 | Jaffe            |\n| DPA      | (6) 0.115$\\pm$0.012 | (6) 0.852$\\pm$0.013 | Jaffe            |\n| FCC      | (3) 0.092$\\pm$0.007 | (3) 0.888$\\pm$0.006 | Jaffe            |\n| LRR      | (2) 0.092$\\pm$0.007 | (2) 0.889$\\pm$0.006 | Jaffe            |\n| Ridge        | (5) 0.115$\\pm$0.014 | (5) 0.853$\\pm$0.015 | Jaffe            |\n| IAR      | (1) 0.108$\\pm$0.003 | (1) 0.871$\\pm$0.003 | BU-3DFE          |\n| LDM      | (6) 0.119$\\pm$0.007 | (6) 0.863$\\pm$0.007 | BU-3DFE          |\n| DPA      | (2) 0.109$\\pm$0.003 | (2) 0.870$\\pm$0.003 |  BU-3DFE          |\n| FCC      | (5) 0.116$\\pm$0.003 | (5) 0.865$\\pm$0.003 | BU-3DFE          |\n| LRR      | (4) 0.115$\\pm$0.005 | (4) 0.866$\\pm$0.004 | BU-3DFE          |\n| Ridge        | (3) 0.109$\\pm$0.003 | (3) 0.870$\\pm$0.003 | BU-3DFE          |\n| IAR      | (1) 0.313$\\pm$0.006 | (1) 0.577$\\pm$0.006 | Natural Scene     |\n| LDM      | (6) 0.362$\\pm$0.012 | (6) 0.538$\\pm$0.010 | Natural Scene     |\n| DPA      | (3) 0.325$\\pm$0.010 | (3) 0.571$\\pm$0.009 | Natural Scene     |\n| FCC      | (5) 0.358$\\pm$0.009 | (5) 0.542$\\pm$0.009 | Natural Scene     |\n| LRR      | (4) 0.336$\\pm$0.010 | (4) 0.563$\\pm$0.010 | Natural Scene     |\n| Ridge        | (2) 0.322$\\pm$0.008 | (2) 0.574$\\pm$0.008 | Natural Scene     |\n| IAR      | (1) 0.357$\\pm$0.016 | (1) 0.555$\\pm$0.014 | Emotion6          |\n| LDM      | (6) 0.371$\\pm$0.014 | (6) 0.544$\\pm$0.012 | Emotion6          |\n| DPA      | (3) 0.360$\\pm$0.016 | (2) 0.552$\\pm$0.013 | Emotion6          |\n| FCC      | (5) 0.365$\\pm$0.013 | (4) 0.548$\\pm$0.012 | Emotion6          |\n| LRR      | (4) 0.365$\\pm$0.013 | (5) 0.548$\\pm$0.012 | Emotion6          |\n| Ridge        | (2) 0.360$\\pm$0.017 | (3) 0.552$\\pm$0.015 | Emotion6          |\n| IAR      | (1) 0.295$\\pm$0.049 | (1) 0.581$\\pm$0.036 | Art Painting |\n| LDM      | (4) 0.320$\\pm$0.040 | (4) 0.557$\\pm$0.027 | Art Painting |\n| DPA      | (5) 0.344$\\pm$0.047 | (5) 0.529$\\pm$0.041 | Art Painting |\n| FCC      | (3) 0.316$\\pm$0.043 | (3) 0.562$\\pm$0.032 | Art Painting |\n| LRR      | (2) 0.316$\\pm$0.043 | (2) 0.563$\\pm$0.032 | Art Painting |\n| Ridge        | (6) 0.345$\\pm$0.046 | (6) 0.528$\\pm$0.039 | Art Painting |\n| IAR      | (4) 0.078$\\pm$0.003 | (4) 0.802$\\pm$0.008 | Music Mood        |\n| LDM      | (3) 0.077$\\pm$0.004 | (3) 0.803$\\pm$0.008 | Music Mood        |\n| DPA      | (5) 0.085$\\pm$0.003 | (5) 0.789$\\pm$0.010 | Music Mood        |\n| FCC      | (1) 0.076$\\pm$0.003 | (1) 0.803$\\pm$0.007 | Music Mood        |\n| LRR      | (2) 0.076$\\pm$0.003 | (2) 0.803$\\pm$0.007 | Music Mood        |\n| Ridge        | (6) 0.086$\\pm$0.004 | (6) 0.789$\\pm$0.009 | Music Mood        |\n| IAR      | (1) 0.409$\\pm$0.011 | (1) 0.584$\\pm$0.010 | M2B               |\n| LDM      | (3) 0.421$\\pm$0.020 | (3) 0.571$\\pm$0.021 | M2B               |\n| DPA      | (5) 0.430$\\pm$0.020 | (5) 0.563$\\pm$0.019 | M2B               |\n| FCC      | (2) 0.419$\\pm$0.016 | (2) 0.574$\\pm$0.016 | M2B               |\n| LRR      | (4) 0.423$\\pm$0.018 | (4) 0.571$\\pm$0.018 | M2B               |\n| Ridge        | (6) 0.431$\\pm$0.020 | (6) 0.562$\\pm$0.020 | M2B               |\n| IAR      | (1) 0.210$\\pm$0.024 | (1) 0.749$\\pm$0.022 | Movie             |\n| LDM      | (4) 0.213$\\pm$0.023 | (4) 0.747$\\pm$0.021 | Movie             |\n| DPA      | (6) 0.213$\\pm$0.025 | (6) 0.746$\\pm$0.023 | Movie             |\n| FCC      | (3) 0.212$\\pm$0.024 | (3) 0.748$\\pm$0.022 | Movie             |\n| LRR      | (2) 0.212$\\pm$0.024 | (2) 0.748$\\pm$0.022 | Movie             |\n| Ridge        | (5) 0.213$\\pm$0.024 | (5) 0.746$\\pm$0.023 | Movie \n\nAccording to the results in the above table, it can be seen that under the Chebyshev distance and intersection similarity metrics, our algorithm achieves state-of-the-art performance on the Jaffe, BU-3DFE, Natural Scene, Emotion6, Abstract Painting, M2B, and Movie datasets. Notably, while it does not attain the best performance on the Music Mood dataset, it remains highly competitive, with performance very close to the top.\n\n***Question 2: On some high-entropy datasets, the performance gap between IAR and other algorithms is not significant, while on low-entropy datasets, IAR shows a noticeable performance gap compared to some SOTA algorithms. The authors should further analyze the underlying reasons for these phenomena.***\n\nThe performance gap is more pronounced on low-entropy datasets than on high-entropy datasets because prediction errors exhibit greater variance in low-entropy datasets compared to high-entropy ones. We demonstrate this claim through data simulation with the following procedure:\n\n1. First, we uniformly generate 10,000 label distributions $[\\boldsymbol z_n]_{n=1}^{10,000}$ with varying entropy levels.  \n2. For each label distribution $\\boldsymbol{z}_n$, we randomly generate 10,000 predicted label distributions and compute their KL divergence, cosine similarity, Chebyshev distance, and intersection similarity with respect to $\\boldsymbol{z}_n$.  \n3. We then partition $[\\boldsymbol z_n]_{n=1}^{10,000}$ into nine groups based on entropy intervals: [0, 0.2], (0.2, 0.4], (0.4, 0.6], (0.6, 0.8], (0.8, 1], (1, 1.2], (1.2, 1.4], (1.4, 1.6], (1.6, $+\\infty$).  \n4. Finally, we compute the variance of all prediction performance metrics within each group.\n\nThe results (shown in the table below) reveal that label distributions with higher entropy exhibit smaller variance in prediction errors. This explains why the performance gap is more noticeable in low-entropy datasets than in high-entropy ones.  \n\n| Entropy Range    | KL Variance | Cosine Variance | Chebyshev Variance | Intersection Variance |\n| ---------------- | ----------- | --------------- | ------------------ | --------------------- |\n| [0, 0.2]         | 0.8555      | 0.0453          | 0.0129             | 0.0158                |\n| (0.2, 0.4]       | 0.6871      | 0.0436          | 0.0128             | 0.0154                |\n| (0.4, 0.6]       | 0.6011      | 0.0411          | 0.0127             | 0.0152                |\n| (0.6, 0.8]       | 0.4107      | 0.0376          | 0.0102             | 0.0149                |\n| (0.8, 1]         | 0.2854      | 0.0321          | 0.0101             | 0.0134                |\n| (1, 1.2]         | 0.232       | 0.0285          | 0.0098             | 0.0128                |\n| (1.2, 1.4]       | 0.1486      | 0.0215          | 0.0069             | 0.0126                |\n| (1.4, 1.6]       | 0.0941      | 0.0115          | 0.0054             | 0.0099                |\n| (1.6, $+\\infty$) | 0.0459      | 0.0044          | 0.0042             | 0.0081                |\n\n***Question 3: The impact of the hyperparameter $\\alpha$ on model performance varies across different datasets. The authors should further analyze the underlying reasons for the hyperparameter $\\alpha$ on model performance varies across different datasets, and provide practical recommendations for its application.***\n\nWe analyzed the underlying reasons for the hyperparameter $\\alpha$ on model performance varies across different datasets, and provided practical recommendations for its application as follows.\n\n**Underlying reasons for the hyperparameter $\\alpha$ on model performance varies across different datasets.** The hyperparameter $\\alpha$ balances the relative importance between the training error (KL divergence between the model-output label distribution and the true label distribution) and the IAR regularization. Since the converged training error (KL divergence value) is influenced by the number of labels, data distribution, and the noise level of the dataset, the optimal weight ($\\alpha$) for the IAR term should be dataset-dependent. Furthermore, variations in data distribution across datasets lead to different degrees of over-uniformity when the training error is converged, consequently requiring different levels of IAR regularization during training. This further necessitates dataset-specific $\\alpha$ values.\n\n**Recommended settings.** As analyzed in lines 249-254 of our paper, we recommend $\\alpha=40$ as a default setting for most datasets, as it approaches optimal performance across diverse scenarios. For further performance optimization, we suggest the following empirical guidelines:\n\n1. When the training set contains predominantly high-entropy label distributions, consider using $\\alpha<40$.\n2. For datasets with predominantly low-entropy label distributions, larger $\\alpha$ values are preferable."}, {"review_id": "xBNCwLWFyU", "reviewer": "Reviewer_21xv", "summary": "This paper identifies and addresses the \"entropy bias\" in Label Distribution Learning (LDL), where existing algorithms perform poorly on low-entropy samples, which are crucial for decision-making. The authors propose two key innovations:\n\n- Inter-anchor Angular Regularization (IAR): A regularization term that penalizes overly cohesive anchor vectors to better capture low-entropy distributions.\n\n- Entropy-Calibrated Aggregation (ECA): An evaluation strategy that balances performance assessment across low- and high-entropy test samples.\n\nThe paper offers both theoretical analysis (including an entropy lower bound theorem) and extensive experiments on eight datasets, demonstrating consistent performance gains, especially on low-entropy samples.", "questions": "1. Can IAR be extended to architectures without explicit anchor vectors, such as attention-based or Transformer models?\n2. How sensitive is the performance to the choice of entropy threshold in ECA? Would a learned threshold work better?", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. Novel identification of entropy bias in LDL, a previously overlooked but practically important issue.\n2. Strong theoretical analysis supporting the IAR formulation, with well-structured theorems and proofs.\n3. Comprehensive experiments, including multiple baselines, evaluation metrics, ablations, and significance testing.\n\nWeaknesses:\n\n1. The ECA evaluation section is mathematically overloaded and may benefit from simplification or partial relocation to the appendix.\n2. There is limited discussion on applicability to non-anchor-based architectures such as Transformer-based LDL models.", "was_revised": true, "final_justification": "All my concerns have been resolved, and I am now inclined to recommend acceptance of the paper.", "rebuttal": "Dear Reviewer 21xv,\n\nThank you for your positive assessment on our work. We sincerely appreciate the time and effort that you have dedicated to evaluating our work. We have carefully considered all the comments and have revised the paper accordingly. Below, we provide a point-by-point response to your suggestions.\n\n***Question 1: Can IAR be extended to architectures without explicit anchor vectors, such as attention-based or Transformer models?***\n\nOur current theoretical framework and methodology are specifically designed for models with explicit anchor vectors, as the core mechanism relies on regularizing the angular separation between these anchor vectors. Therefore, our method is not applicable to models that do not involve anchor vectors. As elaborated in Section 6 of the main text, our proposed inter-anchor angular regularization (IAR) term is not directly compatible with tree-based LDL algorithms since their learning process does not incorporate anchor vectors. On the other hand, our approach actually can be integrated with transformer or attention-based architectures, as the IAR term solely constrains the weight matrix of the neural network's output layer.\n\n***Question 2: How sensitive is the performance to the choice of entropy threshold in ECA? Would a learned threshold work better?***\n\nActually, the entropy threshold does not participate in the model learning process; rather, it serves as a parameter in the model evaluation phase. The choice of this threshold parameter can indeed significantly impact the presented performance. For instance, if the threshold is set too small, the majority of test samples would be treated as high-entropy samples, thereby biasing the evaluation results toward high-entropy samples. To address this problem, our paper proposes treating the threshold as a distribution and computing the mean performance across all possible threshold values. In practice, a more reasonable approach is to determine the threshold based on specific task requirements—that is, defining what entropy level qualifies a sample as high-entropy or low-entropy according to the application's needs. As for whether a learnable threshold could be better, we argue that adaptive threshold would not be particularly beneficial, since the threshold is purely a parameter in model evaluation and does not participate in model training."}, {"review_id": "MCt4QLViAq", "reviewer": "Reviewer_r9iZ", "summary": "The authors demonstrate that many Label Distribution Learning (LDL) methods tend to make significantly larger errors on low-entropy instances compared to high-entropy ones, a disparity that standard aggregate metrics often obscure. To address this, they present Theorem 3.1, which links the cohesion—measured by small pairwise angles—of anchor vectors in softmax models to a lower bound on the entropy of the predicted label distributions. This theoretical insight helps explain why low-entropy samples are particularly challenging to model accurately. To mitigate this issue, the authors introduce an Inter-Anchor Angular Regularizer (IAR) that penalizes overly small angles between anchor vectors, encouraging more dispersed and expressive representations. Additionally, they propose a new evaluation metric, Entropy-Calibrated Aggregation (ECA), which separately averages performance on low- and high-entropy test subsets to provide a more nuanced assessment. Experimental results across eight benchmark datasets using four recent LDL baselines plus ridge regression, along with ablation studies (“ER”, “EW”) and a hyperparameter analysis, show that their approach yields substantial improvements on low-entropy samples without degrading performance on high-entropy ones. Overall, their method achieves the best ECA scores on six out of eight datasets.", "questions": "See W1", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. Connecting anchor geometry to entropy bias is simple yet convincing; the angle-entropy bound is intuitive and mathematically clean.\n2. Clear assumptions and and full proofs\n\nWeaknesses:\n1. All baselines are non-deep or shallow deep models. Demonstrating IAR on a modern backbone (e.g., ViT-LDL) would strengthen the claim of universality.", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer r9iZ,\n\nThank you for your positive assessment on our work. We sincerely appreciate the time and effort that you have dedicated to evaluating our work. We have carefully considered all the comments and have revised the paper accordingly. Below, we provide a point-by-point response to your suggestions.\n\n***Question: All baselines are non-deep or shallow deep models. Demonstrating IAR on a modern backbone (e.g., ViT-LDL) would strengthen the claim of universality.***\n\nWe selected two datasets where instance features are encoded by raw image information to demonstrate the effectiveness of the proposed IAR term in deep learning models. We apply the proposed IAR term to the ViT-LDL algorithm while maintaining strict consistency with the experimental procedure utilized in the main text. The next two tables show the performance of the ViT-LDL model with IAR term and the ViT-LDL model without IAR term. It can be seen from the next two tables that our proposed IAR improves the performance of ViT-LDL in most cases.\n\n|   Jaffe  | ViT-LDL with IAR  | ViT-LDL without IAR |\n| :------------- | :---------------- | :------------------ |\n| KL (LEA)       | 0.024$\\\\pm$0.0159  | 0.0329$\\\\pm$0.0167   |\n| KL (HEA)       | 0.0153$\\\\pm$0.003  | 0.0154$\\\\pm$0.003    |\n| KL (ECA)       | 0.02$\\\\pm$0.0105   | 0.0244$\\\\pm$0.0117   |\n| Cosine (LEA)   | 0.9784$\\\\pm$0.0148 | 0.9699$\\\\pm$0.0171   |\n| Cosine (HEA)   | 0.9851$\\\\pm$0.0029 | 0.985$\\\\pm$0.0032    |\n| Cosine (ECA)   | 0.9817$\\\\pm$0.0095 | 0.9774$\\\\pm$0.0113   |\n| Cheb (LEA)     | 0.0719$\\\\pm$0.0213 | 0.09$\\\\pm$0.0251     |\n| Cheb (HEA)     | 0.0474$\\\\pm$0.0039 | 0.0481$\\\\pm$0.0047   |\n| Cheb (ECA)     | 0.0597$\\\\pm$0.015  | 0.0693$\\\\pm$0.0184   |\n| Intersec (LEA) | 0.921$\\\\pm$0.0219  | 0.9019$\\\\pm$0.0233   |\n| Intersec (HEA) | 0.935$\\\\pm$0.0048  | 0.9354$\\\\pm$0.0056   |\n| Intersec (ECA) | 0.9281$\\\\pm$0.0154 | 0.9187$\\\\pm$0.0175   |\n\n|     Emotion6          |ViT-LDL with IAR  | ViT-LDL without IAR |\n|:--------------|:------------------|:------------------|\n| KL (LEA)        | 0.1094$\\\\pm$0.032  | 0.1091$\\\\pm$0.0267 |\n| KL (HEA)       | 0.1461$\\\\pm$0.0533 | 0.1327$\\\\pm$0.0307 |\n| KL (ECA)        | 0.1372$\\\\pm$0.0379 | 0.129$\\\\pm$0.0234  |\n| Cosine (LEA)    | 0.9712$\\\\pm$0.0038 | 0.9652$\\\\pm$0.0068 |\n| Cosine (HEA)   | 0.9724$\\\\pm$0.0056 | 0.9721$\\\\pm$0.0048 |\n| Cosine (ECA)    | 0.9677$\\\\pm$0.0065 | 0.966$\\\\pm$0.0048  |\n| Cheb (LEA)      | 0.1118$\\\\pm$0.014  | 0.1239$\\\\pm$0.0163 |\n| Cheb (HEA)     | 0.119$\\\\pm$0.0197  | 0.1202$\\\\pm$0.0174 |\n| Cheb (ECA)      | 0.1235$\\\\pm$0.0156 | 0.1284$\\\\pm$0.0118 |\n| Intersec (LEA)  | 0.8771$\\\\pm$0.011  | 0.8652$\\\\pm$0.0137 |\n| Intersec (HEA) | 0.8675$\\\\pm$0.0166 | 0.8665$\\\\pm$0.0151 |\n| Intersec (ECA)  | 0.8635$\\\\pm$0.0132 | 0.8596$\\\\pm$0.0098 |"}, {"review_id": "7v8xbV3xll", "reviewer": "Reviewer_zUMv", "summary": "This paper addresses the problem of entropy bias in Label Distribution Learning (LDL), where models tend to perform poorly on low-entropy samples due to overly cohesive anchor vectors. To mitigate this, the authors propose an Inter-anchor Angular Regularization (IAR) term that penalizes anchor vectors with small angular separation, thereby enhancing the model’s ability to represent low-entropy samples. Additionally, they introduce an Entropy-Calibrated Aggregation (ECA) strategy for fairer model evaluation by separately assessing performance on low- and high-entropy subsets. Experimental results demonstrate the effectiveness of the proposed methods.", "questions": "1.\tWhat is the form of the function $f (\\cdot)$? Does it use only anchor vectors as classifier weights?\n\n2.\tAre the anchor vectors treated as learnable parameters? It would be helpful to clarify the background of LDL.", "limitations": "The limitations are included in Section 6.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1.\tThe paper tackles the entropy bias in LDL, which is interesting.\n\n2.\tThe proposed method is built upon both empirical and theoretical analyses, and is well-motivated.\n\n3.\tThis work also proposes a new evaluation mechanism termed ECA for fairer model evaluation, which sounds reasonable.\n\nWeaknesses\n\n1.\tIn line 42, “the samples with high-entropy” should be “the samples with low-entropy”.\n\n2.\tFor better clarity, it would be beneficial to mention the term “ECA” in Section 4.\n\n3.\tIn the main text, it would be beneficial to reference the Appendices explicitly. For example, after presenting Theorem 3.1, a note should be added indicating that its proof is provided in Appendix A.", "was_revised": true, "final_justification": "All of my concerns have been addressed, and I am inclined to maintain a positive score for this work.", "rebuttal": "Dear Reviewer zUMv,\n\nThank you for your positive assessment on our work. We sincerely appreciate the time and effort that you have dedicated to evaluating our work. We have carefully considered all the comments and have revised the paper accordingly. Below, we provide a point-by-point response to your suggestions.\n\n***Weakness 1: In line 42, \"the samples with high-entropy\" should be \"the samples with low-entropy\".***\n\nAs suggested, we have corrected this typo error in the revised version.\n\n***Weakness 2: For better clarity, it would be beneficial to mention the term \"ECA\" in Section 4.***\n\nAs suggested, we have added an introductory paragraph about ECA at the beginning of Section 4 to elucidate its objectives and main concepts. The details are as follows. In this section, we illustrate the proposed ECA (Entropy-Calibrated Aggregation) strategy to address the entropy bias in conventional model performance evaluation methods. Following the divide-and-conquer principle, the main idea underlying ECA is to evaluate the model performance separately on the low-entropy and high-entropy subsets of test set.\n\n***Weakness 3: In the main text, it would be beneficial to reference the Appendices explicitly. For example, after presenting Theorem 3.1, a note should be added indicating that its proof is provided in Appendix A.***\n\nAs suggested, we have added the necessary references of Appendices in the main text. For example, the proof of Theorem 3.1 is provided in Appendix A; the introduction of datasets is provided in Appendix B.1; more experimental analysis are provided in Appendix B.2.\n\n***Question 1: What is the form of the function $f (\\\\cdot)$? Does it use only anchor vectors as classifier weights?***\n\nThe function $f (\\\\cdot)$ in the paper is a multivariate function that maps the feature vector $\\\\boldsymbol{x}$ to the label distribution. Specifically, it uses a softmax normalization over the dot products between the feature vector $\\\\boldsymbol{x}$ and the anchor vectors $\\\\{\\\\boldsymbol \\\\omega _ m\\\\} _ \\{m=1\\}^M$. The form of $f(\\\\cdot)$ is $softmax([\\\\langle \\\\boldsymbol\\\\omega_m, \\\\boldsymbol x\\\\rangle ] _ \\{m=1\\}^M)$ where $\\\\langle \\\\boldsymbol \\\\omega _ m, \\\\boldsymbol x \\\\rangle$ is the dot product between the anchor vector $\\\\boldsymbol \\\\omega _ m$ and the feature vector $\\\\boldsymbol x$, and $softmax(\\\\cdot)$ ensures the output is a valid probability distribution. \n\nThe function $f(\\\\cdot)$ does not use only anchor vectors as classifier weights. The function $f(\\\\cdot)$ is theoretically compatible with various representation learning techniques, serving as a classifier when combined with architectures like convolutional neural networks for image classification or graph convolutional networks for node classification. Specifically, in typical label distribution learning (LDL) models, the output can be formulated as $softmax([\\\\langle \\\\boldsymbol\\\\omega _ m, \\\\boldsymbol v \\\\rangle ] _ \\{m=1\\}^M)$, where $\\\\boldsymbol v$ represents the feature vector of a sample. In deep learning scenarios, $\\\\boldsymbol v$ is usually derived by passing the raw feature vector $\\\\boldsymbol x$ through a feature extraction network, whereas in non-deep learning settings, $\\\\boldsymbol v$ is often directly set to $\\\\boldsymbol x$. However, in our experiments, we deliberately abstain from incorporating representation learning techniques and instead employ anchor vectors exclusively as classifier weights. This choice is motivated by two key considerations: (1) Most available label distribution datasets are tabular in nature, rendering complex neural network architectures unnecessary. (2) To ensure a fair comparison with state-of-the-art baseline methods, which predominantly do not adopt representation learning techniques, we also avoid a representation learning technique. Thus, our practical implementation of $f(\\\\cdot)$ in experiments relies solely on anchor vectors as classifier weights.\n\n***Question 2: Are the anchor vectors treated as learnable parameters? It would be helpful to clarify the background of LDL.***\n\nYes, the anchor vectors are indeed treated as learnable parameters that play a fundamental role in the model's architecture. These vectors are optimized during minimizing the Kullback-Leibler (KL) divergence between the predicted label distributions and the ground-truth distributions, as formalized in Equation (3) of the paper. The learning process is further refined through the proposed Inter-Anchor Angular Regularization (IAR) term, which explicitly penalizes excessive cohesion between anchor vectors by minimizing their pairwise cosine similarities.\n\nLDL (Label Distribution Learning) is an effective approach to accurately estimate the entire conditional distribution of labels according to a set of feature variables. This task receives increasing attention both in the field of statistics and machine learning, as the information about the entire distribution is crucial in scenarios that are sensitive to risk, extremes, or uncertainty, such as drug efficacy prediction or emotion recognition. Various kinds of techniques, such as model calibration or mixture density neural network can be utilized to estimate the entire conditional distribution using the training samples that are labeled only with the mean or mode of the underlying true conditional distribution, which are beneficial in the tasks where the true label distributions are unavailable. However, there remain a large number of real-world scenarios where the true distributions are readily available. To address this kind of scenarios, LDL is proposed to learn a multivariate regressor that maps a set of feature variables to the entire conditional distribution of labels according to a training set where each instance is labeled with a label distribution. Compared to the cases without true label distributions, LDL is capable of predicting the entire conditional distribution of labels more accurately, as it is directly supervised by the true label distributions."}, {"review_id": "8ne9QhjB2f", "reviewer": "Reviewer_Z4dW", "summary": "This paper points out that in the current field of label distribution learning, the existing algorithms underperform on low-entropy samples, and the existing evaluation metrics of label distribution learning cannot adequately capture the prediction performance of samples with different levels of entropy. Aiming at these two problems, this paper theoretically analyzes the underlying reasons for the underperformance on low-entropy samples, designs a regularization term according to the theoretical results, and proposes a novel evaluation method for label distribution learning.", "questions": "1. In order to make the anchors as dispersed as possible, why not just minimize the inner product of the anchors instead of minimizing the cosine similarity of the two anchors? After all, cosine similarity imposes a greater computational burden compared to inner product?\n2. Are there some empirical suggestions for hyperparameter selection?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: 1) This paper presents the problem of underperformance on low-entropy samples and the bias of evaluation metrics in existing LDL studies, which are widespread and have serious impacts on the downstream decision-making tasks of LDL because low-entropy samples usually harbor expensive misjudgement costs. 2) This paper analyzes the causes of the underperformance problem of existing LDL algorithms on low-entropy samples from the theoretical and empirical viewpoints. The theoretical and visualization results reliably reveal the relationship between the hypothesis space and the low-entropy underperformance of the widely-adopted softmax model, which is instructive for the design of subsequent LDL algorithms. 3) This paper proposes a regularization term based on the results of the theoretical analysis, gives several mathematical properties of the regularization term, and verifies the validity of the regularization term through extensive experiments. 4) This paper proposes a new evaluation method that can effectively capture the prediction performance of samples with different levels of entropy.\n\nWeaknesses: 1) The experimental configurations of this paper are not adequately shown, such as the specific configuration of the optimizer used and the iteration stopping conditions. 2) The writing of the paper could be further improved, such as the mismatch of left and right parentheses in line 196, the lack of period at the end of the sentence in line 197, and in the captions of the subfigures in Fig. 5, Fig. (b) and Fig. (c) show the entropy mean and the entropy variance of the dataset but Fig. (a) shows only the mean value. The paragraph subheadings before line 267 end with a period, while the paragraph subheadings after line 267 end without a period, which is not uniform in format. The dataset name in the image caption in Figure 1 does not match the dataset name in the caption.", "was_revised": true, "final_justification": "The authors have addressed my concerns and thus I maintain my previous rating.", "rebuttal": "Dear Reviewer Z4dW,\n\nThank you for your positive assessment on our work. We sincerely appreciate the time and effort that you have dedicated to evaluating our work. We have carefully considered all the comments and have revised the paper accordingly. Below, we provide a point-by-point response to your suggestions.\n\n***Weakness 1: The experimental configurations of this paper are not adequately shown, such as the specific configuration of the optimizer used and the iteration stopping conditions.***\n\nWe have added more necessary information about the experimental configurations. The optimizer used in our paper is the L-BFGS algorithm implemented by pytorch. The code is shown as follows:\n\n```python\noptimizer = torch.optim.LBFGS(..., lr=1e-3, max_iter=1000, tolerance_grad=1e-5, tolerance_change=1.4901161193847656e-8, history_size=5, line_search_fn='strong_wolfe', max_eval=None)\n```\n\nThe above code shows that the optimization process will be stopped when the change of loss function is smaller than 1.4901161193847656e-8 or the gradient is smaller than 1e-5.\n\n***Weakness 2: The writing of the paper could be further improved.***\n\nWe have improved the writing of our paper. For example, we have added a left parentheses before the \"Art Painting\"; we have added a period at the end of the sentence in line 197; we have added the entropy variance in the caption of Fig. 5(a); we have calibrated the inconsistency of period of the paragraph subheadings.\n\n***Question 1: In order to make the anchors as dispersed as possible, why not just minimize the inner product of the anchors instead of minimizing the cosine similarity of the two anchors?***\n\nThe inner product of anchor vectors is also determined by their norm, which undermines its reliability in preventing over-uniformity. Mathematically, the inner product between anchor vectors equals to their cosine similarity multiplied by their $L_2$ norms. Consequently, the minimization of the inner product could result merely from reduced vector norms rather than improved angular dispersion (i.e., decreased cohesion among anchor vectors).\n\n***Question 2: Are there some empirical suggestions for hyperparameter selection?***\n\nAs analyzed in lines 249-254 of our paper, we recommend $\\alpha=40$ as a default setting for most datasets, as it approaches optimal performance across diverse scenarios. For further performance optimization, we suggest the following empirical guidelines:\n\n1. When the training set contains predominantly high-entropy label distributions, consider using $\\alpha <40$.\n2. For datasets with predominantly low-entropy label distributions, larger values are preferable."}]

Author rebuttal:

Meta-review: This paper identifies and analyzes entropy bias in Label Distribution Learning, where models struggle with low-entropy samples. The authors propose an inter-anchor angular regularization term and an entropy-calibrated aggregation strategy, supported by both theory and extensive experiments. Reviewers found the theoretical analysis clear and the empirical validation convincing, with rebuttal additions (new metrics, ViT results, real-data variance analysis) further strengthening the work. While some issues remain around hyperparameter sensitivity and broader applicability, the paper makes a solid contribution.
Decision: Accept (poster)