Approximately Correct Label Distribution Learning
Authors: Weiwei Li
Haitao Wu
Yunan Lu
Xiuyi Jia
Venue: ICML 2025 poster (2025)
Keywords: label distribution learning
approximate correctness
label polysemy
Primary area: general_machine_learning->supervised_learning

Abstract: Label distribution learning (LDL) is a powerful learning paradigm that emulates label polysemy by assigning label distributions over the label space. However, existing LDL evaluation metrics struggle to capture meaningful performance differences due to their insensitivity to subtle distributional changes, and existing LDL learning objectives often exhibit biases by disproportionately emphasizing a small subset of samples with extreme predictions. As a result, the LDL metrics lose their discriminability, and the LDL objectives are also at risk of overfitting. In this paper, we propose DeltaLDL, a percentage of predictions that are approximately correct within the context of LDL, as a solution to the above problems. DeltaLDL can serve as a novel evaluation metric, which is parameter-free and reflects more on real performance improvements. DeltaLDL can also serve as a novel learning objective, which is differentiable and encourages most samples to be predicted as approximately correct, thereby mitigating overfitting. Our theoretical analysis and empirical results demonstrate the effectiveness of the proposed solution.

Approximately Correct Label Distribution Learning
Weiwei Li Haitao Wu
Yunan Lu Xiuyi Jia
Abstract
label distribution d, represented as a probability simplex, where each label is associated with a real-valued description degree d% By quantifying label polysemy, LDL enables a broader spectrum of applications, including tasks like ob- ject detection (Xu et al., 2023), age estimation (Wen et al., 2023), sentiment analysis (Chen et al., 2020), and diagnostic assistance (Li al., 2023), etc.
Label distribution learning (LDL) is powerful learning paradigm that emulates label polysemy by assigning label distributions over the label space. However, existing LDL evaluation met- rics struggle to capture meaningful performance differences due their insensitivity to subtle dis- tributional changes, and existing L.LI learning objectives often exhibit biases by disproportion- ately emphasizing small subset of samples with extreme predictions. As result, the LDL metrics lose their discriminability, and the LDL objec- tives are also at risk of overfitting. In this paper, we propose DeltaLDL, a percentage of predic- tions that approximately correct within the context of LDL, solution the above prob- lems. DeItaL.DL can serve as novel evaluation metric, which is parameter-fiee and reflects more on real performance improvements. De can also serve as a novel learning objective, which is differentiable and encourages most samples to be predicted as approximately correct, thereby mitigating overfitting. Our theoretical analysis and empirical results demonstrate the effective- ness of the proposed solution.
For years, there are some deep-rooted problems in the field of LDL: 1) existing evaluation metrics for LDL are based some distance/similarity metrics, e.g., Kullback-Leibler divergence (KLD), which have very small values and thus have poor discriminability between superior and inferior models, since the label distribution subject to two con- straints, non-negativity (i.e., du 2 0) and sum-to-one (i.e., Eney d% = 1); 2) existing LDL algorithms mainly focus minimizing the average of some kind of measurement, usually the KLD as well, which may lead to overfitting due to excessive preference for some samples, resulting even worse performance than that calculated from uniform vectors.
To kill two birds with one stone, the idea of approximately correct prediction can be introduced. On the one hand, it enables the design of percentage- baseggeee--dddd-ee metrics, quantifying what proportion of the samples are predicted approximately correctly, which is intuitive and semantically meaningful; on the other hand, it enables the design of noise-robust ob- jectives, allowing the model to sacrifice samples that are particularly difficult to learn, ensuring that most samples are predicted approximately correctly, thereby avoiding overfit- ting to specific subset of data.
1. Introduction
Label polysemy is common phenomenon in real-world scenarios, traditionally emulated by multi-label learning (Zhang & Zhou, 2013), which treats labels in a binary fash- ion. In contrast, label distribution learning (LDL) (Geng, 2016) offers a more refined framework by answering the question: "To what extent does each label y describe the instance x?? This is achieved through the concept of
Contributions and organizational structure In this paper: 1) we conduct a theoretical analysis of the KLD to demon- strate its unsuitability as an evaluation metric/learning ob- jective for LDL (Section 2.1): 2) we propose DeltaLDL, a function that calculates the percentage of label distributions that approximately correctly predicted, exhibiting desir- able properties, making it promising candidate for both metrics and objectives (Section 2.2); 3) based on DeltaL.D,, we propose a novel evaluation metric, the i metric, which possesses superior discriminative power according to our analysis (Section 3); 4) based on DeltaLDL, we propose a novel learning objective and encapsulate it in a new LDL algorithm, named 8-LDL, which can achieve highly compet-
"College Computer Science and Technology, Nanjing Uni- versity of Aeronautics and Astronautics, Nanjing, China "School of Computer Science and Engineering, Nanjing University of Sei- ence and Technology, Nanjing, China 'Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China. Cor- respondence to: Xiuyi Jia <jiaxy @ njust.edu.cn>>
Proceedings of the nnnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Approximatcly Correct Label Distribution Learning
itive performance according our experiments (Sections 4 and 5). Related work and limitations are discussed in Sec- tions 6 and 7, respectively.
Table 1. Case analysis of two L.L. datasets
Case
Dataset
E(KL(u v))
SA-BFGS
LRR
SBU_3DFE
.0707
.0627
.0571
2)
MºB
.6335
.6878
.6664
2. Theoretical analysis
In this section, we conduct a theoretical analysis to argue that the KLD is not suitable as an evaluation metric/learning objective for LDL. Then, we propose our new solution, named DeItaLDL.
where r()) is the gamma function. Assumption 2.1 is cur- rently the most common assumption in the literature of LDL. (Lu & Jia, 2022; He et al., 2024). Based on Equation (5), we have the following proposition.
Preliminaries Let x E X = Rs denote the feature of the instance and d = (d= )5=1 E A denote the label distribution, where с is the number of the labels, Fy is the j-th label in the label space y/ = {リ, -;=1: and
Proposition 2.2. The expectation of the KLD between a Dirichlet distribted random vector u and a uniform vector
1 c
(6)
(1) the (I: 1) dimmensional probability simplex. LDL's goal is to find a mapping given a training set {a:;, d: |m and an LDL model [(; O). the loss function can be defined as
A k-1 A {0 E R*
Iv T = 1, v 2 0}
c
X' - 1-1 More specifically,
is given by
[(KL(u||0))
aj
m
Ei=l 1 ak
()))
dist(s(æ; θ), d.),
j=1
m
v(a + -b 3/
Σ Okk +11)
+
aj
log
e
where @ is the model parameters and dist(-, :) is a distance metric, usually the KL.D. The optimal parameters O* can be obtained by minimizing Equation (2), i.c.
kk=1
Og
(7)
where v-)) is the digamma function. E(KL(u|I can be regarded as the theoretical stochastic KLD performance of LDL model.
θ
arg min (()).
2.1. Issues of existing metries: KLD as an example
Proof. See Appendix A for details.
KLD is defined as
Remark 2.3. To estimate a from given data (d.),- one can employ maximum likelihood estimation (MLE), i.e. minimizing:
KL (1 = Σ U
log Uy 1=1 Un
(4)
which exposes shortcomings in two aspects. First, KLD not bounded and only sensitive to small probability values of the prediction, i.e., KL[(u]) v) E (00+ and KL(u () - 3o if 11 0+ Aj [c]. Most existing LDL methods are based on maximum entropy models or neural networks with softmax as the final mathematical processing, hence it is difficult to trigger its sensitivity. Second, KLD does not adequately capture the changing nature of prob- ability simplexes, i.e., KL(u || v) lacks the significance of changes to tell the difference between superior and inferior models when u. --1, To illustrate the second aspect, we introduce the following assumption and proposition. Assumption 2.1. The ground-truth label distribution d fol- lows Dirichlet distribution, i.e., the probability density function of d is given by
( =
log p(di a)
i=1
ln
aj
In T(as))
(8)
y=1
m
(aj 1) In di
1=
the closed-form solution of which does not exist. Thus, one can employ numerical optimization methods, e.g., L-BFGS, to solve the optimization problem.
With Proposition 2.2 and Remark 2.3, one calculate E(KL("u v)) for each LDI. dataset. Here we conduct a case analysis of two LDL datasets, SBU 3DFE and M²B, and compare the KLD performance of two learned LDL meth- ods, SA-BFGS (Geng, 2016) and LRR (Jia et al., 2023b), both of which exploit average KL.D as the learning objective.
1 p(d; a) B(a) j=1
1 [(ag) a B(a)= r(2;- az)"
II
deg
(5)
2

---

Approximatcly Correct Label Distribution Learning
The results are shown in Table 1. In case 1), since label dis- tributions in SBU 3DFE are non-sparse, the calculated values are very small, lacking discriminability. i.e., considerable performance improvements are only reflected via these nu- anced changes, which is counterintuitive; in case 2), results of SA-BFGS and LRR are worse than the stochastic perfor- mance, indicating that minimizing average KLD sometimes does not work as expected, which may be attributed to the negative contribution of some extremely predicted samples.
metric employed. In this paper, we focus primarily on D(KI,, 1 -), since the KLD is the most widely used evalu- ation metric in the field of LDL. Following the definition of DeltaLDL, one may wonder how it can be utilized as an evaluation metric/learning objective.
3. DeltaLDL as an evaluation metric
To transform D (KLL, :) into evaluation metric, the fol- lowing challenges must be addressed: 1) the metric should be parameter-free, as 5 varies with different label distribu- tions and is difficult to determine; 2) the metric should effec- tively reflect real performance improvements, enabling sig- nificant differentiation between superior and inferior models. To this end, we propose the following definition.
2.2. The proposed solution: DeltaLDL
On the one hand, Proposition 2.2 fully illustrates the non- significance of KLDD an evaluation metric/learning objec- tive for LDL; on the other hand, it provides a baseline of performance improvements to get glimpse of real perfor- mance improvements, which will be discussed in Section 3. Taking inspiration from the analysis of Table 1, we define DeltaLDL as our new solution.
Definition 3.1 (The discriminative power μ)). Suppose that LDL model is evaluated on the test set, where the ground-
truth label distribution matrix is D = (d, itd. D) and the predicted label distribution matrix is D (d.) which is given by the LDL model f; our new evaluation metric based on (KI,,, 6; f). denoted as μ. is defined as
Definition 2.4 (Approximately correct LDL; DLIt Suppose that an LDL model f(: )) is evaluated on fax, di 1 DeltaLDL of a distance metric dist(;,)) is defined as the following function:
PdD
D) A
O(KL, 5; f)dō
6p
1
D (dist, 8; f)
M(dist(d; f(z;; +)) S 5), (9)
1
no
T(KL(d (1 S 5)d8 (11)
1=1
80
where I(() is the indicator function to count the number of approximately correct predictions. DeltaLDL of similarity metric sim(., :) is defined as the following function:
[0, 1|2.
where
8. E (KL(u [])) u D,
(12)
1 12
(sim, 6; f)
Tdsim(d,, [(m;; θ)) 2 8). (10)
and v is a uniform vector (the same as Equation (6)).
Since the test set is available in the evaluation phase, Equa- tion (12) can be calculated as:
Remark 2.5. The following properties hold for D:
D(5 [0, 1/::
n
a d In Cefc.
(dist, suPu,ves= dist(u, v); -) = 1; D (sim, inf inf u sim(u, u. v): 1;
50
%
KL 11
n2
n
(13)
(diit, 5; -) < (dist, 8¹: if S 5';
According to Definition 3.1, p's semantic essence lies in the overall performance of the predictions being approximately correct across different values of d; mathematically, μ is defined as the ratio of the area enclosed by the function D (KL,, 6; .) w.r.t. 8 and the coordinate axes, where 5n' serves as the scaling factor. By adopting an integral form, eliminates dependency on specific addressing the afore- mentioned challenge 1; the introduction of do establishes a baseline via the value based uniform vectors, exclusively capturing performance improvements above this baseline, thus effectively resolving the challenge 2.
(sim, .) 2 (sim, 8': :), if 5 5':
dist(d;, ((x;)) = infu,ees-- dis((u, v). Vi E [n). if
and only if 2 (dist, infu,ves- dist(u, v); f) = 1;
sim d,, f(x,)) suPu,ved--1 sim v). Vi E [n]. if and only if D (sim, supu,vE.- sim(u, v); f) = 1.
As stated in Definition 2.4, any distance/similarity metric can be transformed into a percentage of predictions that are approximately correct within the context of LDL. Further- more Remark 2.5 shows some good properties of Deltal.L.. which are intuitive and easy to proof by readers. These prop- erties indicate that DeltaLDL has excellent potential as an evaluation metric/learning objective, and also highlight the relationship between Deltal.DL. and the distance/similarity
Discriminability analysis For better comparison, we de- fine the real performance improvement based on the KLD/U(;; +), with little symbol abuse.
3

---

Approximatcly Correct Label Distribution Learning
0.6
A p((d.) (dit
Values
0.4
Mo Biu((d)) v)]
111
KL((d) 11 (d))_))
0.2
[/. (1e))
0.0
50
100
150
200
# Sumples (R)
Figure I. N1, No. fa, and μn of DF-LDL w.r.t. the number of samples in the test set of SBU_3DFE.
Definition 3.2 (The real performance improvement based on the KLD). Let Ko = (KL((D || v)), where v is a uniform vector; suppose that an LDL model is evaluated on the test set and the KLD is K1; the real performance improvement based on the KLD is defined as KR - K1 Ko-
distribution matrix is D (d, Ltd. D) and the predicted
label distribution matrix D (d. ) which is given by
the LDL model f: then (KLL, ó; f) can be transformed
into the following basic loss function:
Definition 3.3 (The real performance improvement based on the l metric). Let fo = Bip(D D, v)). where v is a uniform vector; suppose that an LDL model evaluated on the test set and the value of the μ metric is M1; the real performance improvement based on the y metric is defined as PRR
m Co(ó; D, D = m
Ms
a
da.
In
8
das
(14)
which is differentiable if (-)) is appropriate. The above chal- lenge lies in ensuring that D D(KI., 6; as possible while keeping 8 as small as possible. (Note that is difficult to determine, as mentioned in Section 3.) This requirement can be understood geometrically as minimizing the area enclosed by the function (ō; ", w.r.t. 5 and the coordinate axes. We can use adaptive Simpson's 's rule (ASR) (McKeeman, 1962), numerical integration method, for loss function calculation, to solve this challenge. Remark 4.1. From Simpson's 's rule, we have
p1 M4
should be as large
Similarly, readers can get the real performance improve- ment based on other evaluation metries. We train DF-LDL model on the training set of the SBU 3DFE dataset, and eval- uate on the test set by calculating K1, Ko, (4,, and μo with varying numbers of test samples. The results are shown in Figure 1. As can be seen, (AR is much larger than KR. indicating that the μ metric is more sensitive to performance improvements than the KLD, which is consistent with our theoretical analysis. One limitation worth noting that, when there are few samples for testing, the μ metric is unsta- ble, which is the nature characteristic of percentage-basse metrics and can be eliminated by increasing the number of samples. Since De has been successfully trans- formed into an evaluation metric, the j metric, one may wonder how it can be utilized as learning objective.
1: Eold: v, )d8 spu(a, b)
(15)
b- ((((a) + 4ho 6
a (a+b) (b+b) + lo(b)
2
In adaptive Simpson's 's rule, the term Va,b quantifies the error of the basic Simpson's approximation by considering the subdivision at the midpoint. It is defined as:
4. DeltaLDL as a learning objective
To transform (KKL,, :; -) into learning objective, the fol- lowing challenges must be addressed: 1) the objective should not contain the indicator function I(-), as it is dif- ficult to optimize; 2) the objective should be to sacrifice small number of samples that are difficult to learn and ensure that most samples can be predicted as approximately correct. To solve the challenge 1, we introduce smoothing function (-) to replace ([.)) Suppose that an LDL model is training on the training set, where the ground-trus label
Va. - Pola, +ppup,,, -)- (o b), c= b
(16)
2
The adaptive Simpson's 's rule refines the integration by re- cursively subdividing the interval until the error is below a specified tolerance e. Remark 4.2. From the adaptive Simpson's 's rule, we have
[" lo(0: :, )do p(a, b, E) =
For readers who find these definitions obscure, we herehy provide a metaphor: Ko and ni like the accuracy of random selection multi-classifictioon problems.
|Va,a] S 15e, (17) o/w.
pola, c) + polc, b) + vn./15, ( e, 6/2) + p (c. b, f/2),

---

Approximatcly Correct Label Distribution Learning
Algorithm 1 Adaptive Simpson's 's rule: ASR
differentiable and its gradient w.r.t. O is given by
Input: Interval bounds b: emror tolerance e; current integral estimate s, maximum recursion depth
(lo(O)
2 [[KLL(DI) D) 8) o (x (x (D - D)), (21)
Output: Integral estimate s'.
I: c + (/+
where @ is the element-wiss product.
2: [ - (o c):
Equation (15)
3: r 4 (o(c, b);
Time complexity analysis Let X E =="x4 denote the fea- ture matrix and D E mgmx denote the ground-truth label distribution matrix; D = f(x; ; θ);
4: if u
15s or < 0 then
5:
return + + 11/16;
Equation (17)
6: end if'
the overall time cost of 6-LDL. (Algorithm 2) is primarily influenced by the fol- lowing calculations: computing 8 in Equation (19) has complexity of (me); the forward pass step in Equa- tion (20) requires (mgq); each calculation of the KLD,
7: return ASR(a, 6/2, 1, - + ASR(e, 6 4/2, 1):
Algorithm 2 Our proposed algorithm: 8-LDL
Input: Training set ((x,, d.) )= maximum recursion depth Ç, test sample x'.
D C (log D log D), has complexity of O (me); calculating the gradient, an/ae = x (D D), involves complexity of (mgec.. Since the parameter is fixed, to is computed a limited number of times and this number remains relatively small. The overall time complexity of each iteration of 6-LDL (kmqq). where & the number of calculations of Equation (14), attributed Algorithm 1, is linear w.r.t. the number of samples m/labels clarify- ing that our proposed 6-LDL can be applied to large-scale datasets. Then, one may wonder how S-LDL performs in practice.
Output: Label distribution d'.
1: EPS < 10-;
2: @ random initialization;
3: & + +1/m E Σラ=1 d2; In d%
Equation (19)
4: for each epoch to anticipated epoch number do
5: B 6- --randomly split { d,)]== into batches;
6: each batch in B do
7:
D - Da= @). Væ E the batch;
Equation (20)
8:
< (0, 6):;
Equation (15)
9:
f < (S, dg. EPS, so, ();
Algorithm
10: @ update Adam, tracing e;
Equation (21)
11: end for
12: end for
13: d' < [ax'; 9)
Equation (20)
5. Experiments
14: return d';
In this section, extensive experiments are conducted illus- trate the superiority of the μ metric and 6-LDL. Details of all implementations are openly accessible at GitHub.
The rule is summarized in Algorithm 1. Finally, the loss function can be defined as:
5.1. Experimental setup
E(D, D) = (00, 8, EPS).
(18)
Metries & datasets We mainly evaluate LDL models' per- formance via the f t metric proposed in Section 3. We also represent the same metrics suggested in (Jia et al., 2023b), which are Cheby. 4 (Chebyshev distance), Clark + (Clark distance), Can. 4 (Canberra distance), KLD 4 (Kullback- Leibler divergence). Cosine 1 (cosine similarity), Int. t (intersection similarity) and Spear. (Spearman's 's coeffi- cient), respectively. Here (f) indicates "the lower (higher) the better"" We adopt several widely used label distribution datasets, including: M²2B (Nguyen et al., 2012), fbp5500 (Liang et al., 2018), RAF ML (Li & Deng, 2019), SBU_3DFE (Yin et al., 2006). Natural Scene (Geng et al., 2021), Music (Lee et al., 2021) and Painting (Machajdik & Han- bury, 2010).
Implementation details In Equation (14), o(-) is imple- mented as the ReLU function; in Algorithm 2, E is set to 5; in Equation (18),
%
dz In dafc..
(19)
7
j=1
serving as a performance baseline, and EPS can be small positive number, e.g., 10-7 The above loss function can be optimized by gradient descent methods, such as Adam (Kingma & Ba, 2015). The LDL. model ff is implemented by a naive network with the parameter matrix O E Rixe;
Comparison methods & methodology We compare 8-LDL with several LDL methods. On the one hand, methods based on the optimization of the average KLD are mainly compared, including: SA-BFGS (Geng, 2016), LDLLC (Jia et al., 2018), LDLSF (Ren et al., 2019a), LCLR (Ren et al., 2019b), SCL (Jia al., 2019), DPA (Jia et al., 2023a) and
(x: ) = ((x
(20)
where s(-) is the softmax function as the final mathemat- ical processing. The overall algorithm is summarized in Algorithm 2, denoted as 5-L.L..
Remark 4.3. Let Eo(8; D, f(x; θ)) be ee): assume that (-) is the ReLU function; D f(X; θ); to (+) is
Phttps://gitee http::::/giccuubcoom//g/ com/ www.spri//s.eeecocccoo Pa...
5

---

Approximatcly Correct Label Distribution Learning
Table 2. Experimental results on B'B, fbp5500, and SBU_3DFF 3DFE, formatted as "mean + atd"
Algorihmms
Chaby. +
Clark
Can.
KLD
Cosine
Int.
Spear.
N'B
PT Bayes
.5394 1. ..43
1.7487 1. 064
3.6172 168
1.7422 1. 293
.5272 2..03
.4498 L. 042
.5758 1. osa
25.76% .0.4
SCL
.4788 + .11s
1,6358 + 010
3.3495 + 082
.7773 = 022
6842 =.014
.4741 = 0100
$752
26.67% .012
LDLSF
.3855 + .018
1.5441 + 0031
3.0624 + 083
2.0495 + 497
.7363 =.022
.59911 - 0199
.6405 4 022
32.95% + .030
LDLF
.171 2.024
1.5266 +: 048
3.1013 114
5688 -t. 061
75844 = 20000
.56044 1: 028
6984 1: 0.1
44.91 56 ..04
LCCR
.3677 #,022
1,2516 +
2.4208 d:. $5.009
7300 = .0.
77152 = -,06
.000 - 022
6862 1: 020
52.13% + .081
DPA
.3447 +.
1.1734 000
2.2681 + 071
7013 = ..8
.7187 = 220
.6755 220
.636 1: 021
52.95% +:
LDLLC
.3845 +
1,1733 + 030
2.2683 + 070
7003 + 068
71189 =
.0077 -
6939 +
52.99% +
SA-BFGS
.3826 -. 021
1,2228 + 028
2.3520 + 0688
6878 091
.7231 =
6080 - 021
6886 022
53.06% +
LRR
.3793 020
1.1675 1. 028
2.2525 1. 068
6664 1. 008
.7271 .2.5
.6127 o21
.986 1. 024
53.90% 1:
AA-ANN
.3721 .20
1,0566 1. 081
1.9678
067
6179:1 0774
..426 4.02
.6191 1: 120
.7022 1: 026
55.21 1: .026
DE-LDL
.3699 ..09
11011 + 025
2,0883
068
5743 - 0772
- .199
.7049 022
55.36% .0.7
6-LDL
.3599 2.07
+. 023
2,0629 051
4882 -:. .o.
2.00
.633 + 0177
6943 023
58.25
.225
#bp8500
PT-Baaes
165669 + 011
16829 +. 01.
3.1196 1:196:.um 052
15.2111 3.362
.479 2.055
.33413 011
6987 + 010
.08% + .001
SCT.
2822 4000
1.4191 +
2.6654 1: 022
4370 oo.0
8442 = 000
.6339 .004
7314 + 050
38.39% +: 0011
LDLSF
14500 -
1.0405 + 01.
o 1.6992 1001
3391 =0.00
.915 = .002
.381 + 0008
8876 +: 0000
69.42% +: .013
AA-ENN
1515 4. .04
L0482 +: 0155
о 1.7383 1: 031
1819 o13
.4188 =
.315 +:
8860 1: 000
78.19% 4:
LCLR
1386 -.
1,2978 + 000
2.2253 + 023
I1 + 005
.5233 =
8475 +
9009 + 000
84.36% +
SA-BFGS
.369 + 2003
1,2945 + 009
2.2146 + 0222
1084 + 005
.9566 .003
8494 = 0044
.025 + .0.0
84.77% +
LDLF
.1348 1. .04
1,2786 1. 011
2.1763 1. 020
1058 11 00G
.9549 .000
.8503 1: 000
.017 1. 000
85.18% 1: .00s
DELDI
.1346 .00
1,2898 1: 01.
2.2008 1: 020
1055 -t. 0o5
.9548 .00
.8188 0088
9042 1: 004
85.18% + .006
.1347 000
1,2844 + 01.
2.1895 024
1053 nos
=
+ 0033
9038 po1
85.19% .ooe
LDLLC
.1348 +. 0033
1.2844 01.
1894 024
1052 +: 005
.9499 -00
.811 003
9036 po1
85.20% + ooo
LRR
.343 2.03
1.2837
+ 010
2.1871
1.02
1046 004
9552 -002
8516 o003
9040 .004
85.29% .006
6.L.L.
.1300 i.o0s
1,2783 11o
2.1685 1. 1.023
0997 oos
.9566 ..002
.8560 1 .003
.961 100
85.99 % ..ooe
SBU_3DFE
PT PTyBayes
.1289 .00
.0019 +. 009
8600 f. 020
U772 +t .04
9247 =000s
.842 + .004
1144 +: oss
33.40% .:10
AA-ENN
.1722 1:
.001 o00
8281 1: 020
0801 1: 004
9217 =
8489 1: 004
2053 1: 030
4: 36.18%
LDLSF
.1091 +
.0200 +
8224 + 021
0084 + 0200
9402 =
8591 + .00004
3358 + 025
37.61 % +
SA-BFGS
.151 + .044
3706 + .009
.7827 + 020
.067 + 003
.9844 =
.3999 + .044
3001 + 022
39.68% +
LCLR
1061 1. ..04
.3277 1. n00
.7531 1: 020
.0579 11 0038
.9438 0008
8669 1: .04
.3573 1 025
43.42% 1: .020
L.D.ICC
1078 2.0
.3811 1: 000
.7476 1:
0073 -:. .088
.9499 2onss
.877 1 0008
.341 .25
43.49% + .016
.1077 .003
.3580 4: one
.7474 o.1
JU572 003
.003
8662 +: 0033
3446 027
43.56% =t.orr...
LRR
.1078 + 003
.3776 on8
.7470 018
0571 + 003
.9440 = 003
8672 + 003
3421 027
43.56% = 011
SCL
.1111 - .044
.3112 + 008
..123=.018
0573 40 0o3
.9436 .003
8658 - 0033
2052 027
43.61 % -.17
LDLF'
.1023 1oo
.3310 1. 008
.6940 1.01
.0512 1. 003
.9922 i.0os
.8660 1: o003
4D45 1. 022
48.82% d.
DEI.
.0001 1: .00
.3276 11 007
.6817
1: 016
.0493 1: 000
.9511
- 1000
.8800 1: 000
.273 1: 022
49.93% J. o11
6-LDL
.0986= +
.322 +: 00.
.6794 + 0.11
0494 + 0033
.9533 =
.8793 + 003
A345 d: 0:29
50.73% +
LRR (Jia et al., 2023b); on the other hand, methods based on specified structures also compared, including: PT-Bayes (Geng, 2016), AA-KNN (Geng, 2016), LDLF (Shen et al., 2017) and DF-LDL (González et al., 2021). To ensure a fair comparison, for each dataset and for each method we conduct ten-fold experiments repeated 10 times, and the average performance is recorded.
AA-kNN, exposing the limitations of minimizing average KLD; in contrast, 5-LDL achieves the best performance on five metrics, significantly outperforming the other methods. Note that the KLD evaluation of 8- 5-LDL on M is 0.4882, which is lower than the stochastic performance, i.e., 0.6335 in Table 1, which fully demonstrates the effectiveness of our proposed 6-LDL. The results on fbp5500 and SBU 3BU_3DFE are similar, which implies that 8-LDL is effective on different datasets. Remaining results can be found in Appendix B.
5.2. Results and discussion
Label distribution prediction Table shows the represen- tative label distribution prediction results. (o) indicates "8-LDL is statistically superior (inferior) to the comparing methods (pairwise t-test at 0.05 significance level); there is no significant if neither nor is shown; the best and second-best results are highlighted in bold and underline, respectively. In terms of performance on M2B, a consider- able number of tfae-of-tthe-an methods are even inferior to
Regarding the 1a metric, it is always significant on t-test as our expectation. Note that the ranking derived from the μ metric may be different from that derived from the KL.D. An obvious example lies in the comparison between LDLF and DF-LDL on MºB, where the KLD evaluation of LDLF is 0.5688, which is lower than that of DF-LDI,, i.e., 0.5743, while the μ metric shows that DF-LDL is superior to LDLF. This is because the a metric focuses on the percentage of
6

---

Approximatcly Correct Label Distribution Learning
predictions that are approximately correct, which is less susceptible extreme predictions. Compared with other metrics, the y metric has explicit range, is more discrim- inable between superior and inferior models, and is more intuitive, reliable, and interpretable.
180
Thirsee ttez
130
500
336
180
41DI
HUll
122
Abation study & analysis on parameters Here we are interested in the importance of each processing step in 8- LDL, thus an ablation study on M²B is conducted: 1) we replace the loss function with solely fu(%; ; -) to examine the importance of f, and this model is denoted as 6-LDL 2) we replace the ASR with soicly Simpson's 's rule by setting { 0, and this model is denoted as 6-LDL w/o ASR. The results are shown in Figure 2(a), which confirms that each processing step 8-LDI. indispensable. We conduct an analysis on parameters on M²B as well, where the KLD and training time w.r.t. ç are also shown in Figure 2(a). The training time initially increases sharply with the growth of S. but gradually levels off. This is because, as & increases, more samples are able to meet the conditions for halting the computation of the ASR loss in advance. For the balance between training time and performance, { is fixed at in our proposed 6-LDL.
331-
130
300
20-
1.0
0..0
0756
0.5
0.5
(a) KLD/training time w.r.t. 6
(b) Ø(KL, 6; w.r.t. O
Figure 2. (Best viewed in color) Visualized results of the analysis on parameters.
1/0
AM
1.0 610
353
Though 8-LDL is d- S-parameter-fieee
can still analyze the
impact of 8 on ((,, 5; .) with predictions of an experiment. We analyze various (,,, 5; -)s w.r.t. 8 with predictions of one experiment on B,B, results of which are shown in Fig- ure 2(b) and Appendix B. The gray lines indicate the integra- tion interval. In terms of our proposed 8-LDL, (KLL, ó; +) larger that that of other methods when 5 is small, which makes the area under the curve larger. It is worth noting that D (KL, d; -) is monotonically increasing function of 5, which is consistent with our theoretical analysis.
Stadad
(a) Visualization of noise data (b) KLDIR w.E.t noise level
Figure 3. (Best viewed in color) Visualized results of the robust- ness testing of the y metric.
ods, exploiting the same artificial dataset mentioned above. From bottom to top, from left right, the four types of processing in Figure 4(a) are: 1) no treatment; 2) applying Gaussian noise (He et al., 2024); 3) randomly setting de- scription degrees to zero (Xu & Zhou, 2017); 4) randomly emphasizing description degrees (Kou et al., 2023). These treatments correspond to different possible types of noise in label distributions. The prediction results of represen- tative methods are shown in Figures 4(b) to 4(g), and the ground truth, i.e., the original artificial data without noise, is shown in Figure 4(h) for comparison. The consistency of each prediction result reflects the robustness of each method. Although it is hard to visualize the so-called "deci- sion boundary" in the field of LDL, Figure 4 shows that our method predicts accurately and the prediction results are not easily affected by noise.
Robustness testing We now evaluate the robustness of the y metric, exploiting the artificial dataset introduced in (Geng, 2016). First, we generate the artificial dataset with 200 200 samples. Then we continuously apply noise along a cer- tain axis of the features, which is visualized in Figure 3(a). The KLD and metric w.r.t. the noise level are shown in Figure 3(b). The KLD lacks the significance of changes when the noise level is low. Then, sharply increases and becomes highly volatile once the noise level surpasses a certain threshold. This because when the noise increases certain extent, the label distribution will become sparse, triggering the sensitivity of the KLD, which is consistent with our theoretical analysis. The early tiny changes/the later substantial fluctuations in the KLD are detrimental to the discriminableereiablle evaluation of LDL models. In con- trast, the μ metric remains stable and gradually decreases with the noise level. This is because the μ metric is based on the percentage of predictions that are approximately correct, which is less sensitive to noise.
6. Related work
Our work is mainly related to LDL. LDL was initially pro- posed by Geng et al. to tackle the problem of age estimation, and subsequently developed into a novel paradigm of ma- chine learning (Geng, 2016), which now supported by
Next, we evaluate the robustness of 8-LDL and other meth-
7

---

Approximatcly Correct Label Distribution Learning
-10
-10
Noise data
(b) PT-Baaes
(c) AA-ANN
(d) LDLSF
10
13
(e) DF-LDL
(1) LRR
(g) d-LDL
(h) Original data
Figure 4. (Best viewed in color) Visualized results of the robustness testing of 6-LDL and other methods.
theoretical underpinnings (Wang& & Geng, 2019). Over the years, a large number of specialized algorithms for LDL have emerged (Geng et al., 2013; Geng & Hou, 2015; Shen et al., 2017; González et al., 2021).
(Jia al., 2018; 2023a), ADMM (Ren et al., 2019a;b), and gradient-based methods (Jia et al., 2019; 2023b). These methods 1) implicitly address overfitting through ad-hoe regularization, indicating the field of LDL lacks principled solutions; 2) sometimes are worse than average performance deduced by uniform vectors, which could not even be ver- ified before our work. The essential difference between our proposed 8-LDI. and these methods is that it no longer focuses on all samples, but sacrifices part of them by using approximately correct interval.
Regrading evaluation metrics, the average KLD is the most widely used one and it is first considered by Geng, who advocates hierarchical clustering to select representative metries, and demonstrates that Clark and Canberra metrics suffer from oversensitivity to small values. Subsequent stud- ies by Xu & Zhou revealed KLD's 's unreliability with sparse predictions, prompting calls for alternative measures. Met- rics that used in early work therefore directly borrows from existing mathematical foundations, e.g., Topsoe (Ren & Geng. 2017). Sørensen (Ren et al., 2019a) and Spearman's 's coefficient (Jia et al., 2023b), etc. There is little work designing new metrics until recently, e.g., the DPA proposed by Jia et al., which, however, from the empirical results, still lacks discriminability between superior and inferior models. The CAD, QFD and CJS proposed by Wen et al. do not address this issue either, and only work with ordinal label distributions. Our proposed j metric, which based on the percentage of predictions that are approximately correct, is the first to tackle this issue.
Furthermore, the parameter 5 is potentially related to the parameter P in the margin theory, which is exploited in some LDL work (Wang & Geng, 2019; 2021a;b). The difference, however, is that: 1) their work focuses on the problem of objective mismatch, using margin theory to improve the performance of classification tasks, destroying the expected shape of the predicted label distributions, while our work still focuses on LDL tasks; 2) the margin is still a hyperparameter that needs to be set in their methods, while our method is 5-parameter-frre,, which is more convenient for practical scenarios.
7. Limitations and conclusion
Regrading learning objectives, the average KLD and the MSE are the most widely adopted. For example, AA-BP (Geng, 2016), a simple 3-layer network, minimizes MSE but underperforms due to overfitting. The average KL. is exploited in many LDL work based on loss function engi- neering via different optimization methods, including BFGS
Limitations A notable limitation of the proposed 8-LDL algorithm is that it has not yet achieved optimal performance across all metrics, which is close to the theoretical best- case scenario. This is likely due our current work not considering fitting different shapes of label distributions, which is an area that warrants further exploration.
8

---

Approximatcly Correct Label Distribution Learning
Conclusion In this paper, we propose Del percent- age of predictions that are approximately correct in L.L.. DeltaLDL can serve as a novel evaluation metric, which is parameter-free freee and reflects real performance improve- ments; Deltal.DI. can also serve as novel learning objec- tive, which is differentiable and ensures that most samples can be predicted as approximately correct. Our theoretical analysis and empirical results demonstrate the effectiveness of the proposed solution.
He, L., Lu, Y., Li, W., and Jia, X. Generative calibration of inaccurate annotation for label distribution learning. In Proceedings of the AAAI Conference Artificial Intelli- 2024.
gence, pp. 1
Jia, X., Li, W. Liu, J., and Zhang, Y. Label distribution learning by exploiting label correlations. In Proceedings of the AAAI Conference Artificial Intelligence, pp. 3310-3317, 3317, 2018.
Jia, X., Li, Z.,, Zheng, X., Li, W., and Huang. S.-J. Label dis- tribution learning with label correlations on local samples. IEEE Transactions Knowledge and Data Engineering, 2019.
Impact statement
This paper presents work whose goal is to advance the field of machine learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Jia, X., Qin. T., Lu, Y., and Li, W. Adaptive weighted ranking-oriented label distribution learning. IEEE Trans- actions Neural Networks and Learning Systems, 35 (8):1 (8):11302-11 1316, 2023a.
Acknowledgements
Jia, X., Shen, X., Li, Lu, Y., and Zhu, J. Label distribu- tion learning by maintaining label ranking relation. IEEE Transactions Knowledge and Data Engineering, 35 2023b.
This research is partially supported by Shanghai Cen- tral Guidance Science and Technology Development Fund XYDXX20333000000000 the Natural Science Foundation of Jiangsu Province (BK20242045), and the National Natu- ral Science Foundation of China (62176123, 62476130).
Kingma, D. and Ba, J. L. Adam: method for stochastic optimization. In Proceedings of the International Confer- ence Learning Representations, 2015.
References
Chen, S., Wang, J., Chen, Y., Shi, Z.., Geng, X., and Rui, Y. Label distribution learning on auxiliary label space graphs for facial expression recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13984-13993, 2020.
Kou, Z., Wang, J., Jia, Y., Liu, B., and Geng, X. Instance- dependent inaccurate label distribution learning. IEEE Transactions on Neural Networks and Learning Systems, 36(1):1425-1433 2023.
Lee, H., Höger, F., Schoenwiesner, M., Park, M., and Jacoby, N. Cross-cultural mood perception in pop songs and its alignment with mood detection algorithms. In Proceed- ings of the International Society for Music Information Retrieval Conference, 2021.
Geng, X. Label distribution learning. IEEE Transactions on Knowledge and Data Engineering, 28(7)1134-1148, 2016.
Geng, X. and Hou, P. Pre-release prediction of crowd opin- ion on movies by label distribution learning. In Proceed- ings of the International Joint Conference on Artificial Intelligence, pp. 3511-3517, 2015.
Li, S. and Deng, W. Blended emotion in-the-wild: Multi- label facial expression recognition using crowdsourced annotations and deep locality feature learning. Interna- tional Journal of Computer Vision, (27(6-7):884-999)) 2019.
Geng, X., Yin. C., and Zhou, Z.-H. Facial age estimation by learning from label distributions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(10): 2401-2412, 2013.
Li, X., Luo, G., Wang, W., Wang, K., and Li, Curriculum label distribution learning for imbalanced medical image segmentation. Medical Image Analysis, 89: 102911 2023.
Geng, X., Zheng, R., Lv, J., and Zhang, Y. Multilabel ranking with inconsistent rankers. IEEE Transactions on Puttern Analysis and Machine Intelligence, 44(9):5211- 5224, 2021.
Liang, L., Lin, L., Jin, L., Xie, D., and Li, M. SCUT- FBP5500: A diverse benchmark dataset for multi- paradigm facial beauty prediction. In Proceedings of the International Conference on Pattern Recognition, pp. 1598-1603, 2018
González, M., González-Almarro, G., Triguero, I., Cano, J.-R., and García, S. Decomposition-fuuee fusion for label dis- tribution learning. Information Fusion, 66:64-75, 2021.
Lu, Y. and Jia, X. Predicting label distribution from multi- label ranking. In Proceedings of the Annual Conference

---

Approximatcly Correct Label Distribution Learning
Neural Information Processing Systems, pp. 36931- 36943, 2022.
Xu, M. and Zhou, Z.-H. Incomplete label distribution learn- ing. In Proceedings of the 26th International Joint Con- ference Artificial Intelligence, pp. 3175-3181, 2017.
Machajdik, J. and Hanbury. A. Affective image classifica- tion using features inspired psychology and theory. In Proceedings of the ACM International Conference Multimedia, pp. 83-92, 2010.
Yin, L., Wei, X., Sun, Y., Wang. J., and Rosato, M. J. 3D facial expression database for facial behavior research. In Proceedings of the International Conference Auto- matic Face and Gesture Recognition, pp. 211-216, 2006.
McKeeman, W. M. Algorithm 145: Adaptive numerical integration by simpson's rule. Communications of the ACM, 5(12):604, 1962.
Zhang, M.-I.. and Zhou, 7.-H. review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering, (6(8):1819-18) 2013.
Nguyen, T. V., Liu, S., Ni, B., Tan, J., Rui, Y., and Yan, S. Sense beauty via face, dressing, and/or voice. In Proceedings of the ACM International Conference Multimedia, pp. 239-248, 2012.
Ren, T. Jia, X., Li, W., Chen, L.,, and Li, Z. Label distribu- tion learning with label-specific features. In Proceedings of the International Joint Conference on Artificial Intelli- gence, pp. 3318-3324, 2019a.
Ren, T. Jia, X., Li, W., and Zhao, S. Label distribution learning with label correlations via low-rank approxima- tion. Proceedings of the International Joint Conference on Artificial Intelligence, pp. 3325 3325-3331, 2019b.
Ren, Y. and Geng. X. Sense beauty by label distribution learning. Proceedings of the International Joint Con- ference on Artificial Intelligence, pp. 2648-2663 2017.
Shen, W. Zhao, K., Guo, Y., and Yuille, A. Label distri- bution learning forests. In Proceedings of the Annual Conference on Neural Information Processing Systems, pp. 834 843, 2017.
Wang, J. and Geng, X. Classification with label distribution learning. In Proceedings of the International Joint Con- ference Artificial Intelligence, pp.
2019.
Wang, and Geng, X. Label distribution learning machine. In Proceedings of the International Conference on Ma- chine Learning, pp. 10749-10759, 2021a.
Wang, J. and Geng, X. Learn the highest label and rest label description degrees. In Proceedings of the Inter- national Joint Conference Artificial Intelligence, pp. 3097-3103, 2021b.
Wen, C., Zhang, X., Yao, X., and Yang, J. Ordinal label distribution learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 23481- 23491, 2023.
Xu, H., Liu, X., Zhao, Q. Ma, Y., Yan, C., and Dai, F. Gaussian label distribution learning for spherical image object detection. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 1033-1042, 2023.
10

---

Approximatcly Correct Label Distribution Learning
A. Proof of Proposition 2.2
Proof. Let u = (wi, U2, Uo) be a random vector distributed according to a Dirichlet distribution, i.e. u ~ Dir(a)), with parameters = (a1, 02 ... a²). The KLD between u and v is given by KL (| = Ej= 4 log(cu,): ). Then the expectation of the KLD can be decomposed as
E (KL(u]|()) E
Σ u log(cu,)
Σ
E (uz log(cu,))
Σ (B(ug log log uy) + E(uy)loge)) c).
(22)
From the properties of the Dirichlet distribution, the expected value of uy is
B(u,)= = Σ kk=1 ak
(23)
Since Dir((x), we have uy ~ Beta(aj, Ex- ak - az)) Vj E |c]. From the properties of the Beta distribution, we have
c
+ 1 1)
B(u, ug)
aj
bla +1) +
Σ ak
(24)
Ek=1 ax
Combining Equations (22) to (24), the proof is complete.
B. Remaining experimental results
100
100
PT-Bay
PT-Sav
PT-
AA-50N
AA-RSX
JA-ENN
80
LBI
DP-DD
30
DFADI
LRB
80
DF-LDL
LRB
LOLSF
(a) D (Chebyshee, 8; .) w.r.t. 8
(b) (Claakk, 6; -) w.r.t. 8
(c) (CCaberra, 8; w.r.t. 8
003
100
300
PT-3oo
PT-Bor
A4-450 430
AA-AN
LRB
90
LRB
50
DF
DF-1DL
LOLSP
141.
DESLDI.
LCLR
03
0.30
(d) (CCosine, 5; .) w.r.t. 5
(e) (Intersectton,, 8; -) w.r.t. 5
(f) (Spearman, 6; -) w.r.t. 6
Figure 5. (Best viewed in color) Visualized results of the analysis on parameters.

---

Approximatcly Correct Label Distribution Learning
Table 3. Experimental results on RAF ML, Natural Scene, Music, and Painting, formatted as "mean + std" Clark
Algorithms
Cheby.
Cosine
f
Spear. f
14
RAF
SCL
3366 .one
1.5754 2012
3.406 o3.31
.3600 = 011
7080 + 003
55337 2,004
.3372 .026
20.95% = 2,00
Bayes
.382 .222
1.7253 2.oas
3,7506 10s
.8455 123
.7346 d: 024
.5983 1. 2.02s
.5575 ..22
32.15% 1.09
LDLST
1885 .0004
1.5429 1.013
3.0619 di, 034
.256 .000
.9002 1. 005
76772 1. 1,0055
.7713 1.00
48.58% .017
AA-ANN
2643 +
1.4882 =
3.0326 +:
4073 =
.8221 # 000
6637 =
.6041 2.03
49.37% $.12
LDLF
1974 +
1.4077 010
2.7417 #..7 045
.2444 =
.9071 #.005
7769 +
.7494 +:
68. 68.14% ..17
DE-LDI
1546
1.4196 +
2.7375 +
1957 -
.9273 + .00
8087
78177 +
75 75.68% #.01
SABBEGS
1538
1.4199
2.7385 +
.1577 -
.2277 +
8096
.7840 +
75.77% #.010
LCLR
11538 005
1.4199 +
2.7386 +:
.1977 +
.9277 1: 005
8099 .00
.7839 +
75.77% 2.000
LDLLC
1538 000
1.4178
2.7335 + 036
1956 011
.9776 005
8096 000
7842 .110
75.79% 2.010
LRR
1538 .000
1.4185
01015
2.7356
.957 20011
.276 00s
8096 .ons
7841 _010
75.79%
2,000
1. ooo
1.4185
110
03.
.1957 2.011
.9766 005
ons
1.000
75.80%
1100
6.L.I.
+ 2.00
1.4199
2.0
d: 0.8
d: 011
+: 005
2,00
+ ..000
+. 2,010
Sceaee
PT Bayes
7469 1.0
2,9025 1.
8,5810 1.00__..33 033
19,3376 1.911
.446 1. 022
2416 110
.467 1,024
10.19% 1. .017
SCL
3486 +: .017
2.4728 =
6.8978 it
.9999 =
6647 + 000
4180 2.
3368 ..01
25.111 .007
LDLSF
3185 2.1
2.4948 +
6.8440+..or
2.6764 +
.7933 #.01
.443 #
.4188 + 2016
35.02% + 2,000
L.CR
.590 + 016
2,3688 +
6.5106
1,1351 2,00
.6681 0.018
.2558
.697 +
35.33% +
SABBEGS
.323
2.3921 +
6.5725 +
.9399
.7016 0.010
.5480 2.014
.898 +
39.00%
DF-LDL
.307
2.3698
6.4633 +: 074
.7228 0.0
.7323 1: 013
.5670 1:
.5092 1:
42.55% 1: .02
AA-ANN
2883 .144
1,7886 oo0
4.1708
1.0107 0.0
.7391 0.0
5969 2.014
.5319 .020
42.71% .024
LDLLC
2994 1. 2012
2.4214
6.6168 #.7
.7327 2,035
.7504 0.1
.5718
.5251 2011
44.30% .018
2994 1.02
2.4207
1.019
6.6152:11.77
.7327 2,0
.7500 012
5522 1.01
.5522 001G
.018
2012
2.4210
2.18
470
2,0
0.0
5722
.016
.018
LDLF
2842 + 2.12
2.4188
o.0
6.5960 +: 074
6787 + 0.037
7776 014
5899 2.022
.5473 .17
47.47% + ...02
6-LDL
.2700 + 012
2.4123 - 017
6,5501 +: 075
6586 = 033
.7807 + 012
.9744 + 011
5883 - 017
48.96% .188
Nusic
PT-Bayes
1733 +
166 +
2.8681 +
.220 di.
.7888 + 040
6930
.279 +
04.85% ..026
LDLSF
0946
9498 +
2,4003 +:
61668 +330
8818 -:
76778 +
3123 +
23. 9..99% #.055
SABBEGS
1000
.802
2.2770 4:
.1912 1.000
.8665 0..18
.7673 1
.3030 +:
25.57% 4.04
DF-LDL
.1022 .088
8489 007
2,1694 1:
.1719 ..02
.8726 1. 018
.7763
.2688 1: .so
26.18'8 1. .045
LCLR
1000 .008
8644
2.2295 + 182
.8211 0,0
.8738 +
.7733
.3673 - .073
26.23% .049
DPA
.007
8532 0059
= 168
.1758 .00
.8746 010
.7741
.072
77,04% .047
= 007
+: 16B
.1751 +: 027
.8750 017
.7745 1 .10118
.3214 .71
.047
IDLLC
0968 = 007
.818 -.077
2.2063 160
.1751 + 0.027
.8511 0107
7745 #.017
.3212 +000 .070
27.12% + .046
SCL
0776 = 004
76666 + .50
1.9339 +
.1157 0.0
.9150 00s
8065 2 ..0
.4544 o05
32.12 +: .028
LDLF
0775 005
7532 + 054
1,9050 + 1400
1134 #134 014
.9161 000
8096 014
A538 078
34.01% ..45
AA-NNN
0784
7499 + 001
1.9309 +:
.1258 2201
.0077 0.5
8051 +
.3989 +
36.75% #,04
6.L.L.
0733
.1855 1.
1.8221 1:
.1011 1.016
.218 1.
8186 015
.4422 1:
39.35% 1. .ac
Painting
PT-Bayes
7070
2.5747 006
7.0292 =::
7,4050
.337 +: 0s5
2275
.1484 1:
00.07% +: .003
LDLSF
.2853 .28
1,8290
4,3005 1. 204
4,1107 1: 205
.6565 1. 033
.5507
.0047 1
11.89% 1.000
LCLR
2979 .288
1.8632 070
4,5308
220
.0055 215
6679 + 034
5538
.2454 .006
16.23% #.045
BFGS
2965 .028
1.8559 4.073
4,5050 =t: 23T
233
.596 4: 034
.028
.2363 ..89
16.37%
.084
DF-L.DI
.2714 - 021
1,8051
060
4.31
202
+: .00
.6722 082
.576 .23
.2310 .773
..44
SCL
2579 = .019
1.6915
=.007
3,9339 193
.5574 + 042
.7223 017
.940 .019
.1987 + .07
19.36% .024
AA-ENN
2572 021
1.7553
.000
4.1427 182
.100 +: 161
.6987 +: 027
5888 021
.2241 +: 672
19.76'8 1: ..17
DPA
2585 =
1.7561 +
1452 215
5999 +
.7116
.951 4:
.2780 1000--000
20.48% .043
LRR
2587 200
1.7556 +
4. 1.1422 2.2
5988 -
.7118 #.07
.5544
.2782 -
20.48% .040
LDLLC
.2586
1.7560 1.
4.1438 1.
.5994 1.
.7120 J.
.553 1,,22
.2774 1:
20.48% 1.
LDLF
2485
1.7059 +:
3,9700 4:
5343 +:
.7336 02.
6075
.054 J:
22.54% J. .036
6-LDL
.2484
1,7019 +
3.9520
.5288 +
.7377 +:
.6102
.3146 + 23.05% .037
12


Reviews:
[{"review_id": "jpEhwpJyQF", "reviewer": "Reviewer_xDUT", "summary": "This paper addresses the issues in **Label Distribution Learning (LDL)**, notably the limitations of **Kullback–Leibler Divergence (KLD)** as both an evaluation metric and learning objective. The authors propose **DeltaLDL**, a novel framework that introduces the concept of \"**approximately correct**\" label distributions, aiming to measure and optimize the percentage of samples predicted within a reasonable distance from the ground-truth distributions. This is operationalized as both a new **evaluation metric (µ)** and a **differentiable learning objective**. The authors provide theoretical analysis to support their approach, including a critique of KLD's properties via Dirichlet distribution modeling, and propose an algorithm δ-LDL that integrates these ideas. The method is evaluated on a variety of standard LDL datasets, showing improved performance across multiple metrics, and experiments include ablations and robustness tests to validate the effectiveness of their contributions.\n\n**update after rebuttal**\n\nThank the authors for the rebuttal, which has addressed my concerns and I agree to raise my score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Many thanks for your comprehensive comments! Responses to your concerns are as follows.\n\n**Response about $\\delta$:** Our theoretical analysis illustrates that, $\\delta_0$ reflects the worst-case divergence. Therefore, values *larger* than $\\delta_0$ (for distance metrics) would imply tolerating worse-than-random errors, which doesn't make much sense. While *smaller* $\\delta$ could be explored, they require strong assumptions about data training difficulty (e.g., label noise), i.e., we can't decide how small $\\delta$ should be. Without such prior knowledge, $\\delta_0$ provides a neutral starting point. We provide results of $\\mathfrak{D}(\\text{KL}, \\delta; f)$ w.r.t. $\\delta$ on $\\mathtt{SBU\\\\_3DFE}$ & $\\mathtt{Natural\\\\_Scene}$, where $f$ always outputs a uniform label distribution matrix.\n\n$\\mathtt{SBU\\\\_3DFE}$:\n|$\\delta$|0.02|0.04|0.06|0.0851 ($\\delta_0$ of $\\mathtt{SBU\\\\_3DFE}$)|0.1|0.12|0.14|1.172 ($\\delta_0$ of $\\mathtt{Natural\\\\_Scene}$)|\n|-|-|-|-|-|-|-|-|-|\n|$\\mathfrak{D}$|.156|.314|.450|.594|.665|.732|.787|1.|\n\n$\\mathtt{Natural\\\\_Scene}$:\n|$\\delta$|0.0851 ($\\delta_0$ of $\\mathtt{SBU\\\\_3DFE}$)|0.6|0.8|1.0|1.172 ($\\delta_0$ of $\\mathtt{Natural\\\\_Scene}$)|1.4|1.6|1.8|\n|-|-|-|-|-|-|-|-|-|\n|$\\mathfrak{D}$|.0|.238|.240|.481|.650|.662|.777|.797|\n\nKey findings: 1) $\\mathfrak{D}(\\text{KL},\\delta;f_0)$ shows high sensitivity to $\\delta$ changes (non-linear response); 2) optimal $\\delta$ ranges vary significantly across datasets, which is not a mutually referable setting. This sensitivity highlights the dangers of ad-hoc parameter choices, justifying our pursuit of a parameter-free solution.\n\n**Response about $\\varepsilon$:** The tolerance parameter $\\varepsilon$ & maximum recursion depth $\\xi$ both ultimately control iteration limits. However, $\\varepsilon$ is usually set as a fixed small value (1e-7) following numerical analysis conventions. We instead focuse on more adjustable $\\xi$, analyzed in **Fig. 2 (a)**.\n\n**Response about downstream tasks:** We appreciate this constructive suggestion, but directly applying LDL to downstream tasks risks *objective mismatch* (W. & G., '19; '21b). However, we can conduct additional vision experiments in the context of pure LDL.\n\nSetup:\n+ Dataset: $\\mathtt{JAFFE}$ (256 $\\times$ 256 facial images).\n+ Arch.: ResNet-50 backbone.\n+ Training: 10-fold CV, Adam optimizer (lr=1e-4, bs=32, 100 epochs).\n+ Baseline: AA-BP (G., '16) with the same configuration.\n\nResults (**bold** = better):\n||$\\mathtt{Cheby.}$|$\\mathtt{Clark}$|$\\mathtt{Can.}$|$\\mathtt{KLD}$|$\\mathtt{Cosine}$|$\\mathtt{Int.}$|$\\mathtt{Spear.}$|$\\mu$|\n|-|-|-|-|-|-|-|-|-|\n|AA-BP|.0469|**.1651**|**.3368**|.0139|.9858|.9421|**.7949**|85.10%|\n|$\\delta$-LDL|**.0454**|.1654|.3369|**.0122**|**.9880**|**.9431**|**.7949**|**85.83%**|\n\n$\\delta$-LDL achieves better ($\\uparrow$0.73% $\\mu$) or comparable results.\n\n**Response about the complexity:** We provide runtime comparisons ($\\delta$-LDL vs. baselines on $\\mathtt{SBU\\\\_3DFE}$). Results:\n|$\\delta$-LDL|DF-LDL|LDLF|SCL|LRR|DPA|LDLLC|LCLR|SA-BFGS|LDLSF|AA-$k$NN|PT-Bayes|\n|-|-|-|-|-|-|-|-|-|-|-|-|\n|1.x|.16x|1.11x|1.02x|.07x|.06x|.06x|.21x|.01x|3.95x|-|.001x|\n\nRuntimes are normalized to $\\delta$-LDL’s execution time (1.x) for direct comparison. $\\delta$-LDL shows similar training times to SCL/LDLF methods. Note that such comparisons may lack fairness since PT-Bayes, AA-$k$NN, LDLSF, SA-BFGS & LCLR do *not* employ deep learning architectures in their implementations.\n\n**Response about the assumption:** While we use Dir. distributions for theoretical analysis due to their mathematical tractability and natural fit for probability simplex, theoretical assumptions are relaxed in implementation. Precisely because of the consideration of deviations in real-world scenarios, our method does not strictly require this assumption in practice; our method is empirically robust to distribution-free label structures, as demonstrated in experiments.\n\n**Response about the applicability:** We appreciate the suggestion to explore broader applicability. However, *the continuous nature of label distributions differs fundamentally from the discrete logic of multi-label annotations*, making DeltaLDL and its derivatives ($\\mu$ & $\\delta$-LDL) more suited to pure LDL tasks. Let‘s discuss some examples:\n+ Some MLL evaluation metrics, such as *subset accuracy*, inherently involve discretization in their computation. These metrics must be calculated on mini-batch test sets and are unsuitable as optimization objectives, making them irrelevant to the core problems we aim to address. \n+ However, metrics like *Hamming*/*Jaccard* could potentially be applicable. When approximating $\\delta_0$ for these cases, one would need to substitute uniform vectors with a random binary matrix and modify the margin mechanism for discrete outputs.\n\nMulti-label metrics often prioritize accuracy over distributional fidelity, conflicting with LDL’s paradigm. This presents an interesting and meaningful direction for future work.", "claims_and_evidence": "The claim that \"DeltaLDL, based on approximately correct predictions, offers a more discriminative and robust framework.\" is problematic:\n\nThe **definition of \"approximately correct\"** relies on an empirically-derived threshold (δ) whose semantic meaning across domains is underexplored, making the generalizability of these claims **less certain without additional justification or sensitivity analysis**.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria make sense **within the LDL field**, focusing on nuanced distributional accuracy rather than strict one-hot labeling. The replacement of KLD with DeltaLDL as both a metric and loss function is well motivated. The use of **adaptive Simpson's rule** for optimizing the objective is technically creative and well-integrated.\n\nHowever, while the datasets used (e.g., M2B, SBU 3DFE, fbp5500) are standard in LDL, **the lack of evaluation on more challenging or modern datasets (e.g., higher-dimensional, more skewed distributions, language models, or image classification) is a limitation**. Given the paper's ambitious claims about general improvements to LDL, it would strengthen the work to see tests beyond traditional benchmarks, such as larger-scale datasets in domains like NLP or vision where distributional representations are crucial.\n\nRuntime and complexity is not evaluated: While the method is linear in key dimensions, the use of adaptive Simpson’s rule introduces overhead. A quantitative runtime analysis would help assess practical viability.", "theoretical_claims": "The theoretical critique of KLD via Dirichlet expectations is sound and well-developed, including the calculation of expected KLD values under stochastic models. The properties of DeltaLDL (e.g., monotonicity, normalization) are clearly stated and appear mathematically correct.\nHowever, **the threshold δ that underlies DeltaLDL is only heuristically defined**, and **its theoretical grounding (e.g., optimality properties, sensitivity)** is not fully explored. While the integral-based µ metric attempts to mitigate the need for precise δ selection, further theoretical discussion on this aspect would be helpful. Moreover,  the assumptions (e.g., Dirichlet distribution for ground truth) could be limiting in real-world scenarios.", "experimental_designs_or_analyses": "The experimental design is **comprehensive** within LDL:\n- Comparison against strong baselines.\n- Multiple datasets and repeated runs.\n- Robustness evaluations against noise.\n\nNevertheless, there are some **missing experimental angles**:\n1. **Impact of δ and sensitivity analysis**: Since δ plays a crucial role, a more detailed empirical analysis of how δ affects outcomes across datasets would be valuable.\n2. **Lack of real-world downstream tasks**: It would be valuable to see if improvements in label distribution predictions translate into improvements in downstream tasks like multi-label classification, especially in ambiguous domains like NLP or vision where label semantics are inherently overlapping.", "relation_to_broader_scientific_literature": "The paper is well-situated within the **LDL literature**, offering clear advancement over previous KLD-based methods. However, **connections to broader multi-label and distributional learning paradigms are missing**. Particularly:\nRelating the proposed method to multi-label classification [1, 2], distribution-based labels (soft targets) [3] could highlight broader applicability.\n\n[1] Zhu, Feng, et al. \"Learning spatial regularization with image-level supervisions for multi-label image classification.\" Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.\n[2] Bi, Wei, and James Kwok. \"Efficient multi-label classification with many labels.\" International conference on machine learning. PMLR, 2013.\n[3] Zhang, Chang-Bin, et al. \"Delving deep into label smoothing.\" IEEE Transactions on Image Processing 30 (2021): 5984-5996.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths:\n\n- Introducing an “approximately correct” metric and corresponding loss function is a novel contribution.\n\n- The theoretical analysis is thorough and offers insight into why KLD may be problematic in LDL contexts.\n\n- The experiments cover several datasets and compare many baselines, demonstrating consistent trends across different conditions.\n\nWeaknesses:\n\n- The additional computational overhead due to ASR and the smoothing function is not well quantified.\n- The assumptions (e.g., Dirichlet-distributed ground truths) might limit the method’s applicability in real-world settings where distributions deviate from these assumptions.", "other_comments_or_suggestions": "Broader Applicability: Discussion on how the proposed ideas might extend to other related tasks (e.g., multi-label learning) would be beneficial.", "questions_for_authors": "- Training Time and Complexity: Could you provide quantitative comparisons of training time between δ-LDL and standard KLD-based LDL methods, especially for larger datasets?\n\n- Robustness to Distributional Assumptions: How sensitive is the method to deviations from the Dirichlet assumption for the ground-truth label distributions?\n\n- Parameter Sensitivity: While the method is termed δ-parameter-free, there remain parameters (e.g., tolerance in ASR). How do these affect performance, and are there guidelines for setting them?\n\n- Generalization: Have you explored the applicability of the µ metric beyond the LDL framework, such as in multi-label learning or other prediction problems with distributional outputs?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TYRoQ1ys8t", "reviewer": "Reviewer_yrYr", "summary": "This paper focuses on label distribution learning (LDL) and addresses the limitations of existing evaluation metrics and learning objectives. Existing LDL evaluation metrics based on distance/similarity metrics, like Kullback-Leibler divergence (KLD), have poor discriminability due to the constraints of label distributions. Also, existing LDL learning objectives often overfit by emphasizing a small subset of samples, leading to sub-optimal performance. Hence, it proposes DeltaLDL, which can be used as both a novel evaluation metric and a learning objective, aiming to improve the performance and discriminability in LDL.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Many thanks for your precious comments! We have provided point-by-point responses to your questions below.\n\n**Comment 1:** My main concern is whether the new metric proposed in the article can truly distinguish between superior and inferior models. In Eq. (11), there is a coefficient of 1/delta_0 before the integral. What does this coefficient mean? Why does the area under the curve (AUC) need to be multiplied by such a coefficient? Moreover, the smaller this coefficient is, the larger \\mu becomes. This makes one wonder whether it is the subsequent integral, this coefficient, or the interaction between the two that plays a role in the measurement. I think this point needs further explanation.\n\n**Response:** The metric $\\mu$ is *not* a raw AUC but a ratio of areas. Let us explain this better with **Fig. 2 (b)**:\n\n+ Numerator: The integral in Eq. (11) corresponds to the area under the curve, bounded by the gray line and axes.\n\n+ Denominator: $\\delta_0$ represents *the area of the gray rectangle*, reflecting the ideal model’s performance.\n\nThus, $\\frac{1}{\\delta_0}$ normalizes the $\\mu$ metric to [0, 1], ensuring fair comparison (no arbitrary scaling) between metrics. We will clarify this critical point in the manuscript.\n\n**Comment 2:** Setting \\delta_0 as the expected KL divergence between the label distribution and the vector v is somewhat heuristic. It is recommended to give more explanations or conduct more experiments for exploration.\n\n**Response:** Our theoretical analysis illustrates that, $\\delta_0$ reflects the worst-case divergence. Therefore, values *larger* than $\\delta_0$ (for distance metrics) would imply tolerating worse-than-random errors, which doesn't make much sense. While *smaller* $\\delta$ could be explored, they require strong assumptions about data training difficulty (e.g., label noise), i.e., we can't decide how small $\\delta$ should be. Without such prior knowledge, $\\delta_0$ provides a neutral starting point. We provide results of $\\mathfrak{D}(\\text{KL}, \\delta; f)$ w.r.t. $\\delta$ on $\\mathtt{SBU\\\\_3DFE}$ & $\\mathtt{Natural\\\\_Scene}$, where $f$ always outputs a uniform label distribution matrix.\n\n$\\mathtt{SBU\\\\_3DFE}$:\n|$\\delta$|0.02|0.04|0.06|0.0851 ($\\delta_0$ of $\\mathtt{SBU\\\\_3DFE}$)|0.1|0.12|0.14|1.172 ($\\delta_0$ of $\\mathtt{Natural\\\\_Scene}$)|\n|-|-|-|-|-|-|-|-|-|\n|$\\mathfrak{D}$|.156|.314|.450|.594|.665|.732|.787|1.|\n\n$\\mathtt{Natural\\\\_Scene}$:\n|$\\delta$|0.0851 ($\\delta_0$ of $\\mathtt{SBU\\\\_3DFE}$)|0.6|0.8|1.0|1.172 ($\\delta_0$ of $\\mathtt{Natural\\\\_Scene}$)|1.4|1.6|1.8|\n|-|-|-|-|-|-|-|-|-|\n|$\\mathfrak{D}$|.0|.238|.240|.481|.650|.662|.777|.797|\n\nKey findings: 1) $\\mathfrak{D}(\\text{KL},\\delta;f_0)$ shows high sensitivity to $\\delta$ changes (non-linear response); 2) optimal $\\delta$ ranges vary significantly across datasets, which is not a mutually referable setting. This sensitivity highlights the dangers of ad-hoc parameter choices, justifying our pursuit of a parameter-free solution.\n\n**Comment 3:** How can we determine which samples are difficult to learn? Perhaps it would be best to have a warm-up process, or there could be a process of gradually changing the hyperparameter \\delta.\n\n**Response:** The identification of difficult-to-learn samples is gradually facilitated by the \"ReLU + margin\" mechanism during training, since initializing $\\delta$ to $\\delta_0$ is sufficiently inclusive, allowing the model to first learn from easier patterns. While we prioritize simplicity and avoid introducing additional hyperparameters, we acknowledge that a warm-up strategy or adaptive scheduling for $\\delta$ could be explored in future work.\n\n**Comment 4:** In Remark 2.3, please give some explanations that why the closed-form solution of Eq. (8) does not exist.\n\n**Response:** The partial derivative of Equation (8) is given by:\n\n$$\n\\frac{\\partial \\ell_{\\text{MLE}}}{\\partial \\alpha_k} = -m \\left( \\Phi \\left( \\sum_{j=1}^{c} \\alpha_j \\right) - \\Phi (\\alpha_k) \\right) - \\sum_{i=1}^{m} \\ln d_{\\boldsymbol{x}_i}^{d_k}\\text{,}\n$$\n\nwhere $\\Phi(x) = \\frac{\\mathrm{d} \\ln \\Gamma(x)}{ \\mathrm{d} \\Gamma(x)}$ is the digamma function. The equation exhibits highly nonlinear behavior and implicitly forms a globally coupled nonlinear system, making a closed-form solution for Equation (8) intractable.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Yes.", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "The key contributions of the paper are closely related to the fields of label distribution learning. It proposes a new metric to measure the distance between the ground-truth label distribution and estimated label distribution, and transform the metric into a loss function so that we could consider the metric when optimizing the model.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths:\n1. Novel Evaluation Metric: This paper proposes a parameter-free metric, μ, which quantifies the percentage of samples predicted as approximately correct by integrating the Kullback-Leibler divergence (KLD) over a threshold range. This metric avoids the sensitivity issues of traditional KLD and provides a more discriminative measure of performance improvements.\n2. Novel Learning Objective: This paper formulates a learning objective using a smoothed indicator function and adaptive Simpson's rule for numerical integration. This approach encourages most samples to be approximately correct while mitigating overfitting to extreme predictions.\n3. Experiments across multiple datasets (e.g., M2B, fbp5500, SBU_3DFE) show that δ-LDL outperforms baseline LDL methods in terms of both accuracy and robustness, which validates the effectiveness of the proposed method.\n\nWeaknesses:\n1. My main concern is whether the new metric proposed in the article can truly distinguish between superior and inferior models. In Eq. (11), there is a coefficient of 1/delta_0 before the integral. What does this coefficient mean? Why does the area under the curve (AUC) need to be multiplied by such a coefficient? Moreover, the smaller this coefficient is, the larger \\mu becomes. This makes one wonder whether it is the subsequent integral, this coefficient, or the interaction between the two that plays a role in the measurement. I think this point needs further explanation. \n2. Setting \\delta_0 as the expected KL divergence between the label distribution and the vector v is somewhat heuristic. It is recommended to give more explanations or conduct more experiments for exploration.\n3. In Section 4, it mentions “the objective should be to sacrifice a small number of samples that are difficult to learn and ensure that most samples can be predicted as approximately correct.” Then, in the initial stage of training, when the model's parameters are random, how can we determine which samples are difficult to learn? Perhaps it would be best to have a warm-up process, or there could be a process of gradually changing the hyperparameter \\delta.", "other_comments_or_suggestions": "None.", "questions_for_authors": "1.In Remark 2.3, please give some explanations that why the closed-form solution of Eq. (8) does not exist.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "clX8lxgBA1", "reviewer": "Reviewer_RXkW", "summary": "This paper theoretically reveals the deficiency of the KL divergence in learning and evaluating LDL mappings. To address the mentioned shortcomings, this paper proposes a new LDL paradigm, DeltaLDL, which focuses on how many label distributions are approximately correctly predicted. Based on DeltaLDL, this paper proposes a novel evaluation metric (which is theoretically proved that it possesses superior discriminative power) and a novel learning objective (which achieves highly competitive performance). Finally, this paper conducts extensive experiments to demonstrate the effectiveness of the proposal.\n## update after rebuttal\nI read the rebuttal and the other reviews, my major concern is how a specific distance or similarity value represents the closeness between the predicted and ground-truth label distributions and some writting issues. The rebuttal well answer the distance question. The writting issues can be solve in the camera-ready version. Therefore, I raise my score.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Many thanks for your precious comments! We have provided point-by-point responses to your questions below.\n\n**Comment 1:** Non-professional users may not intuitively understand how a specific distance or similarity value represents the closeness between the predicted and ground-truth label distributions.\n \n**Response:** Traditional metrics reflect the closeness from different perspectives. For example, Euclidean distance measures geometric separation, while K-L divergence quantifies distributional differences. Our metric, i.e., Eq. (11), built upon these foundations, remains intuitive. It directly corresponds to an *area ratio*. Let us explain this better with **Fig. 2 (b)**:\n\n+ Numerator of Eq. (11): The integral corresponds to the area under the curve, bounded by the gray line and axes.\n\n+ Denominator of Eq. (11): $\\delta_0$ represents *the area of the gray rectangle*, reflecting the ideal model’s performance.\n\n**Comment 2:** The paper also has some imperfections, such as the arbitrary naming of methods or metrics (e.g., $\\mu$ and DeltaLDL), which does not reflect the characteristics of the methods and metrics. ... It is recommended to assign a name to the evaluation metric to more accurately reflect its characteristics, rather than using the Greek letter ($\\mu$) directly.\n\n**Response:** We appreciate this thoughtful suggestion. While we originally adopted Greek-letter notation $\\mu$ for consistency with established metric conventions like Spearman's $\\rho$ and Kendall's $\\tau$, we agree that more descriptive names would better reflect their characteristics. We will provide the aliases as follows:\n\n+ $\\mu$: *improvement ratio* (clearly indicating performance gain).\n\n+ DeltaLDL: AC-LDL (Approximately Correct LDL).\n\nThese new names better capture the methods' essential features while maintaining readability. Thank you for helping improve our paper's clarity.\n\n**Comment 3:** In Equation (13) and Equation (19), it should be clarified whether the symbol $c$ is also included within the logarithmic operation.\n\n**Response:** In both Equations (13) and (19), the parameter $c$ should indeed be included within the logarithmic operation. We will revise these equations to explicitly show this by adding parentheses.\n\n**Comment 4:** Why should $\\delta_0$ be defined as in Equation (12), and what advantages does this approach have compared to directly allowing users to pre-set a threshold?\n\n**Response:** Our theoretical analysis illustrates that, $\\delta_0$ reflects the worst-case divergence. Therefore, values *larger* than $\\delta_0$ (for distance metrics) would imply tolerating worse-than-random errors, which doesn't make much sense. While *smaller* $\\delta$ could be explored, they require strong assumptions about data training difficulty (e.g., label noise), i.e., we can't decide how small $\\delta$ should be. Without such prior knowledge, $\\delta_0$ provides a neutral starting point.\n\nAllowing non-professional users to set thresholds does *not* provide any advantages. We provide results of $\\mathfrak{D}(\\text{KL}, \\delta; f)$ w.r.t. $\\delta$ on $\\mathtt{SBU\\\\_3DFE}$ & $\\mathtt{Natural\\\\_Scene}$, where $f$ always outputs a uniform label distribution matrix.\n\n$\\mathtt{SBU\\\\_3DFE}$:\n|$\\delta$|0.02|0.04|0.06|0.0851 ($\\delta_0$ of $\\mathtt{SBU\\\\_3DFE}$)|0.1|0.12|0.14|1.172 ($\\delta_0$ of $\\mathtt{Natural\\\\_Scene}$)|\n|-|-|-|-|-|-|-|-|-|\n|$\\mathfrak{D}$|.156|.314|.450|.594|.665|.732|.787|1.|\n\n$\\mathtt{Natural\\\\_Scene}$:\n|$\\delta$|0.0851 ($\\delta_0$ of $\\mathtt{SBU\\\\_3DFE}$)|0.6|0.8|1.0|1.172 ($\\delta_0$ of $\\mathtt{Natural\\\\_Scene}$)|1.4|1.6|1.8|\n|-|-|-|-|-|-|-|-|-|\n|$\\mathfrak{D}$|.0|.238|.240|.481|.650|.662|.777|.797|\n\nKey findings: 1) $\\mathfrak{D}(\\text{KL},\\delta;f_0)$ shows high sensitivity to $\\delta$ changes (non-linear response); 2) optimal $\\delta$ ranges vary significantly across datasets, which is not a mutually referable setting. This sensitivity highlights the dangers of ad-hoc parameter choices, justifying our pursuit of a parameter-free solution.", "claims_and_evidence": "Yes, the claims are supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "Yes, both the methods and evaluation criteria make sense for the problem at hand.", "theoretical_claims": "Yes, I have checked the proof of Proposition 2.2.", "experimental_designs_or_analyses": "Yes, I have checked the experimental designs and analyses. The evaluation metrics and datasets are sufficient, and the comparison methods are state-of-the-art. Additionally, the effectiveness of the proposal is demonstrated by the ablation experiments.", "relation_to_broader_scientific_literature": "In the literature related to support vector regression, the algorithms aim to learn a strip with the minimum width such that as many samples as possible are within the strip. This paper adapts this idea to the label distribution learning paradigm and proposes the concept of \"approximately correct prediction,\" aiming to approximately correctly predict more samples.", "essential_references_not_discussed": "The paper does not neglect the essential works.", "other_strengths_and_weaknesses": "The evaluation metrics proposed in this paper possess strong practicality and offer a new perspective for subsequent research in label distribution learning. Traditional evaluation methods typically measure prediction quality by calculating the distance or similarity between the predicted label distribution and the ground-truth label distribution. \n\nHowever,:\n1. Non-professional users may not intuitively understand how a specific distance or similarity value represents the closeness between the predicted and ground-truth label distributions. \n2. The evaluation approach proposed in this paper aims to calculate the number of samples with a distance less than a certain threshold, which is easy to understand for non-professional users. \n3. The paper provides theoretical support for the proposed evaluation method. \n4. the paper also has some imperfections, such as the arbitrary naming of methods or metrics (e.g., $\\mu$ and DeltaLDL), which does not reflect the characteristics of the methods and metrics.", "other_comments_or_suggestions": "In Equation (13) and Equation (19), it should be clarified whether the symbol $c$ is also included within the logarithmic operation. Besides, it is recommended to assign a name to the evaluation metric to more accurately reflect its characteristics, rather than using the Greek letter ($\\mu$) directly.", "questions_for_authors": "Why should $\\delta_0$ be defined as in Equation (12), and what advantages does this approach have compared to directly allowing users to pre-set a threshold $\\delta_0$?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4cG7lla0II", "reviewer": "Reviewer_KudB", "summary": "To address  such as exisome deep-rooted problems of LDLsting LDL metrics lose their discriminability, and existing LDL objectives are also at risk of overfitting, this paper proposes DeltaLDL, a percentage of predictions that are approximately correct within the context of LDL. Based on DeltaLDL, a novel evaluation metric (the µ metric) and a novel learning objective are proposed. Finally, the authors encapsulate it to propose a new LDL algorithm, named δ-LDL.  The theoretical analysis and empirical results validate the effectiveness of the proposed δ-LDL.\n\n## update after rebuttal\nAfter reading the authors' repsonses, I decide to keep my original score.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Many thanks for your precious comments! We have provided point-by-point responses to your questions below.\n\n**Comment 1:** Some necessary justifications about the existing deep-rooted problems of LDL are needed. In Introduction: \"For years, there are some deep-rooted problems in the field of LDL:\" How to draw this conclusion? Combining with the related work (Section 6), More detailed elaboration is needed.\n\n**Response:** The \"deep-rooted problems\" refer to the following two issues:\n\n+ Poor discriminability of traditional metrics: Early work (Geng, 2016) demonstrated that Clark and Canberra metrics suffer from oversensitivity to small values. Subsequent studies (Xu & Zhou, 2017) revealed KLD's unreliability with sparse predictions, prompting calls for alternative measures.\n\n+ Overfitting from minimizing average measurement: For example, AA-BP (Geng, 2016), a simple 3-layer network, minimizes MSE but underperforms due to overfitting. Current methods implicitly address this through ad-hoc regularization (Ren et al., 2019a; b; Jia et al., 2023a; b), indicating the field of LDL lacks principled solutions.\n\nWe will expand on these points in Section 6 to clarify the motivation for our work.\n\n**Comment 2:** In Robustness testing, shown in Figure 4, why only other five methods were used to compare? In Table 2, the authors comapre the proposed δ-LDL with the existing other eleven methods.\n\n**Response:** We selected representative methods from different performance tiers to maintain *clarity* in the visualization. Including all 11 methods would make the plot overcrowded and hard to interpret. Tables 2 & 3 provide the complete comparison for readers interested in detailed results.\n\n**Comment 3:** In Table 2, the detailed experimental results on different datasets, there is no need to ranking the other eleven competitors, which reduces the readability of the paper.\n\n**Response:** The ranking was intentionally included to demonstrate that while $\\mu$ is derived from KLD, it produces meaningfully different evaluation outcomes than KLD alone. As discussed in **Section 5.2**, this highlights how $\\mu$ addresses known limitations of traditional metrics. We will add a clearer explanatory note in the table caption to improve readability while maintaining this important comparative information.\n\n**Comment 4:** Although the current experiments are good enough, it is more convinced if the authors can explain the reason why these seven datasets are used in the current experiments.\n\n**Response:** The seven datasets were carefully selected to represent diverse real-world LDL applications across multiple domains: Aesthetic perception ($\\mathtt{M}^{\\mathtt{2}}\\mathtt{B}$ & $\\mathtt{fbp5500}$), facial emotion recognition ($\\mathtt{RAF\\\\_ML}$ & $\\mathtt{SBU\\\\_3DFE}$), multi-class image classification ($\\mathtt{Natural\\\\_Scene}$), and artistic emotion perception ($\\mathtt{Painting}$ & $\\mathtt{Music}$). Due to the complexity of human subjective perception, these tasks particularly exhibit the characteristic of label polysemy - a crucial challenge LDL aims to address.\n\n**Comment 5:** In the last paragraph of Introduction, the authors highlight the contributions of the paper. It looks more like contributions and organizational structure.\n\n**Response:** Yes. We will revise the subsection title to better reflect its dual purpose.", "claims_and_evidence": "YES", "methods_and_evaluation_criteria": "YES", "theoretical_claims": "YES", "experimental_designs_or_analyses": "YES", "relation_to_broader_scientific_literature": "This paper proposes DeltaLDL, a percentage of predictions that are approximately correct within the context of LDL, as a solution to some existing deep-rooted problems of LDL. DeltaLDL can serve as a novel evaluation metric and a novel learning objective.", "essential_references_not_discussed": "NO", "other_strengths_and_weaknesses": "Strengths:\n1) The authors conduct a theoretical analysis of the KLD to demonstrate its unsuitability as an evaluation metric/learning objective for LDL.\n2) The authors propose DeltaLDL, which can serve as a solution to some existing deep-rooted problems of LDL.\n3) DeltaLDL can serve as a novel evaluation metric and a novel learning objective. \n4) Finally, the authors encapsulate DeltaLDL to propose a new LDL algorithm, named δ-LDL.  The empirical results validate the effectiveness of the proposed δ-LDL.\n\nWeaknesses:\n1) Some necessary justifications about the existing deep-rooted problems of LDL are needed. In Introduction: \"For years, there are some deep-rooted problems in the field of LDL:\" How to draw this conclusion? Combining with the related work (Section 6), More detailed elaboration is needed.\n2) In Robustness testing, shown in Figure 4, why only other five methods were used to compare? In Table 2, the authors comapre the proposed δ-LDL with the existing other eleven methods.\n3) In Table 2, the detailed experimental results on different datasets, there is no need to ranking the other eleven competitors, which reduces the readability of the paper.\n4) Although the current experiments are good enough, it is more convinced if the authors can explain the reason why these seven datasets are used in the current experiments.", "other_comments_or_suggestions": "1) Some necessary justifications about the existing deep-rooted problems of LDL are needed. In Introduction: \"For years, there are some deep-rooted problems in the field of LDL:\" How to draw this conclusion? Combining with the related work (Section 6), More detailed elaboration is needed.\n2) In Robustness testing, shown in Figure 4, why only other five methods were used to compare? In Table 2, the authors comapre the proposed δ-LDL with the existing other eleven methods.\n3) In Table 2, the detailed experimental results on different datasets, there is no need to ranking the other eleven competitors, which reduces the readability of the paper.\n4) Although the current experiments are good enough, it is more convinced if the authors can explain the reason why these seven datasets are used in the current experiments.\n5) In the last paragraph of Introduction, the authors highlight the contributions of the paper. It looks more like contributions and organizational structure.", "questions_for_authors": "1) Some necessary justifications about the existing deep-rooted problems of LDL are needed. In Introduction: \"For years, there are some deep-rooted problems in the field of LDL:\" How to draw this conclusion? Combining with the related work (Section 6), More detailed elaboration is needed.\n2) In Robustness testing, shown in Figure 4, why only other five methods were used to compare? In Table 2, the authors comapre the proposed δ-LDL with the existing other eleven methods.\n3) In Table 2, the detailed experimental results on different datasets, there is no need to ranking the other eleven competitors, which reduces the readability of the paper.\n4) Although the current experiments are good enough, it is more convinced if the authors can explain the reason why these seven datasets are used in the current experiments.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper studies the problem of metric design for label-distribution learning problems. The motivation is to rectify certain issues with the use of the KL divergence for this problem, notably its limited discriminative ability in the case of ground-truth distributions being drawn from a Dirichlet distribution. The key idea is to propose a normalized version of the KL divergence (Equation 9), analogous to the empirical CDF at a fixed threshold. It is proposed to further consider a suitable average over a range of thresholds (Equation 11) as the final evaluation metric. A suitable convex relaxation (Equation 14, 17) then provides the metric for training.

Reviewers were generally supportive of the paper, finding it to provide an interesting new perspective and contribution to the LDL problem, with both theoretical and empirical grounding. The main concerns were:

(1) the precise choice of threshold -- i.e., $\delta_0$, the average KL divergence to a uniform vector, used to normalize the proposed metric. The author response clarified how this is used to represent the ratio of the proposed area metric against that of a perfect predictor, thus representing a normalized area with $1$ denoting a perfect predictor.

(2) the empirical results not including more larger-scale settings, wherein the computational costs of the method could be an issue. In the response, the authors added an additional vision dataset, and runtime comparisons. The authors also argued that further experiments in downstream tasks might introduce distribution mismatch.

We believe the concerns on the latter point are valid, and may somewhat limit the scope of the work in its present state. Nonetheless, in the context of the existing LDL literature, the work has provided an interesting new perspective with both conceptual and empirical contributions. We thus believe it likely to be of interest and value to the community, and recommend its acceptance. The authors are encouraged to incorporate points from the reviewer discussion in preparing their final manuscript.
Decision: Accept (poster)