Can Class-Priors Help Single-Positive Multi-Label Learning?
Authors: Biao Liu
Ning Xu
Jie Wang
Xin Geng
Venue: NeurIPS 2025 poster ()
Keywords: Multi-Label Learning
Primary area: general_machine_learning

Abstract: Single-positive multi-label learning (SPMLL) is a weakly supervised multi-label learning problem, where each training example is annotated with only one positive label. Existing SPMLL methods typically assign pseudo-labels to unannotated labels with the assumption that prior probabilities of all classes are identical.
However, the class-prior of each category may differ significantly in real-world scenarios, which makes the predictive model not perform as well as expected due to the unrealistic assumption on real-world application.
To alleviate this issue, a novel framework named Crisp, i.e., Class-pRiors Induced Single-Positive multi-label learning, is proposed. Specifically, a class-priors estimator is introduced, which can estimate the class-priors that are theoretically guaranteed to converge to the ground-truth class-priors. In addition, based on the estimated class-priors, an unbiased risk estimator for classification is derived, and the corresponding risk minimizer can be guaranteed to approximately converge to the optimal risk minimizer on fully supervised data.
Experimental results on ten MLL benchmark datasets demonstrate the effectiveness and superiority of our method over existing SPMLL approaches.

Can Class-Priors Help Single-Positive Multi-Labe Learning?
Biao Liu, Ning Xu", Jie Wang, Xin Geng School of Computer Science and Engineering, Southeast University, Nanjing 210096, China
Key Laboratory of New Generation Artificial Intelligence Technology and Interdisciplinary Applications (Southeast University), Ministry of Education, China
"Correspodiing author: xning@seu. edu. cn
Abstract
Single-positive multi-label learning (SPMLL) a weakly supervised multi-label learning problem, where each training example is annotated with only one positive label. Existing SPMLL methods typically assign pseudo-labels labels with the assumption that prior probabilities of all classes are identical. However, the class-prior of each category may differ significantly in real-world scenarios, which makes the predictive model not perform well as expected due to the unrealistic assumption on real-world application. To alleviate this issue, novel framework named CRISP, i.e., Class-pRiors Induced Single-Positivy multi-label learning, is proposed. Specifically, a class- -prioss estimator is introduced, which can estimate the class-priors that are theoretically guaranteed converge the ground- truth class-priors. In addition, based on the estimated class-priors, an unbiased risk estimator for classification is derived, and the corresponding risk minimizer can be guaranteed to approximately converge to the optimal risk minimizer on fully supervised data. Experimental results on ten MLL benchmark datasets demonstrate the effectiveness and superiority of our method over existing SPMLL approaches.
unannotated
1 Introduction
Multi-label learning (MLL) is a learning paradigm that aims to train a model on examples associated with multiple labels accurately predict relevant labels for unknown instances [43] 25). Over the
past decade, MLL has been successfully applied to various real-world applications, including image
annotation (30]. text classification 244] and facial expression recognition (2).
Compared with muuti--llas-ssngll-laaee learning, where each example is associated with a unique label, MLL involves instances that are assigned multiple labels. As the number of examples or categories is large, accurately annotating each label of an example becomes exceedingly challenging. To address the high annotation cost, single-pose kingle-positive multi-label learning (SPMLL) has been proposed
5 38). where each training example is annotated with only one positive label. Moreover, since many examples in multi-class datasets, such ImageNet 42), contain multiple categories but annotated with a single label, employing SPMLL allows for the derivation of multi-label predictors from existing numerous multi-class datasets, thereby expanding the applicability of MIL..
To address the issue that model tends to predict all labels as positive if trained with only positive labels, existing SPMLL methods typically assign pscudo-labels to unannotated labels. Cole et al. updates the pseudo-labels as learnable parameters with a regularization to constrain the number of expected positive labels [5]. Xu al. recovers latent soft pseudo-labed by employing variational label enhancement [38] Zhou et al. adopts asymmetric-tolerant strategies to update pseudo-labed
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

cooperating with an entropy maximizationn loss [45]. Xie et al. utilizes contrastive learning to learn the manifold structure information and updates the pseudo-labels with threshold [33]
These approaches rely on a crucial assumption that prior probabilities of all classes are identical. However, in real-world scenarios, the class-prior of each category may differ significantly. This unrealistic assumption will introduce severe biases into the pseudo-labels, further impacting the training of the model supervised by the inaccurate pseudo-labels. As a result, the learned model could not perform as well as expected.
Motivated by the above consideration, we propose novel framework named CRISP, i.e., Class Blass-pRRors Induced Single-Posien Positive multi-label learning. Specifically, a class class-priors estimator is derived, which determines optimal threshold by estimating the ratio between the fraction of positive labeled samples and the total number of samples receiving scores above the threshold. The estimated class- priors can be theoretically guaranteed converge to the ground-truth class-priors. In addition, based on the estimated class-priors, an unbiased risk estimator for classification is derived, which guarantees the learning consistency (26) and ensures that the obtained risk minimizer would approximately converge to the optimal risk minimizer on fully supervised data. Our contributions can be summarized as follows:
Practically, for the first time, we propose a novel framework for SPMLL named CRISP, which estimates the class-priors and then an unbiased risk estimator is derived based on the estimated class-prirre addressing the unrealistic assumption of identical class-priors for all classes.
Theoretically, the estimated class-priors can be guaranteed to converge to the ground-truth class- priors. Additionally, we prove that the risk minimizer corresponding to the proposed risk estimator can be guaranteed to approximately converge to the optimal risk minimizer on fully supervised data.
Experiments on four multi-label image classification (MLIC) datasets and six MLL datasets show the effectiveness of our methods over several existing SPMLL approaches.
2 Related Work
Multi-label learning is a supervised machine learning technique where an instance is associated with multiple labels simultaneously. The study of label correlations in multi-label learning has been extensive, and these correlations can be categorized into first-order, second-order, and high-order correlations. First-order correlations involve adapting binary classification algorithms for multi-label learning, such as treating each label as an independent binary classification problem [1 27 Second- order correlations model pairwise relationships between labels 9 High-order correlations take into account the relationships among multiple labels, such as employing graph convolutional neural networks to extract correlation information among all label nodes increasing interest in utilizing label-specific features, which are tailored to capture the attributes of a specific label and enhance the performance of the models [1].
[3]. Furthermore, there has been an
In practice, accurately annotating each label for every instance multi-label learning is unfeasible due to the immense scale of the output space. Consequently, multi-label learning with missing labels (MLML) has been introduced (2 MLML methods primarily rely on low-rank, embedding, and
graph-based models. The presence of label correlations implies a low-rank output space (25) which
has been extensively employed to fill in the missing entries in label matrix [35] 40 34 Another
widespread approach is based on embedding techniques that map label vectors to wiw-dimeniional space, where features and labels are jointly embedded to exploit the complementarity between the feature and label spaces [39 31 Additionally, graph-based models prevalent solutions for MLML, constructing a label-specific graph for each label from a feature-induced similarity graph and incorporating manifold regularization into the empirical risk minimization framework [28
In SPMLL, a specific case of multi-label learning with incomplete labels, only one of the multiple positive labels is observed. The initial work treats all unannotated labels as negative and updates the pseudo-labels as learnable parameters, applying a regularization to constrain the number of expected 36 16 is used to recover latent soft labels and train the multi-label classifier [38]. The introduction of an asymmetric pseudo-labe approach utilizes seymmettiictolerrame strategies for pscudo-labels, along with an entropy maximization loss
positive labels [ A label enhancement process (37 22 21

---

45]. Additionally, Xie et al. proposes a label-aware global consistency regularization method, leveraging the manifold structure information learned from contrastive learning to update pseudo- labels [33 Liu et al. investigates the theoretical guarantee of pseudo-label-basee
methods for SPMILL (23 proving the learnability of such methods and proposing mutual label enhancement approach that iteratively refines the label distributions [15] 17 19 T8 14) of samples and optimizes the multi-label classifier.
3 Preliminaries
3.1 Multi-Label Learning
Let X' = IR denote the instance space and y = {0, 13e denote the label space with e classes. Given the MLL training set {( y1)l s S n) where I; E X' is q-dimensional instance and y: E y is its corresponding labels. Here, y; [y] V?. ...,. y/ where y 1 indicates that the j-th label a relevant label associated with a; and y! = indicates that the j-th label is irrelevant æ;, Multi-label learning is intended produce a multi-label classifier in the hypothesis space h E H. : + y that minimizes the following classification risk:
R(h) E (x..))p((..)) [C(h(z), y)],
where L: X' x R+ is multi-label loss function that measures the accuracy of the model in fitting the data. Note that method is risk-consistent if the method possesses a classification risk estimator that is equivalent to R(f) given the same classifier (26]
3.2 Single-Positive Multi-Label Learning
For single-positive multi-label learning (SPMLL), each instance is annotated with only one positive label. Given the SPMLL training set D {(x,, 7:] < i n) where % E 21, denotes the only observed single positive label of z;, For each SPMLL training example %), we use the observed single-positive label vector L, = [1] ..1 E {0, 1] to represent whether j-th label the observed positive label, i.e., 1/ = 1 if j = Yt. otherwise I = The task of SPMLL is to induce multi-label classifier h E H X' + y from 27, which can assign the unknown instance with relevant labels.
4
The Proposed Method
4.1 The CRISP Algorithm
In this section, we introduce our novel framework, CRISP, i.c., Class-pRiors Induced Single- multi-label learning. This framework alternates between estimating class-priors and optimizing an unbiased risk estimator under the guidance of the estimated class-priors.
Firstly, we introduce the class-priors estimator for SPMIL,, leveraging the blackbox classifier f to estimate the class-prior of each label. The class-priors estimator exploits the classifier f to give each input score, indicating the likelihood of it belonging to a positive sample of j-th label. Specifically, the slaas-priors estimator determines an optimal threshold by estimating the ratio between the fraction of the total number of samples and that of positive labeled samples receiving scores above the threshold, thereby obtaining the class class-prior prior probability of the j-th label.
Motivated by the definition of top bin in learning from positive and unlabeled data (PU learning) [0], for given probability density function p(z) and classifier f. define the threshold cumulative
density function (2) = Js. p( (z daz where S, = {ar X (a) 2 =} for all E [0, 1]. (2) captures the cumulative density of the feature points which are assigned value larger than a threshold z by the classifier the j-th label. We now define an empirical estimator of qj (x) as
(2) = 1 Σn= 1(f5 (x:) 2 2) where 1(.) is the indicator function. For each probability density function P" = =p(x = 1), P = p( By = 0) and p = p(x),, we define of = Js, p(æly, = 1 da
and 93 Sss, p( (y 0)dx respectively.
3

---

Algorithm CRISP Algorithm
Input: The SPMLL training set D (m) Sis n), the multi-label classifier f, the number of epoch T' hyperparameters 0 _ 11;
1: for = 1 to T do
2: for j to do
3: Extract the positive-labeled samples set St, {z, : =1,1ii Estimate (z(=) > (x,) 2) and (p(2) - ((f(x) 2) for all
4:
E 1).
5:
Estimate the class class-prior of j-th label by & = 4,(5) (2)
with the threshold induced by Eq. 2
6: end for
7: Update the model f by forward computation and back wock-propagation by Eq. using the
estimated class-priors.
8: end for
Output: The predictive model f.
The steps involved the procedure are as follows: Firstly, for each label, we extract a positive- labeled samples set Sr,- = {ar: 1 = 1, S S n from the entire dataset. Next, with Sty we estimate the fraction of the total number of samples that receive scores above the threshold (=) = 3 Σー (x,) 2 z) and that of positive labeled samples receiving scores above the threshold (²)) nz Exss, (x)(()) 2 2) for all x E [0. 1], where n² = -Sz, is the cardinality labeled samples set of j-th label. Finally, the class-prior of j-th label is estimated by
of the positive-labeled
(3(3) at & that minimizes the upper confidence bound defined in Theorem 4.1
(3)
Theorem 4.1. Define 2n
arg 2i 97 (z))
for every
1. define å the estimated
arg g
(.(=)
log(4/6)
log (4/6) 2n
Assume n²
log 4/8 of (=
(2)
(=)
2n
class-prion f
9 (2)
satisfies with probability at least 5:
4 (3)
log(4/8)
log(4/6)
T (2*)
J
(x))
2n
+
2np
< S S T +1 (1-
o3 (2*)
C2 of (2*)
log(4/0)
log 4/8)
2n
2mp
where c1. C2 2 are constants and T fixed parameter ranging in (0, 1). The proof can found in Appendix A. Theorem 4.1 provides principle for finding the optimal threshold. Under the condition that the threshold satisfies:
(2)
1 (3)
log(4/0)
log(4/0) 2n
2 = arg min 2E(0.1]
2n
the estimated class-prior ř of j-th category will converge the ground-truth class-prior with enough training samples. Practically, to determine the optimal threshold Eq. P we conduct an exhaustive search across the set of outputs generated by the function f for each class. The details can be found in Appendix A.2
After obtaining an accurate estimate of class-prior for each category. we proceed to utilize these estimates as form of supervision to guide the training of our model. Firstly, the classification risk R(f) on fully supervised information can be written as
R(f) = E ((x.))))(((a))) [C((()) w)1 = Σ E Bm-p(ll)) [f(((x))))
(3)
'The datail is provided in Appendix
A.3
A.3

---

In Eq. (3) the loss function [(f(2), y) is calculated for each label separately, which is commonly used approach in multi-label learning:
((f(x). y) = ((1-
+
(4)
By substituting Eq. (4) into Eq. 33. the classification risk R(f) can be written as follows with the absolute loss function
R(1) = Σ 2p(ys = 1) eeppl)))
[((x)] + (Rxxp(() [f(x)]] - plus = 1)) (5)
The rewritten classification risk comprises two distinct components. The first component computes the risk solely for the positively labeled samples, and the second component leverages the unlabeled data to estimate difference between the expected output of the model f and the class-prior マj - -P(yj = 1)
align the expected class-prior outputted by model with the ground-truth class-prior.
During the training process, the prediction of model can be unstable due to insufficiently labeled data. This instability may cause large divergence between the expected class- Mlas-pproor Eff (x) and the ground-trun -ruth class-prior J. even leading to situation where the difference between E (x)] and n turns negative [44]. To ensure non-negativity of the classification risk and the alignment of class-priors absolute function added to the second term. Then the risk estimator can be written as:
Rep (f) = Σ 2n,, Bo-pal))))) [1 (x))) + Bewp(a)) [f T3
(6)
Therefore, we could express the empirical risk estimator via:
{ (P(()))))))) - Rfp(F)
2
Σ (1-f(
mESL;
ΦED
The proposed equation enables the decomposition of the risk over the entire dataset into terms that be estimated using both labeled positive and unlabeled samples.
In MLL datasets, where the number of negative samples for each label significantly exceeds that of positive samples, there is a tendency for the decision boundary to be biased towards the center of positive samples, especially common strategy of assuming unobserved labels negative [5] 38 33 alleviate the issue, we propose a modification of Eq.
rare classes. This bias is further exacerbated in SPMLL due to the
to warm up the model. To
(f) =
2 SL
Σ 2ES1,
1 (x (1I+ee-Mo))N))
1
(f(x))) -T))
(8)
where À is hyyer-paramettr and g (x) represents the logit of j-th label outputted by the network for instance æ and (x) = agg' (x)) where (f-) denotes the sigmoid function.
The algorithmic description of CRISP is shown in Algorithm I
4.2 Estimation Error Bound
In this subsection, estimation error bound is established for Eq. consistency. Firstly, we define the function spaces as:
to demonstrate its learning
G4p (1r, Σ 2n,l, (1- (1))
(ar
Σ ( 15 E FF.
j=1
and denote the expected Rademacher complexity 26 of the function spaces as:
RR (g4))
Em.yy.
sup ,
Σ
o.g (z: 9:)
A. (g%)
Em....
sup
Σ o,g (Dis v.)
yEgur
5

---

Table I: Predictive performance of each comparing method four MLIC datasets in terms of mean average precision (mAP) (mean + std). The best performance is highlighted bold (the larger the better).
VOC
COCO
NUS
CUB
AN
85.546+0.294
64.326+0.200
42,494+0.333
18.656++0.000
AN-LS
87.548+0.137
67.074+0.196
43.616++0.32
16,446=+0.269
WAN
87.138=0.240
65,552+0.171
45.785=0.192 192
14,622++
EPR
85,228=0.444
63,60440.249
45.240=0.333
19.842+00423
ROLE
88.088+0.161
67.022+0.141
41.949+0.205
10.793+00611
EM
88.67410.077
70.636=10009
47.25410.297
20.692-10.527
EM-APL
88.860=1.000
70.758+0.215
47.778+0.181
21.202-+0.792
SMILE
87.314=0.150
70.431+0.213
47.241=0.172
18.611
PLC
88.021+0.121
70.422+0.062
46.211+0.155
21.340=0.237
LI-R
87.784+0.0063
70.078+0.008
48.048
00.926
LL-CP
87.466:+0.031
70.460+0.032
48.000+0.077
19.310=+0.144
LL-CT
87.054:10.214
70.38410.055
47.93010.010
19.012:10.097
BOOSTLUALI-R
89.224+000017
73.272+0.006
49.590=0.021
19. 10.136+00009
ROOSTI.U+LI-CP
88.25
70.820+0.030
47.810+0.166
18. 00.06
BOOSTL.U+LI-CT
88.528++0.053
71.742+0.006
48.216+0.021
17.952++0.007
CRISP
89.820:10.191
74.64010.2119
09.99611.3116
21.650:10.178
where a = fo1.02, 02. nn is n Rademacher variables with oe independently uniform variable taking value in [+1,-1111 -13. Then we have:
Theorem 4.2. Assume the loss function CtP (f (x)) -T,) could be bounded by M, iee, M
Σj=1 (1-f((x))
and CT
(x), y), (f(x),,))) with probability at least - 8, we have: SuP m (f(x),
2/2p c
M
log 1/4 2n
c
log 1
R(fap)-
Σ
Rn (H,) + miny |SL,
+ +4 _ R Rn,) M
C
2n
where C is a constant, fap = men and the true risk minimizer respectively and p = max, 2T,,, Hz = {h= : + (z) E F} and RR (Hz) = (x)) F. SUPREH, 1.2 Ei= h (m:) The proof can be found in Appendix A.5
SEF (fp(f), fr minser R(f) are the empirical risk minimizer
Theorem 4.2 shows that, as - xc, fap would converge to f with an intrinsic error quantified by the Rademacher complexity terms, reflecting the complexity of the hypothesis space. Note that the error a fundamental aspect of the learning problem and remains even fully supervised scenario
26
5
Experiments
5.1 Experimental Configurations
Datasets. In the experimental section, our proposed method is evaluated on four large-scale multi-label image classification (MLIC) datasets and six widely-used multi-label learning (MLL) datasets. The four MLIC datasets include PSACAL vOc 2021 (VOC) [8, MS-COCO 2014 (COCO) 20), NUS-WIDE (NUS) and CUB-200 2011
(CUB) 291; the MLL datasets cover a wide range of scenarios with heterogeneous multi-label characteristics. For each MLIC dataset, 20% of the training set withheld for validation. Each MI.L dataset partitioned into train/validation/test sets at ratio of 80% 10% 10%. One positive label is randomly selected for each training instance, while the validation and test sets remain fully labeled. Detailed information regarding these datasets can be found in Appendix A.7 Mean average precision (mAP) is utilized for the four MLIC datasets 5, (45) and five popular multi-label metrics are adopted for the MLL datasets including Ranking loss, Hamming loss, One-error, Coverage and Average precision
38
The detail is provided in Appendix A.4

---

Table 2: Predictive performance of each comparing method on MLL datasets in terms of Ranking loss (mean + std). The best performance is highlighted in bold (the smaller the better).
Image
Soene
Yeast
CorelSk
Mirflickr
Delicious
AN
0.432=0.067
0.321=0.113
0.383=0.066
0.140=0.000
0.125=0.002
0.131=0.000
AN-LS
0.378=0.041
0.246=0.064
0.365=0.031
0.186=0.003
0.163=0.006
0.213=0.007
WAN
0.354=0.051
0.216=0.023
0,212=0.021
0.129=0.000
0.121=0.002
0.126=0.000
EPR
0.401=0.053
0.291-0.056
0.208=000110
0.139=0.000
0.119=0.001
0.126=0.000
ROLE
0.340:00059
0.074
0.213:00017
0.259
0.182:00.014
0.006-00007
EM
0.47==00044
0.
0.261=0.030
0.155=00002
0.134=0.004
0.164=0.001
EM-APL
0,508=0.028
0.420=0.069
0,245=0.026
0.135=0.001
0.138=0.003
0.163=0.003
SMILE
0.260=0.220
0.161=00045
0.167=0.002
0.125=00003
0.120=00002
0.126=0.000
LI-R
0.346-0.072
0.155-0.002
0,227-0.001
0.114-0.001
0.123-0.003
0.129-0.002
LL-CP
0.329=0.04]
0.148=00017
0.215=0.000
0.114=00005
0.124=0.003
0.160=0.001
Lt-CT
0.227=00199
0.180=0.038
0.238=0.001
0.115=
0.124=0.002
0.160=0.000
CRISP
0.164=0.027
0.112=0.021
0.164=0.001
0.001
0.118:0.001
0.122=0.000
Figure 1: Predicted class-prior of (5 AN- WAN EPR 51 ROLE EM EM-APL 45
SMILE 38 and CRISP on the 3-rd (left), 10-th (middle), and 12-th labels (right) of the dataset Yeast.
Comparing methods. In this paper, CRISP is compared against nine state-of-the-art SPMLL approaches including: 1) AN (5) assumes that the unannotated labels negative and uses binary cross entropy loss for training. 2) AN-LS 5 assumes that the unannotated labels are negative and reduces the impact of the false negative labels by label smoothing. 3) WAN [ introduces a weight parameter to down-weight losses in relation to negative labels. 4) EPR 5 utilizes a regularization to constrain the number of predicted positive labels. 5) ROLE (5) online estimates the unannotated labels as Iearnable parameters throughout training based on EPR with the trick of linear initial. 6) EM (45 reduces the effect of the incorrect labels by the mutroy-maximizze adopts asymmetric-toleraag
loss. 7) EM-APL [45] pseudo-label strategies cooperating with entropy maximyyaaaiiizttion loss and then more precise supervision can be provided. 8) PLC 33 designs a label-aware global consistency regularization to recover the pseudo-labed leveraging the manifold structure information learned by contrastive learning with data augmentation techniques. 9) SMILE (38] recovers the latent soft labels in a label enhancement process to train the multi-label classifier with binary cross entropy loss. Additionally, since the SPMLL problem an extreme case of the MLML problem, we employ a mate-of-theeard MLML. methods as comparative methods: 1) LL 12 treats unobserved labels as noisy labels and dynamically adjusts the threshold to reject or correct samples with a large loss, including three variants LL-R, LL-CT and LL-CP. 2) BOoSTLU [33 apply BoostLU function to the CAM output of the model to boost the scores of the highlighted regions. It can be integrated with LL. The implementation details are provided in Appendix A.6
5.2 Experimental Results
Table D presents the comparison results of CRISP compared with other methods on VOC, COCO, NUS, and CUB. The proposed method achieves the best performance on VOC, CDCO, and NUS. Although does not surpass the top-performing method on CUB, the performance remains competitive. Table record the results of our method and other comparing methods on the MLL datasets in terms of Ranking loss respectively. Similar results for other metrics can be found in Appendix [A. Note that due to the inability to compute the loss function of PLC without data augmentation, we do not report the results of PLC on MLL datasets because data augmentation techniques are not suitable for the MIL datasets. Similarly, since the operations of BOOSTLU for CAM are not applicable to the tabular data in MIL datasets, its results are also not reported. The results demonstrate that our proposed method consistently achieves desirable performance in almost cases (except the result of Mirflick on the metric Average Precision, where our method attains comparable performance
7

---

Vic
30-
696
Can
105
(a) Sensitivity analysis of 6
(b) Sensitivity analysis of
(c) Ablation results
Figure 2: (a) Parameter sensitivity analysis of (parameter 7 is fixed as 0.01); (b) Parameter sensitivity analysis of 7 (parameter is fixed 0.01): (c) The initial data point represents the performance of the proposed CRISP (with class- prssoss estimator). The others are the performance with a fixed value for all class-priors gradually increasing from 0.001 to 0.3.
against SMILE). Table 12 in Appendix A. reports the p-values of the wilcoxon signed-ranks test [6] for the corresponding tests and the statistical test results at 0.05 significance level, which reveals that CRISP consistently outperforms other comparing algorithms (49 of 55 test cases score win). These experimental results validate the effectiveness of CRISP in addressing SPMLL problems.
5.3 Further Analysis
5.3.1 Class-Prior Prediction
Diguureiillustrats the comparison results of the predicted class-priors of CRISP with other methods on the 3-rd (left), 10-th (middle), and 12-th labels (right) of the dataset Yeast. Compared with other approaches, whose predicted class-priors P(ū; 1), which represents the expected value of the predicted, significantly deviate from the true class-priors, CRISP achieves consistent predicted class-priors with the ground-truth class-priors (black dashed lines). Without the constraint of the class-priors, the predicted class-prior probability diverges from the true class-prior as epochs increase, significantly impacting
model's performance. In this experiment, the true class-priors are derived by calculating the statistical information for each dataset. More experimental results about the convergence analyses of estimated class-priors all classes on MLIC datasets are recorded in Appendix A.9 These results demonstrate the necessity of incorporating class dass-priors in the training of the SPMLL model.
5.3.2 Sensitivity Analysis
The performance sensitivity of the proposed CRISP approach with respect to its parameters and during the flas-prrooss estimation phase is analyzed this section. Figures Figures[ajand (andee/aaaadndlilluuuusssa [2]]ilustte the performance of the proposed method on VOC and COCO under various parameter settings, where 8 and t are incremented from 0.001 to 0.1. The performance of the proposed method remains consistently stable across wide range of parameter values. This characteristie is highly desirable as it allows for robust application of the proposed method without the need for meticulous parameter fine-tuning, ensuring reliable classification results.
5.3.3 Ablation Study
Figure 2c| depicts the results of the ablation study to investigate the impact of the class-priors estimator by comparing it with a fixed value for all class-priors. The initial data point represents the performance of the proposed CRISP (with class-priors estimator). Subsequently, we maintain a fixed identical class-priors, gradually increasing it from 0.001 to 0.3. As expected, our method exhibits superior performance when utilizing the class-priors estimator, com- pared with employing fixed class-prior proportion. The ablation results demonstrate the significant enhancement CRISP perfor- mance achieved through the proposed class-priors estimator.
Table 3: Predictive perfor-
mance comparing CRISP with
CRISP-VAL.
Datuset
CRISP-VAL
CRISP
VOC
89.58510.3188
89.820.0.01
COCO
74.435:10.14
74.699-:0.219
NUS
49 49:230=0.113
493996:20.316
19.60011.400
21.650)10.178

---

Table 4: Time cost of class- faas-priors estimation and the whole training time of one epoch.
Table 5: Predictive performance with different updating frequency of class class-priors priors estimation.
VOC
COCO
CUB
VOC
COCO
NUS
CUB
Time of Clss-ppion extimation (min) Whole training time of one epoch (min)
0.24
3.47
6.4
0.45
ChISP-Jer
8937770 251
73 399
29 99-66621.210
19, 19,450891.399
2.19
27.29
49.00
3,89
CRESP
09.83000.10 191
74.4000.211
40 99640.344 316
21,65000, 178
Original image
Attention map
Original Imagee
Attration map
Utserved label
Jöeiiieed labels
Observed label
Ideaiee labels
Mysle
konir
how!
cavi
firk
Bus
di/jinee
Parsoe
Aapopo
bontic
cap
Figure 3: Visualization of attention maps on VOC (left) and COCO (right).
Furthermore, we conduct experiments comparing the performance of CRISP with the approach that estimating the class-priors with the full labels of validation (CRISP-VAL). T shows that the performance of CRISP is superior to CRISP-VALL using the validation which can lead to unstable estimation of the class-priors, thus leading to a suboptimal performance. Similar results are observed in Table 13 of Appendix A.11 for the MLL datasets.
is indeed feasible to estimate the class-prion However, the size of validation set in many datasets is often quite small,
5.3.4 Time Cost of Class-Priors Estimation
In Eq. we have adopted an exhaustive search strategy to find an optimal threshold for estimating class-priors in each training epoch, which may introduce additional computational overhead the algorithm. We conducted experimental analysis on this aspect. As illustrated in the Table the time for class-priors estimation is short compared to the overall training time for an epoch, ensuring that our method remains practical for use in larger datasets. Additionally, to further enhance the speed of our algorithm, we have experimented with updating the class-priors every few epochs instead of every single one in Table 5 The variant of our method, denoted as CRISP-3EP, updates the priors every three epochs and our experiments show that this results in a negligible loss in performance.
5.3.5 Attention Map Visualization
Figure is utilized to visually represent attention maps on COCO, elucidating the underlying mecha- nism responsible for the efficacy of CRISP in discerning potential positive labels. Specifically, for each original image in the first column, attention maps corresponding to the single observed positive label and identified positive labels are displayed in the subsequent three columns. As evidenced by the figures, given the context of a single positive label, the proposed method demonstrates the ability identify additional object labels within the image, even for relatively small objects such as the bottle in the first row, the fork in the second row, and the cup in the final row. These observations indicate that the proposed method can accurately detect small objects with the aid of class-prion This insight further suggests that the proposed method substantially enhances the model's 's capacity to pinpoint potential positive labels.
Conclusion
In conclusion, this paper presents novel approach to address the single-positive multi-label learning (SPMLL) problem by considering the impact class-priors on the model. We propose theoretically guaranteed class-priors estimation method that ensures the convergence of estimated class-prior to ground-truth class class-priors during the training process. Furthermore, we introduce an unbiased risk estimator based on the estimated class- and derive generalization error bound guarantee

---

that the obtained risk minimizer would approximately converge to the optimal risk minimizer of fully supervised learning. Experimental results on ten MLL benchmark datasets demonstrate the effectiveness and superiority of our method over existing SPMI.I approaches.
7
Acknowledgments
This research supported by the Jiangsu Science Foundation (BG2024036, BK20243012), the National Science Foundation of China (62576093, 62206050, 62125602, U24A20324, and 92464301), the Fundamental Research Funds for the Central Universities (2242025K30024) and the Big Data Computing Center of Southeast University.
References
[1] Matthew R Boutell, Jiebo Luo, Xipeng Shen, and Christopher M Brown. Learning multi-label scene classification. Pattern recognition, 37(9)11557-1771 2004.
[2] Shikai Chen, Jianfeng Wang, Yuedong Chen, Zhongchao Shi, Xin Geng, and Yong Rui. Label distribution learning auxiliary label space graphs for facial expression recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13981-13990 Seattle, WA, 2020.
[3] Zhao-Min Chen, Xiu-Shen Wei, Peng Wang, and Yanwen Guo. Multi-label image recognition with graph convolutional networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 5177-5186, Long Beach, CA, 2019.
[4] Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng. NUS- WIDE: a real-world web image database from national university of singapore. Proceedings of the 8th ACM International Conference on Image and Video Retrieval, Santorini Island, Greece, 2009.
[5] Elijah Cole, Oisin Mac Aodha, Titouan Lorieul, Pietro Perona, Dan Morris, and Nebojsa Jojic. Multi-label learning from single positive labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 933 933-942, virtual, 2021.
[6] Janez Demšar. Statistical comparisons of classifiers over multiple data sets. The Journal of Machine learning research, 7:1-30, 2006.
[7] André Elisseeff and Jason Weston. A kernel method for multi-labelled classification. In Advances in Neural Information Processing Systems 14, pages 681-687, Vancouver, BC, Canada, 2001.
[8] Mark Everingham, Luc Van Gool, Christopher K. I. Williams, John M. Winn, and Andrew Zisserman. The pascal visual object classes (VOC) challenge. International Journal of Computer Vision, 88(2)3033338, 2010.
[9] Johannes Fürnkranz, Eyke Hüllermeier, Eneldo Loza Meneía, and Klaus Brinker. Multilabel classification via calibrated label ranking. Machine learning. 1332)1133-153, 2008.
[10] Saurabh Garg, Yifan Wu, Alexander J. Smola, Sivaraman Balakrishnan, and Zachary C. Lipton. Mixture proportion estimation and PU learning: A modern approach. In Advances in Neural Information Processing Systems 34, pages 8532-8544 Virtual. 2021.
[11] Jun-Yr Hang and Min-Ling Zhang. Collaborative learning of label semantics and deep label- specific features for multi-label classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, (4(11):9860-9877 2022.
[12] Youngwook Kim, Jae Myung Kim, Zeynep Akata, and Jungwoo Lee. Large loss matters in weakly supervised multi-label classification. Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 14156-14165. New Orleans, LA, 2022.
10

---

[13] Youngwook Kim, Jae-Myrnee Lue-Myung Kim, Jieun Jeong, Cordelia Schmid, Zeynep Akata, and Jungwoo Lee. Bridging the gap between model explanations in partially annotated multi-label classifica- tion. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition, pages 3408-3417, Vancover, BC, Canada, 2023.
[14) Zhiqiang Kou, Si Qin, Hailin Wang, Jing Wang, Mingkun Xie, Shuo Chen, Yuheng Jia, Tongliang Liu, Masashi Sugiyama, and Xin Geng. Label distribution learning with biased annotations assisted by multi-label learning. In Proceedings of International Joint Conference on Artificial Intelligence, Montreal, Canada, 2025.
[15] Zhiqiang Kou, Jing Wang, Yuheng Jia, and Xin Geng. Inaccurate label distribution learning. IEEE Transactions on Circuits and Systems for Video Technology, (4110:00237-10243 2024.
[16) Zhiqiang Kou, Jing Wang. Yuheng Jia, and Xin Geng. Progressive label enhancement. Pattern Recognition, 160: 11172, 2025.
[17] Zhiqiang Kou, Jing Wang, Yuheng Jia, Biao Liu, and Xin Geng. Instance-dependent inaccurate label distribution learning. IEEE Transactions on Neural Networks and Learning Systems, 36(1):1425-14) 2025.
[18] Zhiqiang Kou, Jing Wang. Jiawei Tang, Yuheng Jia, Boyu Shi, and Xin Geng. Exploiting multi-label correlation in label distribution learning. In Proceedings of International Joint Conference on Artificial Intelligence, pages 4326-43344 Jeju Island, Korea, 2024.
[19] Zhiqiang Kou, Haoyuan Xuan, Jingyu Zhu, Hailin Wang, Ming-kun Xie, Changwei Wang. Jing Wang, Yuheng Jia, and Xin Geng. Tail-aware reconstruction of incomplete label distributions with low-rank and sparse modeling. IEEE Transactions Circuits and Systems for Video Technology, 2025.
[20] Tsung- Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: common objects in context. In Proceedings of 13th European Conference Computer Vision, volume 8693, pages 740-755, Zurich, Switzerland, 2014.
[21] Biao Liu, Ning Xu, Xiangyu Fang, and Xin Geng, Correlation-induced label prior for semi- supervised multi-label learning. In Proceedings of the 41st International Conference on Machine Learning, volume 235, pages 32224-32238 Vienna, Austria, 2024.
[22] Biao Liu, Ning Xu, and Xin Geng. Progressively label enhancement for large language model alignment. In Proceedings of the 42nd International Conference Machine Learning, volume Vancouver, Canada, 2025.
267, pages
[23] Biao Liu, Ning Xu, Jiaqi Lv, and Xin Geng. Revisiting pseudo-label for single-positive multi- label learning. In Proceedings of International Conference on Machine Learning, volume 202, pages 22249-22265. Honolulu, Hawaïi, 2023.
[24] Jingzhou Liu, Wei-Cheng Chang, Yuexin Wu, and Yiming Yang. Deep learning for extreme multi-label text classification. In Proceedings of the 40-th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 115-124, Tokyo, Japan, 2017.
[25] Weiwei Liu, Haobo Wang, Xiaobo Shen, and Ivor W Tsang. The emerging trends of multi-label learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11):7955-79) 7974, 2021.
[26] Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar, Foundations of Machine Learning. Adaptive computation and machine learning. MIT Press, 2012.
[27] Jesse Read, Bernhard Pfahringer, Geoff Holmes, and Eibe Frank. Classifier chains for multi- label classification. Machine learning, 85(3):333-359, 201
[28] Yu-Yin Sun, Yin Zhang, and Zhi-Hua Zhou. Multi-label learning with weak label. In Proceed- ings of the 24th AAAI Conference on Artificial Intelligence, Atlanta, Georgia, 2010.
11

---

[29] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd birds-200-2011 dataset. California Institute of Technology. 2011.
[30] Changhu Wang, Shuicheng Yan, Lei Zhang, and Hong-Jiang Zhang, Multi-label sparse coding for automatic image annotation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 1643-1650, Miami, Florida, 2009.
[31] Kaixiang Wang. Robust embedding framework with dynamic hypergraph fusion for multi-label classification. In Proceedings of the IEEE International Conference on Multimedia and Expo, pages 982-987, Shanghai, China, 2019.
[32] Baoyuan Wu, Zhilei Liu, Shangfei Wang, Bao-Gang Hu, and Qiang Ji. Multi-label learning with missing labels. In Proceedings of the 22nd International Conference on Puttern Recognition, pages 1964-1968, Stockholm, Sweden, 2014.
[33] Ming-Kun Xie, Jia-Hao Xiao, and Sheng-Jun Huang. Label-aware global consistency for multi-label learning with single positive labels. In Advances in Neural Information Processing Systems, virtual, 2022.
(34] Chang Xu, Dacheng Tao, and Chao Xu. Robust extreme multi-label learning. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1284, San Francisco, CA, 2016
[35] Miao Xu, Rong Jin, and Zhi-Hua Zhou, Speedup matrix completion with side information: Application to multi-label learning. In Advances in Neural Information Processing Systems 26, pages 2301-2309, Lake Tahoe, Nevada, 2013.
[36] Ning Xu, Biao Liu, Jiaqi Lv, Congyu Qiao, and Xin Geng. Progressive purification for instance- dependent partial label learning. Proceedings of the 40th International Conference on Machine Learning, volume 202, pages 38551-3856 Honolulu, Hawaii, 2023.
[37] Ning Xu, Yun-Peng Liu, and Xin Geng. Label enhancement for label distribution learning. IEEE Transactions on Knowledge and Data Engineering, 33(4):1632-1666 2021.
[38] Ning Xu, Congyu Qiao, Jiaqi Lv, Xin Geng, and Min-Ling Zhang. One positive label is sufficient: Single-positive multi-label learning with label enhancement. In Advances in Neural Information Processing Systems, virtual, 2022.
[39] Chih-Kuan Yeh, Wei-Chieh Wu, Wei-Jen Ko, and Yu-Chiang Frank Wang. Learning deep latent space for lulti-label classification. Proceedings of the 21 AAAI Conference on Artificial Intelligence, pages 2838-2888 2844, San Francisco, California, 2017.
[40] Hsiang-Fu Yu, Prateek Jain, Purushottam Kar, and Inderjit Dhillon. Large-scale multi label learning with missing labels. In Proceedings of the 31th International Conference on Machine Learning, volume 32, pages 593-601, Beijing, China, 2014.
[41] Ze-Bang Yu and Min-Ling Zhang. Multi-label classification with label-specific feature genera- tion: wrapped approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 4499):5199-5210, 2022.
[42] Sangdoo Yun, Seong Joon Oh, Byeongho Heo, Dongyoon Han, Junsuk Choe, and Sanghyuk Chun. Re-labeling imagenet: From single to multi-labels, from global to localized labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2340-2355 virtual, 2021.
[43] Min-Ling Zhang and Zhi-Hua Zhou. A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering, 26(8): 819-1837, 2013.
[44] Yunrui Zhao, Qianqian Xu, Yangbangyan Jiang, Peisong Wen, and Qingming Huang. Dist- pu: Positive-unlabeled learning from a label distribution perspective. In Proceedings of the IFEF/CVF Conference on Computer Vision and Pattern Recognition, pages 14461-14470, New Orleans. LA, 2022.
12

---

[45] Donghao Zhou, Pengfei Chen, Qiong Wang, Guangyong Chen, and Pheng-Ann Heng. Ac- knowledging the unknown for multi-label learning with single positive labels. In Proceedings of 17th European Conference on Computer Vision, volume 13684, pages 423-440, Tel Aviv, Israel, 2022.
13

---

A Supplementary Material
A.1 Proof of Theorem 4.1
Proof. Firstly, we have:
(x)2) (2)
(2)
- - (=))(()))
(²) (2)
(2)
(z)())(f()) (z)))(())) (=)(=))(())))))))))))))))))))))) (=))
(2)
(5(2)
(2(2) |6 (0)(2) - (+=)1 + (2)))(()) 10)()) - of(2)),
where z is an arbitrary constant in [0, 1]. Using DKW inequality, we have with probability - 8: 2/5 2re
lāy(2) (z)l S
and l0 (z) (z(2) v
2/8 2ng
Therefore, with probability 6:
2 (2)
(2(2
1
log 4/6
(2)
log 4/8
(10)
(2)
(=)
(x)
2n
22)
2nlz
Then, we define:
(z) (=)
1
1og(4/5)
log (4/)
2 = arg min
+
(x)
2n
2n²
4,(=)
x" arg min
=E[0.1] (2))
(2)
93(2*)
R (8)
and
nj (2*)
Next, consider z' E [0, such that (=(²²)
전 of (=*). We now show that 8 < 2'. For any
E [0, 1]. by the DWK inequality, have with probability 1 1-6 6:
log 4/8
(=())
S (3),
V
2m
(11)
log 4/6
(2)
< Oj (2).
2n
Since (=(=") (=(=) [= (2)
we have:
log(4 (4(()
(og(4/8)
9 (2
log(4/0)
(2) > (2)
()(())
(12)
(z*)
2n
2n1
(z*)
2n
Therefore, we have:
1og(4/8)
log(4/8)
+ T
(13)
(z)
2n
2n1
Using Eq. (10 2 and the fact that π.
(z*) of (=*)
S 1 we have:
(2)
(2*))
1
1og(4/6)
log(4/8)
+
+ n.
(2)
(2*)
(3*)
(3(z)
2n
2n
(14)
(2'2)
log (4/5)
log(4/8)
+ (3(z)
+
(x*)
(2*)
2n
2n²
14

---

Furthermore, the upper confidence bound at is lower bounded by:
(z)
T
log(4/5)
log(4/0)
()(()
(=())
2n
2n²
(x
1 +
1
1og(4/6)
log(4/8)
+
(15)
(=t)
(2)
(3()t)
(2)
2n
2nd
(z)²)
(og((/0)
log( (1/()
(2))
(2)
(2))
2n
2n²
Using Eq. 15
at 2 where (2')
-
(2")),
we have:
2+r
(=(=)
1+
log(4/0)
log(4/0)
(²))
()(21)
2n
2mg
(,(2*)
--
1og(4/6)
log(4/0)
(16)
(10)
(5(2')
(5(3*)
2n
2nd
()(²)
1
(V
(1og(4/0)
1og(4/0)
(2)
(2))
2n
2n²
Moreover from Eq.
15 and using definition of have:
(v log(4/8)
(='²)
1
log (4/))
(3)))
(5)
2n
2n
(x)))
1+r (
log(4/0)
log (4/()
(17)
()))
()*)))
2n
2n
(3)
1+T
(4/0)
log(4/8)
+
(5)
(2)
2n
2nj
and hence 8 a..
We now establish an upper and lower bound on 3. By definition of غ, we have:
(2)
1
1og(4/8)
log(4/6)
(2)
(2)
2n
2n
(1())
1 ++
(0gg(4/))
log(4/6)
min =E
(18)
(2)
(2)
2n
2nL
0 (2*)
1+
(o)()
log (4/()
(2*)
()()*)
2n
2nd
Using Eq. (00) 2º, have:
(x))
(2*)
1
(V 1og(4/8)
log(4/8)
(3(3*)
(2))
(2(2
2n
T
2n²
(19)
Then, have:
(2)
2+r
(v log(4/0)
log(4/8)
4
2 n. +
(20)
(2))
2n
2n²
Assume n² 2 2 1/84/6 of (x-)
we have (3()) > of (z*)/2 and hence:
n, - - + 2T
1og(4/6)
(4g(4/))
(21)
(3()
2n
2m
15

---

From Eq. 10 at 2, we have:
(3 (2) (3) (v (Voo(t110)
(33)
(3)
43(3)
1og(4/6)
2n
+
(2) V 2ng
(22)
Since Tj 2 (
we have:
9f (2)
(8)
(
(og((/0)
+ (,(2)
(gg(4/6)
jj
(23)
(3)
(2)
(3)
2n
0f (2) (2)
2n²
Using Eq.
21 and the assumption that n 2 ng 2
log 4/6
we have:
of (x
R (3)
(2) S x + 4+ 2r
log(4/8)
(gg(4/))
+
(2))
2n
2n
(24)
S πz + 4 + 2T v 4 4 27 = 5 + 2T.
Using this in Eq. 23 we have:
3j - (8)
V /log(4/0)
log (4g() 2nl
+ (5 27)
(25)
(2)
(2)
2n
Since S 2', have (B) 2 (x')
z S2 (2*). Therefore, we have:
2 T (x*)
(v 1ogg(//0)
log(440)
m
+ (5 + 27)/ 2n
(26)
2n
(3)
With the assumption that n.. 2 2 1/84/5
we have (3()) 2 of (= 3/2, which implies:
29 (2-)
(2*)
4 2T
1og(4/0)
1og(4/8)
33 (x)
+ (5 (5++2)) 2
f...
(27)
2n
2n²
Note that since Ty S nT. the lower bound remains the same in Theorem
For the upper bound,
with qj 42(2*) n, (2*) + (1 ny (2))) have Tÿ n, + (1 nT of (2*)*
Then the proof is
completed.
A.2 The details of the optimization of Eq. (2)
In practice, to determine the optimal threshold, we conduct an exhaustive search across the set of outputs generated by the function f for each class. For instance, for given class j. and a set of instances 11, Ø2 03 in our dataset, we compute the corresponding outputs 21 = (x1). f (z22), (xa))
The optimal threshold then selected by identifying the value of {22, 22, 23} that minimizes the objective function specified in Equation (2):
(=(=)
1+T
1og(4/8)
log(4/6)
min (R
())
2n
2n
This approach ensures that we find the optimal threshold that minimizes the given expression, as per Eq. (2) across all available output values from the function f³.
16

---

A.3 Details of Eq. 3
R()- = (e v [f((()), y)1
/ m Σ [
Σp(y) J. [(f(x)) y)p(= y)
(28)
(F(s(x), y) (CU
A.4 Details of Eq. (5)
The absolute loss function (zf) (f((x) Vol. when ys = ((f((())) -1- P(x), ((oa and when y, 0. ((f)()))))) = (z)). Then:
R()) = Σ p(y) Ea Eamp(a))))
Σ
Σ
p(ys 11) 1) + p(y; E) (()f(())))) (()
f=1
Σ plys = 11 + (1 - --1p 100 [P())]
j=1
Σ
p(yj
(1-x((x) + Empp(a) [x]]]]
(29)
j=1
-Pyy, E) plx p(p [x))
Σ p(U; 1 E up m
fz + Empp(a)) [x))
-(y, = 1) E up
f (x) - -1 + 1]
Σ 2p(ys = E 1) [1 [zx)] + Empp(r) [f -plj 1).
A.5 Proof of Theorem 4.2
In this subsection, an estimation error bound is established for Eq. 7 to demonstrate its learning consistency. Specifically, The derivation of the estimation error bound involves two main parts, each corresponding to one of the loss terms in Eq. The empirical risk estimator according to Eq. can be written as:
Rop
Σ 2T,
Σ Σ (1-f(x))
+ (f(x) TT)) n RED
j=1 [St,) SES2,
(30)
= RR (f) + RV (f),
Firstly, we define the function spaces as:
G4p = (x, + 27,4 F- ((x11) + - ==1
1=1
17

---

and denote the expected Rademacher complexity [26] of the function spaces as:
R, (g -.
sup Σ ge i=1
ogg (ets 4+)
RR (9 EE
sup gES
Σ
oig (201 4i)
where = {o1. 02, Fn } is n Rademacher variables with independently uniform variable taking value [+1,- Then we have:
Lemma A.I. We suppose that the loss function CtP = Σj= 2Tyly 0 (x)) and S sup JEE least 1 - 8, we have:
Ei -T,) could be bounded by M, ie., M (f (f(x)) and for any O, with probability at
2 Nn (g%p)
M
log 2 2n
Jer
sup (5) R,, S
c
C
+
min, |5z
log 2
(1) - (0) s 29 (g
sup fer
+
2
2n
where R,p (f) = Ei=1 2n, pe [1- f2xx), ( (f) = Emppla)) Ej=1 [f((z)] and
(f)
C' = mini 18 Em= (/) is constant.
Proof. Suppose an example (ar, I) is replaced by another arbitrary example (ær I'), then the change of sur R2, R24/ (f) is no greater than 2n miny (52,1" By applying McDiarmid's inequality, for any 8 > with probility at least
M
M 2 miny |St,)
log
sup R2y - (f) < E feF
- sup R (f) R(.(()) SEF
+
2n
By symmetry, we can obtain
sup (f) - (f) $B FEF
sup R Rf, R2)(()
+ 2 min, Stt,
M
log 2
leF
2n
18

---

Next bound the term E
fer R R (f)
sup SEF
R ( (tp(
EE
sup (f) (f)
EEF
sup Eig (R(((f)) R [n] fEr
BB
sup &EDD
(T) R
/EF
2Th
12 (1 (((a))
2ng
1. f (z.i))
sup
gi
FET
i=1
j=1
Σ R
Li=1
1
E
2
11 (1-
< Ep'
sup
Ti
FEF
11
i=1
2Tz
+ Eń.
1/ (1 (1- f (a:))
/E 1=1 j=1
Σi=1 12
Ep
sup
(2m,l² - (xt)))
SEF
1=1 f=1 f=1
oi 2 (1- (xx)))
+
sup
C
fer 1ml
2
Rn
(gt))
ch
where C' a constant that C min, E, Ei= vil Then we have:
M
log 2
sup fer
R ( ( R2 (11 5 C
Ri (g
2 2min Sty
2n
Similarly, obtain:
log 3 2n
sup R (f) - (f) < 29,, (o + | fEF
Lemma A.2. Define = max, 2T, H, = x f(x)] EF} and R, (Hz)
Eo supneh, En= h (2:))
Then, we have with Rademacher vector contraction inequality:
Hin (gL)) < 22p
Rn (H,)),
Hr (G S /
Rn (H₂,)
y=1
Based on Lemma
A.I and Lemma A.2 we could obtain the following theorem.
Theorem
A.3. Assume the loss function
CL
Σ5 2 f()()
sp
and
CV Ej=1 (f) -T,) could be bounded by M. ie. M
19

---

Table 6: Characteristics of the MLIC datasets.
Dataset
#Training
#Validation
#Testing
#Classes
VOC
4574
1143
5823
20
COCO
65665
16416
40137
80
NUS
120000
30000
60260
81
CUB
4795
1199
5794
312
Table 7: Characteristics of the MLL datasets.
Dataset
#Examples
#Features
#Classes
#Domain
Image
2000
294
5
Images
Scene
2407
294
Images
Yeast
2417
103
14
Biology
Corel5k
5000
499
374
Images
Mirflickr
24581
1000
38
Images
Delicious
16091
500
983
Text
SUPaEX max ( (f))), (f(x), (f))) y)), with probability at least 1 5, we have:
-(fsp)-R(C
4 0
Σ
Hi (0%))
M
log 1 8
49%,, (g M
log
miny
|SL,
2n
2n
<
2v2p
Σ
Nn (Hz)
M
log
+ 4 Σ
Rx (Hl,) + M
log
C
miny |St,]
2n
2n
==1
/=1
Proof.
R((xp)
-R((') = R((sp) - RFp + Rop(f) (R)
<R(fsp) R.p (f) (f*) -ft)-R(f)))
RRp RI (f) + (5*) - -R
+ RYp -Ry, (f) + (f*) RT,
$ 2 sup (f) (f) + sup RR! (f) - -Ry (f1
(f)
fEF
SEF
$
4
Rn (g
M
log
+ 49h, (g + M
log
C
miny |L,
2n
2n
4 2p
M
log
log
Nn (H2)
+ 4v
Rn (HL,) + M
C
miny SLL,]
2n
2n
1=1
1=1
A.6 Implementation Details
During the implementation, we first initialize the predictive network by performing warm-up training with AN solution, which could facilitate learning fine network in the early stages. Furthermore. after each epoch, the class prior is reestimated via the trained model. The code implementation is based on PyTorch, and the experiments are conducted on GeForce RTX 3090 GPUs. The batch size selected from (8, 16] and the number of epochs is set to 10. The learning rate and weight decay are selected from 210-2 10-3 10-4 10 10-5) with validation set. The hyperparameters d and are all fixed as 0.01. All the comparing methods run trials on each datasets. For fairness, we employed ResNet-50 the backbone for all comparing methods.
20

---

Table 8: Predictive performance of each comparing method on MLL datasets in terms of Average Precision (mean std). The best performance is highlighted in bold (the larger the better).
Image
Soene
Yeast
CorelSk
Mirflickr
Delicious
AN
0.534=0.061
0,580=0.100
0.531=0,079
0.217=0.003
0.615=0.004
0,317=0.002
AN-LS
0.574=0.037
0.31-00002
0.538-00.04
0.230=0.002
0.587=0.006
0.261=0.006
WAN
0.576=0.041
0.661=00033
0.698=0.017
0.241=0.002
0.621=0.004
0,315=0.000
EPR
0.539-0.028
0.597-00062
0.710=0.008
0.214=00001
0.314=0.000 0.628-00003
ROLE
0.606:00041
0.700:0.040
0.711=00013
0.203
0.516:0.027
0.030-00.03
EM
0.48-=0.031
0.549=0.103
0.62:=0.029
0.294=0.002
0.614=0.003
0.293=0.001
EM-APL
0,46==0.026
0,44==0.049
0.654=0.040
0,275=0.003
0.589=0.007
0,311=0.001
SMILE
0.670=0.021
0.722=00071
0.751=0.004
0.295=00004
0.629-0.003
0.318=0.001
L1-R
0.605-0.058
0.714-0.035
0.658-0.006
0,268-0.002
0.625-0.001
0,296-0.004
LL-CP
0.595=0.031
0.735=0.028
0.700=0.000
0.259=0.004
0.621=0.007
0,251=0.007
LL-CT
0.600=0.012
0.669=0.052
0.629=0.007
0.258=0.004
0.619=0.004
0.253=0.004
CRISP
0.749=0.037
0.795=0.031
0.758=0.002
0.004=0.003
0.628=0.003
0.319-0.001
Table Predictive performance of each comparing method on MLL datasets in terms of Coverage (mean + std). The best performance is highlighted in bold (the smaller the better).
Image
Scene
Yeast
Corel5k
Mirfiickr
Delicious
AN
0.374-0.050
0.279-0.094
0.707-0.045
0.330-0.001
0.342-0.003
0.653-0.001
AN-LS
0.334_00033
0.217-0.052
0.703_00012
0.44110.009
0.433_000055
0.830_0.016
WAN
0.313-0.040
0.192-0.019
0.512-0.045
0.309-0.001
0.334-0.002
0.632-0.001
EPR
0.352=0.043
0.254=0.046
0.506=00011
0.328=0.001
0.332=0.002
0.637=0.001
ROLE
0.306=0.049 0.157=0.023
0.519=0.026
0.551=0.007
0.448=0.028
0.887=0.004
0.407-0.036 0.28110.096
0.575-0.042
0.382-00005
0.359-0.010
0.753_00004
EM-APL
0.438=0.022
0.360=0.057
0.356=0.045
0.335=0.005
0.369=00005
0.765=0.006
SMILE
0.242.0.014
0.146.:0.037
0.462-00003
0.008
0.028
0.628-00003
LL-R
0.311=00059
0.141=0.017
0.512=00002
0.274=0.002
0.335=0.006
0.622=0.001
LI.CP
0.296100.03
0.136.00016
0.508
0,272 0.008
0.337
0.708-0.004 0.004
L1-CT
0.297-00017
0.161=0.031
0.509=0.001
0.277=0.005
0.335-00003
0.708-0.002
CRISP
0.164:9.001 0.012
0.082 0.018
0.455 0.002
0,276-0.002
0.324 0.001
0.620 0.001
Table 10: Predictive performance of each comparing methods on MLL datasets in terms of Hamming loss (mean + std). The best performance is highlighted in bold (the smaller the better).
Image
Scene
Yeast
CorelSk
Mirfliek
Delicious
AN
0.229=000000
0.176=00.00
0.306:1:0000
0.010=00000
0.127=10.000
0.019=00000
AN-LS
0.209:00.000
0.108=00.044
0.306::0.000
0.010=00000
0.127:10.000
0.019=0.000
WAN
0.41110.060
0.29910.035
0.2851000116
0.156=0.001
0.19110.006
0.102=00000
EPR
0.370=-0.043
0.220=00006
0.234-10.007
0.016=00000
0.136:10.002
0.020=0.000
ROLE
0.256::0.018
0.17620.017
0.279:10.010
0.010=0.000
0.128:10.000
0.019=0.000
EM
0.770+-0.001 0.820=0.003
0.669+-0.025
0.589=00003
0.718++0.010
0.630-0.005
EM-APL
0.707=00088
0.780=0.082
0.6411:0.032
0.648:0.006
0.754:10.017
0.622-00006
SMILE
0.219+0.009
0.182+0.021
0.208 0.000
0.010
0.127++0.001
01.008
LL-R
0.220:10.013
0.162=0.005
0.3121:0001
0.015=00001
0.124:10.000
0.019=0.000
Lt-CP
0.218+00016
0. 0.16440.002
0.306+00.000
0.016=00.011
0.126++0.001
0.019-0.000
Lt-Cr
0.246:00001
0.17620.009
0.3211:0001
0.018=00001
0.124:10.001
0.000
CRISP
0.065
0.140=1.013
0.2114:0000
0.010=0.000
0.121:10.002
0.019=0.000
A.7 Details of Datasets
The details of the four MLIC datasets and the five MLL datasets are provided in Table Table 7 respectively. The basic statics about the MLIC datasets include the number of training set, validation set, and testing set (#Training, #Validation, #Testing), and the number of classes (#Classes). The basic staties about the MLL datasets include the number of examples (#Examples), the dimension of features (#Features), the number of classes (#Classes), and the domain of the dataset (#Domain).
A.8 More Results of MLL Datasets
Table
10 and report the results of our method and other comparing methods on five MIL. datasets terms of Average Precision, Coverage, Hamming loss and One Error respectively.
21

---

Table 11: Predictive performance of each comparing methods MLL datasets in terms of One-error (mean + std). The best performance is highlighted in bold (the smaller the better).
Image
Scene
Yeast
Corel5k
Mirllickr
Delicious
AN
0.708
0.626 123
0.489:0. 194
0.758=0.002
0.358 005
0.410=0.012
AN-15
0.643+00.02
0.578+0.111
0.495+00.30
0.736+0.009
0.360++0.015
0.454-0.013
WAN
0.670=00.000
0.543=0.060
0.239++0.022
0.727=0.012
0.352=+0.010
0,404=0.002
EPR
0.703+00.06
0.615+0.090
0.240+-0.003
0.764+0.000
0.362++0.015
0.440-0.008
ROLE
0.605=00041
0.507=0.066
0.2441:0.005
0.705=0.016
0.525:10.022
0.594=0.006
EM
0.709
0.681=0.119
0.326+-0.079
0.656=00009
0.365-+0.008
0.446=0.009
EM-APL
0.773::0.045
0.812=0.059
0.341+:01099
0.690=0.007
0.434:10.023
0.405=0.006
SMILE
0.533=0.036
0.466=0.117
0.250:0.012
0.650=0.008
0.340:1.0100
0.402=0.005
LL-R
0.597=00.04
0.490=0.054
0.436:000877
0.715=0.006
0.342:10.016
0.543=0.041
Lt.CP
0.629:00.03
0.450=0.051
0.240+-0000
0.731
0.357:+0.016
0.490=0.028
LL-CT
0.6161:00199
0.574=0.074
0.55211.007
0.726=0.022
0.375=:0.012
0.475=0.019
CRISP
0.005
0.311
0.227--0.004
0.646 + 0.006
0.295 0.009
0.402 0.003
Table 12: Summary of the Wilcoxon signed-rant ranks test for CRISP against other comparing approaches 0.05 significance level. The p-values shown in the brackets.
Caner gya
Role
Ar
S
1.-
11-CT
Conoage we
way (G1-)
viNc
wtept (011))
w H
wHe 0431)
wank-1013 (515)
wirch
ward (5.15)
42:3
(670)
wsegd (013)
wiag0.09.3
wixe0 0515)
wa.00.0133
(c000
(860/1813)
+08[0/3133]
Table 13: Predictive performance of CRISP compared with the approach of estimating priors from the validation set (CRIsP-VAL) on the MLL datasets for five metrics.
Metries
Image
Scene
Yeast
Corel5k
Mirflickr
Delicious
Coverage
0.164:0.012
0.082:000088
0.455:10.002
0.206=:0.002
0.324=0.001
0.620=:0.001
Ranking Loss
0.064
0.112:+0.021
0.164:1:0001
0.113=00.01
0.118=0.00
0.122:000000
CRISP
Average Precision
0.749=00037
0.795
0.758:10.002
0.304:10.003
0.628=0.003
0.319=10.001
Homming Loss
0.065=00023
0.140::0.013
0.211110.001
0.010::0.000
0.121=00002
0.019:10.000
OneError
0.026
0.311:000447
0.227:10.004
0,646210.006
0.295
0.402:10.003
Coverage
0.193=0.009
0. 0.101::0011
0.456:+0.004
0.280
0.330=0.001
0.623:10.002
Ranking Loss
0.198=00016
0.116::0.03
0.165110.001
0.114:1.002
0.120=0.001
0.122:10.000
CRISP-VAL
Average Precision
0.725=00004
0.7901:0.028
0.753+10.006
0.294=00008
0.622=00001
0.319=10.001
Homming Loss
0.180=000006
0.141::0004
0.216110.000
0.010=0.,000
0.124=0.001
0,0191000000
OneError
0.395+0.077
0.059+0.050
0,246
0.666
0.314-0.003
0.444-+0.001
(a) VOC
(b) COCO
(c) NUS
(d) CUB
Figure 4: Convergence of मे on four MLIC datasets.
A.9
More Results of MLIC Datasets
Figure illustrates the discrepancy between the estimated class-prior म, and the true class-prior TT, in every epoch on four MLIC datasets. During the initial few epochs, significant decrease in the discrepancy between the estimated class-prior and the true class-prior is observed. After several epochs, the estimated class prior tends to stabilize and converges to the true class-prior. This result provides evidence that our proposed method effectively estimates the class-prior with the only observed single positive label.
A.10 p-values of the wilcoxon signed-ranke test
Table 12 reports the p-values of the wilcoxon signed-ranke test (6) for the corresponding tests and the statistical test results at 0.05 significance level.
22

---

A.11 Ablation results of MLL datasets
Table reports the predictive performance of CRISP compared with the approach of estimating priors from the validation set (CRIsP-VAL) on the MLL datasets for five metrics. The results show that CRISP outperforms CRISP-VAL VAL on almost all the five metrics.
23

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: See the abstract and the penultimate paragraph in introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discussed the potential time complexity issues of the proposed algorithm in the Experiment Section 5.3.4
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]

---

Justification: In Theorem 4.1 and Theorem 4.2 we have comprehensively presented the corresponding assumptions and proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof' provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
See the Section A.6 in Appendix.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
25

---

Answer: [No]
Justification: The code is currently proprietary and not publicly available. However, we have provided detailed information necessary for replicating the experiments.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurIPS code and data submission guidelines (https
http:///guddeeeee (ooeeS/ ssionlolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
See the Experiment Section and Section A.6 Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: For each set of experiments, we conducted five trials. In Table we also performed a significance analysis.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
26

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: See the Experiment Section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
See the Section Impact Statements in Appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
27

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [Yes]
Justification: This paper does not directly release the model, and the training data used does not contain any sensitive content.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: See the Experimtent Section.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the datasets..papersstt wwwppswipitttt.. thoode. com/ datasets package should be provided. For popular datasets, has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
28

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [No]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [No]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
30


Reviews:
[{"review_id": "AxKarhwrcm", "reviewer": "Reviewer_GgxX", "summary": "The focal issue addressed in this paper is single-positive multi-label learning (SPMLL), a variant of the multi-label learning paradigm under weak supervision where each example is labeled with only one positive label, leaving other labels unannotated. This scenario poses a challenge because conventional SPMLL methodologies make the unrealistic assumption that all class priors are identical, which doesn't hold true in practical applications where class-prior distributions can vary significantly. To address this limitation, the paper introduces a novel framework named CRISP (Class-pRiors Induced Single-Positive multi-label learning). CRISP encompasses a class-priors estimator designed to theoretically converge to the actual class-priors of the data. Furthermore, leveraging these estimated class-priors, CRISP formulates an unbiased risk estimator for classification tasks. This estimator ensures that the corresponding risk minimizer is approximately equivalent to the optimal risk minimizer achievable with fully supervised data.", "questions": "1.Will the inclusion of the additional absolute operator of Equation 7 compromise the unbias of the proposed risk estimator?\n2.Why weren’t BOOSTLU and PLC compared on the MLL datasets?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n1.The paper theoretically guarantees that the estimated class-priors will converge to the true class-priors. Furthermore, based on these class-priors, the paper introduces a consistent risk estimator that ensures the corresponding risk minimizer will approximately converge to the optimal risk minimizer on fully supervised data. The solid theoretical foundation lends credibility to the proposed approach.\n2.The paper presents an extensive experimental validation, demonstrating the necessity of addressing class-prior issues and the effectiveness of the proposed method through a comprehensive set of experiments, including ablation studies designed to validate the effectiveness of the proposed class-priors estimator and an analysis of the time complexity.\nWeaknesses\n1.The effectiveness of the method appears to rely in part on the performance of the warm-up classifier; if the initial classifier is too weak, it may undermine the reliability of thresholding and lead to degraded practical performance.\nSome recent baselines are missing, such as “Exploring Structured Semantic Prior for Multi-Label Recognition With Incomplete Labels.", "was_revised": true, "final_justification": "Overall a solid work. My previous concerns on the affect of warm-up classifier and missing baselines have been partially addressed.", "rebuttal": "Thank you for taking the time to review the paper and providing valuable feedback. I appreciate your efforts in ensuring the quality of the research. Regarding your concerns, I would like to provide the following explanations:\n\nFor weaknesses:\n\n> 1. The effectiveness of the method appears to rely in part on the performance of the warm-up classifier; if the initial classifier is too weak, it may undermine the reliability of thresholding and lead to degraded practical performance.\n\nTo evaluate how much our method depends on the quality of the warm-up classifier, we conducted an additional ablation study on VOC and COCO. Specifically, we trained the warm-up model using the AN loss for 1, 2, and 3 epochs to simulate classifiers with varying initial quality. The results are shown below:\n\n| **Epochs** | **AN Loss (VOC)** | **CRISP Final (VOC)** | **AN Loss (COCO)** | **CRISP Final (COCO)** |\n| :--------: | :---------------: | :-------------------: | :----------------: | :--------------------: |\n|     1      |       43.54       |         88.50         |       45.21        |         73.85          |\n|     2      |       76.25       |         89.10         |       56.33        |         74.30          |\n|     3      |       83.21       |         89.93         |       62.11        |         74.91          |\n\nAs shown, our method remains stable even when the warm-up classifier is relatively weak. This suggests that while the initial classifier provides a starting point for thresholding, the overall method is robust to its performance within a reasonable range.\n\n> 2. Some recent baselines are missing, such as “Exploring Structured Semantic Prior for Multi-Label Recognition With Incomplete Labels.\n\nThank you for pointing this out. In the revised version, we have added new baselines, including MIME[1] and GR Loss[2], and the updated comparison results are shown in the table below. Our method outperforms these recent approaches on most datasets, with only a slight gap compared to the best method on CUB. Regarding *“Exploring Structured Semantic Prior…”* and other CLIP-based pseudo-labeling approaches, they introduce extra supervision information beyond single-positive labels, making direct comparison potentially unfair. However, we will explicitly discuss these methods in the Related Work section to provide a more comprehensive overview.\n\n| **Method** |       VOC        |       COCO       |       NUS        |       CUB        |\n| :--------: | :--------------: | :--------------: | :--------------: | :--------------: |\n|    MIME    |   89.20 ± 0.16   |   72.92 ± 0.26   |   48.74 ± 0.43   | **21.89 ± 0.35** |\n|  GR Loss   |   89.42 ± 0.23   |   73.15 ± 0.22   |   49.09 ± 0.02   |   21.36 ± 0.12   |\n|   CRISP    | **89.82 ± 0.19** | **74.64 ± 0.22** | **50.00 ± 0.32** |   21.65 ± 0.18   |\n\n[1] Chen, Yanxi, et al. \"Boosting single positive multi-label classification with generalized robust loss.\" Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence. 2024.\n\n[2] Liu, Biao, et al. \"Revisiting pseudo-label for single-positive multi-label learning.\" International Conference on Machine Learning. PMLR, 2023.\n\nFor Questions:\n\n> 1. Will the inclusion of the additional absolute operator of Equation 7 compromise the unbias of the proposed risk estimator? \n\nThe absolute value is introduced to ensure the non-negativity of the risk and to better align the model-predicted class prior with the ground-truth class prior. This modification does not compromise the unbiasedness of the estimator, as formally shown in Theorem 4.2. Despite the introduction of the absolute operator, we provide a estimation error bound demonstrating that the empirical risk minimizer still converges to the optimal risk minimizer as $n \\to \\infty$. The bound has already accounted for this modification.\n\n> 2. Why weren’t BOOSTLU and PLC compared on the MLL datasets?\n\nAs we explained in lines 224-227 of our paper, due to the inability to compute the loss function of PLC without data augmentation, we do not report the results of PLC on MLL datasets because data augmentation techniques are not suitable for the MLL datasets. Similarly, since the operations of BOOSTLU for CAM are not applicable to the tabular data in MLL datasets, its results are also not reported.\n\nWe hope that our revisions have addressed all of your concerns, but please let us know if there is anything else we can do to improve the manuscript. We would be happy to answer any additional questions or provide any further information you may need."}, {"review_id": "7uEMkWpsam", "reviewer": "Reviewer_5pXo", "summary": "This paper presents a statistical strategy for building stronger supervision signals in weakly-annotated settings by estimating class probabilities with the model which is in the process of being trained, and samples from the data.", "questions": "Here I include both direct questions, and comments to which I hope you will respond as appropriate.\n1. l. 116: \"ratio between the fraction\nof the total number of samples and that of positive labeled samples receiving scores above the threshold, thereby obtaining the class-prior probability of the j-th label\": this was pretty unclear.\n2. l. 123: empirical estimator $\\hat{q}$: since this is estimating an expectation by an un-weighted sum, you need to describe the sampling of $x_i$ from $\\mathcal X$ for this estimator. \n3. eqs 6, 7: the class priors you are using for supervision can not be known in ground-truth. Are you in fact using the estimations as per Algorithm 1?\n4.  you never demonstrate that $\\pi_i = p(y_i=1)$\n5. eq 4: you should mention that this is binary cross-entropy. In general, your thresholded classifier approach only makes sense in contexts where the classifier's outputs can be interpreted as a confidence score for each class. A classifier trained on a hinge loss, for example, would not fit this framework.\n6. the \"class-prior\" vocabulary implies a Bayesian framework, but the notion of updating priors based on data is contradictory to the Bayesian viewpoint, where the prior is fixed a priori. Bayesian data-based updates are typically made on the posterior.\n7. why do you split data differently for MLL and MLIC datasets?\n8. Why are separate statistics being reported for MLL and MLIC (tables 6 and 7). I'd like to know table 6 and 7's reported statistics on both the MLIC and MLL datasets.\n9. Naively, we could estimate $p(y_i=1)$ by looking at the data only and counting the number of times the label $y_i$ is positive and dividing by the total number of observations. Why is the proposed method, which is more of a pseudo-signal since it is dependent on the classifier, better? Empirical evidence here would be most convincing. \n10. l. 240: \"Without the constraint of the class-priors, the predicted class-prior probability diverges from the true class-prior as epochs increase, significantly impacting the model’s performance.\" The claim that model performance is impacted by this phenomenon is not sufficiently supported. Predicted and true probabilities do not necessarily have to be close for the correct discrete decision to be made. The issue of accurate probability prediction is one of model calibration, which has its own specialized performance metrics for this explicit reason; standard performance metrics are often agnostic to calibration.\n11. Figure 1: how do you compute the predicted class-priors for these methods. Presumably, they do not all include explicit class-prior prediction strategies in their published methodologies.", "limitations": "The authors make brief discussion of time complexity. The discussion of time-complexity as a limitation is possible in nearly any modern machine-learning work, and in my opinion does not constitute an earnest attempt to address a work's limitations.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n1. The paper is laid out in a mostly clear way, with an obvious motivation/story\n2. Sensitivity studies are presented and statistical significances are computed\n3. while the notion of supervising multi-label classification training with dataset-level class-frequency statistics is not in itself novel, the presented approach to this end is novel (although as yet not shown to be especially impactful relative to even naive options).\n\nWeaknesses\n1. A crucial step is missing from the method, connecting the ratio $\\pi$ to the class probability. \n2. Some experimental details are missing, as referenced in the comments/questions below", "was_revised": true, "final_justification": "My primary concerns during this discussion process were the use of the word \"prior\", the similar performance between the proposed method and naive sampling, and some experimental procedure related to the estimation of other methods' marginal class probability estimates.\n\nThe authors agreed to shift the text away from \"class-priors\". \n\nThe authors pointed out that the naive-sampling ablation was done over the fully labelled validation set and is not a practical solution to the problem, since the available data during model training is mostly un-labelled. They pointed out that estimating class marginal probabilities by taking the frequency of labels on the train set will be biased, demonstrating theoretically. I asked them to show empirical results to demonstrate this.\n\nWhile the authors did not improve their estimation of other methods' marginal class probability estimates, their clarified procedure is, while not generous, adequate considering it is not a primary result.", "rebuttal": "Thank you for taking the time to review the paper and providing valuable feedback. I appreciate your efforts in ensuring the quality of the research. Regarding your concerns, I would like to provide the following explanations:\n\n> 1. l. 116: \"ratio between the fraction of the total number of samples and that of positive labeled samples receiving scores above the threshold, thereby obtaining the class-prior probability of the j-th label\": this was pretty unclear.\n\nWhat we intended to express is: For label j, we calculate the class-prior $\\pi_j$ as the ratio between the total number of samples and the number of samples with scores above threshold $\\hat z$. For example, if we have 1000 samples in total and 300 samples have scores above $\\hat z$, then the estimated class-prior $\\pi_j = 300/1000 = 0.3$​. We will clarify this in the revision. \n\n> 2. l. 123: empirical estimator : since this is estimating an expectation by an un-weighted sum, you need to describe the sampling of from for this estimator.\n\nThe samples $\\boldsymbol{x}_i$ in our empirical estimator are independently and identically distributed (i.i.d.) samples drawn from the marginal distribution $p(\\boldsymbol{x})$. We will add this clarification to make the estimator more rigorous.\n\n> 3. eqs 6, 7: the class priors you are using for supervision can not be known in ground-truth. Are you in fact using the estimations as per Algorithm 1?\n\nWe confirm that the class priors used in Eqs. (6) and (7) are indeed the estimates obtained via Algorithm 1.\n\n> 4. you never demonstrate that $\\pi_i = p(y_i=1)$\n\nWhile we have described $\\pi_i$ as the class prior probability in the text, we acknowledge that we should have explicitly stated the mathematical relationship $\\pi_i = p(y_i=1)$. We will add this explicit equation in the revised version.\n\n> 5. eq 4: you should mention that this is binary cross-entropy. In general, your thresholded classifier approach only makes sense in contexts where the classifier's outputs can be interpreted as a confidence score for each class. A classifier trained on a hinge loss, for example, would not fit this framework.\n\nWe would like to clarify that our method uses L1 loss (absolute loss) as stated in Eq. (5). We agree that the loss should allow the classifier output to be interpreted as a confidence score, which can be achieved by applying a sigmoid function to the logits.\n\n> 6. the \"class-prior\" vocabulary implies a Bayesian framework, but the notion of updating priors based on data is contradictory to the Bayesian viewpoint, where the prior is fixed a priori. Bayesian data-based updates are typically made on the posterior.\n\nWe clarify that in our setting the true class-prior probabilities are indeed fixed, consistent with the Bayesian framework. What we update during training is the estimate of the class-priors, not the true priors themselves. As the classifier improves, the estimated class-priors become more accurate, and this alternated optimization leads to better performance.\n\n> 7. why do you split data differently for MLL and MLIC datasets?\n\nFor the MLIC datasets, the train/test splits are predefined, so we simply hold out a portion of the training set for validation. In contrast, the MLL datasets do not have predefined splits, so we manually split them into train/validation/test sets. This difference in splitting strategy does not affect the fairness of comparison.\n\n> 8. Why are separate statistics being reported for MLL and MLIC (tables 6 and 7). I'd like to know table 6 and 7's reported statistics on both the MLIC and MLL datasets.\n\nAs mentioned above, the MLIC datasets are pre-defined **image** datasets, so we report their statistics in terms of #Training / #Validation / #Testing / #Classes. In contrast, the MLL datasets are **tabular** datasets without predefined splits, so we report their statistics as #Examples / #Features / #Classes.\n\n> 9. Naively, we could estimate by looking at the data only and counting the number of times the label is positive and dividing by the total number of observations. Why is the proposed method, which is more of a pseudo-signal since it is dependent on the classifier, better? Empirical evidence here would be most convincing.\n\nIn SPMLL, each instance is associated with only one observed positive label, while the remaining labels are unobserved. As a result, a large portion of positive labels is missing, making it impossible to accurately estimate class priors by simply counting the observed positive labels and dividing by the total number of samples. To address this challenge, our method iteratively refines class-prior estimates by leveraging both the classifier outputs and the limited available labels.. Moreover, we provide empirical evidence in the ablation study (Table 3), where we used the frequency of labeled positive samples in the validation set as the class-prior (“CRISP-VAL”). The results show that our proposed method achieves better performance than this naive approach, because our estimator leverages both the classifier’s outputs and the available labels to iteratively refine the class-prior estimates.\n\n> 10. l. 240: \"Without the constraint of the class-priors, the predicted class-prior probability diverges from the true class-prior as epochs increase, significantly impacting the model’s performance.\" The claim that model performance is impacted by this phenomenon is not sufficiently supported. Predicted and true probabilities do not necessarily have to be close for the correct discrete decision to be made. The issue of accurate probability prediction is one of model calibration, which has its own specialized performance metrics for this explicit reason; standard performance metrics are often agnostic to calibration.\n\nWe will revise this statement in the next version. Our intention was to suggest that lack of class-prior alignment may be one factor contributing to poorer performance, rather than claiming a direct causal relationship. Nonetheless, our experiments clearly show that our method achieves better alignment between predicted and true class-priors.\n\n> 11. Figure 1: how do you compute the predicted class-priors for these methods. Presumably, they do not all include explicit class-prior prediction strategies in their published methodologies.\n\nFor all methods, we compute the predicted class-priors as the average predicted confidence score for each class across all samples. Even though most baselines do not explicitly estimate class-priors, their model outputs can still be interpreted as confidence scores, so taking the mean over samples provides a consistent way to compare predicted class-priors across methods.\n\nWe hope that our revisions and additional clarifications have fully addressed your concerns. Please feel free to let us know if further explanations or additional analyses are needed."}, {"review_id": "8U5sGaiMHk", "reviewer": "Reviewer_8tLt", "summary": "This paper proposed method named CRISP for single positive multi-label learning. A class-priors estimator is introduced with   a convergence guarantee.  The author claims that the experimental results demonstrate the effectiveness and superiority of this method over existing approaches.", "questions": "- Please add more results comparison with the recent papers\n\n- Why choose ranking loss? It would be better to add the commonly used metrics including mAP, F1, Recall, etc.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength\n\n- The single positive Multi-label learning is a pratical setting in the classification task since the labeling is usually expensive, it is worthy to spend effort on this task.\n\n- It is great that the author provides theoretical guarantees on the convergence of the estimated class priors (to ground-truth class priors), which makes the approach convincing.\n\n\nWeakness\n\n-  the methods used for comparison are from several years ago, it would be better to make comparison with the latest papers on this topic.\n\nFor example,\n\n[1]. Chen, Yanxi, et al. \"Boosting single positive multi-label classification with generalized robust loss.\" Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence. 2024.\n\n[2] Xing, Xin, et al. \"Vision-Language Pseudo-Labels for Single-Positive Multi-Label Learning.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n\n[3]. Liu, Biao, et al. \"Revisiting pseudo-label for single-positive multi-label learning.\" International Conference on Machine Learning. PMLR, 2023.\n\n[4]. Ding, Zixuan, et al. \"Exploring structured semantic prior for multi label recognition with incomplete labels.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n\n[5]. Kim, Youngwook, et al. \"Large loss matters in weakly supervised multi-label classification.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.\n\n- One thing confused me is the metrics utilized for demonstrating your experiment results and make comparisons. Why choose ranking loss? It would be better to add the commonly used metrics including mAP, F1, Recall, etc.", "was_revised": true, "final_justification": "I have read the author's response and it has resolved all my mentioned concerns. So I will keep the score as borderline accepted.", "rebuttal": "Thank you for taking the time to review the paper and providing valuable feedback. I appreciate your efforts in ensuring the quality of the research. Regarding your concerns, I would like to provide the following explanations:\n\nFor Weaknesses:\n\n> 1. the methods used for comparison are from several years ago, it would be better to make comparison with the latest papers on this topic.\n\nThank you for the valuable suggestions. We note that [5] (LL) has already been included as one of our baselines. In the revised version, we additionally include [1] and [3] as new baselines, and the comparison results are shown below. Our method achieves superior performance on most datasets, with only a slight gap compared to MIME on the CUB dataset. Regarding [2] and [4], both methods rely on CLIP-based pseudo-labeling, which introduces extra supervision information beyond single-positive labels. Direct comparison with our method may therefore be unfair. However, we will discuss these approaches in the Related Work section to provide a more comprehensive overview.\n\n| **Method** |       VOC        |       COCO       |       NUS        |       CUB        |\n| :--------: | :--------------: | :--------------: | :--------------: | :--------------: |\n|    MIME    |   89.20 ± 0.16   |   72.92 ± 0.26   |   48.74 ± 0.43   | **21.89 ± 0.35** |\n|  GR Loss   |   89.42 ± 0.23   |   73.15 ± 0.22   |   49.09 ± 0.02   |   21.36 ± 0.12   |\n|   CRISP    | **89.82 ± 0.19** | **74.64 ± 0.22** | **50.00 ± 0.32** |   21.65 ± 0.18   |\n\n> 2. One thing confused me is the metrics utilized for demonstrating your experiment results and make comparisons. Why choose ranking loss? It would be better to add the commonly used metrics including mAP, F1, Recall, etc.\n\nThank you for the suggestion. In fact, we have already evaluated our method using multiple commonly used metrics, including mAP, Ranking Loss, Hamming Loss, One-Error, and Coverage, as shown in Tables 1, 2, 8, 9, 10, and 11. These metrics jointly assess different aspects of model performance, such as ranking quality, per-label accuracy, and top prediction correctness. We will clarify this in the revised version to avoid confusion.\n\nWe hope that our revisions and additional clarifications have fully addressed your concerns. Please feel free to let us know if further explanations or additional analyses are needed."}, {"review_id": "AnC82veZti", "reviewer": "Reviewer_wnkV", "summary": "The authors propose a novel framework that addresses a key limitation in Single-Positive Multi-Label Learning: the unrealistic assumption that all classes have identical prior probabilities. The method estimates class priors using a threshold-based approach with theoretical convergence guarantees, then derives an unbiased risk estimator for classification. Extensive experiments demonstrate superior performance over existing SPMLL approaches.", "questions": "I have the following two questions. First, how sensitive is the class-prior estimation to the quality of the initial model predictions? Poor initial models might lead to unreliable prior estimates, creating a chicken-and-egg problem. Second, have the authors considered other class-prior estimation techniques beyond the threshold-based approach? Comparison with alternative strategies would strengthen the contribution.", "limitations": "the authors adequately addressed the limitations.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths：\nThe paper provides convergence guarantees for both class-prior estimation (Theorem 4.1) and risk minimization (Theorem 4.2). The theoretical analysis appears sound, though the assumptions are somewhat restrictive. And the consistent improvements across multiple datasets and metrics provide reasonable evidence of the method's effectiveness.\n\nWeaknesses：\nThe warm-up strategy using AN solution seems necessary but lacks theoretical grounding. The related work section reads more like a literature survey than establishing clear connections to the proposed approach.", "was_revised": true, "final_justification": "I have reviewed the authors' rebuttal and discussions with other reviewers. The main concerns have been adequately addressed, and the authors provided convincing responses to the technical questions raised. I maintain my borderline accept recommendation.", "rebuttal": "Thank you for taking the time to review the paper and providing valuable feedback. I appreciate your efforts in ensuring the quality of the research. Regarding your concerns, I would like to provide the following explanations:\n\n\n> 1. The warm-up strategy using AN solution seems necessary but lacks theoretical grounding. How sensitive is the class-prior estimation to the quality of the initial model predictions? Poor initial models might lead to unreliable prior estimates, creating a chicken-and-egg problem.\n\nThe warm-up strategy using AN loss is actually a well-established practice in the SPMLL, adopted by several classical algorithms [1-4]. To address your concern about sensitivity to initial model quality, we conducted an ablation study on VOC and COCO datasets, training the warm-up model for different numbers of epochs (1, 2, and 3) to simulate varying initial classifier quality:\n\n| **Epochs** | **AN Loss (VOC)** | **CRISP Final (VOC)** | **AN Loss (COCO)** | **CRISP Final (COCO)** |\n| :--------: | :---------------: | :-------------------: | :----------------: | :--------------------: |\n|     1      |       43.54       |         88.50         |       45.21        |         73.85          |\n|     2      |       76.25       |         89.10         |       56.33        |         74.30          |\n|     3      |       83.21       |         89.93         |       62.11        |         74.91          |\n\nThe results demonstrate that even with a weak initial classifier (1 epoch, mAP of only 43.54% on VOC), CRISP achieves strong final performance (88.50%). The final performance remains stable across different warm-up qualities, with only minor improvements as the warm-up classifier gets stronger. This suggests that the class-prior estimation mechanism is sufficiently robust to work with imperfect initial predictions and progressively refine them during training.\n\n[1] Label-aware global consistency formulti-label learning with single positive labels\n\n[2] One positive label is sufficient: Single-positive multi-label learning with label enhancement.\n\n[3] Large loss matters in weakly supervised multi-label classification\n\n[4] revisiting pseudo label for single positive multi label learning\n\n> 2. The related work section reads more like a literature survey than establishing clear connections to the proposed approach.\n\nThank you for this valuable feedback. We will revise the related work section in the next version to more clearly establish the connections between our proposed approach and existing methods.\n\n> 3. Have the authors considered other class-prior estimation techniques beyond the threshold-based approach? Comparison with alternative strategies would strengthen the contribution.\n\nWe have actually compared our threshold-based class-prior estimation with an alternative strategy in our ablation study (Table 3). Specifically, we used the frequency of each labeled positive sample appearing in the validation set as the class prior. Our experimental results show that our approach outperforms the alternative strategy, demonstrating the effectiveness of our proposed method.\n\nWe hope that our revisions and additional clarifications have fully addressed your concerns. Please feel free to let us know if further explanations or additional analyses are needed."}]

Author rebuttal:

Meta-review: This paper addresses the problem of single-positive multi-label classification, where every data point is only labeled with a single positive label. It proposes a novel approach to the problem, provides theoretical justification and demonstrates strong results. The rebuttal and discussion (especially the back-and-forth with reviewer 5pXo) played a big role in leading all reviewers to recommend acceptance.
Decision: Accept (poster)