Single-positive Multi-label Learning with Label Cardinality
Authors: Shayan Gharib
Pierre-Alexandre Murena
Arto Klami
Venue: Accepted by TMLR ()

Abstract: We study learning a multi-label classifier from partially labeled data, where each instance has only a single positive label. We explain how auxiliary information available on the label cardinality, the number of positive labels per instance, can be used for improving such methods. We consider auxiliary information of varying granularity, ranging from knowing just the maximum number of labels over all instances to knowledge on the distribution of label cardinalities and even the exact cardinality of each instance. We introduce methods leveraging the different types of auxiliary information, study how close to the fully labeled accuracy we can get under different scenarios, and show that an easy-to-implement method only assuming the knowledge of the maximum cardinality is comparable to the state-of-the-art single-positive multi-label learning methods when using the same base model. Our implementation is publicly available at https://github.com/shayangharib/SPMLL_with_Label_Cardinality.

Published in Transactions on Machine Learning Research (09/2025)
Single-positive Multi-label Learning with Label Cardinality
Shayan Gharib Deportment of Computer Science University of Helsinki
shayon.ghharibShhcsiiik.k
Pierre- Alexre-Ae Murena
piere-aleeanddeemueeaauuuuuu
Human- Centric Machine Learning Research Group Hambury Universiby of Technology
Arto Klami
arto art.kllam@hhelsink..f
Department of Computer Science Universily of Helsinki
Reviewed OpenReview: https:// o/tprrve//perrvieeu. net/ formm? id= XEPPXH2KK
Abstract
We study learning multi-label classifier from partially labeled data, where each instance has only single positive label. We explain how auxiliary information available the label cardinality, the number of positive labels per instance, can be used for improving such hods We consider auxiliary information of varying granularity, ranging from knowing just the maximum number of labels over all instances to knowledge on the distribution of label cardinalitics and even the exact cardinality of each instance. Wc introduce methods leveraging the different types of auxiliary information, study how close to the fully labeled accuracy can get under different scenarios, and show that easy-to-implement method only assuming the knowledge of the maximum cardinality is comparable to the state-of- the-art single-positive multi-label learning methods when using the same base model. Our implementation publicly available at https:/ //tpt/uggbbtt/t//// com/ /twh/haaghaaab//P...... with
Label_ Cardinalidd
1 Introduction
Numeerous methods for learning multi-label classifiess from fully annotated data have been proposed Ridnik et al.) 2023 Zhang Zhou 2013 Parascandolo al. 2016). but for specialized domains full annotation is often too costly (Nguyen et al. 2017 Xie & Huang 2018). Instead, aim to train model with minimal label information ideally, just single positive label which is far less costly than annotating all possible aspects. Algorithms for learning the full multi-label function from such labeling are commonly called single- positive multi-label learning (SPMLL) (Cole et al. 2021), which is a special case of the more general positive and unlabeled (PU) setting (Elkan & Noto 2008
The SPMLL literature has largely focused on the question of how the missing label information is treated, by introducing new losses (Chen et al., 2024 Kim et al. 2022). regularizers (Cole et al. 2021) or algorithmic leveraging new type of complementary information: How additional information about label cardinality can be used for improving SPMLL methods. Here label cardinality refers to the number of positive labels of an instance. Direct access to the cardinality of specific instance is naturally highly useful for instance, knowing an instance has just one label implies that all unobserved labels are negative but is often difficult to obtain in practice. We explain and evaluate alternative ways of using label cardinality to improve SPMLL methods and quantify the possible gains that can be achieved in different setups. Our main focus is in cases where we assume known cardinality distribution, probability distribution over the label cardinalities, and this information to estimate
variants (Xie et al.] 2022). We, instead, focus

---

Published in Transactions on Machine Learning Research (09/2025)
Observed Single-positive Label
+ Cardinality Information
CS
Carcinality Statistios
kiaxx 5
Frequency
CD
Cardinality Distribution
k RJ 3
1C
Instance Cardinality
1g =3
10
Cardinality
Figure 1: In SPMLL, only one positive label (in black) for each instance is observed, yet the task is to predict all labels. We explain how to use different forms of additional information on label cardinality, the number of positive labels, to help this. The information can be on data-level cardinality statistics (CS), instance-level cardinality (IC), or cardinality distribution (CD) over all samples.
instance-level cardinalities. This is done with a newly proposed algorithm that finds the optimal cardinalities based on the current model prodictions, without requiring label information for learning. However, we also study the case where the cardinalities are known exactly, and a setup where only crude summary statistics of the cardinality distribution are observed. Figure illustrates the setup.
Several studics have explored the use of label cardinalities in other contexts. For instance, label cardinality has been used proxy for scene complexity (Hajimirsadeghii al.] 2015), cardinality constraints can be used to enforce models to output a predefined number of labels (Cortes et al. 2024 Amos et al. 2019) and the related concept of cardinality potentials has been considered in count regression problems to encourage predictions to be on the right scale (Swersky et al., 2012 Tarlow et al., 2012). However, within SPMLL literature this aspect has been ignored, even though additional information is most useful when having the least amount of direct label information. The only previous reference to label cardinality is by Cole et al.) (2021), who used the expected number of positive labels on the level of the whole dataset as a basis for a regularizer. This can be as the crudest special case of our principle.
On a technical level, formalize the task by characterizing the different forms of cardinality side informa- tion that could be available, and provide concrete solutions for leveraging them. We introduce three key components and specific method combining them, but the components could be incorporated into various other SPMLL mothods as well.
A new loss that ensures the outputs for the unobserved classes remain informative during training, in contrast the previous losses pushing them towards arbitrary target values (Cole al. 2021 Zhou et al. 2022
A new joint loss for the collection of all unobserved labels of an instance that uses label cardinality either an estimate or known value to encourage predictions consistent with the desired cardinality.
3. A bipartite matching algorithm designed for estimating the label cardinalities of all instances from current model outputs when an estimate of the cardinality distribution is available.
We empirically evaluate the value of the cardinality side information using standard SPMLL benchmark tasks. Already the first component that only requires crude estimates for mean and maximum cardinality achieves mean average precision (MAP) comparable to several recent SPMLL methods (Cole et al.) 2021 Zhou et. al. 2022 Kim et al., 2023 Chen et al. 2024). With richer cardinality information, we can obviously improve the results, and our focus is on quantifying the possible gain and studying the relationship between the accuracy of the cardinality estimates and the final classification performance. For example, for the commonly used NUS-WIDS data (Chua et al.) (2009). knowing the cardinality instance sufficient for almost. matching the accuracy of model trained fully labeled data, and get close even when only
2

---

Published in Transactions on Machine Learning Research (09/2025)
assuming the cardinality distribution. Finally, we show that improving the accuracy of cardinality estimates directly translates to improved final model accuracy.
2
Related Work
To position the work within the broader SPMLL literature, we briefly outline the main approaches and some recent works. The most closely related works are on avoiding false negatives arising from the unobserved positive labels, resolved in our work by leveraging the cardinality information. As will be detailed later, the crudest approach, that assumes all unobserved labels to be negative, pushos the predictions to zero and results in severe problems with false negatives (Cole et al.] 2021). A possible fix of maximizing entropy of the predictions (Zhou al.] 2022) overcompensates the issue, pushing predictions towards 0.5 that is considerably too high for most multi-label learning (MIL)) tasks, resulting in false positives. Closest to work is method that regularizes the predictions using the expected label cardinality (Cole et al., 2021 while also using label smoothing; their regularizer can be seen a special case of our general setup where different kinds of additional information on label cardinality are considered.
Also other approaches for avoiding false negatives have been studied, e.g. by preventing memorization of false negatives (Arpit al. 2017 Kim al. 2022) or by improving retrieval of positive labels by pseudo- labeling as commonly done in other weakly supervised settings (Chen et al., 2023 Wang et al. 2022). In
SPMLL, Xie et. al. (2022) considered pseudo-labeling combined with consistency regularization (Zhou al.,
Laine & Aila 2017 and Liu et al. (2023) employed mutual information bottleneck for preserving label-specifer characteristics during pscudo-labeling.
2003
Other works have studied the related question of label imbalance; the SPMLL setup makes the common imbalance in MLL tasks more extreme. Several methods have been proposed for addressing class imbalance (Ridnik et al. 2021 Lin al. 2020)) and missing labels (Zhang et al.) 2021) for standard MLL,, with extensions for SPMLL. For instance, Chen et al. (2024) developed a noise-robust. loss to account for the imbalance in SPMLL We do not explicitly consider label scarcity, but cardinality information helps balancing the ratio of positive labels closer to the balance of the original MLL task. Finally, some SPMLL methods leverage domain-specific elements, such as class activation maps for images (Kim et al. 2023 Verelst et al., 2023) In contrast, our study focuses on the general task and does not consider any domain knowledge.
3 Problem Setup
Multi-label lcarning In MLL, each g-dimensional data point E X = RR is associated with a binary label vector E y = 13 where C' is the number of classes. A fully labeled dataset D = { y.) < i < N} is available for training. Here, of denotes the label the i-th sample for the j-th class. y = indicates that class j is relevant (positive) for sample i, while of = 0 indicates irrelevance (negative). The objective is to learn a classifier h X' + [0, 1J that minimizes a classification error, so that the outputs fi = h(x;)) are interpreted as probability each class. Our work is agnostic to the functional form h, and in the empirical experimentation we will always use the same model, ResNet-50 (He al. 2016).
Single-positive multi-label learning We follow the common SPMLL formulation, matching e.g. (Xu et al. 2022). Instead of the full label vector y: for data point æ,, we observe single-positive label vector 1, (0, 1 Here, e 1 indicates that the j-th class is the only observed positive label for 2; while { = for all other classes, signifying that their true label values are unknown (not necessarily negative), ie.
E5 1 1. We denole the set of unobserved classes for sample i with U. This results in partially labeled dataset D = 4) 1ilis Nj. Similar to MLL, the task here is to learn a classifier h X + [0, 1J for the original MI.I. task, predicting the full labels ye-
Our methods generalize for other partial labeling setups. We write the equations using a more general notation, where ke == -の 1 labels are assumed observed, even though we run all experiments with kg 1.
3

---

Published in Transactions on Machine Learning Research (09/2025)
C-
50
Gm A 30
O 512
& m L.
2.
240
Seer
1300
See 18
3.00
Frequeccy
12:30
10:0
2000
50
000
2000
9 90000 3536 35 Cardialit
T Cartinlitty
9 9 21
Cantinality
Cardinalty
(a) VOC
(b) COCO
(c) NUS
(d) CUB
Figure 2: Cardinality distributions and statistics for MLL image benchmark datasets. Low-cardinality instances dominate when the labels are objects, but for the attribute-labels of CUB the distribution is clearly different.
Label cardinality This paper analyzes various ways of using label cardinality within the SPMLL frame- work. We denote the true label cardinality instance as k; = - vi, and define three distinct categories of label cardinality information that can be available during training:
1. Instance cardinality (IC): The most informative scenario, with known label cardinality k; for each instance i. We use it mostly as a theoretical construct to estimate the maximum information provided by cardinality information, but we also show how it can be estimated from the cruder statistics while learning the classifier.
2. Cardinality distribution (CD): A more relaxed assumption is that we know (or can estimate) the discrete distribution P(k) of the label cardinality. Figure 2 shows the CD for four common bench- mark datasets, demonstrating a common pattern of decaying probability for higher cardinalities.
3. Cardinality statistics (CS): If obtaining the full CD is not feasible,
also use simple summary statistics of that, most commonly the maximum and mean label cardinality, kmax max, k and k = N >; ki Robust estimators, such as high quantiles instead of the maximum, can be used for additional robust
These categories represent hicrarchy of information content: CD implics CS, and both CD and CS can be computed from TC. In Section 5.2 study how sensitive the proposed met are for noisy estimates of cardinality information.
4
Method
We first briefly outline the standard SPMLL approach of factorizing the lcarning objective into separate losses for the observed and unobserved labels, and the issues of previously proposed losses for the unobserved labels. We then explain our technical solution for leveraging the cardinality information. We introduce two alternative losses for the unobserved labels U,, as the direct replacements for the previous proposals: (i) provides label-specific loss for cases where we only know the cardinality statistics, and (ii)
Section 4.2
Section 4.3
roduces a joint loss the whole U; that uses instance cardinalities as the learning target.
In Section
we explain how instance cardinalities can be estimated based on the cardinality distribution and model predictions, and finally Section 4.5 explains concrete SPMLL method using these components.
4.4
4.1 Background
For MLL with full label vector ye the standard binary cross-entropy (BCE) loss is the most common choice. In the SPMLL setup, the typical losses CCen et al. 2024
Cole et al. 2021 take the same factorized form
LSPMLL
1 NC
Σ
(w (tt) + 1 C" (f!))
(1)
4

---

Published in Transactions on Machine Learning Research (09/2025)
3.0
LAN Ecs EaM
Axu
2.5
12
10
2.0
Density
Density
1.5
1.0
0.5
2
0.0
0 0.2
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.4
0.6
0.8
1.0
probability
probability
Figure 3: The distribution of model outputs J for the unobserved labels on the VOC (left) and NUS (right) datasets. The LAN loss causes false negatives by pulling predictions towards zero and LEM causes false positives by pulling them towards 0.5. Our Les loss encourages predictions consistent with the cardinality statistics. For visual clarity, the y-axis is truncated at roughly 20% of the maximum density of LAN-
but use separate loss components, (++) for observed positive labels and (²( for the unknown ones. Here 1 is the indicator function selecting the correct loss for each predicted probability f.
Since is unosserve, naive choice is to replace it with the observed label vector. Substituting 4 for in the BCE loss gives the Assumed Negative (AN) loss (Cole et al. 2021
CAN - Σ (1 log(S() + 1u-=o log(1 f?))
(2)
However, this loss tends to assign near-zero probabilities all unknown labels (1) 0), shown empirically in Figure 3 This both misleads the model severely and limits the information content of the predictions. To improve this, Zhou et al. (2022 used entropy _ (denoting the total loss as LFM), but since the labels are binary variables, it effectively just changes the target for the probabilities, making them converge towards 0.5 instcad of O; see Figure 3 This implicitly suggests that we have prior probability around 0.5 for
each individual class be positive, which is considerably too high for typical MLL datasets with k2 << for all instances.
4.2 Unobserved Loss Based on Cardinality Statistics
Under this setting, we assume access to the maximum and mean label cardinality, respectively kmax and kg In practico, estimates of these quantitics are sufficient, as demonstrated in Section B.1 To prevent the misinformation induced by the poor largets for Jú, we propose using (²( satisfying three criteria:
Both very low and high fl ineur penalty the model should a priori be certain about the predictions for the unobserved labels.
2. Between these extremes, within a range specified soon, the loss should not favor any specific output there no information about which f to prefer.
3. The range aligns with the statistical information about the frequency of the positive labels, a way of encoding prior information about f(
Any loss satisfying these prevents the predictions from collapsing towards a pre-defined value and drives them be on the right scale. This as such useful we will show in Scction 5.2 that this loss achieves highly competitive performance but informative f will also be needed for our cardinality estimator described in Section 4.4
5

---

Published in Transactions on Machine Learning Research (09/2025)
Concretely, use quadratic penalty for fi outside the given lower and upper thresholds, Ti and Tg. These thresholds are both determined based on the cardinality statistics provide a useful prior for f/. The upper threshold T, is sct by distributing the maximum possible number of labels kmax (a known cardinality statistic) evenly across all unobserved classes, so that it is (kmax 1)/(C - 1), where one is subtracted because of the already observed label. Outputs below the threshold do not provide evidence on the presence of particular label, since they are consistent with a model that predicts each class equally likely for a sample of the highest cardinality. Any f above the threshold means committing to that particular label and is penalized for.
We could, in principle, set the lower threshold Tj similarly based on the minimum cardinality. However, this would make the threshold zero for most MLL datasets even a single instance with only one label is enough to ensure this. It is essential to prevent zero probabilities that indicate strong commitment to non-presence of particular class, and hence Ti needs to be strictly non-negative. Any small constant is likely to work, and as a practical heuristic, we use ke, the expected number of classes. As shown in Figure 2 it is often (but not always) small.
Building on this intuition, we formally define a new loss
8 kimax
ki C
2
kimax
ki
A
C
(Cs(1?) =
li
kg
2
lic
(3)
A
C
C
Otherwise
where kn and kinxx are the average and maximum number of positive labels per instance, ky is the number of observed positive labels for instance i (which is V,ke = 1 in standard SPMLL), C = C - kg is the number of unobserved classes for that instance, and X is a scaling hyperparameter. Outside the non-penalized range defined by the thresholds, we apply a quadratic penalty commonly used in loss function design, but note that other monotonically increasing penalty functions could be used as well. The general loss defined in Equation using L?s denoted by LCs-
4.3 Unobserved Loss Based on Instance Cardinality
The IC hypothesis assumess that we know k; the number of positive labels for each instance i. This implies kg hy - kg labels within the set of unobserved classes U,, again with ke = 1 for the SPMLL setup. Next, we explain how this information can be usod for constructing a joint loss over all of the unobscrved labels for given instance. Instead of factorizing the loss over all of the labels, as in the case of all previously proposed L" we encourage the joint predictions for the unobserved labels to be consistent with the instance cardinality. The overall loss is
C LIc NC ΣΣ I (
Σ Cie(f))
(4)
where a is a scaling hyperparameter and Cy. is the new loss defined jointly for the set U, of the unobserved labels.
The Lig is designed achieve two goals. First, it encourages the total probability mass assigned for probable classes (the classes for which the current f are high, to be defined formally soon) to align with the desired cardinality ky" Second, drives the probabilities for the remaining classes towards zero. Intuitively, this both ensures the model retains sufficiently high probability for enough classes while allowing it to commit to some classes being negative. Formally,
((E
Ci)(f))
+
Σ f! JEN.
(5)
JEn;
6

---

Published in Transactions on Machine Learning Research (09/2025)
Here 3 is scaling hyperparameteer Ω C U, is the set of indices corresponding to the M, = |9,]| classes with the highest predicted probabilities for instance i, and ky is the corresponding target cardinality the number of unobserved classes that should be positive. The first term encourages the sum of the top M, prodicted probabilities to stay close to k", whereas the second term pushes the remaining labels towards low probability. Note that, if kg" = O, we have Q2 = and only the second lerm remains. That is, we explicitly consider all unobserved labels as negative, achieving the intuitive property mentioned in Section |
Remark: Selecting M4 The general formulation leaves open the question of how many of the largest entries are included in the summation. We should always have M; 2 K to ensure we collect the probability mass over at least k; classes. In practice, we want M, > k2 to prevent false negatives when the current model predictions are inaccurate or k; is not necessarily exact. This is because all S & Ω are pushed towards zero. We use two alternative choices in our experiments, to avoid enforcing such false negatives: M2 = 2k, when we know k reliably, and M₁- kmax - kg when we need to estimate the cardinalitics.
Remark: Scaling probabilities Equation 5 aligns the sum of the model outputs with k;, but any monotonic transformation of / could be used instead. Since we lypically have J < 1 even for the correctly predicted classes, it may be beneficial to use a transformation that saturates to one faster. We use a scaled sigmoid that is steeper than the one used for actual model outputs. If we denote by zi the logits that are transformed for f., we use in place of ff transformed outputs f. = 1+exp (-a(z;) where a > 1.
4.4 Estimating Instance Cardinalities
When the label cardinality k; for each instance is not known, we cannot directly use the loss in Equation 5 However, we can replace the known k with any estimate & instead. In principle, any estimator could be used, for instance a model that takes as input either the input vectors a or the model predictions f, and is trained superviscd fashion to provide the estimate. However, this would require knowledge of the true cardinalities k; for some training instances and dedicated training process. Nexl, we introduce a novel algorithm for estimating the instance cardinalities, requiring only knowing the cardinality distribution, probability vector of P(k). The method is computationally light and does not require any training or other supervision; in particular, we do not need to know the true k; for any instance.
We formulate the problem as perfect bipartite matching between two sets of N elements (Cormen et al, (022). The first set corresponds to the instances, represented by a vector E R. of scores. For cach instance, we define s; = Eieo, fi with M, = kmax: corresponding to the quantity that is pushed towards the cardinality in Equation itself. Note that the score is defined based on the current model predictions fi, meaning assumee they carry information about the cardinality. This will be the case for models trained using the proposed Les and Lie losses, but not e.g. for LAN.
The idea is to now use the score as the basis for estimating the cardinality
The other set is constructed based on the cardinality distribution P(k), so that it has the right frequency for each cardinality in increasing order. We denote by e E [o...., NN a vector where the first P(k- = 0)N entries are set to O, the next P(k- = 1)NV entries to 1, etc. It has the right cardinality distribution by construction.
We find the perfect bipartite match II, permutation over the N entries of the latter set, that mininizes E,ss,-1,y)) - 12 That is, we assign for each instance a single entry in v and hence a cardinality estimate = =I, that the total distance between 8; and k is minimized. Even though bipartite matching has cubic complexity for the general case (Kuhn 1955). our problem is simplified special case of Euclidean bipartite matching (Agarwal & Varadarajan 2004 Arora 1997) with ono-dimensional points cach clement is represented by a non-negative scalar (&s or v.). For this problem, we can find the global optimum in ONN log N) time with an algorithm that sequentially matches the candidates (see Appendix A for a proof); Iteratively assign the smallest cardinality for the sample with the smallest s; and remove the corresponding elements from the sets. Since e is ordered by construction, this can be done by sorting 8; and collecting the sorting order in IT.
7

---

Published in Transactions on Machine Learning Research (09/2025)
4.5 Whole Algorithm
Having introduced the technical elements, we now describe practical SPMLL algorithm. We note, however, that each of the elements could be used also in other ways and this specific algorithm is introduced as an example for the purpose of the comparisons. Both of the unobserved label losses (Equation 3 and 5 can be used as plug-in replacements in other SPMLL methods, and the cardinality estimation a algorithm could be replaced with other estimators.
For the CD setup, where the distribution of cardinalitics is assumed known, our general algorithm works as follows and is denoted by CS = CD in the results:
1. First optimize Les for some epochs, for initialization of the model so that f. start being informative, both about the class predictions and about the cardinalitics.
2. Solve h with the sorting algorithm described in Section 4.4 using the known CD and 8, computed based on the current f. for all instances, as the input.
Optimize Lie for more epochs using iy- k kn as the target in Equation After cach epoch,
re-estimate & using the current predictions as in step 2.
For the CS setup, the algorithm simplifics to only the first step above and is denoted as CS in the results. For the IC setup, we use the known ki, skipping the estimation of hi, and denote it by CS + IC.
5
Experiments & Results
5.1
Experimental Setup
Data We use four common SPMLL benchmark datasets for evaluation: Pascal VOC 2012 (VOC) Ever-
ingham et al. 2015 MS COCO 2014 (COCO) (Lin el al. 2014), NUS-Wide (NUS) Chua et al. 2009 and
CUB-200-2011 (CUB) (Wah et al. 2011]. The datasets are used as in the previous literature, e.g. using the 312 binary attributes of CUB as target labels as in Cole et al. (2021); ;; Zhou et al. 2022), with full details
disclosed in Appendix C.2 The cardinality distributions and statistics are shown in Figure 2
The test sctup follows closely previous SPMLL works, e.g. Cole et al. (2021); Zhou et al. (2022). We randomly split each original training data into our training and validation sets, using 20% for validation. The original
validation data is used as the test set. The SPMLL training data is formed by choosing the observed label uniformly at random from the set of true positive labels for each instance, assumed also by the comparison methods. That is, we do not consider the more general sctups with c.g. class-specific weighting studied in the broader positive and unlabeled literature (Elkan & Noto. 2008). The validation and test sets are assumed fully labeled, as in previous works, to enable a fair comparison against the baselines and reduce random variation; see Section t for discussion.
Comparison methods and implementation details We compare against several recent SPMLL meth-
ods: ROLE (Cole et al. 2021), EM+APL (Zhou et al.] 2022). LL-R + BoostLU (Kim et al. 2023). and GR Loss CCen et al. 2024). In addition, we report the results for the weak baseline of AN loss (Cole et al. 2021 and the upper bound of training the same model on fully laboled data.
We use the same classifier for all methods, with ResNet-50 (He et al. 2016) backbone pretrained Ima- geNet (Russakovsky et al. 2015) and technical details matching the implementation of Cole et al. [202 ; the
details of image preprocessing, data augmentation, and the architecture are provided in Appendix C.1 The
models are trained end-to-end using the specific loss of each method with Adam (Kingma & Ba 2015 and the performance is evaluated using MAP averaged over the C classes. For all the methods, the hyperparam- eters are determined with a grid scarch using validation MAP as the metric, as detailed in Appendix C.4
For simplicity, we use fixed batch sizes from Zhou et al. (2022) and learning rate from Chen al. 2024 for all the methods, focusing on validating the method-sppeific parameters.
8

---

Published in Transactions on Machine Learning Research (09/2025)
112
45
15.
125
23
124
23
70
23
20
1.
10
35
20
25
301
355
031 05
50
25
50
10.0 125 1501
323
33
9 15 21
(a) VOC
(b) COCO
(c) NUS
(d) CUB
Figure 4: Cross-plot of estimated vs true label cardinality, with the area of the circles indicating the count: the colors for visual separation and have meaning. Figure T0 in Appendix shows an alternative representation of this data using the conditional distribution of the predicted cardinalities.
5.2 Experiments
We conduct. three main experiments. The first contrasts the proposed methods against the baselines, the second quantifies the value of the cardinality information, and the third validates the methods are robust with respect to estimation error when CS or CD needs to be estimated from data.
Table 1 reports the test set MAP for three variants of our method and all comparison methods, The results are averaged over three runs. The key comparison is between CS and the previous SPMLL methods. Except for the COCO data, where LL-R + BoostLU (Kim et al. 2023) stands out, is comparable or better than all other methods. The method is easy to implement, only requiring replacing the loss for the unobserved labels with one that encourages f to remain on a reasonable scale (see Figure 3) and only very crude cardinality information needed set the two thresholds in Equation we will show the method is insensitive to specific choice in Section B.1
Experiment 1
The other two variants of our method should not be directly compared with the rest, since they require more information. Instead, the attention should be placed on the difference between the three proposed methods. For COCO and NUS, we observe the expected effect: Having access to CD shows as clear improvement, and knowing IC helps more as should. For NUS, accuracy of the fully labeled case. For VOC data all SPMLL methods reach essentially the accuracy of the full label upper bound, and hence there is no room for improvement from the use of cardinality information. For CUB, in turn, observe that even knowing the IC does not help, which already implies that the CD cannot help either.
worth noting that CS = effectively reaches even the
Figure 4 illustrates the cardinality estimators. The area of each circle represents the frequency of a (k, k) pair, and a perfect estimator would result in a diagonal line where the sizes follow the cardinality distribution. The estimates for COCO and NUS are good, consistent with the improvement MAP Table [ For CUB, we also confirm the numerical result; as knowing is does not even help learning better model, natural that cannot estimate them either based on the model outputs.
Experiment 2 Exact label cardinalities naturally more informative than the oncs estimated based the cardinality distribution. To better undersland the difference in information content, we study two related questions: (Q1) How improvements in cardinality estimates translate to improvements in MAP, and (Q2) How accurate the estimator needs to be to still be useful.
We study both questions using simulation experiment. We run the CS = CD method, but instead of using the scores 8; = Ejen, H as the inputs for the estimator described in Section 4.4 we now use 8; = k, + 6, where e ~ // o²). That is, we do not use the actual model outputs for estimating the cardinalities but assume access to a noisy oracle that allows generating estimates of varying quality in a controllable fashion: if x² then we recover the IC scenario of perfect estimates, whereas with 02 - oo we converge a random permutation of the cardinalities. Note that the cardinality distribution constraint is still satisfied even in this case, by the virtue of the estimation algorithm.
9

---

Published in Transactions on Machine Learning Research (09/2025)
Table 1: Test MAP for all methods. The CS method can be directly compared against the previous methods, but our CS + CD and Cs + IC methods assume more information.
Method
VOC
COCO
NCS
CUB
Full label
89.56 0.35
76.43 0.09
52.12 0.09
32.73 : 0.20
AN
85.51 + 0.41
04.33 0.10
42.68
0.09
18.65 = 0.31
ROLE
88.10 :: 0.16
0.89
41.74 0.21
14.52 + 0.47
EM+
0.29
70.92
47.56 = 0.20
21.13 1
LL-R+BoostLU
88.81 : 0.24
73.11
49.76 0.51
19.75 0.23
GR Loss
89.38 + 0.12
71.33
47.08 0.43
20.69 = 0.28
CS (ours)
88.97 + 0.03
71.35
49.23
0.45
21.56 :: 0.08
CS + CD (ours)
89.10 + 0.17
72.07
50.26 0.08
21.58 +
= IC (ours)
89.69 0.16
73.45 0.12
50.92 0.48
21.73 0.35
52
76
755
Noley IC
MAP
lobel
CS
IC
40
IC
72
48
Ful label
T1
47
10
01.
01.5
0.G
0.7
0.5
11.
1.0
0.3
11.4
11.5
DUG
0.7
0.8
0.9
1.0
p(k, R)
p(k, k)
(a) COCO
(b) NUS
Figure 5:
The MAP score improves approximately linearly when the cardinality estimates improved (black), and the real estimator (blue point) aligns well with the simulation model. MAP for full label (green) and CS (orange) setups are provided as scale reference.
We repeat this for a range of o², from very poor estimates to perfect cardinalities. We quantify the quality by computing the Pearson correlation between & and over the N samples, and evaluate the MAP as well. Figure 5 shows that the relationship of these two essentially linear for both COCO and NUS, the two data sets with most potential for leveraging cardinality information. That is, we confirm that improving the imator results in improved MAP, and additionally learn that the transition is gradual all improvements in cardinality estimates are directly reflected in the MAP score, answering Q1. The answer to Q2 is given by the point where this trend intersects the accuracy obtained with CS: For COCO, any estimator with p(k, k) > 0.5 helps, and we need p(k, k) > 0.65 for NUS.
We can also compute p(k, k) for the real method whore s; = -Eee,, fl is used for estimating hy corresponding to the results visualized in Figure ! For both COCO and NUS, the result, indicated by the pair of the correlation and MAP, close to the trend line for the simulation. This confirms the simulation process mimicks the real estimation error and is hence informative for real use cases.
Experiment 3 For the experiments above, we used exact CS and CD to isolate the effect of possible estimation error, but in real use-cases, they need to be estimated. We next show that the method is extremely robust with respect to the estimation errors. The CS method only requires knax for setting Tu. and already an extremely crude guess is enough. For any kmax between 0.5 and 2.5 times the real kmax, we observed virtually no change in MAP. For instance, for COCO, the MAP scores remain between 70.76 and 71.35; see Figure| in Appendix for full details.
CD can be estimated in multiple ways. Figure shows how MAP develops when estimating it from sample of S instances for which k; is annotated (note that still do not require fully annotated samples), using the crudest possible estimator: Set kmax to a maximum of the largest l in the sample and a user-defined upper bound (we used 1. 1.2k as example) and estimate P(k) as the histogram of the S samples with additive
10

---

Published in Transactions on Machine Learning Research (09/2025)
71.5
G3
11.7
Novyy
Nakay
71.6
05
J00
600) Sample
400
Sample
(a) COCO
(b) NUS
Figure 6: MAP progressively improves when using more accurate estimates of cardinality density, here shown as function sample size S with known instance cardinalitics.
Table 2:
Test MAP for all methods using validation data that contains only single-positive label per instance,
referred to "Single" in the table. For case of comparison, the results on fully annotated validation data
("Full") from Table 1 are also included.
Full
Single
Method
COCO
NUS
COCO
NCS
AN
64.33 + 0.10
42.68
0.09
64.13 0.21
42.33 0.54
ROLE
66.89 0.16
4.27
66.95 : 0.09
41.16 0.69
EM+APL
70.92 0.23
47.56
70.76 0.18
47.56 = 0.20
LL- U
73.11 0.21
0.51
70.71 0.11
48.31 = 0.91
GR Loss
71.33 + 0.21
47.08
0.43
71.06 0.13
47.38 0.02
CS (ours
71.35 0.10
49.23 0.45
71.19 + 0.19
49.02 0.19
Laplace smoothing. For NUS, already S 25 is enough for clearly improving over CS and for both data sets increasing S improves MAP as it should. See Appendix [..2 for further details and analysis, showing how MAP relates to estimation error and thus providing information on how alternative estimators (e.g. elicitation from a domain expert) would work.
6 Discussion
Evaluation Some remarks are useful for correct interpretation of the results. All methods in Table D were run by us ensure consistency, explaining differences the scores reported in previous literature. The MAP scores mostly in line with the ones in the original publications, but for the GR Loss Chen et al. (2024), could not replicate the good performance, despite using their code. As explained in Section 5.1 compared only against baselines that use the same unmodified ResNet-50 backbone, to ensure the differences are due to the loss and algorithm changes. Some works report higher MAP for these benchmark datasets, but the numerical accuracies cannot be directly compared since the architecture has significant effect on the overall performance. For instance, Xie al. (0022) reports strong results but also notably higher MAP even for the AN and ROLE baselines, confirming part of the improvement is because of the different base model.
As in all comparison methods, we used fully labeled validation set for determining the hyperparameters to keep results comparable with the previous literature. This slightly exaggerates all accuracies due to the use of data that would not be available in real use cases, but the bias is similar for all methods and in general small, Note that CS, the method we compare against the previous ones, has fewer hyperparameters than most and cannot gain unfair advantage due to tuning. To validate this empirically, Table Tablee[ reproduces Table the two interesting datasots using a validation set with only singlo-positive label per instance. The results are highly similar, except for AL-R+BoostLL that has several hyperparameters dropping notably in MAP. Our CS method ourforrosms all others both datasets.
11

---

Published in Transactions on Machine Learning Research (09/2025)
Value of cardinality information The empirical results can be summarized as follows: (a) CS alone is strong SPMLI method and in practice does not require any additional information; it uses the statistics for setting the thresholds but works even with 50% estimation error for the maximum cardinality. (b) Cardinality distribution, cither known or estimated from a very small sample, improves MAP compared to CS. (c) Improved estimation of the instance cardinalities improves overall performance, with approximately linear dependeney between the estimation quality (measured by correlation) and MAP. (d) The improvements in MAP are typically not large in absolute terms, but it is important to contrast the gain to what could be achieved: For VOC and NUS the results are already effectively at the level of the fully-labeled model.
Finally, wC note that the CS + IC result gives an upper bound for MAP when optimizing the loss in Equation However, it is not necessarily the best way to use this information; e.g. various pseudo-labeling Liu et al. 2023 could be boosted with cardinality information.
methods
Generality We showed consistent good performance four datasets, but with clear differences in how much the cardinality information helps. For NUS and COCO, we can estimate cardinalities well and MAP improves clearly, but for CUB even known cardinalities do not help. We do not have a clear interpretation for this - it may relate to the shape of the cardinality distribution, or to the data itself. For instance, leveraging the cardinality information will be difficult if ky is almost conditionally independent of y for j E U; given the observed labels. This is more likely when k >> kę, which holds for CUB but not for the other data sets. For VOC, there is no room for improvement: already some methods not using any cardinality information match the MAP of the model trained on full labels.
In this work, we focused on the SPMLL setup, but wrote the equations for more general partial labeling scenarios. Appendix D demonstrates that the method works unmodified also in more general scenarios, by showing results for an experiment where two posítive labels are assumed known for each sample, when applicable. The accuracy of all methods naturally improves as it should, but the improvement from using cardinality information remains similar as in the SPMLL setup.
Our technical elements could be integrated into other SPMLL methods. First, Los could be used as strong initialization for any method, not requiring any modifications for the methods themselves. Moreover, since the objective functions in many prior works are factorized as in Equation [ i.e. the loss term for unlabeled classes L" is an additive component to the entire loss function, Lis can be directly used as a plug-in replacement for their unsupervised components, again without modifying the rest of the method. Extensions where the cardinality information is combined with alternative means of improving SPMLL are particularly interesting. For example, EM+ APT, (Zhou et al., 2022) uses psoudo-labeling to improve the learning, and we could replace their entropy-based regularization with Lis in an attempt to guide the pseudo-labeling with cardinality estimates. Recently Hu et al. (2025) proposed using active learning to select partially labeled samples for full annotation, and cardinality information could potentially be used for guiding this choice as well. Evaluation of these idcas is a promising avenue for future work.
7 Conclusion
We introduced novel problem formulation where varying degrees of side information on the label cardinality is used in the MLL tasks with only partial labeling, focusing on the special case of SPMLL. We characterized three concrete forms of side information, explained how they can be used for improving SPMLL performance, and analyzed the problem. Already a minor modification for the loss assumed for unobserved labels, requiring only crudely estimated statistics of the cardinality distribution, gives a well-performing SPMLL method. The accuracy can be progressively improved by leveraging either a distribution of the label cardinalities or instance cardinality estimates.
Acknowledgments
This work was supported by the Rescarch Council of Finland project 353441 and the Flagship programme: Finnish Center for Artificial Intelligence, FCAI The authors acknowledge CSC -I Center for Science, Finland, for computational resources.
12

---

Published in Transactions on Machine Learning Research (09/2025)
References
Pankaj Agarwal and Kasturi Varadarajan, A near-linear constant-factor approximation for Euclidean bi- partite matching? In Proceedings of the Twentieth Annual Symposium on Computational Geometry, SCG '04, pp. 247 252, New York, NY, USA, 2004. Association for Computing Machinery. ISBN 1581138857 doi: 10.1145/99918 9977799955.
Brandon Amos, Vladlen Koltun, and J. Zico Kolter. The limited multi-label projection layer. CoRR, Abs.1190008777, 2019.
S. Arora. Nearly linear time approximation schemes for Euclidean TSP and other geometric problems. In Proceedings 38th Annual Symposium on Foundations of Computer Science, pp. 554 563, 1997. doi:
10.1109/SPCS.19990 1997///69...
Devansh Arpit, Stanislaw Jastrzębski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder Kan- wal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, and Simon Jacoste-Julien. A closer look at memorization in deep networks. In Doina Procup and Yoe Whye Tch (ods.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research,
pp. 233-20- 242. PMLR, 06 11 Aug 2017.
Hao Chen, Ran Tao, Yue Fan, Yidong Wang, Jindong Wang, Bernt Schiele, Xing Xie, Bhiksha Raj, and Marios Savvides. Softmatch: Addressing the quantity-quality tradcoff in semi-supervised learning. In The Eleventh International Conference on Learning Representations, 2023.
Yauxi Chen, Chunxiao Li, Xinyang Dai, Jinhuan Li, Weiyu Sun, Yiming Wang, Renyuan Zhang, Tinghe Zhang, and Bo Wang. Boosting single positive multi-label classification with generalized robust loss. Kate Larson (ed.), Proceedings of the Thirty-Thoin Third International Joint Conference on Artificial Intelli- gence, IJCAT-24, pp. 3825 3833. International Joint Conferences on Artificial Intelligence Organization, 8 2024. doi: 10.24963 3//06///jjai.0000 (tjj/aai.224/4111 Main Track.
Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng. NUS-WIDE: a real-world web image database from National University of Singapore. In Proceedings of the ACM International Conference on Image and Video Retrieval, CTVR '09, New York, NY, USA, 2009. Association for Computing Machinery. ISBN 9781605584805. doi: 10.1145/ (9.1145/1646396166622 1646396.16466552
Elijah Cole, Oisin Mac Aodha, Titouan Lorieul, Pietro Perona, Dan Morris, and Nebojsa Jojic. Multi-label lcarning from single positive labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pallern Recognition (CVPR), pp. 933 933-942, June 2021.
T.H. Cormen, C.E. Leiserson, R.L. Rivest, and C. Stein. Introduction to Algorithms, fourth edition. MIT Press, 2022. ISBN 9780262046305.
Corinna Cortes, Anqi Mao, Christopher Mohri, Mehryar Mohri, and Yutao Zhong. Cardinality- aware set prediction and top-k classification. In Advonces in Neural Information Processing Systems, 2024.
Charles Elkan and Keith Noto. Learning classifiers from only positive and unlabeled data. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 'OS, pp. 213 220, New York, NY, USA, 2008. Association for Computing Machinery. ISBN 9781605581 1934. doi: 011445/1401190.14000
Mark Everingham, SM Ali Eslami, Lue Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The PASCAL visual object classes challenge: A retrospective. International journal of computer vision, 111:98 136, 2015.
Hossein Hajimirsadeghi, Wang Yan, Arash Vahdat, and Greg Mori. Visual recognition by counting instances: A multi--nstanue instance cardinality potential kernel. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015.
G. H. Hardy, J. F. Littlewood, and G. Polya. Inequalities. Cambridge University Press, 1952. 2d ed.
13

---

Published in Transactions on Machine Learning Research (09/2025)
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition.
In 2016 TEEE Conference on Compuler Vision and Pallern Recognition (CVPR), pp. 770-778, 2016. doi: B0.1109/CVPR.20166
Tingxin Hu, Weihang Zhang, Jia Guo, and Huiqi Li. Co-pseudo labeling and active selection for fundus single-positive multi-label learning. IEEE Transactions on Medical Imaging, pp. 1-1, 2025.
Youngwook Kim, Jac Myung Kim, Zeynep Akata, and Jungwoo Lee. Large loss matters in weakly supervised multi-label classification. In Proceedings of the IEFE/CVE Conference on Compuler Vision and Paltern Recognition (CVPR), pp. 14156 14166, June 2022.
Youngwook Kim, Jae Myung Kim, Jieun Jeong, Cordelia Schmid, Zeynep Akata, and Jungwoo Lee. Bridging the gap between model explanations in partially annotated multi-label classification. In 2023 IEEE/CVE Conference on Compuler Vision and Pallern Recognilion (CVPR), pp. 3408 3417, 2023. doi: 10.1109/ CVPR52729.2022
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Represenlations (ICLR), San Diega, CA, USA, 2015.
H. W. Kuhn. The Hungarian method for the assigmment problem. Naval Research Logistics Quarterly, 2 (1-2):83-91 97, 1955. doi: https://doi.orr/1000
Samuli Laine and Timo Aila. Temporal ensembling for semi-supervised learning. In International Conference on Learning Representations, 2017.
M Lin. Network in network. arXiv preprint Xr:: 2013.
Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: Common objects in context. In David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars (eds.), Computer Vision ECCV 2014, pp. 740-755, Cham, 2014. Springer International Publishing. ISBN 978-3-319-10602-1.
Tsung-Yi Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2):318 327, 2020. doi: PO1.//AAAAM........
Biao Liu, Ning Xu, Jiaqi Lv, and Xin Geng. Revisiting pseudo-label for single-positive multi-label learning. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 22249 22265. PMLR, 23 29 Jul 2023.
An Thanh Nguyen, Byron Wallace, Junyi Jessy Li, Ani Nenkova, and Matthew Lease. Aggregating and predicting sequence labels from crowd annotations. In Regina Barzilay and Min-Yer Kan (eds.), Pro- ceedings of the 55th Annual Mecting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 299 309, Vancouver, Canada, July 2017. Association for Compulational Linguistics. doi: 10.18653 1111653//11/P17-0222 URL https::/acclathhol.ge o0tp:0/11.ooo///111//////
Giambattista Parascandolo, Heikki Huttunen, and Tuomas Virtanen. Recurrent neural networks for poly- phonic sound event detection in real life recordings. 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6440 6444, 2016. doi: AAL109//CAASP:20066
Tal Ridnik, Emanuel Ben-Baruch. Nadav Zamir, Asaf Noy, Iamar Friedman, Matan Protter, and Lihi Zelnik- Manor. Asymmetric loss for multi-labed classification. In 2021 IEEE/CU /CVF International Conference on Computer Vision (ICCV), pp. 82 91, 2021. doi: 10.1109/ICCV 45L100991CCV4599222.222 AS0.CCVV.44922222.00000000
Tal Ridník, Gilad Sharir, Avi Ben-Cohen, Emanuel Ben-Baruch, and Asaf Noy. MI-Decoder: Scalable and versatile classification head. In 2023 IEEE/CVE Winter Conference on Applications of Computer Vision (WACV), pp. 32-41, 2023. doi: 10.1109/WACV5dte 2WA2002660..222300002.
14

---

Published in Transactions on Machine Learning Research (09/2025)
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (LJCV), 115(3):211 252, 2015.
doi: 10.1007/s1
Kevin Swersky, Πla Sutskever, Daniel Tarlow, Richard Zemel, Russ R Salakhutdinov, and Ryan P Adams. Cardinality restricted Boltzmanu machines. In F. Pereira, C.J. Burges, L. Bottou, and K.Q. Weinberger (eds.), Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc., 2012.
Daniel Tarlow, Kevin Swersky, Richard S. Zemel, Ryan P. Adams, and Brendan J. Frey. Fast exact inference Conference Uncertainty in Artifi-
for rocursive cardinality models. In Proceedings of the cial Intelligence, UAL'12, pp. 825 834, Arlington, Virginia, USA, 2012. AUAI Press. ISBN 9780974903989.
Thomas Verelst, Paul K. Rubenstein, Marcin Eichner, Tinne Tuytclaars, and Maxim Berman. Spatial consistency loss for training multi-label classifiers from single-label annotations. In 2023 IEEE/CVI Winter Conference Applications of Computer Vision (WACV), pp. 3868 3878, 2023. doi: 10.1109/ HAACV6668.202330000
Catherine Wah, Steve Branson, Peter Welinder, Pictro Perona, and Serge Belongie. The Caltech-ucsd birds- 200-2011 dataset. California Institute of Technology, 2011.
Haobo Wang, Ruixuan Xiao, Yixuan Li, Lei Feng, Gang Niu, Gang Chen, and Junbo Zhao. PiCO: Contrastive label disambiguation for partial label learning. In International Conference on Learning Representations, 2022.
King-Kun Xie and Sheng-Jun Huang. Partial multi-label learning. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1), Apr. 2018. doi: 10.1609/aadd v3211.1441.
Ming-Kun Xie, Jiahao Xiao, and Sheng-Jun Huang. Label-aware global consistency for multi-label learning with single positive labels. S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 18430-18441. Curran Associates, Inc., 2022.
Ning Xu, Congyu Qiao, Jiaqi Lv, Xin Geng, and Min-Ling Zhang. One positive label is sufficient: Single- positive multi-label learning with label enhancement. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Newral Information Processing Systems, volume 35, pp. 21765- 21776. Curran Associates, Inc., 2022.
M. Zhang and Z. Zhou. A review on multi-label learning algorithms. Knowledge and Data Engineering, IEEE Transoctions on, PP(99):1, 2013. ISSN 1041-4347. doi: 00.119/TKDEE.2013.
Youcai Zhang, Yuhao Cheng, Xinyu Huang, Fei Wen, Rui Feng, Yaqian Li, and Yandong Guo. Simple and robust loss design for multi-label learning with missing labels, 2021.
Dengyong Zhou, Olivier Bousquet, Thomas Lal, Jason Weston, and Bernhard Schölkopf. Learning with local and global consistency. In S. Thrun, L.. Saul, and B. Schölkopf (eds.), Advances in Neural Information Processing Systems, volume 16. MIT Press, 2003.
Donghao Zhou, Pengfei Chen, Qiong Wang, Guangyong Chen, and Pheng- Heng. Acknowledging the unknown for multi-label learning with single positive labels. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (eds.), Computer Vision - ECCV 2022, pp. 423 -400, Cham, 2022. Springer Nature Switzerland. ISBN 778-3-031-200523
15

---

Published in Transactions on Machine Learning Research (09/2025)
A Proof for Perfect Bipartite Matching with Greedy Algorithm
Section 4.4 described that the instance cardinalities are estimated as h; 11 where IT is permutation of a cardinality vector a satisfying the cardinality distribution. Next, we show that the permutation II that minimizes the objective B = (s, llv) can be found by sorting the entries of 8. Let's start by re- re-writing the objective as
B (s llv)² -1l) =s s + v II IIv II..
Since IT- = I, the first two terms do not depend on II. Consequently, the task is equivalent to maximizing 28 IIv. According to the rearrangement inequality Hardy et al. 1952), this is achieved when the elements are groedily paired according to their magnitude: the largest clement of is matched with the largest element of u, and on. Since a is in increasing order by construction, the solution found by sorting analogously, with obvious O(N log N) complexity.
B Sensitivity Analysis
B.1 Sensitivity to kimax
The largest cardinality kmax is dataset-dependenn parameter that primarily determines the upper threshold Tu in Equation B of the CS method. Figure / illustrates the CS method's insensitivity to variations in the maximum value kmax, ranging from 0.5 to approximately 2.5 times the actual kmax. without significantly affecting performance. Alternatively, it is possible to use more robust statistics such as the 95% quantile estimator, with the overall robustness accounting for the small bias caused by the estimator.
In Equation 3 the lower threshold Ti is similarly determined by ke- In practice, the method works with any small number that is strictly non-zero, and sctting it at he that for most multi-label datasets is much smaller than the number of classes C' is simply convenient practice.
B.2 Sensitivity to CD Estimation Error
For Experiment 3, estimate the CD from set of S samples for which k, is known. We denote by my the count of given cardinality & in the sample and estimate the density
my +
k S kmax
((k) =
kiax Evere (m, + 5)
otherwise,
where 8 is smoothing factor and kmax is an estimate of the maximum cardinality. For the results reported in Figure 6 used d = 0.1 and kmax = 1.2k but the results would be highly similar for broad range of other choices. Note that when estimating P(k) with an initial kmax that is too small, there may be k in the sample that exceed the presumed maximum. In such cases, kmax should be re-set to the largest k; in the sample.
Figure 8 provides a complementary analysis. For each estimate, we compute the estimation error as Kullback- Leibler (KL) divergence between the estimate and the true cardinality, and here show the MAP as a function of the estimation error. This illustration provides information on how alternative estimators with similar estimation error would work.
B.3 Sensitivity to Mi
The parameters M, determine how many of the largest probabilities f are summed over in the joint loss Equation 5 and when estimating the cardinalities. Figure the results are not sensitive to the choice, plotting the model performance function of M, and plotting the MAP score normalized with the highest
16

---

Published in Transactions on Machine Learning Research (09/202)
87.5
85.0
82.5
MAP
VOC
80.0
COCO
77.5
75.0
72.5
70.0
0.5
1.0
1.5
2.0
2.5
AmEx kmax
Figure 7: Sensitivity analysis of Rimax and equivalently of the upper threshold T, in the CS method.
72.1
50.2
72.0
Noisy CD
71.9
50.0
CS
71.8
CS
CD
MAP
71.7
MAP
49.8
Noisy
71.6
CS
49.6
71.5
71.4
49.4
71.3
49.2
0.00
0.07
0.21
0.28
0.35
0.00
0.08
0.12
0.16
0.20
KL
KL
(a) COCO
(b) NUS
Figure 8: Results of Figure presented from an alternative perspective, as function of the estimation error (KL divergence between the estimate and true cardinality density).
score. Except for M, k; that results in a minor reduction in MAP due to creating false negatives, the results are extremely consistent over the choice with at most 0.5% reduction in MAP. The choice of M, kmax - k2 that only depends on the maximum cardinality is effectively as good as any M; that depends the instance cardinality kip
C Experimental Details
C.1 Model Architecture
We use a ResNet-50 backbone pre-trained on ImageNet. Following the same image processing techniques as in all comparison methods, we resize all input images to 448 x 448 pixels and apply data augmentation of random horizontal flipping (probability 0.5) during training. The output layer of ResNet-50 replacod by a global average pooling (Lin) 2013) followed by a fully connected layer, and the output dimension is adjusted to match the number of classes in the target dataset.
17

---

Published in Transactions on Machine Learning Research (09/2023)
1.0000
0.9975
Normalized MAP
0.9950
0.9925
0.9900
VOC
0.9875
COCO
NUS
0.9850
CUB
1k;
2
3kg
4kg
Kmax
K2
M;
Figure 9: Sensitivity analysis of M, for the CS IC method. Normalized MAP scores computed by dividing the MAP by the highest MAP for each dataset.
20 10
55
(a) VOC
(b) COCO
(c) NUS
(d) CUB
Figure 10: The conditional distribution of predicted cardinalities in Experiment 1.
C.2
Datasets
Table 3 provides full details about the datasets, indicating both the standard information (number of images and C) and the cardinality statistics k and kmax-
Table 3: The details of benchmark datasets used in our experimentation.
Statistics
VOC
COCO
NUS
CUB
20
80
81
312
Training
4574
65665
120000
4795
# Images
Validation
1143
16416
30000
1199
Test
5823
40137
60260
5794
Training
1.46
2.94
1.89
31.4
ke
Validation
1.46
2.92
1.93
31.52
Test
1.43
2.91
1.88
31.53
Training
18
11
71
kimax
Validation
16
11
69
Test
15
13
72
C.3 Training
For CS and CS CD, we train our model for 10 epochs and 20 epochs for = IC, using Adam optimizer and established training parameters from prior work without further optimization. Following Zhou et. al.
18

---

Published in Transactions on Machine Learning Research (09/2025)
Table 4: Best hyperparameters for each method. The parameters a and used in Equation and are
computed based on 7, 7) and Φ using Equation and 7
Dataset
Parameter
Method
VOC
COCO
NUS
CUB
CS
0.35
0.15
0.1
0.01
CS + CD
e
6
5
5
CS +
IC
a
all
2
3
2
2
CS >
CD
le -4
8e 85-5
4e 3
6e 6
CS + IC
9e 3
6e 3
7e 3
8e -6
CS = CD
0.50
0.10
0.50
0.55
CS + IC
0.30
0.50
0.55
0.40
CS = CD
0.65
0.55
0.35
0.35
Φ
CS = IC
0.60
0.60
0.30
0.55
the batch size of 8, 16, 16, and 8 for VOC, COCO, NUS, and CUB respectively. Consistent with Chen et al. (2024). we fix the learning rates to le for VOC, COCO, and NUS, and to 5e for CUB.
2022
C.4
Hyperparameters
We optimize the hyperparameters for each method and dataset by maximizing the validation MAP. Table 4 lists the final choices obtained by the process described below for all of our model variants. For all of the comparison methods, use the same validation protocol and candidate sets reported in the original publications or their associated code-bases; we do not replicate the descriptions here.
First, we note that our code uses slightly different parameterization than the main paper: The weights in Equation and are re- re-parameterized as
(6)
1
n
B
(7)
and the validation is conducted in the space of 7,7, and ø. Instead of full grid search, staged approach to reduce the compulational cost and overfitling the validation data:
We tuned the À parameter of CS within the set (the number of Cs initialization epochs for CS-> CD and CS-> IC, see Section 4.5 for each dataset without requiring separate runs to tune this parameter.
This also provides ew
After determining X, we search for a in (1,2,3,4} for all datasets.
For CS + CD and CS = IC, we perform a grid search of 64 configurations (8 x 8) 0.35, 0.4, 0.45, 0.5, 0.55) for r and in (0.3, 0.35, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65) for ø.
The range of candidates considered for y depends on the case:
For VOC+COCO+NUS and Cs = IC, we scarch (5e-3 3, 66 7e -3 8e - 3,00 33..
For VOC+COCO and cs CD, we search in {6e-5, 75 - - le-4)
For NUS and CS CD, scarch in (2e-3 3e 4e 5e 31
For CUB and both methods, search in (5e-t -6 7e
Se 9e 6
19

---

Published in Transactions on Machine Learning Research (09/2025)
Table 5: Test MAP for all our method variants and the naive baseline (AN) using the training data with a maximum of two labels per sample. The results for Single-positive from Table are duplicated here for the ease of comparison.
Method
VOC
COCO
NUS
CUB
Full label
89.56 + 0.35
76.43 = 0.09
52.12 - 0.09
32.73 + 0.20
Single-positive
AN
85.51 + 0.41
64.33 0.10
42.68 = 0.09
18.65 = 0.31
CS
88.97 + 0.03
71.35 0.10
49.23 = 0.45
21.56 0.08
CS
CD
89.10 + 0.17
72.07 0.08
50.26 0.08
2
CS
IC
89.69 + 0.16
73.45 0.12
50.92 0.48
2
Two-positive
AN
89.07 + 0.17
70.55 0.07
22.22 0.20
CS
90.22 + 0.14 0.14
73.55 0.08
22.76 +.29
CS = CD
90.11 + 0.07
+3.44+0.15
52.38 + 0.14
23.69
CS
= IC
90.39 + 0.12
75.50 + 0.03
52.94 + 0.13
23.94
Beyond SPMLL: General Partial Labeling Setups
Even though our main interest was in the SPMLL setup that requires the least amount of labeling effort and hence has the most potential for benefiting from the cardinality information, we provided the details for a more general formulation where ke labels are observed for the ith sample. To show that the method indeed works also in other setups, we conducted an experiment that is otherwise identical to the Experiment but observe up to two positive labels for each sample, when available, while still considering the rest unobserved even if the sample did not have more than one true positive label. We re-use the hyperparameters (Table 4 from the main experiment to validate also the robustness of these choices, even across a change in the learning task.
Table reports the MAP for the proposod method variants and the AN bascline, and repeats the results for the single-positive setup from Table for convenience. The first, obvious, observation that the MAP improves for every method and data set, is to be expected since know have information. The main result, however, is that the proposed methods retain their advantage: We still have CS outperforming AN in all cases, and both CS = IC is still the best method, with approximately the same improvement over CS in both scenarios.
20


Reviews:
[{"review_id": "Xdi81rehAX", "reviewer": "Reviewer_MrUA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n- The authors address the critical problem of SPML, recognizing that obtaining fully annotated data can be prohibitively expensive or challenging in certain applications.\n- The proposed methods are intuitive and appear straightforward to implement, making them accessible for practical use.\n- The methods exhibit a plug-and-play nature. Adding numerical experiments to validate this characteristic, by integrating these methods with other standard techniques and demonstrating performance improvements, would further strengthen the paper's contributions.\n\n**Weaknesses**\n- Many of the proposed approaches rely on intuition and lack theoretical guarantees. While this is not inherently a drawback if the methods demonstrate significant performance improvements over the state of the art (SOTA), the reported results show that performance is, at best, comparable to baseline methods.\n- The use of a quadratic loss function in (3) is presented as intuitive, but the rationale for this choice is underexplored. It raises the question of whether alternative loss functions, for example, the absolute loss function, were considered. The authors should provide a more detailed justification for their choice, supported by empirical evidence where possible.\n- The reliance on $k_{\\max}$ and $k_e$ as representatives of label cardinality information may lack robustness. It seems plausible that a single outlier could skew these metrics, making the proposed approach vulnerable to datasets with such outliers. This raises concerns about the generalizability of the method across datasets with varying label distributions.\n- Problem-specific hyperparameter tuning using grid search contradicts the plug-and-play nature of the proposed methods. This dependency on tuning undermines the claim of ease of integration.\n- The matching algorithm described in Section 4.4 for estimating instance cardinalities is intriguing. However, it is unclear whether the current model predictions, $\\mathbf{f}_i$, provide sufficient information to compute reliable scores.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors address the problem of Single Positive Multilabel Learning (SPML) by leveraging additional information on label cardinality. Their primary contribution lies in integrating label cardinality information at different levels of granularity into the optimization problem's design. Specifically, they explore three forms of label cardinality information: instance cardinality (IC), cardinality distribution (CD), and cardinality statistics (CS). The paper introduces distinct methods for embedding this cardinality information directly into the optimization objective and evaluates the proposed approach through numerical experiments on benchmark datasets, demonstrating its effectiveness and applicability.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see the previous section.\n\nCan the authors comment on the following question:\nIs it possible to retain the objective functions in (1) as they are in standard methods while incorporating label cardinality information as constraints in the optimization problem? This approach could, in principle, allow leveraging the strengths of established SOTA methods while also integrating the additional insights provided by label cardinality information.\n\nMinor comments:\nIt is better for the readability to use \\citep when citations are not part of the sentence. For example:\n\"Numerous methods for learning multi-label classifiers from fully annotated data have been proposed (Ridnik et al., 2023; Zhang & Zhou, 2013; Parascandolo et al., 2016)...\"\nreads better than \n\"Numerous methods for learning multi-label classifiers from fully annotated data have been proposed Ridnik et al. (2023); Zhang & Zhou (2013); Parascandolo et al. (2016)...\"", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "sr8qQKX5VL", "reviewer": "Reviewer_1tSR", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n*Relevance* The problem is highly relevant in practical applications where labeling data comprehensively is resource-intensive. In many real-world scenarios, annotators can only provide partial label information—typically a single positive label per instance—despite the true class membership being multi-label. The setup reflects a practical relaxation of annotation assumptions.\n\n*Simple Yet Effective Setup*: The authors present a minimal modification to the existing SPMLL pipeline by incorporating cardinality information. Despite the simplicity, the approach is shown to yield strong empirical performance, suggesting robustness and ease of adoption.\n\n*Mathematical Motivation*: The paper provides a clear and structured mathematical formulation of the problem. Different forms of side information on cardinality (IC, CD, CS) are formally defined, and corresponding loss functions are derived with justification. The treatment helps clarify the assumptions and goals, aiding reproducibility and further development.\n\n*Clear Writing* The manuscript is generally well-written and easy to follow. Notably, Section 4 begins with an overview paragraph, which is helpful for navigating the subsequent content—similar summaries for other sections would be useful for a reader and the authors are encouraged to add them. The figure captions are informative, and I appreciate the effort to remain to the point while conveying necessary detail is appreciated.\n\n*Comprehensive Experimental Evaluation* The authors conduct a broad set of experiments across four datasets and multiple baselines. The analysis covers robustness to estimation error, ablations on estimation techniques, and sensitivity analyses.\n\nWeaknesses: \n\n1. *Baseline (Naive) Approach*: The “Assumed Negative” (AN) baseline is perhaps too weak to be informative. Given the known limitations of treating unknown labels as negatives, including it primarily serves to highlight the inadequacy of this assumption. Additionally, the choice of quadratic loss for Lu (in Equation 3) is not fully motivated. Are there alternatives considered (e.g. hinge, BCE)? Why is quadratic preferred?\n\n\n[IMPORTANT CONCERNS]\n\n2. *Cardinality Distribution Assumption*: The paper assumes access to the label cardinality distribution (CD) in some settings. It is unclear how this distribution would be obtained in practice. If estimated, what are its statistical properties, are there any assumptions you make, can it be heavy-tailed? \n\n3. *Learning Cardinalities via Loss*: The claim that the model’s predictions carry information about cardinality under the proposed loss functions (especially LCS and LIC) could use more theoretical support. I understand the bipartite matching would work and optimize the given function, but why do the f_i contain enough information Why does the proposed loss structure lead to meaningful cardinality estimates? Is there any inductive bias introduced that aligns the model outputs with cardinality? This is particularly relevant when no instance-level supervision on cardinality is provided.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces the use of label cardinality information—ranging from crude statistics to full instance-level cardinalities—as auxiliary supervision in single-positive multi-label learning (SPMLL), a setting where each training instance has only one observed positive label. This is a new setting and is interesting to study. They propose different loss functions and a heuristic workaround when only distribution of label cardinality is available. The proposed approach is tested on four SPMLL benchmark datasets and compared with multiple baselines. Results show that even with minimal additional information (e.g., mean cardinality), the method achieves competitive or superior performance, and further improvements are seen with more detailed cardinality knowledge.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Method Design Choices:\n\nSome method design decisions are ad hoc. For example, the way thresholds are derived from summary statistics​  could be further justified.\n\nSection 4.5: The two choices for $M_i$ (number of top-predicted labels) are stated without enough justification. Is there any theoretical or empirical analysis comparing them to other alternates?\n\nWriting and Presentation:\n\nAbstract: The term “simple method” could be clarified—simple in terms of implementation, computational cost, or sample efficiency?\n\nAbstract: “State-of-the-art” should be qualified—within SPMLL or broader weakly supervised MLL?\n\nIntroduction: The sentence “a bipartite matching algorithm...” could benefit from some qualification. It currently implies this method is used universally without constraint, but it only applies under the CD assumption.\n\nRelated Work: It would be useful to clarify under what assumptions pseudo-labeling methods work better or worse than this method. Has any prior work combined pseudo-labeling with cardinality information? Can label cardinality help improve pseudo-label accuracy?\n\nMethod: In Section 4.5, the transition from LCS to LIC involves solving a cardinality estimation problem. The choice to use a greedy matching algorithm is motivated empirically, but could benefit from discussion on when the assumptions of monotonicity/sorting would break.\n\nAdditional Considerations\n\nIs there any benefit in using the predicted label probabilities directly for pseudo-labeling in combination with cardinality constraints (e.g., sampling top-k based on cardinality estimates)?\n\nAny potential use in active learning? For example, could estimated cardinalities guide which samples to annotate further?", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "6vuVftQNoF", "reviewer": "Reviewer_tbHe", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nThe paper is clearly written, making the technical content accessible and easy to follow. \n\nThe hierarchical treatment of cardinality information -- distinguishing between statistics, distributions, and instance-level estimates—offers a structured perspective that could motivate future work in both SPMLL and broader weakly supervised settings.\n\nThe authors provide extensive ablation studies and sensitivity analyses, which enhance the empirical credibility.\n\n\nWeaknesses:\n\nFrom a technical perspective, the novelty of the proposed method is somewhat limited. The approach mainly combines existing techniques with heuristic adaptations, and the loss function design is largely intuition-driven, lacking a deeper theoretical rationale for why or how label cardinality contributes to improved learning under the SPMLL setting. \n\nOn the experimental side, there are several issues that limit the strength of the results. First, although the authors say that their loss function can be used as a plug-in module, they do not show how it works when combined with other existing SPMLL methods. This makes it hard to judge how useful or general the method really is. Second, the experiments do not include some important recent SPMLL baselines, such as the method by Xie et al. (NeurIPS 2022), which makes the comparisons less convincing. Third, all experiments are done only in the single-positive label setting, without testing whether the method can work in more general cases like missing-label learning. Lastly, although the method does improve performance, the gains over existing methods are quite small. This raises concerns about how much practical benefit the method really provides.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper investigates the problem of single-positive multi-label learning (SPMLL), where each training instance is annotated with only one positive label. The authors propose a unified framework that leverages label cardinality information at multiple levels of granularity—from coarse global statistics to fine-grained instance-level estimates—to improve learning in this weakly supervised setting. The core contributions include (1) a cardinality-aware loss that aligns the predictions of unobserved labels with statistical priors, (2) a joint loss that enforces consistency between model outputs and estimated cardinalities, and (3) a bipartite matching algorithm designed to infer per-instance label cardinality from model predictions and global label statistics. The experimental results on benchmark datasets such as NUS-WIDE and CUB demonstrate that the proposed method can narrow the gap between SPMLL and fully-supervised training, while maintaining robustness under noisy cardinality estimates.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please refer to weaknesses", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "V8U9zOdBMz", "reviewer": "Reviewer_98sq", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n1. The paper is well-written and easy to follow. Technical backgrounds, problem formulation, and the definitions of side information are given clearly. Each technical method is introduced with sufficient intuitions, making it easy for readers to understand it. \n2. The proposed methods are simple to implement.  \n3. This is my personal thought, but I believe that trying to use more side information from the data is a promising way to improve all kinds of machine learning tasks in general. This paper is one of such attempts and I appreciate it. \n4. Experiment 2 is of practical value in that it answers the question of how accurate the estimations should be for the method to work. \n\n# Weaknesses\n1. Experimental results do not compare favorably against baseline methods. In Table 1, CS in general is not the best-performing method compared to baselines. \n2. Extending from weakness 1, it is interesting if the proposed method can work together with some more advanced baselines. It would be good if the proposed method can work with baselines and further improve performance. \n3. Figure 4 is slightly hard to interpret. I understand it is trying to show how accurate the label cardinality estimation is, but the visualization is not sufficiently clear. Maybe some other presentation forms can be used (e.g. distribution of estimation error). \n4. It is slightly confusing about how the experimental settings differ between Table 1 and Table 4. The authors may provide some additional discussion.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a framework of methods to use side information to improve the problem of SPMLL (single-positive multi-label learning). The side information discussed are mostly related to label cardinality (i.e. the number of labels per instance), with different granularities, instance cardinality (IC), cardinality statistics (CS), and cardinality distribution (CD). The authors propose a method for each setting, and introduce how to estimate IC with CS (i.e. trying to provide more side information) with an efficient bipartite matching algorithm. Experiments are performed on standard benchmark datasets,", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Discussion/experiments about how the proposed method can work together with baseline methods. \n2. Some better presentation forms for Figure 4. \n3. Discussion on how results Table 1 and Table 4 are different.", "broader_impact_concerns": "Not any.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)