Training image classifiers using Semi-Weak Label Data
Authors: Ankit Parag Shah
Bhiksha Raj
Venue: Submitted to ICLR 2023 (2023)

Abstract: This paper introduces a new semi-weak label learning paradigm which provides additional information in comparison to the weak label classification. We define semi-weak label data as data where we know the presence or absence of a given class and additionally we have the information about the exact count of each class as opposed to knowing the label proportions. A three-stage framework is proposed to address the problem of learning from semi-weak labels. It leverages the fact that counting information is naturally non-negative and discrete. Experiments are conducted on generated samples from CIFAR-10 and we compare our model with a fully-supervised setting baseline, a weakly-supervised setting baseline and a learning from proportion(LLP) baseline. Our framework not only outperforms both baseline models for MIL-based weakly supervised setting and learning from proportion setting, but also gives comparable results compared to the fully supervised model. Further, we conduct thorough ablation studies to analyze across datasets and variation with batch size, losses architectural changes, bag size and regularization, thereby demonstrating robustness of our approach. 

Under review as a conference paper at ICLR 2023
TRAINING IMAGE CLASSIFIERS USING SEMI-WEAK LABEL DATA
Anonymous authors Paper under double-blind review
ABSTRACT
This paper introduces a new semi-weak label learning paradigm which provides additional information in comparison the weak label classification. We define semi-weak label data as data where we know the presence or absence of a given class and additionally we have the information about the exact count of each class as opposed to knowing the label proportions. A three-stage framework is pro- posed address the problem of learning from semi-weak labels. It leverages the fact that counting information is naturally non-negative and discrete. Experiments are conducted on generated samples from CIFAR- and we compare our model with fully-supervised setting baseline, a weakly-supervised setting baseline and a learning from pLoportion(LLP) baseline. Our framework not only outperforms both baseline models for MIL-bbaedd weakly supervised setting and learning from proportion setting, but also gives comparable results compared to the fully su- pervised model. Further, we conduct thorough ablation studies to analyze across datasets and variation with batch size, losses architectural changes, bag size and regularization, thereby demonstrating robustness of our approach.
INTRODUCTION
In traditional fully supervised machine learning setting, training samples are "strongly" supervised, iee every training instance is labeled. In practice. though, strongly labeled data are expensive to collect. An alternate approach or absence of instances of a class in sets of training samples. This form of labelling is particularly useful for data such as images or sounds. For instance, an image, it is relatively easy to annotate if the image contains instances of (say) dogs, but much harder to tag the bounding box of every dog in the image. The former represents a weak label, while the latter is a strong label. So also, in sound recordings it is much easier to merely annotate if a recording includes gun shots (weak label) than to identify the onset and offset times of each instance of a shot (strong label).
collect "weak" labels labels which only indicate the presence
At abstract level, it is useful to think of data such images sound recordings as bags of candidate instances (c.g. candidate regions of the image or candidate sections of the recording). Labels are now assigned to bags, rather than instances. A negative label assigned to a bag indicates that the original data (image or recording) did not contain the target class(es), and hence none of the instances in the bag formed from it are positive for any class. On the other hand, positive label assigned to a bag indicates that some instances in the bag are positive, although it is unknown which or how many. In the real world, it is much easier to collect such bag-level, or weak labels. A number of algorithms have also been proposed to train classifiers with such weak labels Shah et al. (018); Pappas & PPopescu-eelis Belis 2014); Carbonneau al.) (2018); Vanwinckelen et al. (2016). However, there remains gap between the performance of fully supervised model&Kumaa & (201)) Shah jet al. (2018) (trained from data where every instance labelled) and that of the weakly supervised model (trained from data with weak labels). The gap in performance limits the ability to which weak supervision (i.c. supervision with weak labels) can be relied upon.
In this paper, we introduce middle ground between the two settings of full and weak supervision. In many settings, it is possible to annotate count information, which specifies the number of occur- rences of individual classes in a bag, for not much extra effort simple weak labeling that merely tags their presence or absence. For instance, it is often fairly straight-forwan to annotate how many dogs there in an image, if the annotation of exact bounding boxes is not required. Similarly, it is

---

Under review as a conference paper at ICLR 2023
reasonable expect that it is not significant extra effort indicate how many (e.g.) gunshots were heard in an audio recording, it is not required that their exact locations be tagged as well.
We refer to such labels "semi-weak" labels, and the problem of learning from such data as learning with semi-weak supervision, or learning from counts. In our paper, we show classifiers trained using semi-weak labels classify test instances with much greater accuracy than those trained with merely weak labels.
Weak Label
Semi Weak Label
gunshot
punsSoo
gunshot
gunshot
guisshot
alarm
alarm
bellow
Test Data
gunshot
Sound Event
Detector
bellow
Figure 1: An example semi-weak labels for audio event detection. Semi-weak labels give count information, while weak labels only provide information about the presence absence of classes.
Our proposed solution for learning from semi-weak labels is pose the training satisfaction problem. The constraint to satisfy is that the sum of the counts predicted per category in each bag must equal the size the bag. Thus, we first learn to predict whether the instance present in the bag not similar to weak label learning setup and learns expected count of each class in a bag. Then, given the bag size and conditional expected count for each class, we propose solve an optimization problem to translate the real valued expected count to non-negative integer count. Finally, given the predicted count as "ground truth" and the instance level logits, we try to find the best assignment of labels such that the counting requirement is satisfied as well as the likelihood of the bag is maximized.
constraint
LITERATURE REVIEW
2.1 MULTIPLE INSTANCE LEARNING
Learning from bag-level labels is often referred to multiple instance learning (MIL). Paper Car- bonneau al. (2018) provides an extensive summary detailing most aspects of multiple instance learning. Dietterich Dietterich al. (1997) first introduced multiple instance learning where it was applicd for drug activity prediction. Since then, several algorithms have been introduced to address MII.. To deal with label noise, natural solution is count the number of positive instances in bag, and apply threshold to tag a bag as positive. This was summarized in |Fouds Frank (2010) as the threshold-based assumption for multiple instance learning. The threshold-basee assumption, which defined that a bag is positive if and only the number of positive instances is between a range, was the first time that counting information was brought into multiple instance learning. Since then, several efforts Tao et al. (2004b a) were exerted to do bag-level prediction using count-based assumptions. Foulds Frank (2010) extended the assumption and proposed an SVM-based algorithm predict the bag label. One common problem with those methods was scalability; also those methods do not generalize for multi-class classification.
Multiple Instance Regression: MIL regression consists of assigning real value to bag. Com- pared to MIL classification, MIL regression has attracted far less attention in the literature. For MIL regression, one line of research has the assumption that some primary instance contributes largely to the bag label. This motivated sparsity- based approaches that assign sparse weights to instances and use regularization methods like L1, L2 Maataaiiiersraappaaaaa & Popescu-Boc & Lane (2007); Pappas & Popescu-Bas (2017). However, most of these method work only for small-scale data and focus the accuracy of predicted results rather than attempting identify the
(2014); Wagstafff
2

---

Under review as a conference paper at ICLR 2023
primary instances that contribute to the bag label. Subramanian et al. (2016) addressed the identi- fication of primary instances by using a Dirichlet process to group the instances and find clusters. However, this method assumes that the largest cluster defines the label; this method also doesn't 't work for multi-class machine learning problems. Also, those methods do not work properly in our semi-weakly supervised setting as they fail to incorporate the natural property of counts, which is non-negative integer value.
Another relevant mainstream of MIL research is instance-level predic- tion. Maror[Maro & Lozano-Pérez (197) is perhaps the best-known framework for instance-level prediction for MIL. Following this framework, many research ideas have been proposed. The basic ideas of those frameworks are to label the instance dynamically or statically according to the bag la- bel. Training instance-level classifiers is non-trivial as strong labels are unavailable. Recently, many methods have proposed to do hag-level prediction and "hope" that the bag-level accuracy could propagate to instance-level accuracy. [Dietterich et al. (1997); Andrews et al. (2003); Babenko et al. 2008). However, as is discussed in Doran & Ray (2014); Vanwinckelen et al. (2016), this method is sub-optimal. Empirical studies have been conducted in Vanwinckelen et al. (2016) that better bag level prediction doesn't 't promise instance-level prediction. Therefore, the the most successful instance-level prediction models, i.e., Mi and SI-SVMAndrews et al. (2003), discard the bag information as much as possible, and treat each instance individually.
Instance-Level Prediction:
2.2 LEARNING FROM PROPORTIONS
A related approach, previously proposed in the literature, is that of learning from proportions Dulac- Arnold al. 2019); Yu et al. (2013); Musicant al. (007), where the proportion of instances in bag that belong to any class is indicated. When the precise number of instances in individual bags is known, knowing proportions is identical to knowing instance counts of classes in the bag. But in other settings, such as in the image or sound examples mentioned above, it is much simpler, and possibly far more natural to annotate the counts of occurrences of a class than the proportions. For instance, in an audio recording which must be labelled for the occurrence of, say, dog barks, it is more intuitive and natural to state that a dog barked three times in the recording than to say (e.g.) that 20% of the recording comprised the sound of dogs barking. Indeed, the main signature of sound events such as dog barked is their onset, which is generally distinct, whereas it generally hard to spot precisely when the sound terminates. As such, while it is perfectly reasonable to state that the recording includes three barks, it may be infeasible, or even meaningless to specify what fraction of the recording comprised dog barks shown in Figure 2
Quadrianto Quadrianto et al. (2009) proposed the McanMap model which assumes the data follows
an exponential distribution and is conditionally independent of the bags. Fan Fan et al. (2014) and Patrin[Patrini et al. (2014) further refined the loss function of MeanMap and make it applicable for multi-class classification. Yu Yu al. (2014) proposed another line of research and used SVM mod- els to iteratively estimate the instance-level classifier. However, this method suffers from scalability issues when is extended by Amlld et al.] (2019); (Ardehaly & Culotta (2017); Bortsova et al. (2018). The commonal- ity for the based methods for the I.I. problem that they try to use bag-level supervision to perform instance-level estimation such that the estimated distribution is as close as possible to the bag label. Interestingly, [Dulac-Arood, et al.] (2019) introduces another loss function that directly minimizes the instance-level prediction results. They introduced the Optimal Transport algorithm to make the loss function computationally tractable.
multi-class setting. More recently, DL-based method was proposed
3 PROBLEM STATEMENT
Figure I shows example problem setting in audio event detection. Generally, we consider supervised multi-class classification problem, where we define the instance-level data x E X' and label y V, where X' R "xXxx xc! and y {0.... .O.....44 K). Let K be the number of classes, Ng be the number of instances in a bag and N be the number of bags. For each bag by E B = {X), wf Y is the bag-level label. More specifically, vff is the count of class k in bag i. Formally,
3

---

Under review as a conference paper at ICLR 2023
10% -
20%
LLC
LLP
MIL
Unnatural for humans to describe
Natural for humans to describe
Figure 2: Left: We illustrate scenario which natural for human annotate between counts and proportions. Right: The relation between learning from proportion (LLP), learning from count (LLC) and Learning from Weak labels (MIL). The intersection part assumes that the bag size is known, and the count is exact. MIL problem is special case of LLC where the count is equal to 1.
given the bag size, we define the space of labels as
K Σ c- NB}
Y = fc E z
(1)
x B x YB, goal is learn predictor f X' + R that predicts the probability distribution of an instance belonging different classes, ie. fa,) = {p(y: = 0) 0;), p(y = 1/z.)....,,P = K x;)}, where r; a feature vector for instance and is the parameters of the network.
Given series of bags, [(bo ); (b1.yf )..., ... (bN,
3.1 GENERAL LOSS FUNCTION
In this problem, focus instance-level prediction and solve it by viewing the problem risk minimization task. More specifically, given a regression loss function Lreg B x Y - R.+
defined as
N
RN()) = N
1
Σ 1=1
Leeg (x:).
(2)
One intuitive approach is the define divergence function that quantitatively quantify the distance between the predicted bag labels and the ground-truth label. As such, let's define bb, B, the predicted count as Σ (Izz), and we want to minimize the divergence between the predicted counts and the actual counts
(ree(f(x)) = nu
div(, v)
(3)
Usually, learning from counts could formulated as a regression problem. However, for inter-class separation of the features at the bag-level and tackle the problem multi-label bag classification, we used binary cross cross-entropy to recognize different classes existing in a bag. let 1(-) denotes the indicator function, taking values or 0 depending whether its argument is true not, define another loss term as follows
F Fmaz maJieo,1..... (z)
(4)
Lete (f (f = diubce (anbceFFmaac [Fmax: 1 > 0))
(5)
Additionally, we require the loss function to robust to the sparsity of the bag and the generalizabil- ity of the loss function is critical in the learning process. Thus, introduce one L.1 regularization
4

---

Under review as a conference paper at ICLR 2023
term that makes the model perform better when the bag is sparse or "pure". Here, we define sparse- ness based on the bag level counts of each class. Our proposed loss function with the regularization
term is defined as follows
Ry(f)= Lreg + alolle +
Σ mnEb;
1 I1f NB
(6)
4 PRELIMINARY: DLLP APPROACH
DILP Ardehaly & Culotta 2017
approach is a DNN-based model for learning from proportion.
Mathematically, given bag 61. 62, by. suppose that fo (b,,) py bıy) is the vector outputs of the DNN for the j-th instance in bj. Let + is an clement-wise addition operator. Then the posterior bag-level class proportion could be estimated as
N2 + P; NB
Po (y biy)
The final objective, i.e., Laup- is made of distance measurement. Let ci be the counts of class k bag i, of be the counts normalized by bag size, and p be the k-th element the vector P; (the estimated proportion of class k), then the loss is defined as Lallp = K LL P.). KL refers to the commonly used Kullback -eilback-Leibler divergence for measuring the distance between distributions.
PROPOSED METHODS: TWO-STAGE FRAMEWORK
In this section, we introduce two-stage model for learning from counts. It is extended from DLLP Ardehaly & Culotta (017))
5.1
STAGE-1: ESTIMATING CLASS COUNT
5.1.1 POISSON Loss AND EXPECTED COUNT
As mentioned in Section one difference between LLP problem and LLC problem is that the label for LLC problem is discrete. it is not optimal to use KL-divergence to measure the difference between prediction and the true labels. Instead, we propose to use Poisson Loss a Poisson distri- bution based loss function. The Poisson distribution the discrete probability distribution of number of events occurring in given time period, which applies to the LLC setting if we consider the counting is the number of times class instance appears in this bag Letkowskki (2012).
In our framework, we assume that the counting for class j follows a Poisson distribution, i.e. p(c)
k(\,)) = Xie-> count for each class in a bag independent. Given a bag of output the network po (y biy), we define zi as the expected count for class k in bag b,.
kT For simplicity, since we assume the size of the bag is unknown, we assume the
NB
c
pa
(y by)
(8)
So have
E ()) ch
(9)
Then the Poisson loss is defined as the negative log likelihood function
Lreg(y. 2) log(p(ylA)) = --ylog(A) + log(v).
(10)
5

---

Under review as a conference paper at ICLR 2023
Because the last term in Equation 14 would be constant for given bag, it is usually omitted. So the final loss and its gradient is
L(y, 2) x Ā y (og((A)
(11)
VzL 2) = - y/a.
The following are two interesting properties for the Poisson loss. Adapted gradient: distance functions like mean absolute error, the Poisson loss doesn't 't have constant gradient for all input values. Also, when the actual count is large, the gradient value would be relatively smaller. This meets our intuition that when the actual count is very large, an off-by-l error matters less than the actual count is or 2. Asymmetric gradient: the gradient is zero when À = but the gradient different when Å = y and À = + 1, where the absolute gradient would be 1/(y - 1) and 1/(v + 1) respectively. The gradient tends focus on penalizing the under-estimation rather than over-estimation.
Unlike other
In addition, we introduce classification loss, Loie, and a regularizer LL ブj= No | to make our model more robust to sparse bags Tsai & Lin (2099) and ensure the inter-class separability of the learned representations Narayan et al. (2019). Lets is nothing but binary cross-entropy loss measure whether the network can classify if class is present/absent given bag. We use
unified loss function defined as
LLLC = Lreg + Lcss BLL1
(12)
5.1.2 ESTIMATING CLASS COUNT
Though we have Xy as the expected count for class j. it remains a problem to estimate the exact count for each class. Mathematically, let the of E R be the expected count for class j in bag b;,
and the t; E AK- { o E ci .B then the exact count for each class in b, could be obtained by
K log (p(c {
i: arg JEA
Σ
(13)
This constrained convex optimization and we can use any greedy algorithm to get optimal solution. Initializing &, vector of zeros, for each iteration, manually calculate the marginal gain for increasing the E to i 1. We choose to increase the count of class such that the increment maximized, and iterate until the summation of the count vector is equal to NB. By using a heap to track the maximum value, we can design algorithm with time complexity of Ollog(K) NB) as is shown in Algorithm T Appendix
5.2 STAGE-2: ESTIMATING INSTANCE LABEL (DECODER)
In Stage-1, 1, the network outputs the expected count for each class, and then we develop an opti- mization problem to translate the expected count to exact count. In stage-2, given the exact count of each class, and the predicted probability distribution of each instance po po(y(b:y) we devise an assign- ment problem to get the instance-level label. Thus, we have the following linear-sum optimization problem.
No
K log(ri)) vij
k 2ij
L,Vj 10, 1.. 1}
(14)
Σ 2 पかt vk {0, ..
p 0
2 (0,

---

Under review as a conference paper at ICLR 2023
Table 1: Generated Data Summaries
Distribution
Bag Size
lambda
of truining bags
# of testing bags
Avg. Count
Avg. (Std) Sparsity
Dataset Id
0.5
40,000
8,000
1.13
82%(41%)
p0
0.5
22,000
4,500
1.28
68%:(75))
p1
1.2
10,000
2,000
1.63
50.02%((1.553%)
p2
16
2
4,000
1,000
2.28
28.9%
p3
32
3.2
2,000
800
3.65
8.72%(88)
p4
Poisson
5,000 1.2
1,000
3.43
(832%((8%)
p5
1.2
1,000
200
3.56
9%((..%%
p6
4,000
1,000
6.52
(9%((6))
p7
2,000
500
6.32
(6.(6))%)
p8
16
2
1,000
200
6.21
(9%(661)
p9
0.67
10,000
2,000
1,7
58%((11.%)
e0
Exponential
16
0.5
4,000
1,000
2.89
42%((13.9)
cl
Uniform
N/A
10,000
2,000
1.41
(9.219))
u0
16
N/A
4,000
1,000
1.95
18.7%(9.72%))
ul
where pt (y = k)by)) ct is the estimated exact count of class ie in bag and of = 1 if and only if instance j assigned with label k.
EXPERIMENTS AND RESULTS
6.1 DATASETS
We conduct experiments based on the CIFAR-10 dataset. CIFAR-10 dataset consists 50000 train and 10000 test images with each class consisting of 5000 and 1000 training and testing images respectively. We create bags of different sizes from each of the dataset images with and without replacement in analysis. Each bag created based the size ranging from 2, 4, 6, 8, 16,32 bag size. the case the images with the replacement, we make sure that the number of images being repeated is restricted with reuse parameter which is controllable to avoid bag being over- represented by the same image. For each dataset generated with a different bag size, we ensure that the parameter configuration is set such that the maximum amount of the CIFAR-10 10 dataset both during the training and testing phase is used to represent the newly generated dataset with semi- weak labels. As CIFAR-10 dataset has a training to testing image ratio maintained as 5:1, the same ratio is maintained in different datasets generated as shown in table
We generate the dataset with the following distribution: Poisson, exponential and uniform distri- bution. This distribution is based on how the class information is distributed in each given bag. The rationale with the generation of the instances of the bag in the above format was to ensure that the bags are generated based naturally occurring instances in nature for the Poisson dis- tribution, the exponential distribution, and the uniform distribution. The generation process is the following. 1) randomly sample class i with equal probability. 2) sample number n, from the given distribution and then truncate it into the range between O and Nn. 3) sample n. instances from class i. In the case where the number of instances needed would be ber of instances, the instances would be sampled at most two times. For each parameter setting, we sample the bags with 5 different random seeds. All the experiments are conducted on these trials and the final value is obtained by averaging the best performance on the validation set. In the standard setting, to try to generate more balanced data, where each class has some instances in bag. Therefore, we selective choose hyperparameter for the distribution that minimizes the sparsity level of the bag. We define the sparsity of a bag as the number of absent class divided by the number of classes. Particularly, we use the following settings for Poisson distribution [(N 2, = 0.5), (Ng = 4, A 0.5), (Ng = 8, 1.2), (Ng = 16, À = 2.0), (N = 32, = 3.2)], and [(NB = 8, À 0.67), (Ng = 16, À 0.5)] The data summary table is provided in Table For uniform distributed samples, the directly sample NB number of samples from the original dataset and label them with the counting vector.
than the total num-
6.2 ARCHITECTURE
We use Residual Network with 18 layers as our base backbone for extracting features from the image. Given bag of images (Io, -., xN E By LxHC the embedding 6, E BB for each
7

---

Under review as a conference paper at ICLR 2023
instance is firstly extracted using Convolutional Neural Network the feature extractor. Then a linear layer fc R2 - RK is used to create a class activation map. Different pooling layer is applied at the end to generate counting prediction vector and logits for multi-label classification problem. All models are trained using Stochastic Gradient Descent with an initial learning rate of 0.01 for 100 epochs. The learning rate would be manually divided by 10 at epoch 30 and 50. Weight- decay is set to be 5e-4 for training. Standard data augmentations are utilized to avoid overfitting of CIFARIO dataset, including random crop with padding of 4 and random horizontal flip with probability of 0.5.
6.3 BASELINES
Fully-supervised Upper-bound: We want to argue that comparable results could be achieved by using semi-weak label compared to strong labels. Therefore, we also train ResNet18 in a fully supervised setting on CIFARIO. We trained it 250 epochs with initial learning rate of 0. and then divided by 10 in the mid-training. This model could be deemed as an upper-bound of the performance in semi-weak setting. For our analysis with different base network architectures, further use similar hyper-parameter
Weakly Supervised Baseline: consider the loss from counting to be absent and thus we generate the results based on bag level prediction where only presence or absence the class is available.
We produce results with the weakly supervised baseline where
Learning from Proportions: If we set the loss function as KL-loss, the our model without the decoder could be used baseline model represent the performance of modeling the learning from counts as LLP.
6.4 EVALUATION
We use precision evaluate framework for both instance-level prediction and bag-level pre- diction. For bag-level prediction, we compute the precision rate using macro averaging. For fully supervised model,
predict the class label individually and label the bag positive for class there is at lease one predicted instance for class i in this bag. Similarly, for semi-weakly supervised model, we label bag according to the instance-level predicted results.
Table provides a detailed summary of all the generated dataset configurations. We use the fol- lowing settings for the Bag size where NB E [2. 4, 8, 16, 32) and the number of training instances chosen such that we utilize most of the data available in the CIFAR-10 dataset. The average count here represents the aggregated mean of average number of instances per class. We represent the sparsity based on the percentage of classes with zero instance in bag. Our experimental results for all the dataset configurations from Table are summarized in Table 2 To remove the effect of randomness, for each dataset setting, train the model with 5 different random seeds and then average the results across all trials. The comparison between baselines are discussed in the loss ablation section A.7 A.7
Table 2: Benchmarking Results on Different Datasets using Semi-weak label
Dataset
Bag Prec.
Inc. Prec.
Dataset
Bag Prec.
Inc. Prec.
pO
94.24
92.76
p7
64.37
85.20
pl
93.78
92.65
p8
89.92
81.92
p2
93.20
91.21
p9
84.97
69.93
p3
92.92
88.07
e0
90.69
91.05
p4
93.9
71.91
el
87.10
87.65
p5
90.54
87.73
u0
94.86
91,42
p6
77.39
68.91
ul
95.62
87.34
6.5 BASELINE RESULTS
Fully-supervised baseline:Table shows the results obtained with the different classifiers con- structed as the base classifier for the classification with the CIFAR-10 dataset. We find that the bag
8

---

Under review as a conference paper at ICLR 2023
level prediction the dataset p2 is comparable to the fully--upervised dataset which is expected as the semi-weak label dataset will be bounded by the fully supervised setting.
Table 3: Fully supervised upper-bound results with classifier configuration
Classifier
Loss
Bag Prec.
Inc. Prec.
ResNet18
Poisson
94.26
94.40
ResNet34
Poisson
94.67
94.681
ResNet50
Poisson
95.2
96.41
MobileNetV2
Poisson
90.03
89.066
Searriin--rom-prooorrii baseline: For the fhaming-ffom-a crommprropotion baseline, results are shown in Table 7 (KL v.s Poisson) because we use KL-loss to proxy the baseline model of learning-free from- proportion. This baseline achieved 87% instance-level precision, which is less than our proposed model on dataset p3, i.e., 88.07%. This result also holds if we consider bag size equals for dataset p2.
Table 4: Effect of the Choices of Lreg on two standard dataset setting with No {8, 16)
Dataset ID
Lrey
Bag Prec.
Inc. Prec.
p5
KL
90.26
90.40
p5
L1
92.67
90.681
p5
Poisson
93.2
91.21
p3
KL
87.03
87.066
p3
LI
92.01
85.97
p3
Poisson
92.92
88.08
For the Searning-from www---rooo-w-abbbee baseline, we report those number in Table [5] Once we deactivate the counting loss, then the model is converted into the traditional weak label setting. Table 5 without counting loss, the performance is much worse than our proposed semi-weak labeling framework. It only achieves 87% instance-level prediction precision, which is 5% worse than our proposed framework.
Learning-from-weet laak-lbbe baseline:
Table 5: Ablation Study of Removing Regression Loss and Classification Loss. (8 {0, 1.0))
Dataset ID
Bag Prec.
Inc. Prec.
p2
93.20
91.21
p2 (w/o Lets)
93.23
91.17
p2 (w/o Liee))
90.92
87.39
p8
92.92
88.07
p8 (w/o Lota)
62.13
83,92
p8 (w/o Lreg)
41.81
72.04
CONCLUSION
In this paper, we propose a novel machine learning problem, namely, learning from counts. We propose two-stage framework to do instance-level prediction given only a counting vector for bag is available. We generated dataset from CIFARIO for experimentation. We achieve comparable results with the fully-supervised setting and much better results than the weakly supervised setting. Additionally, we introduced a L. regularization term that make our model robust to sparse bags and achieve marginally prediction improvement sparse bags. We believe semi-wor weak labels provide better insights in real-workd tasks where counting information can be easily obtained and plan to extend our work other kinds of data.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Stuart Andrews, Ioannis Tsochantaridis, and Thomas Hofmann. Support vector machines for multiple-instance learning. In Advances neural information processing systems, pp- 577-584, 2003.
Ehsan Mohammady Ardchaly and Aron Culotta. Co-training for demographic classification using deep learning from label proportions. In 20/7 IEEE International Conference on Data Mining Workshops (ICDMW), pp. 1017-1024. IEEE, 2017.
Boris Babenko, Piotr Dollár, Zhuowen Tu, and Serge Belongie. Simultaneous learning and align- ment: Multi-instance and multi-pose learning. 2008.
Gerda Bortsova, Florian Dubost, Silas Ørting, Ioannis Katramados, Laurens Hogeweg, Laura Thom- sen, Mathilde Wille, and Marleen de Bruijne. Deep learning from label proportions for emphy- sema quantification. In International Conference on Medical Image Computing and Computer- Assisted Intervention, pp. 768-776. Springer, 2018.
Marc-André Carbonneau, Veronika Cheplygina, Eric Granger, and Ghyslain Gagnon. Multiple in- stance learning: A survey of problem characteristies and applications. Pattern Recognition, 77: 329-353, 2018.
D. Crouse. On implementing 2d rectangular assignment algorithms. IEEE Transactions Aerospace and Electronic Systems, 52:1679-1696, 2016.
Thomas G Dietterich, Richard H Lathrop, and Tomás Lozano-Pérez Solving the multiple instance problem with axis-parallel rectangles. Artificial intelligence, 89(1-2)311-7, 1997.
Gary Doran and Soumya Ray. A theoretical and empirical analysis of support vector machine methods for multiple-instance classification. Machine learning, 97(1-2):79-102, 2014.
Gabriel Dulac-Amold, Neil Zeghidour. Marco Cuturi, Lucas Beyer, and Jean-Philippe Vert. Deep multi-class learning from label proportions. arXiv preprint arXiv: /2909, 2019.
Kai Fan, Hongyi Zhang, Songbai Yan, Liwei Wang, Wensheng Zhang, and Jufu Feng. Learning a generative classifier from label proportions. Neurocomputing, 139:47-55, 2014.
James Richard Foulds and Eibe Frank. review of multi-instance learning assumptions. 2010.
A. Kumar and B. Raj. Weakly supervised scalable audio content analysis. In 2016 IEEE Interna- tional Conference on Multimedia and Expo (ICME), pp. 1-6, 2016. doi: 10. 1991////...0 7552989.
Jerzy Letkowski. Applications of the poisson probability distribution. Proc. Acad. Business Res. Inst. Conf. pp. 1-11, 2012.
Oded Maron and Tomás Dozano-Pérez. framework for multiple-instance learning. Advances in neural information processing systems, 10:570-574 1997.
David Musicant, Janara M Christensen, and Jamie F Olson. Supervised learning by training on aggregate outputs. In Seventh IEEE International Conference on Data Mining (ICDM 2007), pp- 252-261. IEEE, 2007.
Sanath Narayan, Hisham Cholakkal, Fahad Shahbaz Khan, and Ling Shao. 3c-net: Category count and center loss for weakly-supervise action localization. In Proceedings of the IEEE Interna- tional Conference on Computer Vision, pp. 87799866,,,011
Nikolaos Pappas and Andrei Popescu-Belis. Explaining the stars: Weighted multiple-instance learn- ing for aspect-based sentiment analysis. In Proceedingy of the 2014 Conference on Empirical Methods In Natural Language Processing (EMNLP), pp. 455-466. 2014.
Nikolaos Pappas and Andrei Popescu-Bess Beles.. Explicit document modeling through weighted multiple-instance learnin.- Journal of Artificial Intelligence Research, 58:591-626, 2017.
10

---

Under review as a conference paper at ICLR 2023
Giorgio Patrini, Richard Nock, Paul Rivera, and Tiberio Cactano. (almost) no label cry. In Advances in Neural Information Processing Systems, pp. 190-198, 2014.
Novi Quadrianto, Alex J Smola, Tiberio S Caetano, and Quoe Le. Estimating labels from label proportions. Journal of Machine Learning Research, 10(10), 2009.
Ankit Shah, Anurag Kumar, Alexander G Hauptmann, and Bhiksha Raj. A closer look at weak label learning for audio events. arXiv preprint arXiv:/804.092880 2018.
Saravanan Subramanian, Santu Rana, Sunil Gupta, P Bagavathi Sivakumar, C Shunmuga Ve- layutham, and Svetha Venkateshe. Bayesian nonparametric multiple instance regression. In 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 3661-3666. IEEE, 2016.
Qingping Tao, Stephen Scott, NV Vinodchandran, and Thomas Takeo Osugi. Svm-based general- ized multiple-instance learning via approximate box counting. In Proceedings of the twenty-first international conference Machine learning, pp. 101, 2004a.
Qingping Tao, Stephen Scott, NV Vinodchandran, Thomas Takeo Osugi, and Brandon Mueller. An extended kernel for generalized multiple-instance learning. 16th IEEE International Confer- ence on Tools with Artificial Intelligence, pp. 272-277. IEEE, 2004b.
Kuen Han Tsai and Hsuan Tien Lin. Learning from Label Proportions with Consistency Regular- ization. arXiv, pp. 1-16, 2019, ISSN 233
Gitte Vanwinckelen, Daan Fierens, Hendrik Blockeel, et al. Instance-level accuracy versus bag-level accuracy in multi-instance learning. Data mining and knowledge discovery, 30(2):313-34 341, 2016.
Kiri Wagstaff and Terran Lane. Salience assignment for multiple-instance regression. 2007.
Felix X Yu, Dong Liu, Sanjiv Kumar, Tony Jebara, and Shih-Fu Chang. Svm for learning with label proportions. arXiv preprint arXiv:1306.0880 0886, 2013.
Felix X Yu, Liangliang Cao, Michele Merler, Noel Codella, Tao Chen, John R Smith, and Shih-Fu Chang. Modeling attributes from category- attribt proportions. In Proceedings of the 22nd ACM international conference on Multimedia, pp. 977-980, 2014.
A APPENDIX
A.1
EXAMPLE OF COUNT ASSIGNMENT OPTIMIZATION
Figure gives an example with bag size of and class size If we negate the reward, then the objective becomes minimizing the "cost", which is referred to a classical linear-sum assignment problem in bipartite graphs Crouse (2016).
CIASSe
Figure 3: An example of assignment problem. The nodes on the left represent the unsigned instance and on the right represent the label. There are instances in a bag and with class-0 instances. class-1 instance and class-2 instance. Each edge has an associated reward Pij. which is probability of instance i belonging to class j. Each instance must have one assignment and the goal to maximize the reward.
11

---

1: for 1,2, .e.peoskks do Under review as a conference paper at ICLR 2023
A.2 ALGORITHM
The following algorithm provides the greedy approach chosen to translate from the expected count the exact count.
Algorithm Greedy Algorithm to Translate Expected Count to Exact Count
Input: Number of bag size Nn; Number of classes K; A list of floating points (uo, a1,.. ax]; list probability density function for Poisson distribution [Sag (r), Jas ( Sax (x:)
Output: A list of integers [to, (1, tx] such that Σド Lj = NB-
heap < inl, ind >> be a max-heap that stores pairs. Heap has two methods: g.pop() would pop and return the max value from the heap and g.push(w, i) would add pair of < v > to the heap. The comparison of pair is based the first value in the pair;
Initialize: Let
ret = [0, 00....... ..... .0 be a zero-initialized vector of size K;
1: for i= 2....... do
2:
fa. (ret(i) + + fa, (retli))
3: g.push(r, i)
4: end for
5: for = 2.....NN. NB do
6:
U q.pop()
7:
retli] + ret
+1
8:
r fa. (ret(i) q.push(r, i)
+ fa. (retli))
9:
end for
11:
return rel.
The following section provides the algorithm for our semi-weak label classification system. In theory, the weak label classification system can work for all kinds of input data for classification whereas for the purpose of this paper we run experimental analysis on image classification task.
Algorithm 2 Semi-Weak Label Classification System
Input: batchsize number of bags [[bo. yo ), (b1, v )..., (6v, y loss weight a, and regularization weight B, number of epochs Nepochs and loss function for regression Lreg Output: predicted label for each instance [yo. y1.... y.
2: for i= 1,2, .....N do Compute the class-wise distribution for 24. 4: end for
3:
5: for 1,,,..... do Compute C₁ ΣNz No.
6:
7:
end for
8:
Compute loss, Trreg w.r.t label yg.Ca.
9:
Compute bag-level binary label using max pooling. Compute loss, Lots w.r.t label yB.
11: Compute the regularization term and compute the final loss given a, B
Update networks based on the combined loss.
13:
end for
for 2...NN do for i= 1,2, do
15:
compute the the probability distribution for x, using learned classifier. end for
17:
18:
Enumerate all combinations of counts and get the guess that maximizes the likelihood of
b
19:
Given and the probability distribution, using linear sum max algorithm to get the best assignment yo, y1, .BB
20:
end for
12

---

Under review as a conference paper at ICLR 2023
A.3 RESULTS
A.4 ABLATION STUDY FOR THE DECODER
As is shown in Table 6 instead of using greedy search, we use the proposed three-stage framework to infer the instance-level labels. Results show consistent improvement after using the decoder algorithm. More importantly, when the bag sparse, the counting for each class is high and thus the estimation would have high variance. Therefore, when the bag is less sparse, greedy search works well. But once the bag size increases or the bag becomes sparse, the high variance of the estimated logits would make the greedy solution less attractive. Therefore, the decoder contributes more to the performance of the predicted value -2% as the greedy predictions become unstable.
Table Ablation Study for the Decoder
Dataset ID
Inc. Prec.
p2
91.21
p5
87.73
p7
85.20
p2(+Decoder)
92.08
p7(+Dccder
88.92
p7tDocod)))
87.32
A.5 VARIATION WITH THE NUMBER TRAINING SAMPLES
We investigate the degree to which the size of the dataset relates to the model performance. We progressively sample different number of bags and evaluate them on the test set. We compare the results using different scales of training samples. The dataset settings illustrated in Table 7
Table 7: Scale Testing Results on two standard dataset setting with NB (8, 16)).
Dataset ID
Bag Prec.
Inc. Prec.
p2
93.2
91.21
p5
90.54
87.74
p6
77.39
68.91
p3
92.93
88.08
p8
89.92
81.92
p9
84.97
69.93
A.6 VARIATION WITH THE BATCH SIZES AND ARCHITECTRRES
We conduct experiments variants batch sizes and find different performance characteristics. For all experiments, we train the model with batch size equal to 32, 64, 96, 128 and 256 as seen in Table 8 and bag size equal to 8 and 16 the standard setting. We found that the results with the batch size doesn't 1 provide large variation in the output of the optimization for the bag level training. However, increase the batch size there is drop in performance observed on the instance level prediction.
A.7 EFFECTIVENESS OF DIFFERENT REGRESSION LOSSES
As is proposed for learning from proportions, KL loss is mostly widely used loss function for com- paring the estimated class distribution and the true class distribution. Therefore, also tried to use KL-loss as well L1 loss, which is usually used for multi instance regression as alternative choices.
As is summarized in Table 7 poisson loss function performs better than other loss functions for bag size equal to 8 and 16, in terms of bag level prediction as well as instance-level prediction. This as expected as poisson distribution is naturally defined for non-negative and integer values like
13

---

Under review as a conference paper at ICLR 2023
Table 8: Ablation Study of Using different backbones and different batch sizes dataset p2 Bg
8.
Backbone
Bag Prec.
Inc. Prec.
Resnet (bs=32)
94,036
92.096
Resnet18 (bs=64)
93.20
91.21
Resnet18 88 (bs=96)
93.630
91.944
Resnet18 (bs=128)
93.2
91.23
Resnet18 8 (bs=192)
92.661
90.649
Resnet18 (bs=256)
92.541
90.183
Resnet34 (bs=128)
93.763
91.763
Resnet50 (bs=128)
93.512
91,893
MobileNetV2 (bs=128)
91.474
89.002
counting, Another explanation is that the data is generated from poisson distribution but according to Table 9 even though the dataset is generated from non non-poisson distribution, the poisson loss function still achieve comparable values. This implies that poisson distribution is robust.
A.8 EFFECT OF REGULARIZER WITH SPARSE BAGS
In order to analyze the model performance on bags with different sparsity, we further create bags with higher expected counts for each class generate bags with higher sparsity. The specific setting and data statistics are summarized in Table
Interestingly, we found that the regularized term works really well for sparse bags. It consistently improve the performance thought marginally.
Table 9: Effect of the Regularizer on Sparse Bags (Dataset ID=p7)
3
Bag Prec.
Inc. Prec.
0.0
63.7
84.67
0,01
64.37
85.20
0.1
65,09
85.52
0.5
65.07
85.87
A.9 PERFORMANCES OVER DIFFERENT BAG SIZES
As in table 1 have different bag sizes generated with variation of the parameters lambda and number of training bags to create new copies of dataset with different bag size. Table 2 shows that we increase the bag size the performance remains fairly increasing upto bag size of 8 and then there is a gradual decrease in performance observed with large drop seen with the bag size of 32. Such variation can be attributed to the fact that with the larger bag size the nature of the distribution of the bags to the instance level information per class is artificial and there is a consistent decrease the performance as we increase bag size from 16 32 and beyond.
A. COMPARISON WITH DIFFERENT DISTRIBUTION
Besides generating data from poisson distribution, also train our models with the exponential distribution as well as the uniform distribution. We observe that the results with the poisson distri- bution is better compared the exponential distribution which is in accordance with expectation as the count information in case of the exponential distribution will be sparse than case of the pois- son distribution. The analysis can be reviewed the table case the dataset id e0 and p2 there is a stark difference with the same bag size and sparsity distribution but difference in the distribution of classes in each bag results performance drop from 93. 2% precision 90.69%% precision.
14


Reviews:
[{"review_id": "zDngC06G29m", "reviewer": "Reviewer_CGT7", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors propose a classification method that uses semi-weak labels. In this setting, the authors assume that the number of instances in each class is known in addition to the presence or absence of a given class. To utilize this information, the authors propose a loss function utilizing Poisson distribution. The method is built based on an existing method called DLLP. The authors test their method on Cifar-10 dataset and report comparable results to the method that uses full supervised label information.", "summary_of_the_review": "In general, the paper is written well. However, motivation for using Poisson distribution must be supported with theoretical arguments. In addition, the authors must conduct tests on more realistic and challenging datasets such MS-COCO.", "strength_and_weaknesses": "The main strengths of the paper can be summarized as follows:\ni) In general the paper is written well and the classification setting and the motivation are described in details.\nii) The proposed method seems novel.\niii) Good accuracies are reported on a simple dataset.\nThe main weaknesses of the paper can be summarized as follows:\ni) Some of the used terms and statements should be corrected in the manuscript. The authors consider as a weak label when a label is given to an image yet they assume a label is strong if the bounding box information is given.  This is not correct. If the main goal is just image classification rather than visual object detection, a label associated to an image is enough and that is all we need. Therefore, it is also strong supervision. This kind of labeling can be considered as weak if the goal is visual object detection rather than classification since bounding box information is also necessary for this problem. These issues must be corrected.\nii) The authors use Poisson distribution in order to estimate the number of samples in classes. As written in the text, Poisson distribution is used to estimate the number of events occurring in a given time period, and basically 3 important conditions must be satisfied to use this distribution. One of the is the count of each class must be independent and the authors assume that is satisfied. What about other conditions and I wonder why this distribution is the best choice for his problem. Why not to use exponential distribution as in other methods (Quadrianto etal, 2009; Patrini et al. 2014)? More arguments must be given to support this claim.\niii) In the experimental setting, the authors use both discrete random variable distribution (passion) as well as continuous random variable distributions (exponential and uniform distributions) to generate the dataset splits. But, they argued that continuous distributions cannot be used for this goal. Please comment on this.\niv) The major limitation of the paper is the weakness of the experiments. The authors conduct tests only on a simple dataset Cifar-10. The images in this dataset have only one label which is far from real life. In fact, real-world images have multiple labels (i.e., many object may exist in images, for example there may be people, cars, buildings etc. in a given image). Therefore, instead of this simple datasets, it is better to conduct tests on more realistic datasets such as COCO, NUS-WIDE etc. In that case, more reliable results can obtained. \nv) As a last comment, the improvement over the methods using label proportions is very minor, only 1.07%. This may not have any significancy and the performance difference may due to the chance factors.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "nESFMXUTzb0", "reviewer": "Reviewer_vbs2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a method to train a classification model in a new weakly supervised framework. Similarly to the scenario of label proportions, the authors impose the assumption that a bag of instance samples is equipped only with class-frequency counts without annotating each instance. The information of label counts is then leveraged to formulate an objective loss function by means of Poisson loss. In the experiments, the authors build a benchmark dataset based on Cifar-10 by following the weakly-supervised setting, and empirically evaluate the proposed method.", "summary_of_the_review": "Due to lack of clarity and technical novelty, this paper leans toward rejection.", "strength_and_weaknesses": "## Strength\n+ Weak annotation of label counts is effectively exploited in the Poisson model to build a loss function.\n+ The authors construct a weakly annotated dataset by assigning class counts to a bag of image samples drawn from Cifar-10.\n\n## Weakness\n### Limited technical novelty.\n- Simply introducing a well-known Poisson loss in Eq.(10) lacks novelty, compared to the prior works of label proportion [Dulac-Arnold+19, Ardehaly+17].\nIn formulating the loss, it is hard to understand the sentence of \"since we assume the size of the bag is unknown, we assume the count for each class in a bag is independent\".\nIn this weakly-supervised setting using label counts, the size of bag is accessible and in particular is constant in the experiments of Sec.5.\nThus, there lacks an analysis or discussion to explain why such an independence assumption is valid.\nAs to the probabilistic formulation for label counts/proportions, an efficient approach is also proposed for small-sized bags in [R1].\n\n[R1] Denis Barucic and Jan Kybic. FAST LEARNING FROM LABEL PROPORTIONS WITH SMALL BAGS. ICIP2022.\n\n- The gradient in Eq.(11) would be unstable for \\hat{\\lambda} ~ 0. How do the authors cope with the issue? It would be necessary to analyze the stability of training based on the loss.\n\n- L1-regularization for the posterior f(x_i) in Eq.(6) is unclear since it seems to make no sense in the loss. The posterior p(y|x) is by definition subject to a unit-sum constraint of \\sum_y p(y|x), and thus its L1-norm always results in 1, which does not regularize the training at all.\n\n- It is difficult to recognize the efficacy of the post-processing to finally predict bag-level/instance-level label in Secs.5.1.2&5.2.\nHow well does it actually work in comparison to the simplest approach of argmax over the instance logits?\n\n- Related to the post-processing, the second constraint in Eq.(14), \\sum_j x_{ij}^k = \\bar{c}_i^k, is unclear. \n\\bar{c}_i^k is defined as real-value prediction while \\sum_j x_{ij}\\k is a binary-value (one-hot) variable to indicate a class label, thereby both of which are not mathematically compatible in the equality.\n\n### Poor presentation.\n* Mathematical forms are poorly presented in this paper.\nFor example, $\\bigoplus$ usually stands for \"direct-sum\" but is used as just a summation, which is quite confusing; the authors should use a standard $\\sum$, following a mathematical convention. \nBesides, mathematical notations are employed in an inconsistent and confusing way across Secs.3~5.  Such a messy mathematical presentation makes it quite hard to follow the technical content.\n\n* The experiments are organized in an unclear way.\nWhile methods are evaluated on image classification tasks, the authors rarely mention the task in Secs.1~5 except for the \"title\"; it is even apart from the task in Fig.1 that the authors enthusiastically discussed. So, the motivation for the image classification tasks is unclear.\nFor constructing benchmark datasets, bags are sampled in a confusing fashion. In the previous works such as [Dulac-Arnold+19], the same type of dataset is built on the basis of Cifar-10 and so the authors should follow the experimental protocol, which also makes it possible to fairly compare the method with the previous approach.\nOtherwise, one could have a doubt that such a confusing data construction based on Poisson prior is designed for bringing an unfair bias toward the proposed method.\n\n* In experimental evaluation, the comparison methods are unclearly described. For example, in Table 3, what does the loss of \"Poisson\" mean? To train a fully supervised model using instance-level annotation, it is a common way to apply the Softmax cross-entropy loss in this Cifar-10 task.\n\n* Unclear effectiveness of the proposed counting-based weak supervision.\nThis paper lacks sufficiently convincing stuffs to show the efficacy of the weakly-supervised classification tasks based on label counts that the authors present in this paper.\nThe story about acoustic event detection depicted by Fig. 1 and its related discussion seems to be convincing.\nBut, the technical part regarding the proposed method in Secs.3~5 is far from the detection task since the method is built upon the assumption that \"sum of counts per category in each bag must equal the bag size\" (page 2).\nThe acoustic detection task goes beyond the assumption and thus the method is inapplicable to the task.\nPractical validity and feasibility of the assumption is not fully demonstrated in this paper; practically speaking, when we count class labels in a bag, we would have to identify a class category of each instance, being equivalent to demanding instance-level supervision.\nSuch an inconsistent discussion makes this paper difficult to follow.\nBesides, without demonstrating the proposed weakly-supervised task in a feasible and convincing way, it is also difficult to find the meaningful benefit to address the task.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "kiCGKh1ks3D", "reviewer": "Reviewer_b2Pz", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper considers a special case of multiclass classification problem. Here, bag level labels and counts of each class are available. The paper proposes a two-stage algorithm to solve the new defined problem.", "summary_of_the_review": "I recommend to reject as there are many aspects of the paper can be improved, such as the motivation, the presentation of the proposed algorithm and the organization of the paper.", "strength_and_weaknesses": "Strength\n- This paper draws attention on considering new problem settings for multiclass classification.\n\nWeakness\n- I could not understand the motivation of the problem setting. The authors give an example of annotating multiple dogs in an image. However, considerting image classification, an image containg multiple dogs is the instance level data, then what are bags in this case? Concrete examples should be give to motivate the problem setting.\n- Knowing exact numbers of instances for each class seems like a very strong annotation. Are there actual cases that users know exact numbers but still not knowing labels for each instance?\n- Sections 3.1, proposing a loss function, seems to be off the story and not related to the later part of the paper.\n- The part describing the two-stage algorithm is hard to follow. I could not see a clear story to know what to do next.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject