Weakly Supervised Clustering by Exploiting Unique Class Count
Authors: Mustafa Umit Oner
Hwee Kuan Lee
Wing-Kin Sung
Venue:  (2020)
Keywords: weakly supervised clustering
weakly supervised learning
multiple instance learning

Abstract: A weakly supervised learning based clustering framework is proposed in this paper. As the core of this framework, we introduce a novel multiple instance learning task based on a bag level label called unique class count (ucc), which is the number of unique classes among all instances inside the bag. In this task, no annotations on individual instances inside the bag are needed during training of the models. We mathematically prove that with a perfect ucc classifier, perfect clustering of individual instances inside the bags is possible even when no annotations on individual instances are given during training. We have constructed a neural network based ucc classifier and experimentally shown that the clustering performance of our framework with our weakly supervised ucc classifier is comparable to that of fully supervised learning models where labels for all instances are known. Furthermore, we have tested the applicability of our framework to a real world task of semantic segmentation of breast cancer metastases in histological lymph node sections and shown that the performance of our weakly supervised framework is comparable to the performance of a fully supervised Unet model.

Published as a conference paper at ICLR 2020
WEAKLY SUPERVISED CLUSTERING BY EXPLOITING UNIQUE CLASS COUNT
Mustafa Umit Oner Hwee Kuan 2.23,, & Wing-Kin Sung!.s 1,5
School of Computing. National University of Singapore, Singapore 117417, *A*STAR Bioinformatics Institute, Singapore 138671, "Image and Pervasive Access Lab (IPAL),
CNRS UMI 2955, Singapore 138632, "Singapore Eye Research Institute, Singapore 169856, A STAR Genome Institute of Singapore, Singapore 138672 (umitoner, ksung)&comp edu sg. {Leehk] a-star.....r... sg
ABSTRACT
A weakly supervised learning based clustering framework is proposed in this pa- per. As the core of this framework, we introduce a novel multiple instance learning task based on bag level label called unique class count (uce), which is the number of unique classes among all instances inside the bag. In this task, no annotations on individual instances inside the bag are needed during training of the models. We mathematically prove that with a perfect uce classifier, perfect clustering of individual instances inside the bags is possible even when no annotations on indi- vidual instances are given during training. We have constructed neural network based uce classifier and experimentally shown that the clustering performance of our framework with our weakly supervised uce classifier is comparable to that of fully supervised learning models where labels for all instances are known. Fur- thermore, have tested the applicability of our framework to real world task of semantic segmentation of breast cancer metastases in histological lymph node sections and shown that the performance of our weakly supervised framework is comparable to the performance of a fully supervised Unet model.
INTRODUCTION
In machine learning, there are two main learning tasks on two ends of scale bar: unsupervised learning and supervised learning. Generally, performance of supervised models better than that of unsupervised models since the mapping between data and associated labels is provided explicitly supervised learning. This performance advantage of supervised learning requires a lot of labelled data, which is expensive. Any other learning tasks reside in between these two tasks, performances. Weakly supervised learning is an example of such tasks. There are three types of supervision in weakly supervised learning: incomplete, inexact and inaccurate supervision. Multiple instance learning (MIL.) is a special type of weakly supervised learning and a typical example of inexact supervision (Zhou] (017). In MIL, data consists of bags of instances and their corresponding bag level labels. Although the labels are somehow related to instances inside the bags, the instances are not explicitly labeled. In traditional MIL, given the bags and corresponding bag level labels, task is to learn the mapping between bags and labels while the goal is to predict labels of unseen bags (Dietterich et al. 1997 Foulds & Frank 2010).
their
In this paper, we explore the feasibility of finding out labels of individual instances inside the bags only given the bag level labels, i.e. there no individual instance level labels. One important application of this task is semantic segmentation of breast cancer metastases in histological lymph node sections, which is a crucial step in staging of breast cancer (Brierley et al.) 2016).. In this task, each pathology image of a lymph node section is bag and each pixel inside that image is an instance. Then, given the bag level label that whether the image contains metastases or not, the task is to label each pixel as either metastases normal. This task can be achieved by asking experts to exhaustively annotate each metastases region in each image. However, this exhaustive annotation process is tedious, time consuming and more importantly not part of clinical workflow.

---

Published as a conference paper at ICLR 2020
Input is bag containing n instances of different classes of objects, i.c. a bag with uce-k. Individual instance labels are unknown.
Given bags corresponding bag level ucc labels, UCC model learns predict ucc of unseen bags.
Output is predicted aee label. UCC model tries minimize loss between truth ucc labels and predicted ucc labels during training.
? ?
? ?
?
?
UCC
RCCC
?
?
?
?
?
Trained UCC model is used a feature extractor and unsupervised clustering is performed extracted features of individual instances inside the bag.
6 6
8 8
0
0 O
8
Weakly supervised clustering framework. Our framework (green dashed line) consists of the UCC model (magenta dashed line) and the unsupervised instance clustering branch.
Figure 1:
In many complex systems, such in many types of cancers, measurements can only be obtained at coarse level (bag level), but information at fine level (individual instance level) is of paramount importance. To achieve this, we propose weakly supervised learning based clustering framework. Given a dataset consisting of instances with unknown labels, our ultimate objective is to cluster the instances in this dataset. To achieve this objective, we introduce novel MIL task based on new kind of bag level label called unique class count (uce), which is the number of unique classes or the number of clusters among all the instances inside the bag. We organize the dataset into non-empty bags, where each bag is subset of individual instances from this dataset. Each bag is associated with a bag level uce label. Then, our MIL task to learn mapping between the bags and their associated bag level ucc labels and then to predict the ucc labels of unseen bags. We mathematically show that a uce classifier trained on this task can be used to perform unsupervised clustering on individual instances in the dataset. Intuitively, for a uec classifier to count the number of unique classes a bag, it has to first learn discriminant features for underlying classes. Then, it can group the features obtained from the bag and count the number of groups, the number of unique classes.
Our weakly supervised clustering framework is illustrated in Figure [ It consists of a neural net- work based ucc classifier, which is called as Unique Class Count (UCC) model, and unsuper- vised clustering branch. The UCC model accepts any bag of instances as input and uses uce labels feature extractor and unsuper-
for supervised training. Then, the trained UCC model is used vised clustering is performed on the extracted features of individual instances inside the bags in the clustering branch. One application of our framework is the semantic segmentation of breast can- cer metastases in lymph node sections (see Figure 4 The problem can be formulated as follows. The imput is a of images. Each image (bag) has a label of uecl (image fully normal or fully metastases) or ucc2 (image is a mixture of normal and metastases). Our aim is to segment the pixels (instances) in the image into normal and metastases. UCC model can be trained to predict uce labels of individual images in fully supervised manner, and the trained model can be used to ex- tract features of pixels (intances) inside the images (bags). Then, semantic segmentation masks can be obtained by unsupervised clustering of the pixels (each is represented by the extracted features) into two clusters (metastases or normal). Note that uce does not directly provide an exact label for each individual instance. Therefore, our framework is weakly supervised clustering framework.
Finally, we have constructed uoe classifiers and experimentally shown that clustering performance of our framework with our uce classifiers is better than the performance of unsupervised models and comparable to performance of fully supervised learning models. We have also tested the per- formance of our model the real world task of semantic segmentation of breast cancer metastases in lymph node sections. We have compared the performance of our model with the performance of popular medical image segmentation architecture of Une (Ronneberger al. (0055) and shown that our weakly supervised model approximates the performance of fully supervised Unee mode i
Hence, there are three main contributions of this paper:
1. We have defined unique class count as bag level label in MII. setup and mathematically proved that a perfect uce classifier, in principle, can be used to perfectly cluster the indi- vidual instances inside the bags.
Code and trained models: http:://bttttt./ ly/uni sus.s/scounucccsssuuuu
2

---

Published as a conference paper at ICLR 2020
2. We have constructed neural network based uce classifier by incorporating kernel density estimation (KDE) (Parzen 1962) as a layer into our model architecture, which provided us with end-to-end training capability.
3. We have experimentally shown that clustering performance of our framework is better than the performance of unsupervised models and comparable to performance of fully super- vised learning models.
The rest of the paper is organized such that related work is in Section 2 details of our weakly supervised clustering framework are in Section 3 results of the experiments on MNIST, CIFARIO and CIFARI00 datasets are in Section 4 results of the experiments in semantic segmentation of breast cancer metastases are in Section 5 and Section concludes the paper.
2 RELATED WORK
This work is partly related to MIL which was first introduced in (Dietterich et al., 1997) for drug activity prediction. Different types of MIL were derived with different assumptions (Gärtner al.
2002
Zhang & Goldman 2002 Chen et al. 2006 Foulds 20088 Zhang & Zhou 2009|/Zhon et al,
2009
which are reviewed in detail in (Foulds & Frank 2010), and they were used for many different
applications such as, image anmolaaioo/categooizzatiooooneeo
(Chen & Wang 2004 Zhang et al.,
2002
Tang et al.] 20100), text categorization (Andrews et al.) 2003] Settles et 2008 ) spam detec-
tion Jorgensen et al. (2008), medical diagnosis (Dundar et (007), face/object detection (Zhang
al. Felensszzze et al.) 2010) and object tracking (Babenko et al.) 2011).
In MIL, different types of pooling layers are used to combine extracted features of instances inside the bags, such as max-pooling and log-sum-exp pooling (Ramon & De Raedt 2000 Zhou Zhang 2002 Wu et al.] [015 Wang et al.] (018). On the other hand, our UCC model uses KDE layer in order to estimate the distribution of extracted features. The advantage of KDE over pooling layers is that it embeds the instance level features into distribution space rather than summarizing them.
There are also methods modeling cardinality and set distributions (Liu et al. 2015; Brukhim & Globerson 2018 Kipf et al. 2018). However, cardinality of set and ucc completely different from each other. It is also important to state that ucc is obviously different from object/crowd count- ing (Idrees al. 2013 Arteta al., 2014 Zhang al. 2013 2016) since the task in object/crowd counting is to count the instances of the same type of object or people.
Lastly, we compare clustering accuracies of our models with clustering accuracies of unsupervised baseline models: K-means (Wang et al., 2013) and Spectral Clustering (Zelnik-Manor & Perona 2005); state of the art unsupervised models: JULE (Yang al., 2016), GMVAE (Dilokthanakul et al. 20) DAC (Chang et al. 2017). DEPICT (Ghasedi Dizaji et al. (017)) and DEC (Xie et al. 2016 and state of the art semi-supervised models: AAE (Makhzani et 2015), CatGAN (Sprin-
genberg 2013), LN (Rasmus et al. 2013) and ADGM (Maaløe et al. 2016
3 WEAKLY SUPERVISED CLUSTERING FRAMEWORK
In this section, we state our machine learning objective and formally define our novel MIL task, which is the core of our weakly supervised clustering framework. Finally, we explain details of the two main components of our framework, namely UCC model and unsupervised clustering branch.
Objective: Let X [21, , Fn be dataset such that each instance X, to a class, but its label is unknown. In this paper, we assume that total number of classes is known. Hence, each instance x; is endowed with an underlying. but unkown, label L(x;) 4 e ... KK. Further assume that for each class k E 21,2,.. ... K), there exist at least one element I; E X' such that L(x,) = 4 k. Our eventual objective is to derive a predicted class label i, for each instance I, that tends towards underlying truth class 4, i.c. i, + L(x+) 4-
X' belongs
3.1 A NOVEL MIL TASK
In this novel MIL task, unique class count is used as an inexact, weak, bag level label and is defined in Definition I Assume that are given subsets oe C X, < = 1,2, ... N and unique class counts
3

---

Published as a conference paper at ICLR 2020
Na; Vog. Hence, MIL dataset is D {(11 (o.)) ).... (oN Nox)). ). Then, our MIL task is to learn the mapping between the bags and their associated bag level ucc labels while the goal is to predict the ucc labels of unseen bags.
Definition Given subset oe C. X, unique class count, aa defined as the number of unique classes that all instances in the subset og belong i.e. noc = [z) E octl. Recall that each instance belongs to an undertying unknown class.
Given a dataset D, our eventual objective is to assign a label to each instance 2; E X' such that assigned labels and underlying unknown classes are consistent. To achieve this eventual objective, a deep learning model is designed such that the following intermediate objectives can be achieved while is being trained on our MIL task:
1. Unique class count: Given an unseen set oc. the deep learning model, which trained on D, can predict its unique class count Ta; correctly.
2. Labels on sets: Let ofure and of be two disjoint pure sets (Definition such that while all instances in of pure belong to one underlying class, all instances in og belong to another class. Given ofure and ofure, the deep learning model should enable us to develop unsupervised learning model to label instances in oeure and of ofure belonging to different classes. Note that the underlying classes for instances in the sets unknown.
3. Labels on instances:
Given individual instances E X, the deep learning model should enable us assign label to each individual instance 2i such that instances with dif- ferent/same underlying unknown classes are assigned different/same labels. This is the eventual unsupervised learning objective.
Definition 2 A set a is called a pure set if its unique class count equals one. All pure sets is denoted by the symbol uue in this paper.
3.2 UNIQUE CLASS COUNT MODEL.
In order to achieve the stated objectives, we have designed deep learning based Unique Class Count (UCC) model. Our U C'C model consists of three neural network modules (0temnres 0ams a and can be trained end-to-end. The first module 0 extracts features from individual instances; then distributions of features are constructed from extracted features. The second module 6am is used to predict uce label from these distributions. The last module Ousener is used construct autoencoder together with Oremmee so as to improve the extracted features by ensuring that extracted features contain semantic information for reconstruction.
Formally, for 34 E oç, i = 2,, ..... locl). feature extractor module 0 extracts J features (fad : f2.i
f.de 0 (x,) for each instance x; E σç. As a short hand, we write the operator reamue as operating element wise on the set to generate a feature matrix 0 = faa with matrix elements fi.à E IR, representing the jth feature of the 7th instance. After obtaining features for all instances Øe. kernel density estimation (KDE) module used to accumulate feature distribu- tions ho: (h (v), h2g (v), n/ (v)). Then, hog is used input to distribution regression module 6mm to predict the uce label, Mr. = Oan (hrr) as a softmax vector (ii : - Con- currently, decoder module Oueetter in autoencoder branch used to reconstruct the input images from the extracted features in unsupervised fashion, Z; Osscater Ofsaare
Hence, UCC model,
main modules of which are illustrated Figure [(a), optimizes two losses concurrently: 'uce loss' and 'autoencoder loss". While 'uce loss' is cross-entropy loss, 'autoencoder loss' is mean square error loss. Loss for one bag is given in Equation] I
1 loel
T
k log noe
+ +(1 -
(x; 22)²
where [0,
(I)
|aşl
A=1
uoc loss
autoenender lass
4

---

Published as a conference paper at ICLR 2020
(a)
decoder
autoencoder loss
cl,i Jog
Lart (1
12 E 0g0
feature
KDE
Odrn
M
Saz
alm kd(1)
loss
(b)
I =
-u 1-1 -
0 Jeature
Sa
Clustering
0: = (x11,, E 22,,=
Weakly supervised clustering framework. (a) UCC model: 0 extracts features, shown in colored nodes. KDE module obtains feature distribution for each feature. Then, am pre- dicts the uce label N Concurrently, decoder module Aaeaeerr in autoencoder branch reconstructs the input images from the extracted features. (b) Unsupervised clustering: Trained feature extractor, Oceamee is used to extract the features of all instances X' and unsupervised clustering is performed
Figure 2:
on extracted features. Note that 4 is clustering label of æ; E X.
3.2. KERNEL DENSITY ESTIMATION MODULE
In UCC model, input is set og and output is corresponding uce label foc. which does not de- pend on permutation of the instances in occ KDE module provides UCC model with permutation- invariant property. Moreover, KDE module uses the Gaussian kernel and differentiable, so our model can be trained end-to-end (Appendix AA). KDE module also enables our theoretical analysis thanks to its decomposability property (Appendix BB. Lastly, KDE module estimates the probability distribution of extracted features and enables 0am to fully utilize the information in the shape of the distribution rather than looking at point estimates of distribution obtained by other types of pooling layers (Ramon & De Raedt 2000 Zhou & Zhang
2002 Wang al., 2018) (Appendix C.6).
3.2.2 PROPERTIES OF UNIQUE CLASS COUNT MODEL
This section mathematically proves that the UCC model guarantees, in principle, to achieve the stated intermediate objectives Section 3.1 Proof propositions are given in Appendix
Proposition 1
Let og, og be disjoint subsets of X with predicted unique class counts Noç = Nog 1.
the predicted unique class count of oç Ug is no, 2, then boş * hog
Definition 3 A perfect unique class count classifier takes in any set a and output the correct pre- dicted unique class count 7r V/o-
Proposition 2 Given a perfect unique class count classifier. The dataset X can be perfectly clus- = 1,2, ,K, such that X = - of and of pure = {xz1a1 E
tered into K subsets og pure X. ((x,) E
Proposition 3 Given a perfect unique clasy count classifier. Decompose the dataset X into K subsets of pure ; 1,... ...... K, such that of pure (N./2 X, [(xx) E). Then, hagure + hagure for
Suppose we have perfect uee classifier. For any two pure sets qure and of which consist of instances two different underlying classes, uce labels must be predicted correctly by the perfect uce classifier. Hence, the conditions of Proposition [ are satisfied, so we have hagure * hngure. Therefore, we can, in principle, perform an unsupervised clustering on the distributions the sets without knowing the underlying truth classes of the instances. Hence, the perfect wee classifier en- ables us to achieve our intermediate objective of "Labels on sets". Furthermore, given perfect uce classifier, Proposition 2 states that by performing predictions of ucc labels alone, without any
5

---

Published as a conference paper at ICLR 2020
knowledge of underlying truth classes for instances, can in principle perform perfect cluster- ing for individual instances. Hence, a perfect uce classifier enables us to achieve our intermediate objective of "Labels on instances".
3.3 UNSUPERVISED INSTANCE CLUSTERING
In order to achieve our ultimate objective of developing an unsupervised learning model for cluster- ing all the instances dataset X, we add this unsupervised clustering branch into our framework. Theoreticallly, we have shown in Proposition 3 that given a perfect uce classifier, distributions of pure subsets of instances coming from different underlying classes are different.
In practice, it may not be always possible (probably most of the times) to train a perfect uce classifier, so we try to approximate it. First of all, we train our ucc classifier on our novel MIL task and save our trained model (Oteauure 0am Odconder). Then, we use trained feature extractor O to obtain feature matrix fx = sauure (X)). Finally, extracted features are clustered in an unsupervised fashion, by using simple k-means and spectral clustering methods. Figure (bb) illustrates the unsupervised clustering process in our framework. A good feature extractor Oremure is of paramount importance in this task. Relatively poor Oggane may result in a poor unsupervised clustering performance in practice even if we have strong 9am. To obtain a strong Oseauuee we employ an autoencoder branch, so as to achieve high clustering performance in our unsupervised instance clustering task. The autoencoder branch ensures that features extracted by f contain semantic information for reconstruction.
4 EXPERIMENTS ON MNIST AND CIFAR DATASETS
This section analyzes the performances of our UCC models and fully supervised models in terms of our eventual objective of unsupervised instance clustering on MNIST (10 clusters) (LeCun al.) 1998), CIFAR10 (10 clusters) and CIFAR 100 (20 clusters) datasets (Krizhevsky & Hinton] 2009).
4.1 MODEL ARCHITECTURES AND DATASETS
To analyze different characteristics of our framework, different kinds of unique class count models were trained during our experiments: UCC, UCC2+, UCC,.1 and UCC These unique class count models took sets of instances as inputs and were trained on uee labels. While UCC and UCC2+ models had autoencoder branch in their architecture and they were optimized jointly over both autoencoder loss and ucc loss, UCC and UCC2t a= models did not have autoencoder branch in their architecture and they were optimized over uce loss only (i.e. a = 1 in Equation D The aim of training unique class count models with and without autoencoder branch was to show the effect of autoencoder branch in the robustness of clustering performance with respect to ucc classification performance. UCC and UCC,=1 models were trained on bags with labels of uee1 to uce4. On the other hand, UCC2+ and UCC2+ a=1 models were trained on bags with labels uee2 to uce4. Our models were trained on uce labels up to ucc4 instead of ucc10 (ucc20 in CIFAR100) since the performance was almost the same for both cases and training with ucel to uco4 was much faster (Appendix C.2 Please note that for perfect clustering of instances inside the bags, it is enough to have a perfect uce classifier that can perfectly discriminate uce1 and ucc2 bags from Proposition [ The aim of traininig UCC2+ and UCC2+ models was experimentally check whether these models can perform as good as UCC and U'C.== models even if there is no pure subsets during training. In addition our unique class count models, for benchmarking purposes, we also trained fully supervised models, SallySupervised, and unsupervised autoencoder models, Autoencoder. F Superviid models took individual instances as inputs and used instance level ground truths as labels during training. On the other hand, Auloencoder models were trained in an unsupervised manner by optimizing autoencoder loss (i.e. a = 0 in Equation T important to note that all models for dataset shared the same architecture for feature extractor module and all the modules in our models are fine tuned for optimum performance and training time as explained in Appendix C
We trained and tested our models on MNIST, CIFARIO and CIFAR 100 datasets. We have Xmnistttrs Vanistt and Xmansst,tet for MNIST; Xeifar10, Neifarlo val and Xeifar10,teed for CIFAR 10; and Xes Ceifar100.trs Keifar100, vat and X Reifar100,te for CIFAR100. Note that tr, val and test subscripts stand for 'training'. "validation" and 'test' sets, respectively. All the results presented in this paper were obtained on hold-out test sets X Xmmist, X Neifar10,test and 101 F SuperSupee

---

Published as a conference paper at ICLR 2020
Table Minimum inter-class JS divergence values, ucc classificatton accuracy values and clustering accuracy values of our models (first part), baseline and state of the art unsupervised models (second part) and state of the art semi-supervised models (third part) on different test datasets. The best clustering accuracy values for each kind of models (weakly supervised (our models), unsupervised. semi-suppervsed) are highlighted in bold. ('x': not applicable, ' missing')
min. JS divergence
uCC acc.
clustering acc.
mnist cifar10
cifar100
mnist
cifar10
cifar 100
mnist
cifarI0
cifar100
UCC
0.222
0.097
0.004
1.000
0.972
0.824
0.984
0.781
0.338
UCC
0.251
0.005
0.002
1.000
0.936
0.814
0.984
0.545
0.278
UCC.=1
0.221
0.127
0.003
1.000
0.982
0.855
0.981
0.774
0.317
UCC2+
0.023
0.002
0.003
0.996
0.920
0.837
0.881
0.521
0.284
Autoencoder
0.101
0.004
0.002
X
0.930
0.241
0.167
FullySSpeervise
0.283
0.065
0.019
X
X
0.988
0.833
0.563
JULE (Yang et al.
2016)
0.964
0.272
0.137
GMVAE (Dilokthankkul et al. 2016)
0.885
DAC (Chang et al. (2017)
0.978
0.522
0.238
DEC (Xie et al. 2016))
0.843
0.301
0.185
DEPICT (Ghssedi Dizaji et al. 2017)*
0,965
Spectral Mann & Perona 2005
0.696
0.247
0.136
K-means WWang et al. 2015)
0.572
0.229
0.130
ADGM (Maalge al. 2016)
0.990
Ladder Networks (Rasmus et al.
2015
0.989
0.796
AAE (Makhzani et al. (015))
0,981
CaIGAN (Springenberg) 2013)
0.981
0.804
"ooels do not separate training and testing data, i.c. their results are not on hold-out test sets.
models took individual instances as inputs and were trained on instance level ground truths. Unique class count models took sets of instances inputs, which were sampled from the power sets 2mmntttt. 2- and a and were trained on ucc labels (Appendix C.2). While all the models were trained in supervised setup, either ucc labels instance level ground truths, all of them were used to extract features for unsupervised clustering of individual instances.
4.2 UNIQUE CLASS COUNT PREDICTION
Preceeding sections showed, in theory, that a perfect uce classifier can perform 'weakly' super- vised clustering perfectly. We evaluate uoe prediction accuracy of our unique class count models in accordance with our first intermediate objective that unique class count models should predict woe labels of unseen subsets correctly. We randomly sampled subsets for each uec label from the power sets of test sets and predicted the ucc labels by using trained models. Then, we calculated the uce prediction accuracies by using predicted and truth uce labels, which are summarized in Table I] (Appendix C.3). We observed that as the task becomes harder (from MNIST to CIFAR100), also classifier. Moreover, UCC and UCCa=1 models,
becomes harder to approximate the perfect in general, have higher scores than their counterpart models of UCC and UCC2+ a=1, which is ex- pected since the ucc prediction task becomes easier at the absence of pure sets and models reach to early stopping condition (Appendix C.T) more easily. This is also supported by annother interesting, yet reasonable, observation that UC models have higher uce accuracies than UCC2t, models thanks to the autoencoder branch which makes 2C harder to reach to early stopping condition.
4.3 LABELS ON SETS
Lensen-Shannal (JS) divergence Lin 1991) value between feature distributions of two pure sets consisting of instances of two different underlying classes is defined as inter-class JS divergence in this paper and used for comparison on 'Labels on sets' objective of assigning labels to pure sets. Higher values of inter-class IS divergence are desired since it means that feature distributions of
7

---

Published as a conference paper at ICLR 2020
1.00
0.90
K-mnaan Spectral
K-menns
K-menn
0.90
clustering ac ocraay
Spectral
clustering accuracy
0.30
Specoml
0.80
UCC 11
0.00
UCC
UCC,
UCC
UCC
UCC
0.50
0.00
0.20
0.50
MINIS1
0.30
CIEARLU
CIF CIF1R000
0.40
0.6
0.6 UCC accuracy
UCC accuracy
UCC accuracy
Figure 3: Clustering accuracy vs ucc accuracy plots of UCC and UCC,=1 models together with k-means and spectral clustering accuracy baselines on MNIST, CIFAR1O and CIFAR 100 datasets.
pure sets of underlying classes are far apart from each other. The features of all the instances in particular class are extracted by using a trained model and feature distributions associated to that class obtained by performing kernel density estimation pair of classes, inter-class JS divergence values are calculated (Appendix C.4. For a particular model, which is used in feature extraction, the minimum of these pairwise inter-class JS divergence values is used as a metric in the comparison of models. We have observed that as the task gets more challenging and the number of clusters increases, there drop in minimum inter-class JS divergence values, which is summarized in Table
these extracted features. Then, for each
4.4 LABELS ON INSTANCES
For eventual objective of 'Labels on instances', have used 'clustering accuracy' compar- ison metric, which is calculated similar to Ghasedi Dizaji et al. (2017). By using our trained models, extracted features of individual instances of all classes in test sets. Then, performed unsuper- vised clustering over these features by using k-means and spectral clustering. We used number of classes in ground truth number of clusters (MNIST: 10, CIFAR1O. 10: 10, CIFAR100: 20 clusters) during clustering and gave the best clustering accuracy for each model in Table (Appendix C.5)
In Table I1 compare clustering accuracies of our models together with baseline and state of the art models in the literature: baseline unsupervised (K-means (Wang et al.) 2015). Spectral Clus- tering (Zelnik-Manor Peronal 2005)): state of the art unsupervised (JULE (Yang et al. 2016
GMVAE (Dilokthanakul et al. 2016). DAC (Chang et al. [2017). DEPICT (Ghasedi Dizaji et al. 2017. DEC (Xie et al., (0166) and state of the art semi- cami-supervised (AAE (Makhzani et al. 2015).
CatGAN (Springenberg 2015), LN (Rasmus et al., 2015), ADGM (Maaløe et 2016)). Clustering performance of our unique class count models is better than the performance unsupervised models in all datasets and comparable performance of fully supervised learning models in MNIST and CI- FARIO datasets. The performance gap gets larger CIFAR 100 dataset as the task becomes harder. Although semi-supervised methods use our models perform on par with AAE and CatGAN models and comparable to LN and ADGM models on MNIST dataset. ADGM and LN even reach the performance of the F FullySupereescl model since they exploit training with 'exact' labeled data. On CIFAR10 dataset, LN and CatGAN models are slightly better than our unique class count models; however, they use 10% of instances with 'exact' labels, which is not a small portion.
part of the dataset with 'exact' labels during training,
In general, our UCC and UCC. models have similar performance, and they better than their counterpart models of UC CCC+ and UCC2+ due to the absence of pure sets during training. How- ever, the real world tasks, the absence of pure sets heavily depends on the nature of the problem. In our task of semantic segmentation of breast cancer metastases in histological lymph node sec- tions, for example, there are many pure sets. Furthermore, we observed that there is performance gap between 2CC2+ and a= models: UCC2+ models perform better than UCC2+ models thanks to the autoencoder branch. The effect of autoencoder branch also apparent in Figure 3 which shows clustering accuracy vs uce accuracy curves for different datasets. For MNIST dataset, while UCC model gives clustering accuracy values proportional to uce accuracy, UCC, model cannot reach to high clustering accuracy values until it reaches to high accuracies. The reason that autoencoder branch in UCC helps frame module to extract better features during the initial phases of the training process, where the uce classification accuracy is low. Compared other
8

---

Published as a conference paper at ICLR 2020
ucc ((normal) Ree1 [metastases) IcC22 focundary)
uce2 (boundary)
ucc2 (ooundary)
Input Image
Ground Truth Mask
UCC erament Predicted Mask
Unet Predicted Mask
K-means Predicted Mask
Figure 4: Example images from hold-out test dataset with corresponding ucc labels, ground truth masks and predicted masks by UCC, UCC segments Unet and K-means clustering models.
datasets, this effect is more significant in MNIST dataset since itself is clusterable. Although toencoder branch helps in CIFARIO and CIFARI00 datasets as well, improvements in clustering accuracy coming from autoencoder branch seems to be limited, so models UCC and UCC.=1 follow nearly the same trend in the plots. The reason that CIFAR10 10 and CIFAR 100 datasets are more complex than MNIST dataset, autoencoder is powerful enough contribute extract discrimant features, which is also confirmed by the limited improvements of Autoencoder models baseline performance in these datasets.
SEMANTIC SEGMENTATION OF BREAST CANCER METASTASES
Semantic segmentation of breast cancer metastases in histological lymph node sections is cial step in staging breast cancer, which is the major determinant of the treatment and progno- sis (Brierley et al. 2016). Given the images of lymph node sections, the task is to detect and locate, iee. semantically segment out, metastases regions the images. We have formulated this task in our novel MIL framework such that each image is treated as a bag and corresponding uce label is obtained based on whether the image is from fully normal or metastases region, which is labeled by ucel, or from boundary region (i.e. image with both normal and metastases regions), which is labeled by ucc2. We have shown that this segmentation task can be achieved by using our weakly supervised clustering framework without knowing the ground truth metastases region masks of im- ages, which require experts to exhaustively annotate each metastases region in each image. This annotation process tedious, time consuming and more importantly not part of clinical workflow.
We have used 512 x 512 image crops from publicly available CAMELYON dataset (Litjens et al.) 2018) and constructed our bags by using 32 x 32 patches over these images. We trained our unique class count model U U'CSsegment tractor and conducted unsupervised clustering over the patches of the images the hold-out test dataset obtain semantic segmentation masks. For benchmarking purposes, have also trained fully supervised Unet model (Ronneberger et al., 2015). which is well-known biomedical image segmentation architecture, by using the ground truth masks and predicted the segmentation maps in
labels. Then, we used the trained model feature

---

Published as a conference paper at ICLR 2020
Table 2:
Semantic segmentation performance statistics of UCC UCCsegment, segments Unet and K-means clus- tering methods on hold-out test dataset.
TPR
FPR
TNR
FNR PA
UCC segment (weakly supervised)
0.818
0.149
0.851
0.182 0.863
Unet (fully supervised)
0.860
0.126 0.874 0.140 0.889
K-means (unsupervised baseline)
0.370
0.271
0.729
0.630 0.512
the test set. The aim of this comparison was to show that at the absence of ground truth masks, our model can approximate the performance of a fully supervised model. Moreover, we have obtained semantic segmentation maps in the test dataset by using k-means clustering ample images from test dataset with corresponding ground truth masks, ucc labels and predicted masks by different models are shown in Figure 4 (Please see Appendix D.I for more details.)
baseline study, Ex-
Furthermore, we have calculated pixel level gross statistics of TPR (True Positive Rate), FPR (False Positive Rate), TNR (True Negative Rate), FNR (False Negative Rate) and PA (Pixel Accuracy) over the images of hold-out test dataset and declared the mean values in Table 2 When we look at the performance of unsupervised baseline method of K-means clustering. it obvious that semantic segmentation of metastases regions in lymph node sections is not an easy task. Baseline method achieves very low TPR value of 0.370 and almost random score of 0.512 in PA. On the other hand, both our weakly supervised model UC segment and fully supervised model Unet outperform the baseline method. When we compare our model UCCsegmannt with Unet model, we see that both models behave similarly. They have reasonably high TPR and TNR scores, and low FPR and FNR scores. Moreover, they have lower FPR values than FNR values, which is more favorable than vice-versa since pathologists opt to use immunohistochemisti (IHC) to confirm negative cases (Bejnordi et al. 2017). However, there is a performance gap between two models, which is mainly due to the fact that Unet model is fully supervised model and it is trained on ground truth masks, which requires exhaustive annotations by experts. On the contrary, UCC segmett model is trained uce labels and approximates to the performance of the Unet model. uèe label is obtained based on whether the image is metastatic, non-metastatic or mixture, which is much cheaper and easier to obtain compared to exhaustive mask annotations. Another factor affecting the performance of UC'C U'CC'semmet model is that ucel labels can sometimes be noisy. It is possible to have some small portion of normal cells in cancer regions and vice-versa due the nature of the cancer. However, our UCC segment is robust to this noise and gives reasonably good results, which approximates the performance of Unet model.
(Appendix D.2
6 CONCLUSION
In this paper, we proposed a weakly supervised learning based clustering framework and introduce novel MIL task as the core of this framework. We defined ucc as a bag level label in MIL setup and mathematically proved that perfect ucc classifier can be used to perfectly cluster individual instances inside the bags. We designed neural network based uce classifer and experimentally showed that clustering performance of our framework with our ucc classifiers are better than the performance of unsupervised models and comparable to performance of fully supervised learning models. Finally, we showed that our weakly supervised unique class count model, UCC can be used for semantic segmentation of breast cancer metastases in histological lymph node sections. We compared the performance of our model UCC with the performance of Unet model and showed that our weakly supervised model approximates the performance of fully supervised Unet model. In the future, want to check the performance of our UCC model with other medical image datasets and use to discover new morphological patterns in cancer that had been overlooked in traditional pathology workflow.
ACKNOWLEDGEMENTS
This work is supported by the Biomedical Research Council of the Agency for Science, Technology, and Research, Singapore and the National University of Singapore, Singapore.
10

---

Published as a conference paper at ICLR 2020
REFERENCES
Stuart Andrews, Joannis Tsochantaridis, and Thomas Hofmann. Support vector machines for multiple-instance learning. In Advances neural information processing systems, pp- 577-584, 2003.
Carlos Arteta, Victor Lempitsky, J Alison Noble, and Andrew Zisserman. Interactive object count- ing. In European conference on computer vision, pp. 504-518 Springer, 2014.
Boris Babenko, Ming-Hsuan Yang, and Serge Belongie. Robust object tracking with online multiple instance learning. IEEE transactions on pattern analysis and machine intelligence, 33(8):1619- 1632, 2011.
Babak Ehteshami Bejnordi, Mitko Veta, Paul Johannes Van Diest, Bram Van Ginneken, Nico Karssemeijer, Geert Litjens, Jeroen AWM Van Der Laak, Meyke Hermsen, Quirine F Manson, Maschenka Balkenhol, al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. Jama, (18822)::119-22100 2017.
James Brierley, Mary Gospodarowicz, and Christian Wittekind. TNM classification of malig- nant lumours. John Wiley & Sons, 2016.
Nataly Brukhim and Amir Globerson. Predict and constrain: Modeling cardinality in deep structured prediction. In International Conference on Machine Learning, pp. 658-666 2018.
Jianlong Chang, Lingfeng Wang. Gaofeng Meng, Shiming Xiang, and Chunhong Pan. Deep adap- tive image clustering. In Proceedings of the IEEE International Conference Computer Vision, pp. 5879-5887, 2017.
Yixin Chen and James Z. Wang. Image categorization by learning and reasoning with regions. Journal of Machine Learning Research, 5(Aug):913-939, 2004.
Yixin Chen, Jinbo Bi, and James Ze Wang. Miles: Multiple-instance learning via embedded instance selection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(12):1931-1947 2006.
Thomas G Dietterich, Richard H Lathrop, and Tomás Lozano-Pérez. Solving the multiple instance problem with axis-parallel rectangles. Artificial intelligence, 89(1-2):31-11, 1997.
Nat Dilokthanakul, Pedro AM Mediano, Marta Gamelo, Matthew CH Lee, Hugh Salimbeni, Kai Arulkumaran, and Murray Shanahan. Deep unsupervised clustering with gaussian mixture varia- tional autoencoders, arXiv preprint arXiv:1611.02648 2016.
Murat Dundar, Balaji Krishnapuram, RB Rao, and Glenn M Fung. Multiple instance learning for computer aided diagnosis. In Advances in neural information processing systems, pp. 425 432, 2007.
Pedro F Felzenszwalb, Ross B Girshick, David McAllester, and Deva Ramanan. Object detection with discriminatively trained part-based models. IEEE transactions on pattern analysis and ma- chine intelligence, 32(9): 1277-1627116455 2010.
James Foulds and Eibe Frank. A review of multi-instanne learning assumptions. The Knowledge
Engineering Review, 25(1):1-25. 2010.
James Richard Foulds. Learning instance weights in multi-instance learning. PhD thesis, The University of Waikato, 2008
Thomas Gärtner, Peter A Flach, Adam Kowalczyk, and Alexander J Smola. Multi-instance kernels. In ICML, volume 2, pp. 7, 2002.
Kamran Ghasedi Dizaji, Amirhossein Herandi, Cheng Deng, Weidong Cai, and Heng Huang. Deep clustering via joint convolutional autoencoder embedding and relative entropy minimization. In Proceedings of the IEEE International Conference on Computer Vision, pp. 5736-5745, 2017.
11

---

Published as a conference paper at ICLR 2020
Haroon Idrees, Imran Saleemi, Cody Seibert, and Mubarak Shah. Multi source multi-scab scale counting in extremely dense crowd images. In Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2547-2554, 2013.
Zach Jorgensen, Yan Zhou, and Meador Inge. A multiple instance learning strategy for combating good word attacks on spam filters. Journal of Machine Learning Research, 9(un): 2008.
Andreas Kipf, Thomas Kipf, Bernhard Radke, Viktor Leis, Peter Bonez, and Alfons Kem- per. Learned cardinalities: Estimating correlated joins with deep learning. arXiv preprint arXiv: 00077, 2018.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Tech- nical report, Citeseer, 2009.
Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner, et al. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1 1):2278-2333 1998.
Jianhua Lin. Divergence measures based on the shannon entropy. IEEE Transactions Information theory, 37(1):145- 151, 1991.
Geert Litjens, Peter Bandi, Babak Ehteshami Bejnordi, Oscar Geessink, Maschenka Balkenhol, Peter Bult, Altuna Halilovie, Meyke Hermsen, Rob de Loo, Rob Vogels, et al. 1399 h&e- stained sentinel lymph node sections of breast cancer patients: the camelyon dataset. GigaScience, 7(6):giy065, 2018.
Henry Liu, Minghin Xu, Ziting Yu, Vincent Corvinelli, and Calisto Zuzarte. Cardinality estimation using neural networks. In Proceedings of the 25th Annual International Conference on Computer Science and Software Engineering, pp. 59. IBM Corp., 2015.
Lars Maalde, Casper Kaae Sønderby, Søren Kaae Sønderby, and Ole Winther. Auxiliary deep gen- erative models. arXiv preprint arXiv: 2016.
Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow, and Brendan Frey. Adversarial autoencoders. arXiv preprint arXiv:1511.0564: 2015.
Emanuel Parzen. On estimation of probability density function and mode. The annals of mathe- matical statistics, 33(3): 1962.
Jan Ramon and Lue De Raedt. Multi instance neural networks. 2000.
Antti Rasmus, Mathias Berglund, Mikko Honkala, Harri Valpola, and Tapani Raiko. Semi- supervised learning with ladder networks. In Advances in neural information processing systems, pp. 3546-3554, 2015.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedi- cal image segmentation. In International Conference Medical image computing and computer- assisted intervention, pp. 234-241. Springer, 2015.
Burr Settles, Mark Craven, and Soumya Ray. Multiple-instanee active learning. In Advances in neural information processing systems, pp. 1289-1296. 2008.
Jost Tobias Springenberg. Unsupervised and semi-supervised learning with categorical generative adversarial networks. arXiv preprint arXiv:1511.0633 2015.
Jinhui Tang, Haojie Li, Guo-Jun Qi. and Tat-Seng Chua. Image annotation by graph-based inference with integrated multiple/single instance representations. IEEE Transactions on Multimedia, 12(2): 131-141, 2010.
Jaanfnn Wang, Jingdong Wang. Jingkuan Song, Xin-Shun Xu, Heng Tao Shen, and Shipeng Li.
Optimized cartesian k-means. IEEE Transactions Knowledge and Data Engineering, 27(1): (1): 180-192, 2015.
Xinggang Wang. Yongluan Yan, Peng Tang, Xiang Bai, and Wenyu Liu. Revisiting multiple instance neural networks. Pattern Recognition, 74:15-24, 2018.
12

---

Published as a conference paper at ICLR 2020
Jiajun Wu, Yinan Yu, Chang Huang, and Kai Yu. Deep multiple instance learning for image clas- sification and auto-annotation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3460-3469, 2015.
Junyuan Xie, Ross Girshick, and Ali Farhadi. Unsupervised deep embedding for clustering analysis. In International conference on machine learning, pp. 478-487, 2016.
Jianwei Yang, Devi Parikh, and Dhruv Batra. Joint unsupervised learning of deep representations and image clusters. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5147-5156, 2016.
Sergey Zagoruyko and Nikos Komodakis. 17X1v:1605.07146, 2016.
Wide residual networks.
arXiv preprint
Lihi Zelnik-Manor and Pietro Perona. Self-tuning spectral clustering. In Advances in neural infor- mation processing systems. pp. 1601-1608, 2005.
Cha Zhang, John Platt, and Paul A Viola. Multiple instance boosting for object detection. In Advances in neural information processing systems, pp. 1417-1424, 2006.
Cong Zhang, Hongsheng Li, Xiaogang Wang, and Xiaokang Yang. Cross-scen crowd counting via deep convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 833 841, 2015.
Min-Ling Zhang and Zhi-Hua Zhou. Multi-instance clustering with applications multi-instance prediction. Applied Intelligence, (1)):477668, 2009.
Qi Zhang and Sally A Goldman. Em-dd: An improved multiple-instance learning technique. In Advances in neural information processing systems, pp. 1073-1080, 2002.
Qi Zhang, Sally A Goldman, Wei Yu, and Jason E Fritts. Content-based image retrieval using multiple-instance learning. In ICML, volume 1, pp. 2. Citeseer, 2002.
Yingying Zhang, Desen Zhou, Siqin Chen, Shenghua Gao, and Yi Ma. Single-image crowd count- ing via multi-column convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 589-597, 2016.
Zhi-Hua Zhou. A brief introduction to weakly supervised learning. National Science Review, 5(1): 44-53, 2017.
Zhi Hua Zhou and Min-Ling Zhang. Neural networks for multi-instance learning. In Proceedings of the International Conference on Intelligent Information Technology, Beijing, China, pp. 2002.
Zhi-Hua Zhou, Yu-Yin Sun, and Yu-Feng Li. Multi-instance learning by treating instances as non- iid samples. In Proceedings of the 26th annual international conference on machine learning, pp- 1249- 1256. ACM, 2009.
13

---

Published as a conference paper at ICLR 2020
A KERNEL DENSITY ESTIMATION
Kernel density estimation is a statistical method to estimate underlying unknown probability dis- tribution in data (Parzen 1962). It works based fitting kemelss at sample points of an unknown distribution and adding them to construct the estimated probability distribution. estimation process is illustrated in Figure
Kernel density
i decoder
(x,)
foc
111
(v)
Ann 122(())
Ja
1x E cz)
Seature
KDE
Odm
Ma;
pa
Lnsi no (v)
Ann (1((1)
faz
x(v
f
K(V So.
0
(1(())
Σ
|oc1
X1
X2
X3
X1
Mag
1=1
fot
Ai
(v)
2i
(v)
0.3
0.2
0.8
:
f
M
M
W (v)
0.2
1.0
0.3
0.9
fla
h (v)
0.9
0.7
Figure 5: KDE module the Gaussian kernel (N(U ( for each extracted feature for a sample is illustrated with colored curves and previously accumulated kernels are shown in gray. Estimated fea- ture distributions, which are obtained by employing Equation 2 are sampled some pre-determined intervals and passed to 0am.
A.1 KDE MODULE IS DIFFERENTIABLE
The distribution of the feature 1y (v) is obtained by applying kernel density estimation the extracted features få' as in Equation 2 In order to be able to train our unique class count model end-to-end, we need to show that KDE module is differentiable, so that we can pass the gradients from One O during back-propagation. Derivative of hgg (v) with respect input of KDE
14

---

Published as a conference paper at ICLR 2020
module, fag. can be obtained in Equation
loc 1
(n- (0-52())
hgg (v) = locl i=1 2TT
(2)
0he (v)
1
20 (0- (b6)
0fdc
locl 0² 2
After showing that KDE module is differentiable, can show the weight update process for Oreaure module in our model. Feature extractor module Oreamic is shared by both autoencoder branch and uce branch in model. During back proppprppggttion phase of the end-to-end training process, the weight updates of 0 festure comprise the gradients coming from both branches (Equation 5). Gradients coming from autoencoder branch follow the traditional neural network back-propagation flow through the convolutional and fully connected layers. Different than that, gradients coming from uee branch (Equation 6) also prokaprop through the custom KDE layer according to Equation
Loss
LosSuce + (1 a) Lossag where E [0. 1]
(4)
autoencoder loss
ÖLoss
Lossuce
Loss
a1
(5)
resture
reammm
30 teaure
gradients for Oreatue
gradients from uce branch
gradients from sutoencoder branch
Loss uec
Lossuee thag
aJc
X 30 teatune
(6)
a0 teature
buck-propagation through KDE layer
B PROOFS PROPOSITIONS
Before proceeding the formal proofs, it is helpful emphasize the decomposability property of kernel density estimation here.
For any set, σc, one could partition into set of M disjoint subsets og of Ud. where da for vb. is trivial show that distribution hig (v) is simply linear combination of distributions thz (v), A 1,2, ... M (Equation 7). As direct consequence, one could decompose any into pure subsets. This is an important decomposition which will be used in the proofs of propositions later.
10/1 |0çl
(v)
Σ A=1
Woy Ny
(v), Vj where Wor
Now, we can proceed to formally state our propositions.
Definition 1
Given subset aç C X, unique class count, Ma is defined as the number of unique classes that all instances in the subset σę belong to, i.e. No = [[((x))) E &cHl. Recall that each instance belongs to an underlying unknown class.
Definition 2 A set a is called a pure set if its unique class count equals one. All pure sets are denoted by the symbol que in this paper.
15

---

Published as a conference paper at ICLR 2020
For any dç C. X, the unique class count no; of og does not depend the number of instances in oç belonging to a certain class.
Proposition B.
Proof: This conclusion is obvious from the definition of unique class count in Definition
Proposition B. 0m is non-linear.
Proof: We give a proof by contradiction using Proposition B Suppose 0an is linear, then
aam (ha,))
Oam(wcho: We wghog)
= wcDam(hoc)) ) + we ug (mhoe)
weño; + W5Tor no.
Hence, 0an linear only when Equation & holds. However, by Proposition B D (fraare Omn) should count correctly regardless of the proportion of the size of the sets l0cl and og Hence, Equation cannot hold true and am by contradiction cannot be linear.
Proposition B. 3 Let oc, ag be disjoint subsets of X with predicted unique class counts fe. and To respectively. Let No be the predicted unique class count of oy = ac Uog. If hog = hag then No. fo; Mo.g
Proof: The distribution of set au can decomposed into distribution subsets,
ha = wehho + wehhog where we + urg hag = hag = haw = hog
(9)
(10)
Hence, 1o. = nla = Mas-
Proposition 1 Let o. og be disjoint subsets of X with predicted unique class counts Moç Nog 1. If the predicted unique class count of o, o U og is fo. = 2, then hog * hag
Proof: Proof of this proposition follows immediately from the contra-positive of Proposition
Definition 3 A perfect unique class count classifier takes in any set a and output the correct pre- dicted unique class count no
Nl.
Proposition 2 Given a perfect unique class count classifier. The dataset X can be perfectly clus- tered into K subsets of oburc, " 1,2, K, such that X' = U pure and opure = {2,|2, X, f(xx) E}
First note that this proposition holds because the "perfect unique class count classifier" is a very strong condition. Decompose X' into subsets with single instance and then apply the unique class count on each subset, by definition, unique class counts of all subsets are one. Randomly pair up the subsets and merge them their union still yield unique class count of one. Recursively apply merging on this condition until no subsets can be merged.
Proof:
Proposition
3 Given a perfect unique class count classifier. Decompose the dataset into K ç = 1,... K, such that o = (xi12, E X, C(xz) E). Then, hagure + haguro for
subsets
pur
A.
Proof: Since in Proposition 1 the subsets are arbitrary, it holds for any two subsets with unique class count of one. By pairing up all combinations, one arrives this proposition. Note that for perfect unique class count classifier, 1).
C DETAILS ON EXPERIMENTS WITH MNIST AND CIFAR DATASETS
C.I DETAILS OF MODEL ARCHITECTURES
Feature extractor module A f has convolutional blocks similar to the wide residual blocks in Zagoruyko & Komodakis (2016). However, the parameters architectures. number of con- volutional and fully connected layers, number of filters in convolutional layers, number of nodes in
16

---

Published as a conference paper at ICLR 2020
fully-connected layers, number of bins and a value in KDE module, were decided based models' performance and training times. While increasing number of convolutional layers or filters were not improving performance of the models substantialy, they were putting heavy computation burden. For determining the architecture of 0an. we checked the performances of different number of fully connected layers. As the number of layers increased, the uce classification performance of the mod- els increased. However, we want Deanmm to be powerful, so we stopped to increase number of layers as soon as we got good results. For KDE module, we have tried parameters of 11 bins, 21 bins, = 0.1 and ō 0.01. Best results were obtained with 11 bins and o = 0.1. Similarly, we have tested different number of features at the output of Oeeauue module and we decided to use 10 features for MNIST and CIFAR10 datasets and 16 features for CIFAR I0FAR100 dataset based on the clustering performance and computation burden.
During training, loss value of validation sets was observed as early stopping criteria. Training of the models was stopped if the validation loss didn't 't drop for some certain amount of training iterations.
For the final set of hyperparameterss and details of architectures, please see the code for our experi- ments: http://bit.....
C.2 DETAILS OF DATASETS
We trained and tested our models MNIST, CIFARIO and CIFARIO0 datasets. While MNIST and CIFARIO datasets have 10 classes, CIFARIO 100 dataset has 20 classes. For MNIST, we randomly splitted 10,000 images from training set as validation set, we had 50,000, 10,000 and 10,000 images in our training Ammist,rr- validation Xmnisst,vl and test sets Xmniiet, teat, respectively. In CI- FARIO dataset, there are 50,000 and 10,000 images with equal number of instances from each class in training and testing sets, respectively. Similar to MNIST dataset, we randomly splitted 10,000 images from the training set as validation set. Hence, we had 40,000, 10,000 and 10,000 images in our training Xei 10,,tr validation foifort0.val and testing 10sfar10,test sets for CIFARIO, respec- tively. In CIFARIO0 dataset, there are 50,000 and 10,000 images with equal number of instances from each class in training and testing sets, respectively. Similar to other datasets, we randomly splitted 10,000 images from the training set as validation set. Hence, we had 40,000, 10,000 and 10,000 images in our training Kesfar100,tr. validation Keifar100,eal and testing Xci Neifarl00,test sets for CIFARIO, respectively.
FupleSupervised models took individual instances inputs and were trained on instance level ground truths. Xmmistt,r, Keifarl0.tr and Kai were used for training of F FullySuperwiste models. Unique class count models took sets of instances as inputs and were trained on uce labels. Inputs to unique class count models were sampled from the power sets of MNIST, CIFARIO and CIFAR100 datasets, i.e. 2mmmiiat.rr 2*N.sfar10.rr and 2* For MNIST and CIFARIO datasets, the subsets (bags) with 32 instances and for CIFAR100 dataset, the subsets (bags) with 128 instances are used in our experiments. While UCC and UCC, models are trained on ucel to ucc4 labels, UCC2+ and UCC2+ a=1 models are trained on uec2 to uce4 labels.
Our models were trained on vee labels up to uee4 instead of ucc10 (uce20 in CIFAR100) since the performance was almost the same for both cases in our experiment with MNIST dataset, results of which are shown in Table 3 On the other hand, training with uecl to uce4 was much faster than uoel to ucc10 because the ucc label gets larger, the number of instances in a bag is required to be larger in order to represent each class and number of elements in powerset also grows exponentially. Please note that for perfect clustering of instances, it is enough to have perfect ucc classifier that can discriminate wee1 and uce2 from Proposition 2
All the results presented in this paper were obtained hold-out test sets Xmmistt tests Kcifar10,tes. and Xes far test
17

---

Published as a conference paper at ICLR 2020
Table 3: Clustering accuracy comparison of training unique class count models with uce labels of ucc1 to uce4 and ucel to ucc10 on MNIST dataset.
clustering aceuracy
ucel to uce4 ucel to wcc10
UCC
0.984
0.983
UCC
0.984
0.982
C.3 CONFUSION MATRICES FOR ucc PREDICTIONS
We randomly sampled subsets for each ucc label from the power sets of test sets and predicted the uce labels by using trained models. Then, calculated the ucc prediction accuracies by using predicted and truth wee labels, which are summarized in Table Here, we show confusion matrices of UCC and UCC++ models MNIST, CIFARIO and CIFAR100 datasets as examples in Figure and respectively.
1,000
DXNKI
0.000
0,000
Mpe2
1.000
0.000
0,000
Taue label
U0c2
0000
1000
0.000
0.000
Frue label
0.000
0.000
1.000
0.000
0.000
0.000
1.000
0.2
10ce4
0.0
0.0
8802
1ec4
Predicted label
Predicted label
(a) UCC
(b) UCC
Figure 6: Confusion matrices of our UCC and UCC2+ models for uce prediction MNIST.
0.993
0.007
0.000
0,000
0.967
0.024
0.010
True labe!
11,000
0.972
0.023
0.005
Frue label
0.6
0.042
0887
0.071
0.4
V003
0.000
0.001
0.938
0.061
ucc/
0.000
0.000
0.013
0.987
UE04
0.005
0.042
0.953
0.2
ucc4
uo22
Predicted
Predicted label
(a) UCC
(b) UCC
Figure 7: Confusion matrices our UCC and 2 models for uce prediction on CIFAR10.
18

---

Published as a conference paper at ICLR 2020
ucc1
0.331
0.037
0.046
0.036
0
UEC2
0.827
0.125
0.047
1rue label
1pc2
Froe label
0.6
0.073
0.239
UCC3
0.001
0.0.32
0.741
0.224
ucc+
0.023
0.051
0.937
100-4
0.000
0.009
0.078
0.913
0002
1x04
0002
Predicted lahel
Predicted label
(a) UCC
(b) UCC
Figure 8: Confusion matrices of our UCC and UCC models for uce prediction on CIFARI00.
C.4 FEATURE DISTRIBUTIONS AND INTER-CLAS IS DIVERGENCE MATRICES
The features of all the instances a particular class are extracted by using a trained model and feature distributions associated to that class obtained by performing kernel density estimation these extracted features. Then, for each pair of classes, inter-class IS divergence values are calcu- lated. We show inter-class JS divergence matrices for our u'ullySupervised and UCC models on MNIST test dataset in Figure We also show the underlying distributions for F FullySupereised and UCC models Figure and 11 respectively.
2))(
D ()
1.0
1.0
C.000
0,570
0,563
0.565
0.169
0.166
0:457
0.178
0.373
0.159
0.000
0.608
0,395
0.476
0,576
C,530
0.399
0,532
0:437
0.503
1-5070
(0000
0.377
0.562
0.661
0473
0.650
0.464
0.557
0.64
1- CLAR
(M00
0.504
0.495
ILS49
0,589
0.538
0.47
0.4R7
0LSSS
K.563
0.377
0,000
0.551
0.169
0,657
0.645
0.157
0.535
0.499
0.305
I1.00
0.000
0.376
0,508
C.49
0.360
0,365
0.781 0.366
C.566
0.562
0,551
0.000
0.647
0/449
0,470
0.293
0.530
0.454
CJ176
0.496
0.376
0.000
0.455
C/112
0.405
0.158
0.339 0.477
0.6
4-44
-04
0.4
C-457
-.2
0.2
0.557 0.535 0.556
0:455
0,781
0.359
0312 C.324 0.281
CA45 0.643 0.459 0:454 0.362
C..003
0.364 0:477 0.287 C.444 0:403
0.0
0.0
(a) Fullyyupervised
(b) UCC
Figure 9: Inter-class IS divergence matrix calculated over the distributions of features extracted by our FulliSupe and U UCC models on MNIST test dataset.
19

---

Published as conference paper at ICLR 2020
Figure 10: Distributions of extracted features by our Fully S upervised model on MNIST test dataset. Each column corresponds a feature learned by model and each row corresponds to an underlying class in the test dataset.
91
81
61
81
B1
61
00100000
0000300
000000
0000000
0000000
01
81
61
01
81
81
61
61
81
20
0000005
0000000
0000000
80
10
80
83
80
81
81
81
01
81
81
01
81
41
20

---

Published as a conference paper at ICLR 2020
Figure 11: Distributions extracted features by our UCC model on MNIST test dataset. Each column corresponds to feature learned by model and each row corresponds to an underlying class in the test dataset.
91
81
61
81
B1
61
61
41
10
80
20008
30000
30000
30008
50088
80008
00008
00008
50358
80008
81
61
91
81
81
61
61
41
35388
30803
30803
35368
35888
30888
30868
36868
50353
35858
10
82258
30858
32858
30008
32858
30858
35358
30008
35358
35358
83288
58288
52888
32288
52288
58288
52888
88288
58385
52888
20
20
23
20
32828
33328
52358
30358
32328
32228
32228
32328
52328
35328
10
28
22
33828
58858
53888
22288
58888
50888
85888
80288
58358
85888
10
10
32803
50808
30008
30008
30008
80008
53358
35088
35008
35888
10
18
80
30205
20205
35288
32205
30203
35205
30208
30305
11
61
01
11
1t
01
81
11
1300S
32253
52853
350d5
$28d5
328d3
35255
35308
52353
32805
21

---

Published as a conference paper at ICLR 2020
C.5 K-MEANS AND SPECTRAL CLUSTERING ACCURACIES OF OUR MODELS
We performed unsupervised clustering by using k-means and spectral clustering and gave the best clustering accuracy for each model on each dataset in Table 1 in the main text. Here, present all the clustering accuracies for our models in Table
Table 4: Clustering accuracy values of our models with K-means and Spectral clustering methods on different datasets, Best value for each model in each dataset highlighted in bold.
MNIST
CIFARIO
CIFAR100
K-means
Spectral
K-means
Spectral
K-means
Spectral
UCC
0.979
0.984
0.781
0.680
0.338
0.261
UCC
0.977
0.984
0.545
0.502
0.278
0.225
UCC
0.981
0.984
0.774
0.635
0.317
0.249
UCC
0.881
0.832
0.521
0.463
0.284
0.237
Autoencoder
0.930
0.832
0.241
0.230
0.167
0.140
Spplrvsie
0.988
0.106
0.833
0.464
0.563
0.328
C.6 UCC MODELS WITH AVERAGING LAYER AND KDE LAYER
KDE layer is chosen as MII. pooling layer in UCC model because of its four main properties, first three of which essential for the proper operation of proposed framework and validity of the propositions in the paper:
1. KDE layer is permutation-invarian i.c. the output of KDE layer does not depend on the permutation of its inputs, which is important for the stability of 6am module.
2. KDE layer is differentiable, so UCC model can be trained end-to-end.
3. KDE layer has decomposability property which enables our theoretical analysis (Ap- pendix (
4. KDE layer enables amm to fully utilize the information in the shape of the distribution rather than looking at point estimates of distribution.
Averaging layer (Wang et al. 2018) can be an alternative to KDE layer in UCC model. We have conducted additional experiments by replacing KDE layer with "averaging layer' and compare the clustering accuracy values of the models with averaging layer and the models with KDE layer in Table
MIL pooling layer, which also has the first three properties,
Table 5: Clustering accuracy values of the models with averaging layer and the models with KDE layer.
clustering acc.
mnist
cifar10
UCC (KDE layer)
0.984
0.781
UCC (Averaging layer)
0.987
0.638
UCCa=1 (KDE layer)
0.981
0.774
UCC.=1 (Averaging layer)
0.943
0.508
22

---

Published as a conference paper at ICLR 2020
D DETAILS ON SEMANTIC SEGMENTATION TASK
D.I DETAILS OF MODEL AND DATASET
Our model segment has the same architecture with the UCC model in CIFARIO dataset, but this time we have used 16 features. We have also constructed the Unet model with the same blocks used in segment model in order to ensure fair comparison. The details of the models can be seen in our code: http://bitt http:://///nnneecasssssooooo lasscount
We have used 512 x 512 image crops from publicly available CAMELYON dataset (Litjens et al. 2018). CAMELYON dataset is public Whole Slide Image (WSI) dataset of histological lymph node sections. also provides the exhaustive annotations for metastases regions inside the slides which enables to train fully supervised models for benchmarking of our weakly supervised unique class count model.
We randomly crop 512 images over the WSIs of CAMELYON dataset and associate ucc label to each image based on whether it fully metastases/normal (uce1) or mixture (ucc2). We assigned uce labels based provided ground truths since they readily available. However, please note that in case no annotations provided, obtaining uce labels is much cheaper and easier compared to tedious and time consuming exhaustive metastases region annotations. We assigned uce1 label an image if the metastases region in the corresponding ground truth mask is either less than 20% (i.c. normal) or more than 80% (i.e metastases). On the other hand, we assigned ucc2 label an image if the metastases region in the corresponding ground truth mask is more than 30% and less than 70% (i.e. mixture). Actually, this labeling scheme imitates the noise that would have been introduced if uce labeling had been done directly by the user instead of using ground truth masks. Beyond that, uee1 labels this task can naturally be noisy since is possible have some small portion of normal cells in cancer regions and vice-versa due to the nature of the cancer. In this way, we have constructed our segmentation dataset consisting of training, validation and testing sets. The images in training and validation sets cropped randomly over the WSIs in training set of CAMELYON dataset and the images in testing set are cropped randomly over the test set of CAMELYON dataset. Then, the bags in MIL dataset to train UCC JCCSeement model constructed by using 32 x 32 patches over these images. Each bag contains 32 instances, where each instance is a 32 x 32 patch. The details of our segmentation dataset are shown in Table 6
We have provided the segmentation dataset under "/ddata/cameloon"" folder inside our code folder. If you want to this dataset for benchmarking purposes please cite our paper (referenced later) together with the original CAMELYON dataset paper of Litjens et al. (2018).
Table 6: Details of our segmentation dataset: number of WSIs used to crop the images in each set, number of images in each set and corresponding label distributions in each set
uccl
ucc2
normal metastases
total
mixture
# of images
# of WSIs
Training
461
322
783
310
1093
159
Validation
278
245
523
211
734
106
Testing
282
668
950
228
1178
126
We have given confusion matrix for uec predictions of our UCC UCCesgment eegment model in Figure Unet model, we have shown loss curves of traininig and validation sets during training Figure 13
12 For
23

---

Published as a conference paper at ICLR 2020
ucc1
0.74
0.26
0.7
True Label
0.6
0.5
0.4
UCC2
0.12
0.88
0.3
ucc1
UCC2
Predicted Label
Figure 12: Confusion matrix of our UCC, segment model for uce predictions on our segmentation dataset.
UCCsegment
train loss
0.38
val loss
loss
0.32
0.26
0.20
0
20000
40000
60000
iteration
Training and validation loss curves during training of our U net model. We have used the best model weights, which were saved at iteration 58000, during training. Models starts to overfit after iteration 60000 and early stopping terminates the training.
Figure 13:
D.2 DEFINITIONS OF EVEALUTION METRICS
In this section, we have defined our pixel level evaluation metrics used for performance comparison of our weakly supervised UCC eegment model, fully supervised Unet model and unsupervised base- line K means model. Table shows the structure of pixel level confusion matrix together with basic statistical terms. Then, our pixel level evaluation metrics TPR (True Positive Rate), FPR (False Positive Rate), TNR (True Negative Rate), FNR (False Negative Rate) and PA (Pixel Accuracy) are respectively.
defined in Equation] II
and 15
Table 7: Structure of pixel level confusion matrix together with basic statistical terms
Ground Truth
Positive (P)
Negative (N)
Positive (P)
True Positive (TP)
False Positive (FP)
Predicted
Negative (N)
False Negative (FN)
True Negative (TN)
TP
TPR TP+FN
(11)
24

---

Published as a conference paper at ICLR 2020
FP
FPR = FP+TN
(12)
TNR TN TN FF
(13)
FN FNR FN+TP
(14)
TP+TN
PA T +FN
(15)
25


Reviews:
[{"review_id": "r1e8JoT2tS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a MIL clustering method. The proposed MIL setup is called \"unique class count (ucc)\", this is, for a bag os samples ucc is the number of clusters in the bag. The method learns the features of the samples using two losses: an autoender loss and the ucc loss. Once trained on a dataset the method can perform clustering on classes (classify) better than a fully unsupervised clustering algorithm and worse than a fully supervised model. The method is evaluated on MNIST, CIFAR10, CIFAR100  and on binary breast cancer segmentation.\n\nThe main table of the paper (Table 1) compares the results of the proposed methods only with unsupervised methods and a fully supervised one. The results are reasonable. However, not a fair comparison. More fair comparisons can be found in Table 6 of appendix C.6 page 22 where the method is compared to semi-supervised methods. In this case, the proposed method performs worse than the semisupervised methods.\n\nUCC model uses bags of sizes from 1 to 4. Assuming uniform distribution, 25% of the samples are fully labeled. The semi-supervised methods from Table 6 how many labeled samples do they use? Having that small bag samples the problem is quite easy. Moreover, during training, I guess that the same sample can go to different bags. Is this right? If so, the annotation time would be very big. And also, one could trivially get the full label of each sample.\n\nThe article is excessively long. 10 pages for the main article and 12 extra pages for supplementary material which are needed for understanding the paper. Many of the definitions are redundant and there is excessive detail in explanations that can be expressed more simply. \n\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "SyedjL7pYH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a new type of weakly supervised clustering / multiple instance learning (MIL) problem in which bags of instances (data points) are labeled with a \"unique class count (UCC)*, rather than any bag-level or instance-level labels.  For example, a histopathology slide (the bag), consisting of many individual pixels to be labeled (the instances) could be labeled at the bag level only with UCC = 1 (for only healthy or only metastatic) or UCC = 2 (for mixed / border case).  The paper then proposes an approach for clustering instances based on the following two-step approach: (1) a UCC model is trained to predict the UCC given an input bag, and (2) the features of this learned UCC model are used in an unsupervised clustering algorithm to the get the instance-level clusters / labels.  The paper also provides a theoretical argument for why this approach is feasible.\n\nOverall this paper proposes a (1) novel and creative approach that is well validated both (2) theoretically and (3) empirically, and relevant to a real-world problem, and thus I believe should be accepted.  In slightly more detail:\n\n- (1) MIL where we are given bag-level labels only is a well-studied problem that occurs in many real world settings, such as the histopathology one used in this paper.  However to this reader's knowledge, this is a new variant that is both creative and motivated by an actual real-world study, which is exciting and alone warrants presentation at the conference in my opinion.\n\n- (2) The theoretical treatment is high-level, but still serves a clear purpose of establishing feasibility of the proposed method- this modest and appropriate purpose serves the paper well.\n\n- (3) The empirical results are thorough---e.g. the use of the two loss components for the UCC model are appropriately ablated, there are a range of baseline approaches compared to, multiple evaluation points are provided (i.e. both UCC prediction and final clustering metrics), and a real world use case is presented--and the results are impressive.\n\nSome comments to improve the paper:\n- The connection of the proposed UCC approach to the motivating histopath example should be explicitly stated upfront to help the reader understand how this method could be used and how it makes sense!\n- It seems that the KDE element of the UCC model was chosen in part to enable the theoretical analysis?  If so, this should be clearly stated to help the reader understand the design rationale.\n- Either way, it seems that a different approach than the KDE layer could have been taken- this should be added to the ablation experiments", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "BygIZGO3YS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors present a novel weakly-supervised multiple instance learning model based on a bag level label called unique class count(UCC). The core content of the method is to learn mapping between bags and their associated bag level ucc labels and then to predict the ucc labels of unseen bags. \nPositive:\n(1) The authors use the unique class count(UCC) in the bag as a weak, bag level label to design a deep learning based UCC model for extracting features and clustering the individual instances in the unseen bags. They also construct a new optimization objective function by combining ucc loss and autoencoder loss. This is a novel weakly-supervised clustering method rarely seen.\n(2) In this paper, a large number of experiments show the effectiveness of the algorithm in different data sets and semantic segmentation of breast cancer metastases.\nNegative:\n(1) The authors use KDE method to estimate the distribution of extracted features as the input of distribution regression module, but it is not clear how the description of KDE method updates on the parameters of the auto-encoder.\n(2) The proofs of some propositions in Appendix B are very obvious and redundant. The authors’ proofs are not enough to prove whether the designed UCC classifier is perfect.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a weakly supervised learning algorithm, motivated by its application to histopathology. Similar to the multiple instance learning scenario, labels are provided for bags of instances. However instead of a single (binary) label per bag, the paper introduces the setting where the training algorithm is provided with the number of classes in the bag (but not which ones). Careful empirical experiments on semantic segmentation of histopathology data, as well as simulated labelling from MNIST and CIFAR demonstrate the usefulness of the method. The proposed approach is similar in spirit to works such as learning from label proportions and UU learning (both which solve classification tasks).
http://www.jmlr.org/papers/volume10/quadrianto09a/quadrianto09a.pdf
https://arxiv.org/abs/1808.10585

The reviews are widely spread, with a low confidence reviewer rating (1). However it seems that the high confidence reviewers are also providing higher scores and better comments. The authors addressed many of the reviewer comments, and seeked clarification for certain points, but the reviewers did not engage further during the discussion period.

This paper provides a novel weakly supervised learning setting, motivated by a real world semantic segmentation task, and provides an algorithm to learn from only the number of classes per bag, which is demonstrated to work on empirical experiments. It is a good addition to the ICLR program.
Decision: Accept (Poster)