Multi-Instance Causal Representation Learning for Instance Label Prediction and Out-of-Distribution Generalization
Authors: Weijia Zhang
Xuanhui Zhang
Han-Wen Deng
Min-Ling Zhang
Venue: NeurIPS 2022 Accept (2022)
Keywords: multi-instance learning
variational autoencoder
causal representation

Abstract: Multi-instance learning (MIL) deals with objects represented as bags of instances and can predict instance labels from bag-level supervision. However, significant performance gaps exist between instance-level MIL algorithms and supervised learners since the instance labels are unavailable in MIL. Most existing MIL algorithms tackle the problem by treating multi-instance bags as harmful ambiguities and predicting instance labels by reducing the supervision inexactness. This work studies MIL from a new perspective by considering bags as auxiliary information, and utilize it to identify instance-level causal representations from bag-level weak supervision. We propose the CausalMIL algorithm, which not only excels at instance label prediction but also provides robustness to distribution change by synergistically integrating MIL with identifiable variational autoencoder. Our approach is based on a practical and general assumption: the prior distribution over the instance latent representations belongs to the non-factorized exponential family conditioning on the multi-instance bags. Experiments on synthetic and real-world datasets demonstrate that our approach significantly outperforms various baselines on instance label prediction and out-of-distribution generalization tasks.

Multi-Instance Causal Representation Learning for Instance Label Prediction and Out-of-Distribution Generalization
Weijia Zhang', Xuanhui Zhang", Han-Wen Deng', Min-Ling Zhang'
School of Computer Science and Engineering. Southeast University, Nanjing 210096, China 2 School of Information Management, Nanjing University, Nanjing 210023, China zhangwj hetpsw/sseeee..co zhangxhdo@@ 163.com, [denghw.zhangml] @seu.edu.cn
Abstract
Multi-instance learning (MIL) deals with objects represented as bags of instances and can predict instance labels from bag-level supervision. However, significant performance gaps exist between instance-level MIL algorithms and supervised learners since the instance labels are unavailable in MIL. Most existing MIL algo- rithms tackle the problem by treating multi-instance bags harmful ambiguities and predicting instance labels by reducing the supervision inexactness. This work studies MIL from new perspective by considering bags auxiliary informa- tion, and utilize it to identify instance-lewe causal representations from bag-level weak supervision. We propose the CausalMIL. algorithm, which not only excels at instance label prediction but also provides robustness to distribution change by synergistically integrating MIL. with identifiable variational autoencoder. Our approach is based on a practical and general assumption: the prior distribution over the instance latent representations belongs the non-factorized exponential family conditioning on the multi- multi-instance bags. Experiments on synthetic and real-world datasets demonstrate that our approach significantly outperforms various baselines on instance label prediction and out-of of-distribution generalization tasks.
Introduction
Supervised learning has achieved great success in many applications. However, an important limitation of existing supervised learning algorithms is that they model complex objects single feature vector and thus cannot make fine- grained predictions without the corresponding level of supervision, e.g.. localizing the region of interest within an image from coarse- coarse-grained image-level supervision. Unfortunately, acquiring fine grainedinn labels often not only a tedious task but also prohibitively expensive, especially for applications that require high level of domain-specific expertise such as drug activity prediction [8] and medical image classification [24].
Multi-Instance Learning (MIL) [8] is weakly supervised learning paradigm originally proposed for drug activity prediction, where the task is to predict whether a molecule is suitable for binding to target receptor. Since a molecule can take many low-energy conformations and its suitability for making drugs depends on some specific but unknown conformations, objects in MIL are represented by groups of instances called bags where each instance is described by its own feature vector, instead of represented by a single feature vector as standard supervised learning. Because only molecule- level drug binding suitability is known human experts, MIL algorithms only coarsely supervised at the bag-level, while fine-graine grained instance labels are unknown.
*Coresponding author
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

B
The inexactness MIL supervision causes the main difficulty of instance label prediction [42]. Under 2 S
2 5 4
O 2 12846622 666667 6
i
Zij
2 4 444441
in
2 184156791
X11
Xij
XC EiL
Z
Vi1
Zjj
Vij
iin
Vij
Standard Dosti-lnsttnce Assumption
Figure I: (Left) Our proposed graphical model. Shaded nodes denote observed variables and white nodes represent the latent ones. Dashed lines denote edges that may exist depending on the bags, while solid lines indicate relationships that invariant with respect the hags. (Right) Multi-instance bags of MNIST and FashionMNIST images and their reconstructions from =" (each class per row) inferred by CausalMIL.
Utilizing the bag-level supervision, the prediction tasks MIL are two-fold: predicting the bag labels, e.g., whether molecule is suitable for making drugs a human organ contains cancerous cells; and predicting the instance labels, e.g., which specific molecular conformations are suitable or which particular cells are cancerous. Although theoretical results have demonstrated the feasibility of learning accurate instance concepts from bag labels [9], the empirical performances of weakly supervised instance label predictions far behind their fully supervised counterparts.
the standard multi-instance assumption [11], the bag labels only inform the learner that instances from negative bags are negative, while the exact labels for instances in the positive bags are unknown. Most previous instance-level MIL algorithms solve the above problem by disambiguating the bag labels, i.c., finding the most positive instance from positive bags [28, 35] or utilizing the attention mechanism to infer how the different instances contribute the bag label [15, 31, 29].
In this work, we propose utilize the information hidden in the the latent factors that generate the observed instances, and utilize the bag labels for encouraging disentanglement among the latent factors. Specifically, we consider each observed instance as generated from instance-specific causal factors zi, and bag-inherited non-causal factors = as depicted in Figure (left). The -peacii factors = are responsible for capturing information that is causal to the instance labels, while the -ng-inhrrtted factors zij capture spurious information that may be inherited from the bags and correlates with the instance labels. The bag labels are considered the effects of their instance labels according to the standard multi-instance assumption.
bags for identifying
By identifying and disentangling the latent factors, our model brings two advantages over existing MIL algorithms. Firstly, as the identified latent factors capture semantic meaningful causal representations are of much lower dimensions than the observed instances, their identification significantly reduces the difficulties of inferring instance labels from bag supervision. Secondly, since only the instance- specific factors zú, are causes to the instance labels Yij. disentangling 2G, from 2f, remove spurious correlations between B, and Yıj- Therefore, identification and disentanglement of 2f, is not only beneficial for instance label predictions since 2f, will vary among bags, but also useful for distribbbtiiinnonnnn (OOD) generalization since the factors that are the effects of instances labels are excluded from the predictive model [4, 1, 23]. It is worth noting that although B, and yij are unconditionally independent, they become conditionally dependent because previous algorithms that do not identify or disentangle the latent factors implicitly condition on either Ziy or 2 (i.e., the collider descendants of the collider in v-structure [26)).
Our identifiability result builds upon the recent identifiable variational autoencoder (iVAE) [17], but extends to more flexible non-factorized prior distribution conditioning on the bag information instead of the factorized prior conditioning on single instance assumed the original iVAE, Furthermore, we propose to utilize permutation invariant set transformation network for aggregating information

---

from instances within the same bag to facilitate identifiability. Because the set transformation network universally approximates any set function [39], instances within the same bag are not only treated as sources of supervision inexactness but also effectively contribute to identifiability via conditioning the prior distribution. We summarize our key contribution as follows:
We propose novel CausalMIIL algorithm that utilizes the bag information with permutation invariant set transformation networks to theoretical guarantee the identifiability of latent factors under non-factorized conditional exponential family priors;
We propose general graphical model covering many real-world MIL applications, and show that the causal and non-causal factors of the instance labels can be disentangled from the identified latents based on bag-level supervision.
Using variety of datasets, we show that CausalMIL significantly outperforms existing MIL algorithms on instance label prediction tasks, and achieves better out out-of-dee oi-tiiitritution generalization capabilities when compared to supervised invariant algorithms.
Preliminaries
2.1 Notations
Let X' = T denote the instance space and y = = denote the label space. The training data contains samples organized in n training bags B = {BI,... B,,... B,), where each bag is set that contains different numbers of n. instances, i.c.. B, {I11.-. Bzj, Lin, } with æy E X. In the rest of this paper, we will also use B, denote the auxiliary information contained in the bag by abuse of notation. The learning algorithm is provided with bags B, and their associated bag labels Y; E y during training. Although there exists instance labels Vij E y for each Ziy. the instance labels are wnknown to the learners. We follow the standard multi-instane assumes that positive bags contain least one positive instance, and negative bags contain only negative instances. It is worth noting that the assumption only requires bags to be i.i.d., the instances within the same bag can be dependent.
assumption [11] which
2.2
VAE and Identifiability
We briefly introduce Variational Autoencoder (VAE) and its identifiability results. VAE can be considered as the combination of generative latent variable model and an associated inference model, where both models are parameterized by neural networks [18]. Specifically, VAE learns the joint distribution po(x, z) = (x (((z]=))] pe) where pe x) is the conditional distribution of observing æ given z, the set of generative parameters, and po (=))) = IT IT (P)(()
the factorized
prior distribution of the latents. By introducing an inference model qø ar), the set of parameters & and can be jointly optimized through maximizing the evidence lower bound (ELBO) on the marginal likelihood pæ()):
L = B
- DRL (gs x))(((z))
(1) where DKL. denotes the KL-divergence between the approximation and the true posterior, and L is lower bound of the marginal likelihood po (x) because of the non-negativity of the KL-divergence.
log (x) ( xz) Il [z) < log (z)),
Recently, it has been shown that VAEs with unconditional prior distributions pe (z) are not identifiable [22], but the latent factors can be identified with a conditionally factorized prior distribution pe u)) over the latent variables to break the symmetry, where u is an additionally observed variable [17].
3 Methods
3.1 CausalMII.
We now formally describe our graphical model depicted in Figure 1 (left). For any given bag B; we model its observed instances Biy as generated from unknown latent factors Zij = (a. =f,) Rd with d < m. We let zi, correspond to the causes of the instance label Bij, i.c., the line stroke that
3

---

makes a digit represent the Arabic number 'I' in Figure (right). In contrast 2 captures other factors related to the observed but are not causes of the instance label, ie.. the angle, thickness, and color of the line stroke that are not causal to 'I'. The solid lines of the graphical model indicate that the causal relationships are invariant across all bags, and the dashed lines denote that their relationships may vary or even be absent across different bags. The bag label Y, is determined by the labels of its instances Yij according to the standard multi-instance assumption. To sum up, we posit that the graphical structure in Figure should satisty the following assumptions:
Assumption 1. (a) The bag labels are determined by its instances according to the standard multi- instance assumption; (b) the causal graph in Figure 1 is a Divected Acyclic Graph (DAG); (c) given the latent factors Zij, the observed instances are independent of the labels and the bags: Tij 11 Yij, Y;, B,lzig. i.e., the generation mechanism p(Zig kis)) is invariant across bags; (d) Vis 11 B, Zig i.e, the mechanism between the causal factors 2ig and Yig is invariant across bugs.
We now discuss the practicality of Assumption 1. Firstly, the standard multi-instance assumption commonly used in a wide range of MIL applications such as medical image analysis [20], fine- ane-grained sentiment analysis [3], and sound event detection [36]. Secondly, modeling the causal structures as DAGs are common in causal discovery [26, 27J. To the best of our knowledge, most of the causality-based machine learning literature assumes the causal structure to be acyclic. Thirdly, it also makes sense that generating mechanism from latent factors to observed instances p(Zr,)122) is invariant since causal mechanisms it would be impossible to infer # from æ for any test instances. Lastly, the fourth assumption is not only commonly adopted in invariant and causal representation learning literature [4, 23], but also suitable for MIL applications. For example, when diagnosing whether a cell from a tissue is cancerous, the causal mechanism of diagnosing cell should be invariant across bags, while the non-causal factors caused by the patient or equipment differences may often change [40].
considered to be stable across environments [30]. Otherwise,
From now on we will drop the subscript to avoid cluttering the notations. In the following discussion we will show that when the underlying data generating mechanism satisfies Assumption 1, the latent variables z can be identified up to permutation and affine transformations if the conditional prior distribution p(z B,y) y) belongs to a general exponential family distribution:
Assumption 2. The prior distribution of the latent factors p(z) B. y) follows a general exponential
family with its parameter specified by an arbitrary function X(B,y) v) and sufficient statistics T(z)
(Tr(z)² (=x(== Here the sufficient statistics T=2) is defined by the concatenation of T = famill family and the outputs of a neural networks (2) with universal approximation power. Then, the probability density can be written as:
(T((2)) Ta(za)TJ7 TT from a factorized exponential
2(2) (x))
PT, B, =
(2)
C(B,y)
where Q(z) is the base measure and C(B,y) is the normalizing constant.
The above assumption is inspired by the recent advancement of iVAE [17]. However, there are two differences. Firstly, because Figure allows for connections among the components Zi within z as long as the generative model remains a valid DAG, our non-factorized conditional prior assumption more general than the factorized distribution assumed by iVAE. In Equation 2, we utilize a neural networks NNN(2") for capturing the interactions among components of 2. Secondly, observed auxiliary information is provided by multi-instance bag B which may consist arbitrary number of instances instead of single scalar vector-valued instance it is natural to ask (1) if identifiability still holds for non-factorized priors, and (2) how to aggregate information from bags of varying sizes for breaking the symmetry of the prior distribution?
additionally
used by iVAE. Therefore,
To answer the above questions, we first describe our generative model and discuss its identifiability. Formally, let us consider the following generative models under Assumption 2:
(0 2 =| v) = (=|=)) (2
(3)
pf (a| = (= f())),
(4)
where Equation 3 describes the generative process of x given the underlying latent factors 2, along with the bag context B and the instance label y. Equation 4 implies that the observed representation is an additive noise function, i..., x f(z) + where is independent of f or z.

---

Following standard VAE and iVAE derivation, the evidence lower bound for each instance in bag B of the above generative model can be written as:
LELBO = (z [log (z| + Pr.
(5)
- log qs æ B, y)] + log p(B)
(6)
Note that the ELBO in Equation 6 contains an additional term of the bag prior log p( B) that do not affect identifiability but improves the estimation for the conditional prior [25]. Our main identifiability results can now be described as:
Theorem 1. Assume we observe instances from mulli-instanne bags sampled according to Equation 2-4. Furthermore, assume that the following holds:
i. The set {x E X: (2(x=== 0} has measure zero, where Pe is the characteristic function of the density Pe defined in Equation 4.
ü. The function f is injective and all of its second-order cross partial derivatives exist.
ii. The sufficient statistics T are twice differentiable.
iv. There exist k +-1 1 distinct bags (B", ......... (B", 1B v²) such that the k x k matrix E is invertible, where k is the dimension Tf and E is defined as:
E = =x((B,,))
A(B", )..,.... X(B", - ,))))
(7)
Then, the parameters A (f. T, A) are identifiable up to an equivalence class induced by permuta- tion and componentwise transformations.
is interesting to observe that condition (iv) of Theorem is easier to satisfy in MIL than standard supervised learning. Intuitively, the condition requires that the auxiliary information should "break the symmetry" in the representation space the model could learn, An intuitive analogy is inferring an object's s shape from its shadow: if we only observe know its shape; however, if we observe multiple objects under similar lighting conditions (from a bag of instances), we may identify the lighting. If we observe an object under different lighting conditions (from many bags of instances), we may identify the underlying shape. Another analogy is to consider the bags H&E stained histopathology images and the instances as image patches: in one bag, we observe cells under the same staining process, which provides information regarding the staining procedure; in different bags, we observe cancerous cells under different staining procedures, which makes it possible to infer the causal representations of cancerous cells. In standard supervised learning, instances are independent with no auxiliary information given. In MIL, instances within bags are naturally dependent and organized in bags, which may then provide the necessary auxiliary information for latent identifiability. We provide the proof for Theorem 1 in the Appendices.
shadow of the object, it's difficult to
Technically, condition (iv) can be restated as requiring the vectors (A(B), (y')
A(B", v'),..., A(B y²* - y))) to be independent. Therefore, the instance label y becomes
unnecessary there exist l: ++11 distinct bags such that = (A(B)) X X(B"). ..........B")) A(B*) A(B")
of size k x l is invertible, which is especially attractive in MIL where the instance labels are not available but the bags are abundant.
Two practical issues need to be addressed before applying the identifiability theorems to solve MIL problems. The first hurdle lies in how to characterize the auxiliary information for conditioning the prior distribution of z. This is different from standard iVAE where the auxiliary information is readily provided in a single-instance u since the auxiliary information in MIL are provided in the form of instances sampled from bags. To make things worse, the auxiliary information also cannot be derived from the bag labels or the multi-instance assumption. This is because they only tell the bag labels and the relationship between instances labels and bag labels, instead of the generative process of the bags and their instances.
To solve the above problem, we propose to utilize a trainable permutation invariant function parame- terized by deep neural networks. Formally, define:
(8) where p and e are arbitrary continuous functions implemented through neural networks, pool the sum operator over the instances transformed by the neural network ø, and finally p is another neural
B: = net( (#..... Fin, }) plpool( (o( ((.1) , .( ))),
5

---

network that operates on the pooled transformations. An important property of the transformation defined in Equation 8 is that is capable to express any permutation invariant function that operates on set inputs [39]. Since multi-instance bags are both permutation invariant and contain instances of varying sizes [15], is adequate to model its bag information as a set function of the instances.
The second missing piece of the puzzle is how to separate x" from =" in the identified latent factors 2, since we would only require z" for the downstream tasks. To solve this problem, we take a different approach from a typical VAE-based disentanglement algorithm where their goal is to ensure the components of 2. i.e., 2p for p = 1. 1.... d, are independent and correspond meaningful semantics disentanglement
[13]. Although component-win general, due to the fact that that a more pragmatic approach is to utilize the bag labels for separating the causal factors x" of the instance labels from the non-causal ones 2t without requiring additional information.
is desirable for causal representation learning in bags are only weakly supervised at the bag level, we argue
We now revisit the graphical model in Figure 1 to discuss the benefit of not pursuing component-wise disentanglement and using only 2 for instance classification. The instance labels y are correlated with both = and z", and thus the entirety of x is predictive of y. Therefore, for supervised learning tasks that assume i.i.d. training and test data, using z is preferable for prediction. However, in MIL our goal is to predict the instance labels from unseen test bags. Since the bag-inherited factor =" may be different among training and test bags, using z" would negatively affect prediction. Furthermore, labels from bag-level weak supervision,
since instance classifiers in MIL must infer instance- excluding spurious correlations and including only the low dimensional causal representation also reduces the difficulty in learning the instance classifier. Another important benefit of separating =n from 2t is that it also promotes OOD generalization. By considering bags as environments and the non-causal factors z" as the effects of environment-induced biases, using 2 and excluding = improves prediction robustness since ply|2 ) is invariant across bags as specified by Assumption ((d), which also coincides with the goal of invariant causal prediction algorithms such as [4, 1, 23).
If the instance labels are known, then our goal of disentanglement can be straightforwardly achieved by utilizing the PC algorithm [33] to learn Markov equivalent class of DAGs and find the direct causes as done in [23]. Unfortunately, this is not the case for MIL where only bag labels are known. To solve this problem. we propose to utilize the bag label Y with the ELBO in Equation 6 using a bag-wise maximum operator. Specifically, we optimize for the following target
Ca log p (a= ==) + a log Pw (Y|=") KI-(1<<=" "0=", B))lPPAA== [B]
+ log po [ [ KL (L (2 B) (B)].
where: x" arg max Pw P (Y and pw (Y
linear classifier with a as its weighting hyper- parameter. It is also worth noting that the above ELBO is written with respect to multi- bag instead of the instances. Furthermore, the last two terms can be obtained by deriving an ELBO for logPo( Po (B) in Equation 6 [25].
In other words, the reconstruction, classification, and conditional KL terms in the ELBO are optimized for only one instance per bag, and the instance is chosen using a maximum operator to satisfy the standard multi-instance assumption. The formulation of Equation is effective for learning 2" while ignoring x because of the following two reasons. Firstly, as the ELBO is optimized per mini-batches consisted of many multi-instance bags using the reparameterizztiion trick [18], the optimization process will force the linear classifier Pw (yl2z) (2) to predict using the 2" and ignore 2"; otherwise, the linearity of Pw (y 2) will not be able to predict the instance labels since z" are different for different bags in the mini-batch. Moreover, because the reconstruction probabilities are evaluated at only one latent factor for each bag, the encoder (2|e, B) is encouraged to encode only 2 since it carries the content causal information for reconstructing the positive x and remains invariant across different bags in the mitc-batch. To see this, consider the opposite scenario where the reconstruction losses are evaluated for every instance in the bag, the encoder would also encode x" since it remains the same within that bag. Therefore, CausalMIL is targeted to learn z" that are causal to the positive instances of a bag, and ignores both the non-causal factors z" and the causal factors for negative instances, as illustrated by the reconstructions in Figure (right) and Figure 2.
It is worth noting that there exist two types of approximate inference in CausalMIL. The first one, common to almost all VAE-based algorithms, the amortization gaps between the ELBOs and the marginal log-likelihoods caused by amortizing the variational parameters over the entire dataset, instead of optimizing for each training sample individually. The second one, specific to CausalMIIL, is the approximation gap of only inferring the log- likelihood of one instances per each bag. However, as

---

Table I: Results of instance label prediction performances MNIST, FashionMNIST, and KuZushjiMNNIS
bags. The reported means std are the macro averaged 10 one-vs-rest datasets, respectively.
MNIST Bags
FashionMNIST Bags
KuzushiJiMNISTT Bags
F-score
AUC-PR
F-score
AUC-PR
F-score
AUC-PR
mi-NET Attn-MIL
0.59511.078
0.702.1.165
0.251:1.29
0.329-1.450
0.50
0.616:1.164
0.712:4.147
0.776:1.222
0.398:1.244
0.53411.144
0.638
0.654:1.209
KSA-MIL
0.775++092 0.845..122
0.545
0.617--.300
0.7074.088
0.782+1.116
MIVAE
0.901#.035 0.921:10066
0.701
0.733#1277
0.779:..47
0.838=.102
CausalMILL
0.966-t.018
0.981+1.018
0.748 .206
0.822++.168
0.833+0..109
0.934:10075
corroborated by our experiment results, these approximation behaviors does not prevent the success of CausalMIL instance label prediction and OOD generalization.
4 Experiments
In this section, we first evaluate the instance label prediction performances of CausalMIL against MIL baselines including mi-Net [35], Attention- bhesse MIL (Attn-MIL) [15]. Kernel Self-Attention-basee MIL. (KSA-MIL) [29], Multi-Instance Variational Autoencoder (MIVAE) [40]. Then, we evaluate the out-of-distribution generalization ability of CausalMIL by comparing with supervised learning algorithms including ERM and variants, Invariant Risk Minimization (IRM) [4], IRM with Game Theory (IRM GAME) [1], Invariant Causal Representation Learning (iCaRL) [23].
We implemented CausalMIL using PyTorch and conducted most of the experiments with a single NVIDIA RTX3090 GPU. Detailed settings, parameters, and data for reproducing the results are provided in the Appendices. The implementation code is publicly available at https: https://githaan github.. com/ Wei wtwpaaanggg//aaaal//aaaa
4.1 Instance Label Prediction
4.1.1
Datasets
Our quantitative evaluation first utilizes 30 multi-instance instance classification datasets generated from the MNIST [19], FashionMNIST [38], and KuzushijMMNIST [7] datasets. The bag construction procedure similar to the multi-instance bags generation from the 20 Newsgroup corpus [43]. Specifically, we create the multi-instance MNIST-bags such that each bag contains number of images where its bag size is drawn from Gaussian distribution with fixed mean and variance. The bag is positive if contains a target digit, e.g., '4', and negative if otherwise. Using 10 different digits as targets, obtain 10 multi-instance MNIST-bag datasets. Similarly, we construct 10 FeshionMNNST--ags and 10 KuzzshijMMNITT-baa
We also evaluate CausalMIL on a hematoxylin and cosin (H&E) stained Colon Cancer histopathology task [32]. Histopathology from whole-slide images is an important application of MIL since super- vised predictions require pathologists to provide pixel-level annotations which is extremely expensive and time-consuming. The dataset contains 100 images obtained from tissues of either normal malignant patient tissues. For each image bag, the instances are generated as patches 27 x 27 pixels using markings of major nuclei for each cell. A total amount of 22,444 instances (-220 instances per bag) are provided with ground truth instance labels, i.e. whether the cell is epithelial. A bag is labeled positive it contains least one epithelial patch and negative if otherwise.
4.1.2 Quantitative Results
For quantitative evaluation, we use the area under the Precision- Recall curve (AUC-PR), instead of area under the ROC curve due class imbalance at the instance level. For example, although the original MNIST datasets are balanced, only -5% of the instances are positive in the MNIST-bags. We also report the f-scores of the compared methods. For the methods that directly output instance prediction such as mi-NET, MIVAE, and CausalMII., the results are obtained from thresholding their predicted scores. For methods based on the attention mechanism, such as Attn-MII. and KSA- MIL, the results are obtained by first normalizing the instance-to-bag attention weights to [0, and
7

---

Table 2: Instance label prediction performances of the Colon Cancer dataset. Experiments repeated for
times the average standard deviation 5-fold cross validations are reported.
Method
F-score
AUC-PR
mi-Net
0.392##017
0.491#.028
Attn-MIL
0.466#.037
0.536#.014
KSA-MIL.
0,510+-0029 0.578++.025
MIVAE
0.675+.033 0.747+
CausalMIL
0.8331.024
0,8781.046
/ 0//200
0 2
3 6
4
2
5
8
4 6
S
Figure 2: MNIST digits and CausalMIL reconstructions for digits '4', '7', '9", Figures are best viewed when zoomed. FashionMNIST objects and CausalMIL reconstructions for 'dress", 'sandal', and 'handbag'. More results can be found in the Appendices. Figures best viewed when zoomed.
applying the threshold. As there is usually no instance label available in real application scenarios, we use 0.5 uniformly as the classification threshold.
Table 1 shows the quantitative results MNIST, FashiOnMNIST, and KuzushijiMNIST bags. CausalMIL performs significantly better than the compared methods on both metrics. Note that the large standard deviations for FashionMIIIST-bage
because some fashion objects are much more difficult to classify (e.g., distinguishing shirts from pullovers and coats). Furthermore, we can see that attention-based MIL algorithms exhibit significant performance variations even for MNIST-bags. A possible reason is that when the data satisfy the standard multi-instance assumption, learning weighted average of the individual instance contributions is not efficient. Among the compared algorithms, CausalMIL achieves the best performances with the lowest standard deviations.
Table 2 shows the instance prediction performances of the compared algorithms the Colon Cancer dataset. The result trends are similar to previous ones. For discriminative deep learning- based algorithms, iitentiionpoolingg (Attn-MIL. performs better than max-pooling (mi-Net), while incorporating the self-attention mechanism (KSA-MIL) further improves the performances. A recent VAB-based generative MIL algorithm (MIVAE) performs better than the discriminative algorithms but still significantly worse than CausalMIL because it does not learn meaningful representations and does not provide identifiability guarantee.
4.1.3 Qualitative Results
We now qualitatively evaluate the inferred latent factors of CausalMIL by examining the recon- structions of decoder Py (arl2) using the inferred latent factors in Figure 2. For example, in the first row we show the original images and reconstructions of MNIST-bags using '4', '7', and '9' as positive instances. In the reconstructions of '4' we can see that CausalMII. learns the semantic causal characteristies of the positive instances while ignoring the others. The '4's are reconstructed with
8

---

Table 3: Classification performances distributionally biased ColoredMNIST and ColoredFashionnNNNST
datasets in terms of accuracy + standard deviation. CausalMII. (ours) is weakly supervised whereas the other approaches are supervised.
ColOTedMNIST
ColoredFashionMNISST
Train
Test
Train
Test
ERM
0.849+40002
0.105+.007
0.832+E.010
0.225+.007
ERMI
0.84&+0000
0.109+.005
0.813+.014
0.333+0089
ERM2
0.850=1.002
0.101#.002
0.844#.019
0.132+0008
ROBUST MIN MAX
0.843=1004
0.1
0.828+.001
0.292+.086
IRM
0.593:1.044
0.628:1096
0.750+1.003
0.553#.124
IRM GAME
0.634#.011
0.599+.027
0.690#.101
0.702+.015
iCaRL
0.706=#000
0.688#.007
0.750=10044
0.736*.006
MIVAE
0.810#0005
0.156++0033
0.823+.003
0.284#.100
CausalMIL
0.919110022
0.8924.002
0.8731.007
0.816
"U" shape at the top and a short line at the bottom, while other digits are reconstructed to noise. The "U" shape of '4' is clear because it crucial for the positive instance; however, the line the bottom is blurry since it can be written in different angles and the angles are irrelevant. The reconstructions from FashionMNIST bags and KuzushijiMNNSS-bbaie datasets also corroborate the above results.
4.2 Out-of-Distribution Generalization
To further validate whether CausalMIL identifies the causal representations z", we evaluate CausalMIL on dustof-distribution (OOD) generalization tasks and compare supervised algorithms designed specifically for OOD generalization [4, 1, 23].
We utilize two widely-adopted biased classification tasks, ColoredMNIST and ColoredFashhonMNNIST that are commonly used in the invariant risk minimization literature [4]. The datasets are constructed following the same procedure as IRM, where the task is classify whether a digit less than 5. The instance labels are first binarized and randomly flipped with 25% probability. Then, the images are colored by green with probabilities Pe which vary across environments, while the rest Pe of the images are colored by red. There are three environments (two training, one test) where Pe = 0.1, 0.2, and 0.9, respectively. Therefore, simply predicting colors instead of digits will lead to high accuracy in the training environments, but the correlation is reversed in the test environment. For CausalMIL, we construct multi-instaace bags based on the flipped instance labels utilizing instances from both training environments. Only the bag labels are provided to CausalMIL during training, while the instances from the test environment are used for testing.
We compare the instance label prediction performances of CausalMIL against two categories of supervised algorithmss. The first category includes standard Empirical Risk Minimization using CNN trained with both (ERM) and each of the training environment (ERMI and ERM2), and ROBUST MIN MAX which minimizes the maximum loss across different environments. The second category are specifically designed OOD generalization algorithms including Invariant Risk MinimizattinnIIMM [4], IRM GAME [1], and iCaRL [23]. Considering that most of the baseline results come from iCaRL [23], we set CausalMIL to use the same convolutional structure as [23] for fair comparison.
The results are reported in Table 3. We can that CausalMIL. performs significantly better than all compared supervised methods. There are two reasons behind this. The first reason is that CausalMIL. effectively forces the biases into x" by disentangling the causal from non-causal factors, and uses only = for prediction. Although previous supervised OOD generalization algorithms, i.c., IRM and iCaRL, aim to identify and only use the causes of y to improve robustness, they mainly rely on the instance-level labels for identifiability and utilize the scarcely available environments for robustness. Unfortunately, as the instance- their approaches are less effective. On the contrary, in CausalMIL we rely on the abundance of bag information for identification. The second reason is that the formulation of MIL is naturally more robust to instance-level label noises as positive bags naturally consist of both positive and negative instances. Furthermore, previous theoretical results have also shown that generative MII. algorithm [9] is effective even when negative bags are falsely labeled as positive.
labels are biased from the training environments,

---

5
Related Work
Most MII. algorithms can be categorized into two groups according to whether they work at the bag space or at the instance space. Bag space algorithms [43, 37, 41, 10] work by embedding multi-instance bags into single feature vector representation and then solving the single-instance learning problem in the embedded space, and therefore are capable of predicting instance labels. Instance space MIL algorithms aim to directly separate the positive instances from the negative ones [2, 16, 12, 34]. These algorithms can be used to predict instance labels, although not all of them are explicitly designed for the lask.However, these algorithms only work with pre-computed features.
Recently, several deep learning-bass -haned MII. algorithms have been proposed by utilizing permutation- invariant pooling operations, and can be used for instance label prediction. For example, [35] used max-pooling layer with fully connected neural network; [15] introduced the attention mechanism as permuutaionninvariian MIL pooling operation and used the attention weights for instance label prediction; [29] proposed to integrate self-attention with the attention mechanism for capturing the non-i.i.d. information among instances; [21] utilized Gumbel reparametrization and proposed an algorithm for the generalized MIL assumption. Unfortunately, none of the above methods learn semantically meaningful representations.
Unsupervised learning of meaningful representations with VAE-based models has attracted much attention [13, 6]. Since it has been shown that unsupervised disentanglement is theoretically impos- sible without inductive biases [22], various methods have been proposed for learning disentangled representations using different forms of weak supervisions. One form of weakly-supervised dis- entanglement learning that is closely related to MIL is group-based VAE [5, 14]. Similar to MIL where instances are organized into bags, group-based basup algorithms require the objects to be divided into groups: those within the same group share the same content but have different styles that are independent of the contents, while those among different groups should have different contents.
Recently, MIVAE [40] is proposed based on connections between MIL and group- based disentangled representation learning which utilizes VAE for capturing the dependencies among instances as the shared content factors. However, MIVAE also does not learn semantically meaningful latents, nor does it provide identifiability on what latents it actually learns because of its unconditional prior. Please refer to the Appendices for a more detailed comparison between CausalMIL and MIVE.
6 Conclusion
In this work, we proposed CausalMIL, an algorithm that learns semantically meaningful causal representations from multi-instance bags and the corresponding bag-level supervision. The learned representations not only significantly improve the performances of traditional MII. tasks such as instance label prediction, but also exhibits notable performances in supervised learning tasks such as OOD generalization. Qualitative and quantative evaluation results show that CausalMIL performs significantly better than existing state-of-the-an deep MIL algorithms semi-synthetic and real- world datasets.
There are some limitations of our method and many future directions worth investigating. For example, CausalMIL is only designed for the standard see if the results extend to other assumptions in multi-instanne learning. A possible way for such extension utilize the Gumbel reparameterization as discussed in [21]. Another possible direction would he developing an extension to learning from positive unlabeled (PU) examples, another weakly supervised learning problem that closely related to MIL.
assumption. would be interesting to
Acknowledgments and Disclosure of Funding
The authors wish to thank the anonymous reviewers for their constructive comments and suggestions. This work was supported by the National Science Foundation of China (62176055, 62206047, 72204110) 10). We thank the Big Data Center of Southeast University for providing the facility support on the numerical calculations in this paper.
10

---

References
[1] Kartik Ahuja, Karthikeyan Shanmugam, Kush Varshney, and Amit Dhurandhar. Invariant risk minimization games. In Proceedings of the 37th International Conference on Machine Learning. PMLR 119, pages 145-155, 2020.
[2] Stuart Andrews, Ioannis Tsochantaridis, and Thomas Hofmann. Support vector machines for cultipleeinstance learning. In Advances in Neural Information Processing Systems 15, pages 577-584, 2002.
[3] Stefanos Angelidis and Mirella Lapata. Multiple instance learning networks for fine tne-grained sentiment analysis. Transactions of the Association for Computational Linguistics, (6):17-3) 34, 2018.
[4] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk mini- mization. arXiv preprinl, arXiv: 1907.02893, 2019.
[5] Diane Bouchacourt, Ryota Tomioka, and Sebastian Nowozin. Multi-level variational autoen- coder: Learning disentangled representations from grouped observations. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, pages 2095-2102, 2018.
[6] Ricky T. Q. Chen, Xuechen Li, Roger Grosse, and David Duvenaud. Isolating sources of disentanglement in variational autoencoders. In Advances in Neural Information Processing Systems 31, page 2615-2625, 2018.
[7] Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep learning for classical Japanese literature. arXiv preprint, arXiv:1812.0177 2018.
[8] Thomas G. Dietterich, Richard H. Lathrop. and Tomás Lozano-Pérez. Solving the multiple instance problem with axis-parallel rectangles. Artificial Intelligence, 89(1-2):31-71, 1997.
[9] Gary Doran and Soumya Ray. Multiple-instance learning from distributions. Journal of Machine Learning Research, 17(128):150, 2016.
[10] Lei Feng, Senlin Shu, Yuzhou Cao, Lue Tao, Hongxin Wei, Tao Xiang, Bo An, and Gang Niu. Multiple-instance learning from similar and dissimilar bags. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, page 374-382, 2021.
[11] James Foulds and Eibe Frank. A review of multi-instance learning assumptions. The Knowledge Engineering Review, 25(1):1-25, 2010.
[12] Manuel Haussmann, Fred A. Hamprecht, and Melih Kandemir. Variational bayesian multiple instance learning with gaussian processes. In 2017 IEEE Conference on Computer Vision and Pattem Recognition, pages 6570-6579 2017.
[13] Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. VAE: Learning basic visual concepts with a constrained variational framework. In Proceedings of the 5th International Conference on Learning Representations. https://oonneeeiew.netttouum//np
2017.
[14] Haruo Hosoya. Group-based learning of disentangled representationswith generalizability for novel contents. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pages
2019.
[15] Maximilian Ilse, Jakub Tomezak, and Max Welling. Attention dlentioo-bbseed deep multiple instance learning. In Proceedings of the 35th International Conference on Machine Learning, PMLR 80, pages 2127-2136, 2018.
[16] Melih Kandemir and Fred A. Hamprecht. Instance label prediction by dirichlet process multiple instance learning. Proceedings of the 30th Conference on Uncertainty Artificial Intelligence, pages 380-389, 2014.

---

[17] llyes Khemakhem, Diederik P. Kingma, Ricardo Pio Monti, and Aapo Hyvärinen. Variational autoencoders and nonlinear ICA: A unifying framework. In Proceedings of the 23rd Inter- national Conference on Artificial Intelligence and Statistcs, PMLR 108, pages 2007-2017, 2020.
[18) Diederik Kingma and Max Welling. Auto-encoding variational bayes In Proceedings of the 2nd International Conference Learning Representations. http://openeevee..effffnmmiiici hittformmmmdd333XX9099999922 2014.
[19] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2333 1998.
[20] Bin Li, Yin Li, and Kevin W Eliceiri. Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14318-14328, 2021.
[21] Xin-Chun Li, De-Chuan Zhan, Jia-Qi Yang, and Yi Shi. Deep multiple instance selection. Science China Information Sciences, 64(3), 2021.
[22] Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Raetsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. Challenging common assumptions in the unsupervised learning of dissettaggled representations. In Proceedings of the 36th International Conference on Machine Learning, PMLR 97, pages 41 4124, 2019.
[23] Chaochao Lu, Yuhuai Wu, José Miguel Hemández-Lobato, and Bernhard Schölkopf. Invariant causal representation learning for oul-of-distriiutioo generalization. In International Conference on Learning Representations.
2022.
[24] Ming Y. Lu, Tiffany Y. Chen, Drew F.K. Williamson, Melissa Zhao, Maha Shady, Jana Lipkova, and Faisal Mahmood. Al-based pathology predicts origins for cancers of unknown primary. Nature, 594: 106-1001,, 6 2021.
[25] Graziano Mita, Maurizio Filippone, and Pietro Michiardi. An identifiable double vae for disentangled representations. In Proceedings of the 38th International Conference Machine Learning, PMLR 139, pages
2021.
[26] Judea Pearl. Causality. Cambridge University Press, 2009.
[27] Jonas Peters, Peter Bühlmann, and Nicolai Meinshausen. Causal inference by using invariant prediction:
identification and confidence intervals. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 7855)947-1112, 2016.
[28] Jan Ramon and Lue De Raedt. Multi-instance neural networks. Proceedings of the ICML- 2000 Workshop on Attribute-valee and Relational Learning, pages 53-60, 2000.
[29] Dawid Rymarczyk, Adriana Borowa, Jacek Tabor, and Bartosz Zielinsk. Kernel self-attention for weakly- weakly-supervised image classification using deepmultiple instance learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1721
2021.
[30] Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning. Proceedings of the IEEF, 109(5):612-633) 2021.
[31] Xiaoshuang Shi, Fuyong Xing, Yuanpu Xie, Zizhao Zhang, Lei Cui, and Lin Yang. Loss-based attention for deep multiple instance learning. Proceedings of the 34th AAAI Conference on Artificial Intelligence, pages 5742-5749, 2020.
[32] Korsuk Sirinukunwattana, Shan Ahmed Raza, Yee-Wah Tsang, David R. J. Snead, Ian A. Cree, and Nasir M. Rajpoot. Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Transactions on Medical Imaging, 35(5): (115):119--20000 2016.
12

---

[33] Peter Spirtes, Clark Glymour, and Richard Scheines. Causation, Prediction, and Search. The MIT Press, 2000.
[34] Fulton Wang and Ali Pinar. The multiple instance learning gaussian process probit model. In Proceedings of The 24th International Conference Artificial Intelligence and Statistics, PMLR 130, pages 3034 3042, 2021.
[35] Xinggang Wang, Yongluan Yan, Peng Tang. Xiang Bai, and Wenyu Liu. Revisiting multiple instance neural networks. Pattern Recognition, 74:15 24, 2018.
[36] Yun Wang, Juncheng Li, and Florian Metze. A comparison of five multiple instance learning pooling functions for sound event detection with weak labeling. In 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 31-35, 2019.
[37] Xiu-Shen Wei, Jianxin Wu, and Zhi-Hua Zhou. Scalable algorithms for multi-instance learning. IEEE Transactions on Neural Networks and Learning Systems, 2874)9975-987, 2017.
[38] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-MNIST: novel image dataset for benchmarking machine learning algorithms. arXiv preprint, 1.XXiv:1708.0774477 2017.
[39] Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander Smola. Deep sets. In Advances in Neural Information Processing Systems 30, 2017.
[40] Weijia Zhang. Non-i.i.d multi-instance learning for predicting instance and bag labels using variational autoencoder. In Proceedings of the 30th International Joint Conference Artificial Intelligence, pages 3377-3383, 2021.
[41] Weijia Zhang, Lin Liu, and Jiuyong Li. Robust multi-instance learning with stable instances. In Proceedings of the 24th European Conference Artificial Intelligence, pages 1682-1689, 2020.
[42] Zhi-Hi Hua Zhou. A brief introduction to weakly supervised learning. National Science Review, 5(1):44-53, 2017.
[43] Zhi-Hua Zhou, Yin Sun, and Yu-Feng Li. Multi-instance learning by treating instances as non-i.i.d. samples. In Proceedings of the 26th International Conference Machine Learning, pages 1249-1256, 2009.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [N/A]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes] In the Appendix.
3. If you experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e-g., with respect the random seed after running experi- ments multiple times)? [Yes]
13

---

(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you using existing assets (e.g., code, data, models) curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material as a URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you'le 're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "yTq-NdOq5CM", "reviewer": "Reviewer_s6pT", "summary": "This paper synergizes identifiable variational autoencoder with multi-instance learning, an important weakly supervised learning problem. By utilizing the instances in the multi-instance bags as auxiliary information, the proposed method provably identifies the latent factors of the positive instances up to affine transformations. The proposed method then utilizes the inferred latents and achieves significantly better results in downward tasks such as instance label prediction and out-of-distribution generalization. Empirical evaluations of qualitative latent reconstructions support the identifiability claim. Quantitative results on several benchmark datasets also show that the proposed method is significantly better at predicting instance labels and out-of-distribution generalization than the baselines.", "questions": "1.\tWhat is the scope of domains for the proposed causal graph in Figure 1? Is it applicable to weakly-supervised image classification problems? Discussing some practical problems for which this causal graph is suitable would be preferable.\n2.\tAt the high level, how does the proposed VAE-based MIL method compare to the methods that are based mainly on attention, such as [16] and its follow-up works? As VAE-based MIL algorithms is very different from the current trend of attention-based MIL algorithms, what are the considerations when choosing one over another?\n", "limitations": "Yes. The authors have discussed that the method only applies to the standard multi-instance assumption. Solving the identifiability problem in other MIL assumptions remains to be explored.\n\n", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n--\tThis paper presents a novel method for integrating multi-instance learning with identifiable latent representation learning.\n--\tThe paper is well written and nicely presented.\n--\tThe qualitative latent reconstruction results are novel and interesting, and the quantitative results show significant improvement over the baselines.\n--\tThe out-of-distribution generalizability brought by the identified latent factors broadens the impact of MIL methods to other machine learning subfields.\n\nOverall, the strength of this paper is in the formulation of multi-instance learning as an identifiable VAE with auxiliary information problems. This provides a novel perspective for MIL band motivates better methods not only for instance label prediction but also for out-of-distribution generalization. The focus on multi-instance learning is important as it has been much overlooked compared to how prevalent it is in important real-world applications such as whole-slide medical imaging and fine-grained prediction, and the focus on identifiability has been shown to be useful to further the MIL methodology. In terms of writing, it nicely addresses the challenges of MIL and identifiability and builds up its case coherently. Identifiability is first analyzed under the correct graphical model, then relaxed to accommodate the multi-instance assumption. The model assumptions are clearly stated as well as the assumptions about the data generating process.\n\nWeakness:\n--\tSome minor issues in the text could be improved with further proofreading. Please see the detailed comments below.\n\nMinor Points:\nLine 147, “In Equation 2”, -> “In Equation 2,”\nLine 160, “can then be written as” -> “can be written as”\nLine 226, “and thus ensures identifiability” -> “and to ensure identifiability”\nLine 263, “Appendix” -> “Appendices”\nIn Table 2, “TargetedMIL” should be in bold font.\nLine 628 (in the Appendix), “Table ??” should be “Table 3”.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "v36CVg-7opp", "reviewer": "Reviewer_WkwK", "summary": "The authors investigate a generative model for MIL data where instances are generated from bag level and instance level latent factors. They develop variational auto encoders to model this generative process and show an identifiability condition. Experimental results are presented on some synthetic MI datasets and one real world problem which show advances on the state of the art.\n", "questions": "see above", "limitations": "see above", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "\n+ Paper attempts to explore the use of causal models in MIL, which is a nice, not well explored direction.\n+ The experimental results are very impressive and it seems the approach can work well in the real world domain investigated.\n\n- The paper is dense and hard to read. There are many typos. \n- The paper borrows heavily from two prior papers, one on identifiable VAEs and one on a multiple instance VAE. It was not clear to me that the advances made beyond these were significant in algorithmic or theoretical terms. While the identifiability result is nice, it was not clear if it required analytical tools that were novel beyond the known result for iVAE. The distinction between factorized/non-factorized seems minor; it is not clear how significant this is. There was also very little comparison to MIVAE, although the graphical models are extremely similar. It is stated that the reason this approach could do better than MIVAE was identifiability, but why that is and why the MIVAE was not identifiable was not clear to me. In general, I found the discussion and comparison to prior work lacking both in the theory and in the experiments. This is one significant area for improvement.\n- It is not clear if the assumptions in theorem 1 make sense for MIL, especially (iv). Not enough intuition is provided to help understand the necessity/sufficiency of the assumptions.\n- in Eq (8), where is the \"weighting hyper-parameter alpha\"?\n- \"The maximum operator of Equation 8 is fundamentally different from the pooling operators used\"---this was not very convincing to me.\n- Only one experiment is done with an instance labeled MI datasets. There are many MIL datasets available with instance labels. The authors should do additional experiments with actual MI datasets rather than synthetic data such as digit recognition. \n- From the appendix, it is not clear how hyperparameters were chosen. There did not seem to be a common tuning process for all algorithms.\n- As mentioned above, the comparison with MIVAE was not convincing to me. The ablation study in the appendix is helpful but the claim \"this is caused by not having identifiability\" lacks justification, and does not really explain why. Further explanation is needed to explain the differences between these approaches. \n- No information is given about causal interpretations on the clinical dataset. Are causal relationships found clinically validated?\n-Limitations are not really included except one line about the standard MIL assumption, which is not a sensible limitation in as much as every model must make some such assumption. \n\nTo summarize, I found the approach promising, but the paper has significant room for improvement.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "md4bbzox3t8", "reviewer": "Reviewer_RjbB", "summary": "This work solves multi-instance learning (MIL) via utilizing bag-level weak supervision. Specifically, the authors propose a general graphical model which disentangles each instance as generated from instance-specific factors and bag-inherited factors. The proposed TargetedMIL algorithm is solved by identifiable variational autoencoder (iVAE).", "questions": "1. Does this result apply to collective MIL assumption proposed in [1]? The collective assumption assumes that several instances work together to determine the bag label. This assumption is explored in a related work that generalizes Attn-MIL [2]. Hence, could this work generalize to this MIL assumption?\n\n[1] Multiple instance learning: A survey of problem characteristics and applications. Pattern Recognition.\n[2] Deep multiple instance selection. Sci. China Inf. Sci.\n", "limitations": "Yes", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n1. The graphical model seems novel and rational. Under the standard assumption of MIL, instance labels are not observed and their labels determine the bag-level label. Hence, decoupling instance labels to both instance-specific factors and bag-level factors seems reasonable.\n2. A novel TargetedMIL algorithm is proposed. TargetedMIL takes advantage of permutation invariant set transformation networks to identify the latent factors and learn instance feature representations.\n3. The paper organization is well and clear to understand. Theoretical results are provided.\n\nCons:\n1. The utilized datasets are slightly simple. MNIST, FaMNIST, and KuzushijiMNIST are simple datasets that are not so representative to verify the effectiveness of the proposed method. More complex datasets should be explored.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "ZpFNe_I7OY", "reviewer": "Reviewer_A4sn", "summary": "This paper proposes a causal representation learning algorithm for multi-instance learning, called TargetedMIL, which aims to identify invariant causal representations of instances from bag-level weak supervision. TargetedMIL separates each instance into a causal and a non-causal parts and estimates labels for each instance. Experiments on synthetic and real-world datasets demonstrate the effectiveness of the proposed algorithm for two tasks. ", "questions": "How to ensure conditions (b) and (c) in Assumption 1 always hold?", "limitations": "My major concern is about the reasonability of the proposed algorithms. The conditions in assumptions used in this paper seems very strict that may be violate in real world. Furthermore, the latent factors identified by the proposed algorithm are hard to evaluate which may violate the assumptions.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1.\tThis work is well-motivated with theoretical analyses. \n\n2.\tThe assumptions used are described in detail.\n\n3.\tThe experiments are conducted on real-world datasets the empirical results outperform some classical methods.\n\nWeaknesses:\n\n1.\tThe conditions (b) and (c) in Assumption 1 may not hold in real-world applications. \n\n2.\tLack of theoretical analyses about estimation errors using Equations (7) and (8), while Theorem 1 only holds in ideal environment.\n\n3.\tLack of analysis of computational complexity.\n\n4.\tThe relationship between the two tasks (instance label prediction and OOD Generalization) is unclear.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept