Title: Are Multiple Instance Learning Algorithms Learnable for Instances?
Venue: NeurIPS 2024
Accepted: yes
Keywords: Multiple instance learning

Learnability

Learning theory

Probably Approximately Correct
Abstract: Multiple Instance Learning (MIL) has been increasingly adopted to mitigate the high costs and complexity associated with labeling individual instances, learning instead from bags of instances labeled at the bag level and enabling instance-level labeling. While existing research has primarily focused on the learnability of MIL at the bag level, there is an absence of theoretical exploration to check if a given MIL algorithm is learnable at the instance level. This paper proposes a theoretical framework based on probably approximately correct (PAC) learning theory to assess the instance-level learnability of deep multiple instance learning (Deep MIL) algorithms. Our analysis exposes significant gaps between current Deep MIL algorithms, highlighting the theoretical conditions that must be satisfied by MIL algorithms to ensure instance-level learnability. With these conditions, we interpret the learnability of the representative Deep MIL algorithms and validate them through empirical studies.

Are Multiple Instance Learning Algorithms Learnable for Instances?

Abstract

Multiple Instance Learning (MIL) has been increasingly adopted to mitigate the high costs and complexity associated with labeling individual instances, learning instead from bags of instances labeled at the bag level and enabling instance-level labeling. While existing research has primarily focused on the learnability of MIL at the bag level, there is an absence of theoretical exploration to check if a given MIL algorithm is learnable at the instance level. This paper proposes a theoretical framework based on probably approximately correct (PAC) learning theory to assess the instance-level learnability of deep multiple instance learning (Deep MIL) algorithms. Our analysis exposes significant gaps between current Deep MIL algorithms, highlighting the theoretical conditions that must be satisfied by MIL algorithms to ensure instance-level learnability. With these conditions, we interpret the learnability of the representative Deep MIL algorithms and validate them through empirical studies.

1 Introduction

The performance of supervised learning models is greatly influenced by the amount of labeled data [1]. While various models utilizing large-scale datasets have achieved excellent performance, the cost and time of labeling have emerged as issues, especially in domains requiring expert knowledge. For example, in the case of pathology images, detailed labeling requires a significant amount of expert time [2].

To address these issues, multiple instance learning (MIL) techniques have been introduced. MIL encompasses all methodologies that learn to predict the labels of instances by learning from the labels of bags composed of instances [3, 4, 5, 2, 6, 7, 8, 9, 10]. This approach allows for detecting disease areas in pathology images through whole-image labeling without requiring the labeling of regions within the image. This can significantly reduce overall labeling costs and maximize the efficiency of the method [2, 9]. As illustrated in Figure 1, if at least one instance in a bag is positive, the bag is positive, and if all instances are negative, the bag is negative in MIL. To achieve instance-level learning, it was necessary first to confirm that learning at the bag level could be performed at a high level. Therefore, traditional MIL research focused primarily on the feasibility of learning at the bag level rather than the instance level [11, 12, 13, 14], and some studies validated instance-level learning only for specific algorithms [15].

Recently, with the advancement of deep learning technologies, traditional MIL has evolved into Deep MIL, enabling more effective extraction of features from individual instances and consideration of interactions between instances, leading to significant improvements in prediction performance. Despite these advancements, MIL research still predominantly focuses on learning at the bag level, with a lack of exploration into the feasibility of instance-level learning [10, 17, 18].

In this study, we propose a new framework to theoretically validate that Deep MIL can learn at the instance level, overcoming the aforementioned issues. The contributions of this study are summarized as follows:

The proposed theoretical framework derives conclusions about the instance-level learnability of Deep MIL algorithms, assuming that MIL algorithms are learnable at the bag level (Assumption 1).

Utilizing the probably approximately correct (PAC) learning theory, we divide the hypothesis space of the dataset into two cases: 1) each instance being statistically independent and 2) the general case without any constraints on instance distributions, including statistically dependent instances. We theoretically derive the necessary and sufficient conditions for learnability in each hypothesis space (i.e., Condition 4 and Condition 7).

By applying the derived conditions to the existing representative types of Deep MIL, we verify their instance-level learnability.

Through Theorem 10 and Theorem 11, we show that additional information (e.g., positional information of each time point in time series, medical records provided alongside pathology images for better disease diagnosis) beyond the features directly extracted from the original data must act as weights in the independent hypothesis space of each instance.

This paper is organized as follows. Section 2 defines the problem. Section 3 outlines the conditions for Deep MIL to be instance learnable and provides theoretical proof. Section 4 evaluates existing Deep MIL algorithms and validates the results through experiments. Finally, Section 5 concludes the paper.

2 Problem Definition

Notation 1 (MIL domain spaces) Given the feature space for an instance X insti ⊂ R d and the feature space for N instances X := {X inst1 , X inst2 , ..., X instN } , along with the label space Y := { 1 , ..., k } , we can define the joint distribution D XY on X × Y . Here, X insti ∈ X insti , X := ( X inst1 , X inst2 , ..., X instN ) ∈ X , and Y ∈ Y are random variables. The MIL instance domain D X inst i Y represents the joint distribution of individual instances and their respective labels. The MIL bag domain D XY is the joint probability distribution composed of multiple instance domains D X inst i Y . Here, the term 'domain' refers to these joint distributions.

Based on Notation 1, the MIL Problem can be defined as in Definition 1.

Definition 1 (MIL problem) Given a training dataset S := { (( x 1 1 , ..., x 1 n ) , y 1 ) , ..., (( x m 1 , ..., x m n ) , y m ) } drawn IID from the joint distribution D XY , the goal of MIL is to learn a classifier f bag for the data and ( f inst 1 , ..., f inst n ) such that for any arbitrary bag random variable x := ( x 1 , ..., x n ) drawn from the marginal distribution D X : 1) It should be able to classify the class corresponding to the bag x . 2) It should be able to classify the class corresponding to the i th instance x i of the bag. Here, a random sample ( x, y ) := (( x 1 , ..., x n ) , y ) of the training data represents a bag consisting of n instances drawn from D XY , with a total of m bags. The training data for the i th instance is given by S inst i := { ( x 1 i , y 1 ) , ... ( x i m , y m ) } . □

The MIL problem defined in Definition 1 is performed within the following MIL hypothesis spaces.

Notation 2 (MIL hypothesis spaces) 1) The bag hypothesis space H bag ⊂ { h bag : X → Y } is a set of hypothesis functions h bag that classify bags with the correct labels. Here, h bag = f ( X ) , where f ∈ F bag . 2) The i th instance hypothesis space H insti ⊂ { h insti : X insti → Y } is a set of hypothesis functions h inst i that classify the i th instance with the correct labels using the feature space of the i th instance. Here, h inst i = f ( X insti ) , where f ∈ F insti . Here, F bag and F insti are sets of functions that take a bag or the i th instance as input and generate the corresponding labels.

According to Notations 1, 2, and Definition 1, the risk for a bag in MIL is defined as in Definitions 2 and 3.

Definition 2 (Bag Risk) If the loss for a bag in the MIL algorithm is given by ℓ bag ( h bag ( x ) , y ) , the risk for a bag is defined as follows:

Definition 3 (Instance Risk) If the loss for the i th instance in the MIL algorithm is given by ℓ inst i ( h inst i ( x i ) , y ) , the risk for the i th instance is defined as follows:

Based on the definitions of bag risk and instance risk in Definition 2 and 3, the learnability for bags and instances can be defined as in Definition 4 and Definition 5.

Definition 4 (PAC Learnability of Bag) Given the domain space D XY and the bag hypothesis space H bag ⊂ { h bag : X → Y} , the MIL algorithm A is said to be learnable on H bag with respect to D XY if, for all domains D XY ∈ D XY , the following condition is satisfied:

Here, ϵ represents the acceptable error between the learning algorithm and the actual optimal hypothesis. In contrast, δ represents the confidence level that the learning algorithm will return accurate results within a certain error range. Both ϵ and δ have a range of 0 < ϵ, δ < 1 .

Definition 5 (PAC Learnability of Instance) When given the i th instance domain space D Xinst i Y and instance hypothesis space H insti ⊂ { h insti : X → Y } , the algorithm A is said to be learnable over H insti from D Xinst i Y if it satisfies the following for all domains D X inst i Y ∈ D Xinst i Y :

Based on the relationship between Definitions 4 and 5, according to Theorem 1, if a MIL algorithm is not learnable with respect to bags, it is not learnable with respect to instances.

Theorem 1 If MIL algorithm A satisfies Condition 1, then this algorithm is not PAC learnable for any instance domain space D Xinst i Y and instance hypothesis space H insti ⊂ { h insti : X → Y } .

Condition 1 The MIL algorithm A is not PAC learnable for the given domain space D XY and bag hypothesis space H bag ⊂ { h bag : X → Y } :

Proof : The proof of Theorem 1 is conducted in Appendix C.1.

According to Theorem 1, MIL algorithms that are not learnable for bags do not guarantee learnability for instances. Therefore, in this study, we discuss the learnability of instances under Assumption 1.

Assumption 1 The MIL algorithm is PAC learnable for bags.

Based on the definitions, theorem, and assumption above, we can formulate the definition of when the proposed Deep MIL is learnable for instances as follows:

Definition 6 If the MIL algorithm satisfies Condition 2, it is learnable for instances.

Condition 2 The Deep MIL algorithm A must exhibit equivalent PAC learnability for bags and instances:

Definition 6 under Assumption 1 ensures that if Condition 2 is satisfied, the algorithm is guaranteed to be learnable for instances. On the other hand, MIL algorithms that do not satisfy Condition 2 cannot guarantee learnability for instances, even if they successfully learn for bags. Therefore, Condition 2 becomes a necessary and sufficient condition for MIL algorithms to be learnable for instances.

3 Proposed Theoretical Framework

In this study, we propose a theoretical framework to verify whether a given MIL algorithm satisfies Condition 2 according to Definition 6. For some Deep MIL algorithms [5, 6], instances are assumed to belong to independent bag domain spaces. Therefore, we address the problem by distinguishing between the independent bag domain space D Ind XY and the general bag domain space D Gen XY . 1) D Ind XY refers to bag domain spaces where all instances within a bag are statistically independent. 2) D Gen XY refers to a bag domain space that includes both D Dep XY , where interactions or dependencies exist among instances within a bag, and D Ind XY . That is, D Ind XY ∪ D Dep XY = D Gen XY .

3.1 Overview

Figure 2 shows the final summary of the definitions, relationships, and results of the theorems that comprise the theoretical framework proposed in this study.

Priori Learnability Theorems for Instances

Theorem1:Relationshipbetween learnabilityfor Bags and learnability for Instances

Consequence: If an MIL algorithm is not PAC learnable for bags, it cannot be PAC learnable for instances.

Assumption 1: The MIL algorithm is PAC learnable for bags

Theorem 4: PAC Learnability for Bags in Dind

XY

Consequence: For MIL algorithms to be PAC learnable in Den, the use of

Consequence:All MIL pooling techniques arePAC trainable on DInd toBag.

an attention mechanism is necessary.

(Instance,Embedding,Attention,Additive,Conjunctive-Pooling)

Example: In drug activity prediction,MIL analyzes various independent chemical

(Attention,Additive,Conjunctive-Pooling)

structures of a drug candidate to predict whether the candidate is active against a disease.

Example:In review-sentence analysis,MIL considers the relationships between

sentences within a review to predict the overall sentiment of the review.

General Bag Domain Spaces Den = Did U DReP

XY

Independent Bag Domain Spaces Dd

Theorem 2: Instance Learnabilityfor Dd

Theorem 8:

Theorem 9:

Learnability of

Non-Learnability when Instance Hypothesis

Theorem 6:

Theorem 7:

Conjunctive-Pooling for

(Attention,Additive-Pooling)

Learnability of

Non-Learnability of

Instances

Instance-Pooling for

Embedding-Pooling for

Instances

Instances

Theorem 10:

Theorem 11:

Non-Learnability of RNN-Based

Non-Learnability of

Feature Extraction for Instances

Positional Encoding for

Instances

Learnability Theorems for Instances

3.2 PAC Learnability for Independent Bag Domain Spaces

The definition of D Ind XY is provided in Definition 7.

Definition 7 The independent Bag Domain Space D Ind XY is defined as a domain Space that encompasses all instance spaces while satisfying Condition 3:

Condition 3 For each Instance Space D X inst i Y within D Ind XY , there must exist a corresponding bag domain space D XY ∈ D Ind XY , which should be determined as the union of each instance domain space, as follows:

Since D Ind XY satisfies Condition 3, implying independence among instances, the hypothesis space for each instance is unaffected by other instances. In this case, for MIL algorithms to be learnable for instances, they must satisfy Condition 4 according to Theorem 2.

Theorem 2 If a MIL algorithm satisfies Condition 4 in D Ind XY , it is learnable for instances.

Condition 4 The risk of the optimal hypothesis for D Ind XY must ensure that it equals the sum of the risks of the optimal hypotheses for individual instance spaces within D Ind XY :

Consequence : If Condition 4 is satisfied, then Condition 2 is also satisfied. Hence, Condition 4 becomes a necessary and sufficient condition for learnability for instances in D Ind XY .

Proof : The proof of Theorem 2 is provided in Appendix C.2.

3.3 PAC Learnability for General Bag Domain Spaces

The definition of D Gen XY is provided in Definition 8.

Definition 8 The general bag domain space D Gen XY is defined as a domain space that encompasses all instance spaces while satisfying Condition 5 and 6:

Condition 5 For every instance space D X inst i Y within D Gen XY , there exists a corresponding bag domain space D XY ∈ D Gen XY , determined as the sum of each instance domain space.

Condition 6 D Gen XY is formed using weights α i ∈ (0 , 1) to reflect the importance of relationships among instances. Each instance domain space D X inst i Y should be defined along with its weight α i as follows:

Since D Gen XY satisfies Condition 5 and Condition 6, implying the existence of relationships among instances, the hypothesis space for each instance is influenced by other instances. In this case, for MIL algorithms to be learnable for instances, they must satisfy Condition 7 according to Theorem 3.

Theorem 3 If a MIL algorithm satisfies Condition 7 in D Gen XY , it is learnable for instances.

Condition 7 The risk of the optimal hypothesis for D Gen XY must ensure that it equals the weighted sum of the risks of the optimal hypotheses for individual instance spaces within D Gen XY :

Consequence : If a MIL algorithm satisfies Condition 7, it also satisfies Condition 2. Hence, Condition 7 becomes a necessary and sufficient condition for MIL algorithms to be learnable for instances in D Gen XY .

Proof : The proof of Theorem 3 is provided in Appendix C.3.

4 Theoretical Verification of Existing Deep MILs

4.1 Classifications of Existing Deep MIL Methodologies

The Deep MIL algorithms proposed so far are primarily categorized based on 1) whether they perform Aggregation at the Embedding-level [5, 6, 2, 19, 20] or at the Instance-level [5, 21, 22, 23, 9, 8, 10, 24]. Additionally, they can be further classified into 5 types of pooling techniques according to 2) whether they do not use an attention mechanism [5, 21, 22, 6], perform Aggregation by multiplying attention weights at the embedding-level [2, 19, 20], or perform aggregation by multiplying attention weights at the instance-level [23, 9, 10, 24], as shown in Table 1.

A detailed explanation of the pooling techniques is provided in Appendix A.2.

4.2 Theoretical Verification

4.2.1 Relationship between Attention Mechanism and Learnability for Bag

The application of an attention mechanism to MIL algorithms depends on the range of the domain space that the MIL algorithm can learn from. This implies that the feasibility of PAC learning for bags may vary depending on whether attention is applied or not.

Theorem 4 In D Ind XY , the MIL algorithm is PAC learnable for bags.

Proof : The proof of Theorem 4 is conducted in Appendix C.4.

Theorem 5 In D Gen XY , MIL algorithms that aggregate independent hypothesis spaces for each instance must utilize attention satisfying Condition 8 to be PAC learnable for bags.

Condition 8 The hypothesis space H bag att of Attention MIL should be equal to the sum of independent hypothesis spaces h inst i multiplied by attention weights Att i :

Proof : The proof of Theorem 5 is conducted in Appendix C.5.

Validation : Experimental validation is performed in Section 4.4.1.

Consequence: Theorems 4 and 5 are not direct theorems about the learnability of MIL for instances. However, the learnability for bags is a prerequisite for MIL algorithms to be able to learn from instances. According to Theorem 4, in PAC learnable D Ind XY , all MIL algorithms satisfy Assumption 1 regardless of the presence of attention mechanisms. On the other hand, in D Gen XY , according to Theorem 5, the application of attention mechanisms satisfying Condition 8 becomes a necessary condition for MIL algorithms to be learnable for instances.

4.2.2 Verification Learnability for Instances by MIL Pooling Method

The type of pooling technique used in Deep MIL algorithms becomes a determining factor in whether they are learnable for instances when they are learnable for bags. In this case, to verify whether a Deep MIL algorithm is learnable for instances, a definition of Lemma 1 should be established in advance, extending Condition 4 to ensure that no additional hypothesis space is included for instances.

Lemma 1 Condition 9 serves as a necessary condition for the learnability of instances, when the hypothesis space for the i th instance of a MIL algorithm is H inst i ∪ H add i . Here, H inst i represents the hypothesis space for the i th instance, and H add i denotes the hypothesis space for the i th instance generated through elements outside the i th instance.

Condition 9 H add i must be a subset of H inst i :

Proof : The proof of Lemma 1 is conducted in Appendix C.6.

In the case of instance-pooling, as attention mechanisms are not utilized, Condition 8 is not satisfied. Therefore, in D Gen XY , the algorithm is not learnable for bags, and consequently, not learnable for instances either. However, according to Theorem 6, the algorithm is learnable for instances in D Ind XY .

Theorem 6 In D Ind XY , MIL algorithms that perform instance-pooling are PAC learnable for instances.

Proof : The proof of Theorem 6 is conducted in Appendix C.7.

Unlike Instance-Pooling, where the hypothesis for bags is combined from instance-level hypotheses, Embedding-Pooling does not combine bag hypotheses from instance-level hypotheses. As a result, it does not satisfy Condition 9, leading to a scenario similar to Theorem 7.

Theorem 7 MIL algorithms that perform Embedding-Pooling are not learnable for instances.

Proof : The proof of Theorem 7 is conducted in Appendix C.8.

Consequence: In D Ind XY , Deep MIL algorithms exhibit reproducibility when they avoid overfitting to easy information at the bag-level and effectively learn positive instances at the instance-level. Reproducibility in this context refers to the learnability of positive instances. They demonstrated experimentally that Deep MIL algorithms using Instance-Pooling, such as mi-Net [5] and Causal MIL [21, 22], exhibit reproducibility, while those using Embedding Pooling, such as Mi-Net [5], do not. However, Raff et al. [18] failed to provide a theoretical explanation for these results. In contrast, this study theoretically demonstrated that Instance-Pooling is learnable for instances while Embedding-Pooling is not, through Theorems 6 and 7. This provides theoretical support for the experimental findings of Raff et al. [18].

To compute the attention applied to each instance's features in Attention Pooling and Additive Pooling, the bag's features X are used as input. Multiplying attention weights to the features at the feature level results in additional hypothesis space h add i formed by the attention operations on each instance. As a consequence, since Condition 9 is not satisfied, according to Theorem 8, the algorithm becomes not PAC Learnable for instances.

Theorem 8 If the MIL algorithm does not adhere to Condition 10, it is not learnable for instances.

Condition 10 The risk R inst i for the i th instance should be as follows:

Here, H bag -level i denotes the hypothesis space for the i th instance generated through bag-level features, and H bag -level i := { h bag -level : X → Y} .

Consequence : Attention-Pooling and Additive-Pooling multiply attention weights to each instance's feature-level. As a result, the hypothesis space for the i th instance includes, in addition to InstancePooling, h bag -level = { h | h = f ( X bag ) , f ∈ F bag } . In other words, it incorporates the hypothesis space for bag-level features into the prediction for the i th instance. This, according to Theorem 8, renders it not learnable for instances.

Proof : The proof of Theorem 8 is conducted in Appendix C.9.

Validation : Experimental verification of Theorem 8 is demonstrated in Section 4.4.2.

On the other hand, Conjunctive-Pooling does not multiply attention at the feature level of instances, but rather at the prediction level of instances. Therefore, predictions are made based on individual features of instances, resulting in H bag -level i = ∅ , satisfying Condition 10. Additionally, Theorem 9 demonstrates that Conjunctive-Pooling operates in a manner that is learnable for instances.

Theorem 9 When MIL algorithms use Conjunctive-Pooling for aggregation in D Gen XY , they are learnable for instances.

Consequence: According to Theorem 9, Conjunctive-Pooling becomes the unique pooling technique learnable for instances in D Gen XY . Specifically, since D Gen XY includes D Ind XY by definition, ConjunctivePooling becomes a methodology that satisfies all cases.

Proof : The proof of Theorem 9 is conducted in Appendix C.10.

Validation : Experimental validation for Theorem 9 is presented in Section 4.4.2.

Javed et al.[9] demonstrated that the contribution of instances in MIL algorithms performing AdditivePooling is proportional to the shapley value[25]. However, their proof contained an error where the feature multiplied by attention was mistakenly assumed to be the feature of the i th instance. Our theoretical framework identifies that when attention is applied to features, it fails to satisfy Condition 10, leading to the algorithm being not learnable.

In this study, we confirmed that the error causing failure to satisfy Condition 10 also appears in studies proposing MIL based on Conjunctive-Pooling [23, 10]. Details on this are explained in Section 4.3.1.

4.3 Additional Considerations

4.3.1 Rethinking Position Dependencies of Instances on Deep MILs

Text data and time series data exhibit temporal dependencies, while image data often has spatial dependencies. Therefore, research in relevant fields has utilized neural networks capable of capturing dependencies, such as RNNs and CNNs [26, 27, 28, 29, 30, 31], or additional positional encoding on extracted features to enhance performance [32, 33, 34, 35, 36, 37]. Following this trend, Deep MIL studies have also employed RNN-based neural networks or positional encoding during the feature extraction process to capture temporal dependencies of instances for performance enhancement [23, 19, 10]. However, as per Theorem 10 and Theorem 11, these approaches render the models unable to learn from instances.

Theorem 10 If the MIL algorithm extracts features of instances through RNN-based neural networks for aggregation, it is unable to learn from instances.

Proof : The proof for Theorem 10 is conducted in Appendix C.11.

Validation : The experimental validation for Theorem 11 is presented in Section 4.4.3.

Theorem 11 If the hypothesis space H Pos -Encode i generated through positional encoding values for the i -th position of the MIL algorithm is not a subset of H inst i , then the algorithm is not PAC learnable for instances.

Proof : If H Pos -Encode i ̸⊂ H inst i , the algorithm fails to satisfy Condition 9, rendering it not learnable for instances.

Validation : Experimental validation for Theorem 10 is shown in Section 4.4.3.

Consequence: According to Theorem 11, using values outside of an instance's features in the prediction process makes it unlearnable. Therefore, positional encoding should not be used in the process of predicting instances.

According to the investigation conducted in this study, it was observed that existing Deep MIL research using Conjunctive-Pooling, such as MILNET [23] and MILLET [10], all utilized RNN-based neural networks for feature extraction or performed positional encoding on instance features. Therefore, they failed to satisfy Condition 9, making them unable to learn about instances. Furthermore, these theoretical findings provide a basis for the ablation study conducted by Early et al.[10] on time series data classification problems, where Conjunctive-Pooling and positional encoding were applied to predict the class at each time step. In essence, Early et al.[10] demonstrated that while positional encoding contributed to improving prediction performance at the bag level, it acted as a detrimental factor in predicting instances. When utilizing external information such as positional dependencies through supplementary weighting factors like attention operations alone, Condition 9 may be satisfied, thereby enabling learnability concerning instances.

4.3.2 Learnability for Instances in Each Dimension for Multidimensional Deep MILs

In the real world, data is often composed of bags consisting of multi-dimensional instances rather than simple instances of uniform dimensions. For instance, in video data, each frame(images)

is composed of patches in multi-dimensional structures for each frame dimension. Similarly, in multivariate time series data, each timestamp is composed of multivariate data points. In the case of multi-dimensional instances, Multi-dimensional Deep MIL (MD-MIL) methodologies have emerged to perform predictions on lower-level instances based solely on labels at the top level of the bag. These methodologies apply aggregation recursively, performing aggregation on data in the first dimension and sequentially on subsequent dimensions. Existing MD-MIL approaches have employed Embedding-Pooling or Attention-Pooling for each dimension's instances. However, these pooling methods have been shown by Theorems 7 and 8 to be incapable of learning about instances. Therefore, existing MD-MIL methodologies are not suitable for learning from multi-dimensional instances.

Therefore, to design methodologies capable of learning from multi-dimensional data, it is essential to use aggregation techniques that are capable of learning about instances, based on Theorem 6 or 9. Additionally, to ensure learnability from the top-level bag and bags in each dimension, the attention operation's results should be set to be within the instance's hypothesis space based on whether the relationships among instances in each dimension are independent or dependent. Through the experiments in Appendix E.2, we confirmed that Conjunctive-Pooling can capture the relationships between instances across different bags in an MD-MIL architecture, leading to performance improvements. These findings confirm that our proposed theoretical framework serves as a valuable guide in designing learnable models.

4.4 Experimental Validation

In this section, we conduct the following experimental validations to demonstrate whether existing Deep MIL approaches are learnable for instances based on the theorems: 1) (Theorem 5) : Demonstrating the learnability of the attention mechanism for bags in D Gen XY . 2) (Theorem 8, 9) : Showing that multiplying attention at the feature level is not learnable for instances. 3) (Theorem 10, 11) : Demonstrating that inputting position-related values into instance positions is not learnable for instances. As this study assumes an environment where bags are PAC Learnable, we preprocess the MNIST dataset to match the difficulty level of each experiment. For the validation of Theorem 10 and 11, we use the WebTraffic dataset from Early et al. [10], which is a synthetic time-series classification dataset. Detailed experimental settings can be found in Appendix D.

4.4.1 Experimental Validation of Theorem 5

To validate Theorem 5, we conducted experiments on MIL algorithms [5, 21, 38, 2, 39, 40, 19, 9, 10] representing each pooling technique. Table 2 compares prediction performance on the synthetic datasets, detailed in Appendix D.1. This reveals that Instance-Pooling based MIL algorithms [5, 21, 38], which do not apply weights to the hypothesis space, degraded learning performance for bags in D Gen XY . In contrast, the other algorithms [2, 39, 40, 19, 9, 10] that apply weights to the hypothesis space through the attention mechanism demonstrate superior performance, which is even comparable to a none-pooling-based method that, by using fully connected layers, preserves all instance-level information without any loss during prediction. This experimentally validates Theorem 5.

4.4.2 Experimental Validation of Theorems 8 and 9

To assess whether the algorithm [2, 39, 40, 19, 9, 10] is learnable for instances when weights from an attention mechanism that satisfies Condition 8 are multiplied at the feature level, we compared the predictive performance on instances by adjusting the variance of attention weights.

Table 3 shows the results on the synthetic dataset, which details in Appendix D.2. Attention-Pooling based [2, 39, 40, 19] and Additive-Pooling based [9] MIL algorithms, where attention weights are multiplied at the feature level and aggregation is performed, showed significantly lower predictive performance for instances compared to bags. In particular, SA-AbMILP [40] and TransMIL [19], which perform iterative attention operations, show a significant performance gap between bag-level predictions and instance-level predictions. This demonstrates that the Attention-Pooling process does not guarantee the learnability of MIL. In contrast, Conjunctive-Pooling based MIL algorithms [10] exhibited a much smaller difference in predictive performance between bags and instances than other algorithms. This validates, in accordance with Theorem 9, that Conjunctive-Pooling is learnable for instances. Additionally, experiments adjusting the variance of attention to determine its impact on the discrepancy between bag and instance performance are detailed in Appendix E.1.

4.4.3 Experimental Validation of Theorems 10 and 11

Table 4 compares the performance of Conjunctive MIL under various conditions, reflecting information on positional dependency: 1) Applying attention and prediction on features extracted via Positional Encoding or a GRU-layer (All), 2) Using positional information only for attention operations (Att), 3) Using positional information only for prediction (Predict), and 4) The baseline model without any positional adaptations (Default).

Comparative results on the WebTraffic dataset's instance prediction performance reveal that configurations All and Predict, which provide additional positional information to features as in Theorem 10 and 11, showed poorer prediction performance than Default. Particularly, RNN, which reflects more additional information than positional encoding, was found to significantly degrade performance. However, the Att configuration, which utilized positional information solely for attention operations, achieved better performance than Default. This demonstrates that incorporating helpful information such as positional data, into MIL should be selectively applied to attention computations only.

5 Conclusions

In this study, we proposed a novel framework to theoretically validate that deep MIL can learn at the instance level, overcoming the aforementioned issues. The proposed theoretical framework derived conclusions about the instance-level learnability of deep MIL algorithms, assuming that MIL algorithms are learnable at the bag level. Utilizing the PAC learning theory, we theoretically derived the necessary and sufficient conditions for learnability in each hypothesis space. This provides theoretical guidance for building learnable MIL models in various domains. The practical application of our proposed framework to real-world MIL scenarios is presented in Appendix F. Limitations and future work are explained in Appendix G.

References

Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning. MIT press, 2018.

Maximilian Ilse, Jakub Tomczak, and Max Welling. Attention-based deep multiple instance learning. In International conference on machine learning, pages 2127-2136. PMLR, 2018.

Oded Maron and Tomás Lozano-Pérez. A framework for multiple-instance learning. Advances in neural information processing systems, 10, 1997.

Thomas G Dietterich, Richard H Lathrop, and Tomás Lozano-Pérez. Solving the multiple instance problem with axis-parallel rectangles. Artificial intelligence, 89(1-2):31-71, 1997.

Xinggang Wang, Yongluan Yan, Peng Tang, Xiang Bai, and Wenyu Liu. Revisiting multiple instance neural networks. Pattern Recognition, 74:15-24, 2018.

Alessandro Tibo, Manfred Jaeger, and Paolo Frasconi. Learning and interpreting multi-multiinstance learning networks. Journal of Machine Learning Research, 21(193):1-60, 2020.

Yu Tian, Guansong Pang, Yuanhong Chen, Rajvinder Singh, Johan W Verjans, and Gustavo Carneiro. Weakly-supervised video anomaly detection with robust temporal feature magnitude learning. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4975-4986, 2021.

Hui Lv, Zhongqi Yue, Qianru Sun, Bin Luo, Zhen Cui, and Hanwang Zhang. Unbiased multiple instance learning for weakly supervised video anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8022-8031, 2023.

Syed Ashar Javed, Dinkar Juyal, Harshith Padigela, Amaro Taylor-Weiner, Limin Yu, and Aaditya Prakash. Additive mil: Intrinsically interpretable multiple instance learning for pathology. Advances in Neural Information Processing Systems, 35:20689-20702, 2022.

Joseph Early, Gavin Cheung, Kurt Cutajar, Hanting Xie, Jas Kandola, and Niall Twomey. Inherently interpretable time series classification via multiple instance learning. In The Twelfth International Conference on Learning Representations, 2024.

Peter Auer, Philip M Long, and Aravind Srinivasan. Approximating hyper-rectangles: Learning and pseudo-random sets. In Proceedings of the twenty-ninth annual ACM symposium on Theory of computing, pages 314-323, 1997.

Philip M Long and Lei Tan. Pac learning axis-aligned rectangles with respect to product distributions from multiple-instance examples. In Proceedings of the ninth annual conference on Computational learning theory, pages 228-234, 1996.

Avrim Blum and Adam Kalai. A note on learning from multiple-instance examples. Machine learning, 30:23-29, 1998.

Boris Babenko. Multiple instance learning: algorithms and applications. View Article PubMed/NCBI Google Scholar, 19, 2008.

Gary Doran and Soumya Ray. Multiple-instance learning from distributions. Journal of Machine Learning Research, 17(128):1-50, 2016.

Weiming Hu, Chen Li, Xiaoyan Li, Md Mamunur Rahaman, Jiquan Ma, Yong Zhang, Haoyuan Chen, Wanli Liu, Changhao Sun, Yudong Yao, et al. Gashissdb: A new gastric histopathology image dataset for computer aided diagnosis of gastric cancer. Computers in biology and medicine, 142:105207, 2022.

Sivan Sabato and Naftali Tishby. Multi-instance learning with any hypothesis class. The Journal of Machine Learning Research, 13(1):2999-3039, 2012.

Edward Raff and James Holt. Reproducibility in multiple instance learning: A case for algorithmic unit tests. Advances in Neural Information Processing Systems, 36, 2024.

Zhuchen Shao, Hao Bian, Yang Chen, Yifeng Wang, Jian Zhang, Xiangyang Ji, et al. Transmil: Transformer based correlated multiple instance learning for whole slide image classification. Advances in neural information processing systems, 34:2136-2147, 2021.

Saul Fuster, Trygve Eftestøl, and Kjersti Engan. Nested multiple instance learning with attention mechanisms. In 2022 21st IEEE International Conference on Machine Learning and Applications (ICMLA), pages 220-225. IEEE, 2022.

Weijia Zhang, Xuanhui Zhang, Min-Ling Zhang, et al. Multi-instance causal representation learning for instance label prediction and out-of-distribution generalization. Advances in Neural Information Processing Systems, 35:34940-34953, 2022.

Weijia Zhang, Lin Liu, and Jiuyong Li. Robust multi-instance learning with stable instances. In ECAI 2020, pages 1682-1689. IOS Press, 2020.

Stefanos Angelidis and Mirella Lapata. Multiple instance learning networks for fine-grained sentiment analysis. Transactions of the Association for Computational Linguistics, 6:17-31, 2018.

Jaeseok Jang and HYUK-YOON KWON. TAIL-MIL: Time-aware and instance-learnable multiple instance learning for multivariate time series anomaly detection. In The 39th Annual AAAI Conference on Artificial Intelligence, 2025.

Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30, 2017.

Jun Zhang and Kim-Fung Man. Time series prediction using rnn in multi-dimension embedding phase space. In SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No. 98CH36218), volume 2, pages 1868-1873. IEEE, 1998.

Jun Zhang and Kim-Fung Man. Time series prediction using rnn in multi-dimension embedding phase space. In SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No. 98CH36218), volume 2, pages 1868-1873. IEEE, 1998.

Gang Liu and Jiabao Guo. Bidirectional lstm with attention mechanism and convolutional layer for text classification. Neurocomputing, 337:325-338, 2019.

Rahul Chauhan, Kamal Kumar Ghanshala, and RC Joshi. Convolutional neural network (cnn) for image detection and recognition. In 2018 first international conference on secure cyber computing and communication (ICSCCC), pages 278-282. IEEE, 2018.

Kai Zhang, Wangmeng Zuo, Shuhang Gu, and Lei Zhang. Learning deep cnn denoiser prior for image restoration. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3929-3938, 2017.

Yanan Sun, Bing Xue, Mengjie Zhang, Gary G Yen, and Jiancheng Lv. Automatically designing cnn architectures using the genetic algorithm for image classification. IEEE transactions on cybernetics, 50(9):3840-3854, 2020.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.

Minghao Chen, Houwen Peng, Jianlong Fu, and Haibin Ling. Autoformer: Searching transformers for visual recognition. In Proceedings of the IEEE/CVF international conference on computer vision, pages 12270-12280, 2021.

Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106-11115, 2021.

Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.

Rui Xu, Xintao Wang, Kai Chen, Bolei Zhou, and Chen Change Loy. Positional encoding as spatial inductive bias in gans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13569-13578, 2021.

Kan Wu, Houwen Peng, Minghao Chen, Jianlong Fu, and Hongyang Chao. Rethinking and improving relative position encoding for vision transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10033-10041, 2021.

Pei Liu and Luping Ji. Weakly-supervised residual evidential learning for multi-instance uncertainty estimation. In Forty-first International Conference on Machine Learning, 2024.

Xiaoshuang Shi, Fuyong Xing, Yuanpu Xie, Zizhao Zhang, Lei Cui, and Lin Yang. Lossbased attention for deep multiple instance learning. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 5742-5749, 2020.

Dawid Rymarczyk, Adriana Borowa, Jacek Tabor, and Bartosz Zielinski. Kernel self-attention for weakly-supervised image classification using deep multiple instance learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1721-1730, 2021.

Yan Xu, Tao Mo, Qiwei Feng, Peilin Zhong, Maode Lai, I Eric, and Chao Chang. Deep learning of feature representation with multiple instance learning for medical image analysis. In 2014 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 1626-1630. IEEE, 2014.

Dimitrios Kotzias, Misha Denil, Nando De Freitas, and Padhraic Smyth. From group to individual labels using deep features. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 597-606, 2015.

Yunjie Ji, Hao Liu, Bolei He, Xinyan Xiao, Hua Wu, and Yanhua Yu. Diversified multiple instance learning for document-level multi-aspect sentiment classification. In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pages 7012-7023, 2020.

Jun Wang and Jean-Daniel Zucker. Solving multiple-instance problem: A lazy learning approach. 2000.

Waqas Sultani, Chen Chen, and Mubarak Shah. Real-world anomaly detection in surveillance videos. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6479-6488, 2018.

Oren Z Kraus, Jimmy Lei Ba, and Brendan J Frey. Classifying and segmenting microscopy images with deep multiple instance learning. Bioinformatics, 32(12):i52-i59, 2016.

Lei Zhou, Yu Zhao, Jie Yang, Qi Yu, and Xun Xu. Deep multiple instance learning for automatic detection of diabetic retinopathy in retinal images. IET Image Processing, 12(4):563-571, 2018.

Yongluan Yan, Xinggang Wang, Xiaojie Guo, Jiemin Fang, Wenyu Liu, and Junzhou Huang. Deep multi-instance learning with dynamic pooling. In Asian Conference on Machine Learning, pages 662-677. PMLR, 2018.

Seung-Kyu Hong, Jae-Seok Jang, and Hyuk-Yoon Kwon. Enhancing performance of transformer-based models in natural language understanding through word importance embedding. Knowledge-Based Systems, 304:112404, 2024.

Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106-11115, 2021.

Jong Seong Park, Jeong-Ha Park, Jihyeok Choi, and Hyuk-Yoon Kwon. Learning with correlation-guided attention for multienergy consumption forecasting. IEEE Transactions on Industrial Informatics, 2024.

Joseph Early, Christine Evers, and SArvapali Ramchurn. Model agnostic interpretability for multiple instance learning. In International Conference on Learning Representations, 2022.

Hang Zhao, Yujing Wang, Juanyong Duan, Congrui Huang, Defu Cao, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, and Qi Zhang. Multivariate time-series anomaly detection via graph attention network. In 2020 IEEE international conference on data mining (ICDM), pages 841-850. IEEE, 2020.

Taehee Kim, Jae-Seok Jang, and Hyuk-Yoon Kwon. Correlation-driven multi-level learning for anomaly detection on multiple energy sources. Applied Soft Computing, 159:111636, 2024.

Yingying Zhang, Desen Zhou, Siqin Chen, Shenghua Gao, and Yi Ma. Single-image crowd counting via multi-column convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 589-597, 2016.

Wen Liu, Weixin Luo, Dongze Lian, and Shenghua Gao. Future frame prediction for anomaly detection-a new baseline. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6536-6545, 2018.

Marc-André Carbonneau, Veronika Cheplygina, Eric Granger, and Ghyslain Gagnon. Multiple instance learning: A survey of problem characteristics and applications. Pattern Recognition, 77:329-353, 2018.

Zhi-Hua Zhou and Jun-Ming Xu. On the relation between multi-instance learning and semisupervised learning. In Proceedings of the 24th international conference on Machine learning, pages 1167-1174, 2007.

Ragav Venkatesan, Parag Chandakkar, and Baoxin Li. Simpler non-parametric methods provide as good or better results to multiple-instance learning. In Proceedings of the IEEE International Conference on Computer Vision, pages 2605-2613, 2015.

Wu-Jun Li et al. Mild: Multiple-instance learning via disambiguation. Ieee transactions on knowledge and data engineering, 22(1):76-89, 2009.

Jaume Amores. Vocabulary-based approaches for multiple-instance data: A comparative study. In 2010 20th International Conference on Pattern Recognition, pages 4246-4250. IEEE, 2010.

Yixin Chen, Jinbo Bi, and James Ze Wang. Miles: Multiple-instance learning via embedded instance selection. IEEE transactions on pattern analysis and machine intelligence, 28(12):19311947, 2006.

Zhouyu Fu, Antonio Robles-Kelly, and Jun Zhou. Milis: Multiple instance learning with instance selection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(5):958-977, 2010.

Shengye Yan, Xiaodong Zhu, Guoqing Liu, and Jianxin Wu. Sparse multiple instance learning as document classification. Multimedia tools and applications, 76:4553-4570, 2017.

Dan Zhang, Yan Liu, Luo Si, Jian Zhang, and Richard Lawrence. Multiple instance learning on structured data. Advances in Neural Information Processing Systems, 24, 2011.

A Related Works

A.1 Multiple Instance Learning

Multiple Instance Learning (MIL), a type of weakly supervised learning, is a methodology where labels are assigned to a bag of instances rather than individual instances, allowing for predictions about individual instances based on the bag's label. MIL was first introduced in the field of chemistry to identify active molecules and has since been applied to various domains, including medical image analysis, text data processing, time-series data analysis, and video anomaly detection [3, 4, 41, 2, 9, 42, 23, 43, 44, 10, 45, 7, 8].

In medical image analysis, accurate labeling of cancer cell locations requires significant time and effort. However, it is possible to use MIL to predict cancer cell locations with labels for the entire image. In text data, documents or reviews are treated as a single bag, and MIL allows for more detailed analysis by predicting the attributes of each instance within a document or review. MIL is also effectively used in time-series data and video anomaly detection for cause analysis and anomaly detection.

A.2 Type of Pooling in Multiple Instance Learning

Deep MIL methodologies can be broadly classified based on the pooling mechanism into the following categories: 1) Instance-Pooling, 2) Embedding-Pooling, 3) Attention-Pooling, 4) Additive-Pooling, and 5) Conjunctive-Pooling.

The functional architecture of MIL incorporates several key components designed for processing instances within a bag. The feature extraction function for the i th instance is denoted by f i ( X i ) . Furthermore, we utilize classifier functions to make predictions based on these features: p i ( X i ) targets the features of the i th instance, while p ( X ) handles features aggregated at the bag level. The attention weight assigned to the i th instance is expressed as A i . Additionally, the aggregation function, pivotal for our pooling techniques, is represented by g ( X ) . The various pooling methods utilized in our framework are described as follows:

(Instance-Pooling)

Instance-pooling performs predictions for each instance individually and then uses the max or mean operator to aggregate the results:

(Embedding-Pooling)

Embedding-pooling obtains features for each instance and then uses the max or mean operator to aggregate these features to obtain a feature representation for the bag, on which predictions for the bag are made:

(Attention-Pooling)

Attention-pooling multiplies the features of each instance by attention weights and then performs embedding-pooling using the summation operator on the weighted features:

(Additive-Pooling)

Additive-pooling multiplies the features of each instance by attention weights, obtains individual predictions for each instance, and then performs a summation operation to make predictions for the bag:

(Conjunctive-Pooling)

Conjunctive-pooling multiplies the individual predictions of instances by attention weights and performs aggregation through a weighted sum:

A.2.1 Attention Mechanism

In early deep MIL, the role of the deep network was to better extract the features of instances and to accurately predict instances through pooling operations in an end-to-end manner [5, 21, 22, 6, 46, 47, 48].

Early deep MIL models had the limitation that the bag-level prediction does not accelerate the instance-level prediction because predictions for instances were performed separately from predictions for the bag. To overcome this limitation, attention mechanisms have been used in various MIL methodologies [2, 23, 19, 9, 10, 20]. In other deep learning research [32, 35, 49, 50, 51], attention mechanisms have shown significant performance improvements by capturing relationships within the data. Due to these characteristics, attention weights, based on the features of all instances, assign higher weights to important instances for bag predictions, making them useful for instance-level predictions as well [2, 19, 20]. These weights have been shown to improve the prediction performance of instances [23, 9, 10].

A.2.2 Level of Aggregation

Deep MIL requires aggregating the values of instances to perform predictions for the bag. Early deep MIL models performed individual predictions for instances and then aggregated these predicted values using differentiable pooling operators [5]. However, the approach of making individual predictions for instances was not effective in improving the prediction performance for the bag. To enhance bag prediction performance, new methods were proposed that perform pooling operations on features extracted from each instance to obtain a feature representation for the bag and then learn the bag. These methods apply the classifier used for the bag to instances [5] or cluster the features of instances to find similar instances [6].

In the case of deep MIL utilizing the attention mechanism, early methods enhanced performance by multiplying the features of each instance by attention weights, performing mean pooling to extract a feature representation for the bag, and using that feature to train the bag [2, 19, 20]. This approach suggested that instances with higher attention weights contributed more to the bag's prediction. However, while feature-level aggregation allowed for effective bag predictions, it did not facilitate detailed predictions for individual instances. Conversely, applying attention and aggregating the prediction results for instances achieved high performance [9, 10]. Due to this trend, recent research prefers aggregating the prediction results for instances.

A.3 Theoretical Study of Multiple Instance Learning

According to Babenko et al. [14], early theoretical research on MIL focused on simple algorithms such as axis-aligned rectangles, which facilitated the derivation of theoretical results [11, 12]. Blum et al. [13] proposed an algorithm that illustrated the relationship between supervised learning and MIL, showing that if a supervised learning algorithm can learn a concept space, MIL can also learn within that space. Sabato et al. [17] addressed the limitations of previous MIL research, which often derived theoretical results for specific domains or simple hypothesis classes, by presenting a generalized theoretical framework applicable to various hypothesis classes and complex data structures. This framework allowed the evaluation of learnability for bags without being limited to specific algorithms. Raff et al. [18] argued that deep MIL algorithms do not adhere to standard MIL assumptions and are unsuitable for real-world environments. They proposed an algorithm unit test to verify adherence to these assumptions, ensuring that models can accurately predict bags in real-world scenarios.

However, all theoretical research proposed on MIL thus far has focused solely on whether MIL can learn bags as effectively as supervised learning without addressing the primary purpose of MILpredicting individual instances. Therefore, to address this issue, this study proposes a theoretical framework that can universally evaluate whether an algorithm is learnable at the instance level.

B Notations

The key notations used in Definitions, Theorems, Conditions, and Proofs in this study are summarized in Table 5.

C Theoretical Proofs

C.1 Proof of Theorem 1

Theorem 1 If MIL algorithm A satisfies Condition 1, then this algorithm is not PAC learnable for any instance domain space D Xinst i Y and instance hypothesis space H insti ⊂ { h insti : X → Y } .

Condition 1 The MIL algorithm A is not PAC learnable for the given domain space D XY and bag hypothesis space H bag ⊂ { h bag : X → Y } :

Proof The performance of MIL algorithms on bags directly depends on the performance of the instances composing those bags. Therefore, the probability that a bag's performance does not reach the optimal hypothesis is greater than or equal to the probability that each individual instance does not reach the optimal hypothesis:

If Condition 1 is satisfied, the probability that the bag's performance does not reach the optimal hypothesis can be expressed as follows:

Ultimately, if the bag is not learnable, then one or more instances become unlearnable.

C.2 Proof of Theorem 2

Theorem 2 If a MIL algorithm satisfies Condition 4 in D Ind XY , it is learnable for instances.

Condition 4 The optimal hypothesis for D Ind XY must be equal to the sum of optimal hypotheses for each individual instance space within D Ind XY :

Proof First, let's assume that the MIL algorithm satisfies Condition 4 in D Ind XY . Based on this assumption, we apply Definition 4 to verify the learnability of MIL in the independent bag hypothesis space.

The learnability in the independent bag hypothesis space means that the algorithm is likely to return predictions within an acceptable error margin from the true optimal hypothesis:

Here, S represents the training data, and A ( S ) denotes the predictions returned by the MIL algorithm. R D ( h ) represents the actual risk of hypothesis h . Now, this probability is expanded as follows:

Now, this probability can be expressed as follows:

In the above equation, N represents the number of instances.

This equation implies that the learnability of bags and instances becomes equivalent. Since we assumed that bags are learnable according to Assumption 1, Condition 4 becomes a necessary and sufficient condition for the learnability of instances.

C.3 Proof of Theorem 3

Theorem 3 If a MIL algorithm satisfies Condition 7 in D Gen XY , it is learnable for instances.

Condition 7 The optimal hypothesis for D Gen XY must guarantee that it is equal to the weighted sum of optimal hypotheses for each individual instance space within D Gen XY :

Proof To demonstrate that the learnability of bags is equivalent to the learnability of instances, we need to show that the following conditions are satisfied:

This equation represents that the weighted sum of risks for bags and instances exceeds the permissible error ϵ with probability δ or less. Therefore, if this equation is satisfied, the learnability of bags and instances becomes equivalent.

In other words, it indicates that the probability that the difference between the risk for each instance and the weighted sum of the entire bag risks is within the permissible error ϵ N is at least 1 -δ .

Through the inequality for the risk of each instance, we can obtain the following:

To prove this, let's define the random variable X inst i as the difference in risk for each instance:

Let's set the range of each X inst i to [ -M,M ] , and define c i as the mean of X inst i . c i becomes 0 due to cancellation between E [inf R D Gen XY ] and E [ α i inf R inst i ] .

Let's define the function f ( X inst 1 , X inst 2 , . . . , X inst N ) as the optimal risk over the general bag domain D Gen XY :

In this setup, the instances within the bag are not independent; instead, they interact and contribute collectively to the overall bag risk. To capture the effect of each instance on the bag's optimal risk, we introduce weights α i that represent the contribution of each instance's risk to the total risk. This allows us to define the expected value of f as the weighted sum of the optimal risks of each instance domain:

This expression reflects the dependencies and interactions between instances, allowing us to represent the general bag risk as a function of each instance's weighted risk.

In the general bag domain space, where one or more instances may have dependent relationships, we apply the Azuma-Hoeffding inequality.

Since f ( X inst 1 , X inst 2 , . . . , X inst N ) represents the risk of the bag as stated earlier, the equation can be written as follows:

The t term in | inf R D Gen XY -E [inf R D Gen XY ] | represents the deviation from the expected value, so it can be replaced by the permissible error ϵ for learnability, yielding the following:

Finally, it can be expressed as follows:

c i represents the mean of errors. Since Deep MIL algorithms are trained to minimize the mean error, ∑ N i =1 c i ≃ 0 .

This ensures that the sum of c i values remains within a certain range, guaranteeing that it is smaller than the probability of being trained within the permissible error range ϵ , denoted as δ .

Therefore, under the assumption of successful learning, Condition 2 is satisfied. Thus, a well-trained MIL model becomes learnable for instances.

C.4 Proof of Theorem 4

Theorem 4 In D Ind XY , the MIL algorithm is PAC learnable for bags.

Proof The PAC learnability of D Ind XY implies that every concept space C D Ind XY can be approximated by hypothesis spaces H D Ind XY .

Since D Ind XY assumes that predictions for all instances are independent, predictions for bags become the union of predictions for instances.

The hypothesis space H bag non -att for Non-Attention MIL algorithms, which do not use Attention, is also the union of predictions of instances:

Thus, H bag non -att becomes equivalent to the hypothesis space H D Ind XY of PAC Learnable D Ind XY :

Therefore, in the case of PAC Learnable scenarios, Non-Attention MIL is learnable.

The hypothesis space H bag att of Attention MIL encompasses the hypothesis space H bag non -att of Non-Attention MIL:

Thus, if D Ind XY is PAC Learnable, Attention MIL is also learnable.

Ultimately, if D Ind XY is PAC Learnable, then all MIL algorithms can learn from bags.

C.5 Proof of Theorem 5

Theorem 5 In D Gen XY , MIL algorithms that aggregate independent hypothesis spaces for each instance must utilize attention satisfying Condition 8 to be PAC learnable for bags.

Condition 8 The hypothesis space H bag att of Attention MIL should be equal to the sum of independent hypothesis spaces h inst i multiplied by attention weights Att i :

Proof The PAC Learnability of D Gen XY implies that all concept spaces C D Gen XY of D Gen XY can be approximated by the hypothesis space H D Gen XY of D Gen XY :

H D Gen XY represents the union space of hypotheses H Gen insti with dependencies within the same bag. The optimal hypothesis within each instance space can be expressed as:

Combining these optimal hypotheses yields:

From previous proofs, it's shown that the Azuma-Hoeffding inequality satisfies:

Thus, the hypothesis space H Gen bag for D Gen XY can be expressed as:

In proving the learnability of bags for D Gen XY , it's shown that the hypothesis space for Non-Attention MIL algorithms is:

The hypothesis space H D Gen XY extends beyond the range of hypotheses that can independently predict all instances, incorporating dependencies with weighted hypotheses α i :

Thus, Non-Attention MIL Algorithms are not learnable from D Gen XY .

However, according to Condition 8, the hypothesis space of Attention MIL Algorithms is:

Here, replacing Att i with α i makes H bag att identical to H D Gen XY :

Thus, Attention MIL is learnable from D Gen XY in terms of bags.

C.6 Proof of Lemma 1

Lemma 1 Condition 9 serves as a necessary condition for the learnability of instances, when the hypothesis space for the i th instance of a MIL algorithm is H inst i ∪ H add i . Here, H inst i represents the hypothesis space for the i th instance, and H add i denotes the hypothesis space for the i th instance generated through elements outside the i th instance.

Condition 9 H add i must be a subset of H inst i :

Proof

When H inst i ̸⊂ H add i := { h add i : X add i →Y} , the hypothesis space H bag for bags is as follows:

In this case, using the Azuma-Hoeffding Inequality to compare the deviation between R bag and each R inst i , we get:

If h add i is not a subset of h inst i , additional hypothesis space exists, resulting in a deviation between the risk of the Bag-level hypothesis and the risks of each Instance.

Hence, the deviation between R bag and R inst i is greater than or equal to 0:

Therefore, it fails to satisfy the necessary and sufficient condition for the learnability of instances in D Ind XY . The independent bag domain space implies that all weights in the general bag domain space are equal, thus the relation is as follows:

Hence, if it's not learnable in the independent bag domain space, it means it's not learnable in the general bag domain space. Therefore, through the proposed theoretical framework, Condition 7 becomes a necessary condition for MIL.

Thus, Condition 9 becomes a necessary condition for learning instances in MIL.

C.7 Proof of Theorem 6

Theorem 6 In D Ind XY , MIL algorithms that perform instance-pooling are PAC learnable for instances.

Proof

First, the learnability of MIL algorithms on instances in D Ind XY means the algorithm satisfies Condition 4:

Performing Instance-Pooling means forming bag-level hypotheses by independently combining hypotheses at each instance level. The error at the bag level derived from instance-level pooling can be defined as follows:

We can express the error at each instance level as:

Now, comparing the two errors, we have:

Thus, performing pooling on instance predictions satisfies Condition 4.

This proves that MIL algorithms can learn from instances in D Ind XY .

C.8 Proof of Theorem 7

Theorem 7 MIL algorithms that perform Embedding-Pooling are not learnable for instances. Proof

In Embedding-pooling, features of each instance are combined using an aggregation function g :

Here, f i ( X i ) represents the feature of each i th instance, and g is a function that integrates features to generate a single vector. Consequently, H add i includes hypotheses based on F ( X ) , making H add i dependent on the features of all instances, f 1 ( X 1 ) , f 2 ( X 2 ) , . . . , f n ( X n ) :

On the other hand, H inst i produces results dependent solely on the i th instance feature:

A function h ∈ H add i using F ( X ) cannot belong to H inst i . This is because functions in H inst i only use X i as input, while h is based on F ( X ) , i.e., the combined result of all X i features. Since

functions in H add i have more complex dependencies beyond the scope of H inst i , the following inequality holds:

Therefore, MIL algorithms using Embedding-pooling fail to satisfy Condition 9, rendering them incapable of learning from instances.

C.9 Proof of Theorem 8

Theorem 8 If the MIL algorithm does not adhere to Condition 10, it is not learnable for instances. Condition 10 The risk R inst i for the i th instance should be as follows:

Proof

If h bag -level is not empty, then the following holds:

For every δ > 0 , there exists at least one predictor belonging to h bag -level that can make the lower bound of R bag greater than the sum of the lower bounds of ∑ N i =1 R inst i . This implies that if h bag -level is not empty, it fails to satisfy Condition 9. Thus, h bag -level must be empty for Condition 4 to hold.

As a result, Condition 10 also becomes a necessary condition for MIL algorithms to be learnable from instances.

C.10 Proof of Theorem 9

Theorem 9 When MIL algorithms use conjunctive pooling for aggregation in D Gen XY , they are learnable for instances.

Proof

Condition 7 being satisfied implies learnability from instances:

Here, α i represents the weight for the i th instance, satisfying the following conditions:

Given Assumption 1 where the MIL algorithm is assumed to be PAC learnable for bags, R bag becomes the lower bound of prediction error for bags, and R inst i becomes the lower bound of prediction error for each instance. Thus, Condition 7 can be expressed as:

For Conjunctive Pooling, the optimal hypothesis h bag for bags is:

So, the same conclusion applies.

The lower bound for the optimal hypothesis h inst i for the i th instance is:

Therefore, the lower bound of prediction error for the optimal hypothesis h bag for bags is:

Ultimately, MIL algorithms using Conjunctive Pooling satisfy Condition 7 and are learnable from instances.

C.11 Proof of Theorem 10

Theorem 10 If the MIL algorithm extracts features of instances through RNN-based neural networks for aggregation, it is unable to learn from instances.

Proof When extracting features through an RNN, the hypothesis space of the i th instance's features includes not only the hypothesis spaces based on the i th instance's information but also those based on the information of preceding instances up to the i th one:

Therefore, when extracting features of instances through neural networks like RNNs, if H insti := { h insti : X insti → Y } , then H bag -level ( x ) := { h bag -level : X → Y} ̸ = ∅ :

̸

This means that when performing MIL using features extracted through RNNs, it fails to satisfy Condition 10, a necessary condition for learning from instances. Hence, it becomes incapable of learning from instances.

Therefore, it fails to satisfy Condition 10.

As a side note, in the case of extracting features through bidirectional RNNs, the size of the hypothesis space would be:

D Experimental Setting

D.1 Experimental Validation of Theorem 5

Theorem 5 explains that Instance pooling MIL, which aggregates instance hypothesis spaces independently, does not satisfy Condition 8 and thus is not learnable with respect to D Gen XY . In contrast, the Conjunctive Pooling model, which multiplies weights by the hypothesis space of instances to reflect the relationships among them, is learnable.

To validate this, we conduct comparative experiments on the following synthetic datasets, assuming D Gen XY outside of D Ind XY . Each bag in the dataset is labeled based on the relationships between the ten instances constituting the bag, and one of four labels is assigned accordingly.

Label 1 : If the bag contains both 3 and 5, the bag's label is 1.

Label 2 : If the bag contains 1 but not 7, the bag's label is 2.

Label 3 : If the bag contains 1 and also contains 7, the bag's label is 3.

Label 0 : Any bag that does not meet the criteria for other labels is assigned the negative label 0.

Under these labeling assumptions, the MNIST dataset's training data is split into training and testing datasets in an 8:2 ratio. Then, ten images are randomly selected to form a bag, and labeling is performed according to the assumptions.

Label:0

8

7

9

9

7

9

Label: 1

8

7

2

3

7

5

S

5

Label: 2

力

2

h

3

3

8

Label:3

3

8

7

6

7

To extract features of the synthetic dataset, which assumes each MNIST image as an instance, all models use the feature extractor structure described in Table 6. First, mi-Net [5], Attention MIL [2], Additive MIL [9], and Conjunctive MIL [10] performed Aggregation through basic Instance, Attention, Additive, and Conjunctive-pooling operations, respectively. We used the same classifier structure used to perform the pooling operations described in Appendix A.2. The employed attention mechanism is the gated attention mechanism, which is commonly used in MIL research [2]. Second, Causal MIL [21], MIREL [38], Loss-Attention [39], SA-AbMILP [40], and TransMIL [19] performed additional tuning on pooling for performance improvement. We implemented them using the official code provided by each respective paper 23456 . Third, for MIREL [38], although the prediction modules

for the bag and the instances operate separately, the validation is adjusted to fit the nature of MIL. Specifically, the bag-level prediction module is not used, and instead, the pooled instance predictions are used for bag-level evaluation. Fourth, for Conjunctive MIL [10] and TransMIL [19], while each original paper utilizes positional information that affects the predictions, this factor is excluded from our experiments, as it does not influence the outcomes in our setup.

The operational processes and theoretical validation for Causal MIL [21], MIREL [38], LossAttention [39], SA-AbMILP [40], and TransMIL [19] are performed in Appendix F.

The hyperparameter settings for the models used in the experiments are shown in Table D.

Since the labels for the bags are multi-class, we used macro-F1 score (Macro-F1), micro-F1 score (Micro-F1), and weighted-F1 score to measure multi-class classification performance. To ensure generalized results, each algorithm was trained ten times with different initializations, and the average performance was measured. All experiments conducted in this study were performed on an Intel(R) Xeon(R) Silver 4210R 40 Core CPU @ 2.40 GHz, 32GB RAM, and NVIDIA RTX A5000.

D.2 Experimental Validation of Theorems 8 and 9

Theorem 8 and Theorem 9 demonstrate that Attention MIL [2] and Additive MIL [9], which perform attention multiplication on instance features, are not capable of learning at the instance level, whereas Conjunctive MIL is capable of instance-level learning.

To empirically validate this, it is necessary to compare the prediction performance of a Deep MIL model on bags with its performance on instances. Therefore, experiments should be conducted on datasets that have labels for both bags and instances. In this study, we use the labeling criteria for the synthetic dataset explained in Appendix D.1.

The feature extractor and hyper-parameter settings followed the model structure used in Appendix D.1. The performance measurement method also adhered to the approach specified in Appendix D.1. However, due to the nature of MIL, which involves learning whether a specific instance is positive for each class, we also calculated the average AUROC performance for each class and measured the overall AUROC.

To analyze the impact of the attention mechanism, we adjusted the temperature parameter ( τ ) used in the attention operations to three values: 0.5, 1, and 2. This process was repeated ten times to calculate the average performance. The average performance of each algorithm for the bag and the average performance difference between the bag and instances are discussed in Section 4.4.3. The performance analysis based on the adjustment of τ is detailed in Appendix E.1.

Label: 0

0

0

0

0

0

0

0

0

0

0

3

9

8

3

b

2

2

Label: 1

0

0

0

0

1

0

0

0

0

1

0

a

4

5

8

3

Label: 2

0

0

0

0

2

0

0

0

0

0

2

y

A

4

3

Label: 3

0

0

0

0

0

0

0

0

3

0

V

4

D.3 Experimental Validation of Theorems 10 and 11

Theorem 10 and Theorem 11 demonstrate that using an RNN layer to capture positional dependencies or performing positional encoding can act as factors that hinder the learnability of instances.

To experimentally validate this, we use the WebTraffic dataset [10], which provides label information for each time point in a time-series sliding window data. Both the model structure and hyperparameters are set identical to those in Early et al. [10], and models are configured based on the extent to which features with positional encoding and RNN are used, as following four models 7 :

Amodel that does not use features with positional encoding and RNN for instance prediction and attention operations (Default)

A model that uses features with positional encoding and RNN for both instance prediction and attention operations (All)

A model that uses features with positional encoding and RNN only for attention operations (Att)

A model that uses features with positional encoding and RNN only for instance prediction operations (Predict)

All experiments conducted in this study were performed on an Intel(R) Xeon(R) Silver 4210R 40 Core CPU @ 2.40 GHz, 32GB RAM, and NVIDIA RTX A5000.

To evaluate the instance-level prediction performance of deep MIL models, we used the normalized discounted cumulative gain (NDCG@n) and the area under the perturbation curve with random orderings (AOPCR) metrics, as employed by Early et al. [52, 10]. To reliably measure the performance of each model, we repeated the evaluation 10 times and calculated the average performance. AOPCR assesses the interpretability of time-series data by measuring the decrease in model predictions based

on the importance of time points. Starting from the most important time point, the time series is rearranged in order of importance. AOPC is then calculated by removing instances one by one from the most important time points and evaluating how the predictive performance changes.

Here, the MoRF function removes time points in the given order of importance. Instead of removing each time point individually, time points are grouped into blocks corresponding to 5% of the total time series, and perturbations are applied until 50% of the time series is removed, limiting the number of model calls to a maximum of 10. Finally, AOPCR is calculated by comparing the average AOPC value for the given order of importance to the average AOPC value for random orderings to achieve normalization:

Here, T represents the number of repetitions of random orderings. In this study, T is set to 3 following the settings of Early et al. [10].

NDCG@n is a metric used to evaluate instance predictions by comparing them to the ground truth ordering for a specific class. Instances within a bag are classified as supporting instances, neutral instances, or refuting instances based on their labels. The ideal prediction order for instances should be supporting instances, neutral instances, and then refuting instances for the given class. NDCG@n is calculated by ranking the instance predictions { ϕ 1 , . . . , ϕ k } for the given class from highest to lowest importance and comparing this order to the ground truth order as follows:

Here, IDCG is the ideal discounted cumulative gain, which normalizes the score based on the value of n , where n is the number of instances with a positive label.

D.4 Experimental Validation of MD-MIL

In this section, we conduct additional experiments in an extended scenario of Video Anomaly Detection (VAD) to demonstrate that the framework proposed in this study assists in designing optimal algorithms for realistic MD-MIL scenarios. The traditional V AD problem has been addressed as a type of 1D-MIL problem, as shown in Figure 5 (a), where the objective is to detect anomalous snippets based on labels for the entire video. In this context, each snippet should reflect temporal dependencies.

Moreover, the conventional VAD problem can be extended as an MD-MIL problem, as depicted in Figure 5 (b). The problem aims to detect both anomalous snippets and anomalous patches within each snippet based on labels for the entire video. In this case, the patches that constitute a snippet must not only reflect relationships with other patches within the same snippet but, much like multivariate time series data [53, 54], also account for temporal dependencies with corresponding patches in other snippets. Therefore, the V AD problem in an MD-MIL context requires consideration of relationships not only among instances within the same bag but also with instances at the same position across different bags.

To design a model that is capable of learning about instances while reflecting relationships under this complex problem setting, the following requirements must be satisfied:

The VAD problem in MD-MIL should reflect relationships among complex multi-dimensional instances. Therefore, this should be addressed by a learnable MIL method (i.e., Conjunctive-pooling) on D Gen XY .

(a) The video data structure consisting of 1- dimensional-instances.

(b) The video data structure consisting of multi-dimensional-instances.

Additional information about relationships between instances, such as position encoding, should be used in the attention computation process. This ensures learnability about instances and can improve performance, as demonstrated in Section 4.4.3. Based on this, attention operations should incorporate this information to reflect temporal relationships with patches at the same position in other snippets for each patch detection.

To validate these requirements, we measure and compare the performance between the following three types of MD-MIL models:

None-Attention : This model predicts for higher-dimensional bags through Instance-pooling in each dimension. It independently models the relationships between snippets composing a video and patches composing the same snippet.

Attention : This model predicts for higher-dimensional Bags by performing Conjunctive-Pooling based on attention weights calculated within each dimension (same bag). Through this structure, it can make predictions reflecting the relationships between snippets composing a video and patches composing the same snippet.

Cross-Attention : To reflect the temporal dependencies of patches at the same position in different snippet bags, this model computes attention weights based on features of each patch computed through a bidirectional GRU-layer and performs Conjunctive-pooling. Through this structure, it can make predictions reflecting the relationships between snippets composing a video, patches composing the same snippet, and patches at the same position in different snippets.

All models are trained using anomaly labels at the video level. The encoder structure for computing instances is identical across models, as shown in the Table 8, with differences only in pooling and attention operations.

The hyperparameter settings for the models used in the experiments are presented in the following Table 9.

For this study, we utilized the ShanghaiTech dataset [55, 56], a well-established benchmark dataset for Video Anomaly Detection (VAD). As illustrated in Figure 5 (b), each snippet was pre-divided into four patches. To extract features from these patches, we employed a pre-trained I3D-ResNet model 8 . This model was used to extract snippet features for each patch, which were then utilized in our experiments.

E Additional Experimental Results

E.1 Experimental Validation of Theorems 8 and 9

The results of adjusting the variance of attention by applying τ in the softmax operation to determine how much attention affects the error between bag prediction performance and instance performance in deep MIL are presented in Table 10. While all Attention-pooling MIL models show high performance for bags, they exhibit low learning performance for instances. Conjunctive-pooling MIL, satisfying Theorem 9, shows very little error between bag performance and instance performance, with instances sometimes even outperforming bags. However, in the case of Additive-pooling MIL, as τ increases and the variance among instances decreases, AUROC between bags and instances decreases. This result suggests that for Additive-pooling MIL to be learnable for instances, the strength of the attention must be weak. In other words, the closer the structure is to instance predict-level pooling, the more likely it is to be learnable for instances.

E.2 Experimental Validation of MD-MIL

The experimental results indicate that the Cross-Attention method achieved the best performance, followed by the Attention-driven model, while the None-Attention-driven model showed the least favorable performance. These results demonstrate that 1) attention weights are necessary to capture the relationships between video snippets and patches within the same snippet, and 2) when learning relationships between patches in different snippets (i.e., bags) are required, Cross-Attention-driven model, which utilizes such information only during attention computation, is desired.

F Interpretation of MIL Algorithms through Our Theoretical Framework

In this section, we demonstrate how the proposed theoretical framework can interpret actual existing MIL algorithms. To further validate our framework's capability to assess instance-level learning in MIL algorithms, we extended our analysis beyond the basic pooling techniques discussed earlier. We additionally selected five MIL algorithms that apply additional tuning to the existing pooling methods for performance improvement. These algorithms were chosen to demonstrate how our framework can generalize to MIL approaches that enhance standard pooling operations through additional optimization techniques.

Causal MIL [21]: This algorithm utilizes the bag-level features as external variables to identify independent latent variables directly related to the prediction of each instance's feature. It then

performs instance-pooling based on these latent variables. Since Causal MIL derives predictions using individual instance features and performs instance-pooling based on these values, it can be learnable on instances under the space D Ind XY .

Multi-Instance Residual Evidential Learning (MIREL) [38]: This algorithm is composed of two primary modules: 1) a module that predicts the bag label using the average feature of the instances that constitute the bag and 2) a module that performs predictions on individual instances based on their features. MIREL calculates the residual between the bag prediction and the instance predictions, which is indirectly used to perform instance-pooling. The instance prediction module of MIREL

operates based on the instance features themselves without applying weighted aggregation through additional information. Thus, it can be learnable on bag predictions under the space D Ind XY :

(a) The loss function of MIREL can be expressed as the sum of two loss terms generated by the bag and instance prediction modules:

L bag = ℓ bag (ˆ y bag , y bag ) , L inst = ∑ N i =1 ℓ inst (ˆ y i , y i )

L total = L bag + λL inst , where λ is a weighting factor.

ˆ y bag is the prediction made by the bag prediction module; y bag is the true label for the bag; ˆ y i is the label predicted by the instance prediction module; y i is the label inferred for each instance based on ˆ y bag and residual evidence.

(b) The risks for the bag, instance, and total can be defined as follows:

R bag = E ( X,Y ) ∼ D Ind XY [ ℓ bag (ˆ y bag , y bag )]

R inst i = E ( X,Y ) ∼ D Ind Xinst i Y [ ℓ inst (ˆ y i , y i )]

R total = R bag + λ ∑ N i =1 R inst i

(c) Since our theoretical framework evaluates instance learnability under Assumption 1, we can express this as follows:

inf R total = inf R bag + λ inf ∑ N i =1 R inst i

R bag is learnable under D Ind XY . Therefore, inf h ∈ H R D Ind XY = inf R bag holds.

inf ∑ N i =1 R inst i is learnable with respect to the instance labels generated by the bag prediction module. Therefore, inf h ∈ H R D Ind XY = λ inf ∑ N i =1 R inst i .

λ is a constant, inf h ∈ H R D Ind XY = inf ∑ N i =1 R inst i . Therefore, Condition 4 holds.

Therefore, the algorithm is learnable for instances.

Loss-based Attention for Deep Multiple Instance Learning (Loss-Attention) [39]: This algorithm directly connects the attention mechanism with the loss function to simultaneously learn instance weights, instance predictions, and bag predictions. Specifically, Instance weights are calculated based on the loss function. The parameters of the fully connected layer for bag prediction are shared with the instance weight calculation. A regularization term consisting of instance weights and cross-entropy functions is introduced to improve instance recall. A consistent cost is added to smooth the learning process. Through this approach, Loss-Attention can more effectively learn the importance of instances and improve bag classification performance compared to existing attention-based MIL methods.

Loss-Attention [39] aims to simultaneously optimize the hypothesis spaces for both instances and bags by sharing parameters between them during the learning process. However, in D Gen XY , according to Condition 8, while the Loss-Attention algorithm is learnable for bags, it is not learnable for

instances.

Self-Attention Attention-based MIL Pooling (SA-AbMILP) [40]: This algorithm enhances the reflection of relationships between instances compared to traditional Attention MIL [2] by employing a self-attention mechanism to compute instance features, which are then used for attention-pooling. Since SA-AbMILP uses transformed instance features that reflect relationships with other instances via self-attention for prediction and attention-pooling, it does not satisfy Condition 10 and is not learnable for instances.

Transformer-based MIL (TransMIL) [19]: This algorithm employs a Transformer module to better capture the relationships between instances. The Transformer module extracts a cls token feature based on the information of instances that reflect their relationships with other instances. TransMIL performs prediction on the bag using the cls token feature, a form of attention-pooling. Since TransMIL uses positional encoding features, its learnability for instances cannot be guaranteed by Theorem 11. Moreover, its attention-pooling does not satisfy Condition 10 and is not learnable for instances.

G Limitations and Future works

According to the MIL survey by Carbonneau et al. [57], recent major issues in MIL have been defined as label ambiguity [58], label noise [59, 60], complex bag composition [61, 62, 63], and handling non-i.i.d. data [64, 65]. This study has only addressed the learnability of instances and did

not cover the applicability of these issues. In future work, we plan to utilize the theoretical framework developed in this study to devise countermeasures for issues such as label ambiguity and label noise, aiming to enhance the performance of MIL algorithms through theoretical research. Furthermore, we will seek solutions for problems like complex bag composition and handling non-i.i.d. data based on the theoretical framework of MIL algorithms. Additionally, noting that existing MD-MIL algorithms [6, 20] have extended non-learnable MIL algorithms to multi-dimensional cases, we will devise new MD-MIL methodologies based on the learnable MIL algorithms by our proposed theoretical framework.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The main contributions of the paper are presented in the abstract and introduction.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We have shown the Limitation and Future work in Appendix G.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The Theorems and Lemmas proposed in the paper are derived from the Definitions and Assumptions defined through Notation. Detailed proofs for each Theorem are provided in the Appendix.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: In this study, to validate several final Theorems derived from the initial Theorems, we created appropriate experimental environments for each Theorem and conducted experiments. The results from these experiments provided empirical validation for the Theorems.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: In this study, experiments were conducted using the open datasets MNIST and the WebTraffic dataset [10]. For MNIST, since it was used to create synthetic datasets tailored to the experimental environment, we submit the code for data preprocessing and model training. For the WebTraffic dataset [10], as experiments were conducted by swapping models in the code published by Early et al. [10], we submit the code related to the model structures.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We have specified the experimental setting and test details in Appendix D.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: Although we did not include error bars, we ensured the reliability of the experiments by repeating each experiment 10 times and taking the average of the performance results. This approach was taken to enhance the statistical reliability of our results.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We specified the sentence 'All experiments conducted in this study were performed on an Intel(R) Xeon(R) Silver 4210R 40 Core CPU @ 2.40 GHz, 32GB RAM, and NVIDIA RTX A5000.' in the section describing the experimental settings.

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Our research proposes conditions that must be theoretically guaranteed for the MIL model to be learnable on instances. This study poses no ethical issues; rather, it is expected to ensure the reliability of MIL algorithms.

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Our research proposes conditions that must be theoretically guaranteed for the MIL model to be learnable on instances. Since MIL research is utilized in fields requiring reliable predictions, such as pathological image diagnosis and video anomaly detection, satisfying these conditions is essential. In this regard, our research has a positive societal impact.

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our research proposes a theoretical framework to evaluate whether MIL algorithms are learnable on instances, which is not associated with such risks.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: For experiments extending existing research, we have cited the respective studies and provided the GitHub link in the section describing the experimental settings.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: We provide the code for dataset preprocessing, model training, and the model itself as additional submissions.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This study did not involve crowdsourcing.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This study did not involve crowdsourcing or research with human subjects.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to discuss the theoretical learnability of multi-instance learning. It first gives the necessary conditions for bag-level learnability and instance-level learnability. Then it discusses how the results can be used to verify the learnability of existing MIL algorithms. Finally, some empirical experiments are adopted to validate the theoretical findings.\n\nSTRENGTHS:\nThis paper discusses the theoretical aspects of multi-instance learning (MIL). As MIL has attracted increased interest for applications in medical image analysis and time series analysis, it is nice to have some time to pause the development of heuristic algorithms and think about the theoretical foundations.\n\nWEAKNESSES:\n1. Many statements are not rigorous and this significantly reduces readability. For example, in Condition 4 the text refers to that the optimal hypothesis for independent instance domain must equal the sum of hypotheses for the individual instances. However, the formula is about the risks and they are not equivalent. In theorem 4, it just says \"the MIL algorithm\" without specifying which algorithm or the defining characteristics of the algorithm, which is not rigorous.\n\n2. For the general bag domain discussed in the paper,  it is not the multi-instance bags that allow dependency among instances as commonly discussed and adopted by previous literature. According to the paper, this is essentially a weighted sum of the instances. It is better to claim this as such, instead of claiming it as a \"general\" bag domain, which it is not.\n\n3. The categorization of pooling methods, as discussed in the appendix, is not consistent with how they are used in the referenced paper. See questions for details.\n\n4. The empirical validation is somewhat lacking. Although this is mainly a theoretical discussion, I don't think the empirical are well designed to support the theoretical claims.\n\nQUESTIONS:\n1. In Section A.2, the attention pooling is different from Ilse et al., 2018, which is the attention-based MIL that most follow-up works are based on. Instead, additive pooling is actually the attention mechanism in Ilse et al., 2018. Why?\n\n2. I am not very clear about the multi-dimension MIL discussed briefly in 4.3.2. Can you explain this sentence \"For instance, in video data, each frame is composed of patches in multi-dimensional structures for each frame dimension.\"?\n\n3. In Table 2, the title is validation of theorem 4, but in the text it says validating theorem 5?\n\n4. Also in Table 2, do you have results for more recently proposed instance pooling approach, such as [21]? This also applies to the results in Table 3.\n\nLIMITATIONS:\nSome limitations are discussed in the appendix. However, it seems to contain some grammatical mistakes which hinder understanding. For example, what is the meaning of this sentence: \"to address the problem that current MD-MIL algorithms [6, 20] extend non-learnable algorithms for instances to multidimensional cases\"?", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-06-21"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper mainly studies the instance-level learnability of common weakly-supervised MIL algorithms. With the PAC theoretical framework, it shows the conditions for MIL algorithms to be learnable for instances. Two general cases of instance distribution, IID instances and non-IID instances, are discussed and analyzed in the proposed theoretical framework. In addition, the proposed theoretical framework covers a wide range of MIL algorithms, which could provide valuable insights for many MIL-based applications in the real-world.\n\nSTRENGTHS:\n- This paper is overall well-written and easy to follow. All key definitions and theorems’ implications are given and explained clearly. \n- This paper studies an interesting and valuable problem, *i.e.*, instance-level learnability in weakly-supervised MIL. This problem is still in the stage of empirical exploration, so a formal study from a theoretical perspective, which this work presents, is needed and could be valuable to the community of deep MIL. \n- The proposed theoretical framework covers a wide range of MIL algorithms, which could provide valuable insights for many MIL-based applications in the real-world.\n\nWEAKNESSES:\n- In Definition 6, the sufficient condition for instance-level PCA learnability, i.e., Condition 2, is given. However, the authors do not prove that Condition 2 is necessary. From my understanding, the instance-level PCA learnability cannot directly deduce Condition 2, as it (Eq. 7) contains an additional bag-level constraint. \n- The authors mainly study one type of MIL algorithm in which pooling (instance embedding level or instance score level) is only performed once. In fact, there is another type of MIL algorithm with multiple pooling operations [1, 2, 3]. Most of these algorithms aim at making instance-level predictions and could be viewed as a combination of the classical algorithms discussed in the paper. The authors are encouraged to discuss them and indicate whether these MIL algorithms could also be incorporated into the proposed theoretical framework. \n\nMinor issues:\n- Table 3: Please consider using a clearer way to present the performance of instance-level prediction and bag-level prediction, as well as their difference. \n- To meet the basic publication requirements of NeurlPS, the authors are encouraged to carefully and repeatedly check those easy-to-avoid errors in their paper, *e.g.*, line 346 - “pr,edication” and “Additve” on page 33. \n\nReferences: \n\n[1] Shi et al., Loss-based attention for deep multiple instance learning. AAAI, 2020.\n\n[2] Liu et al., Weakly-Supervised Residual Evidential Learning for Multi-Instance Uncertainty Estimation. ICML, 2024.\n\n[3] Wang et al., Rethinking Multiple Instance Learning for Whole Slide Image Classification: A Bag-Level Classifier is a Good Instance-Level Teacher. IEEE Transaction on Medical Imaging, 2024.\n\nQUESTIONS:\nSee my comments above.\n\nLIMITATIONS:\nLimitations have been indicated by the authors and written in the paper. There is no additional limitation that needs to be explicitly highlighted here.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-03"}

comments: SUMMARY:
The paper provides theoretical considerations regarding Multiple Instance Learning.

STRENGTHS:
Theoretical consideration supported by the experimental results.

WEAKNESSES:
There are no images depicting the intuition behind those definitions and theorems. 

Paper is extremely hard to follow because it is not giving the intuition and implication for practitioners.

QUESTIONS:
Is it possible to improve presentation of the paper?

LIMITATIONS:
Discussed.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-08

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe main contributions of the paper are theoretical:\n- Proof of the fact that MIL algorithms that are not learnable for bags do not guarantee learnability for instances.\n- Using PAC learning theory (under some assumptions) learnability conditions are derived. These conditions are sufficient and necessary.\n- Basic experiments are carried out to support the theoretical findings empirically.\n\nThe most important results of the paper are theorems 1, 3, 5, 7, 10, 11.\n\nSTRENGTHS:\nThe strongest part of the paper is the theoretical framework and mathematical setup resulting in some key results about PAC learnability at the instance level for various MIL algorithms.\n- Explicit results connecting bag learnability with instance learnability.\n- The paper derives theoretical guarantees about which kinds of pooling maintain PAC learnability for instances.\n- I think the paper could have broader impact in the development of instance level MIL methods.\n- The paper despite being quite dense with theorems and definitions flows reasonably well and is understandable when read. I checked the proofs and I can follow the arguments line by line. I appreciate that the authors take the time to set up clear notation and an proper mathematical formulation of their problem definition.\n\nWEAKNESSES:\nDespite the strengths of the paper there are some weaknesses.\n- From a motivation standpoint, it has been the case for a while now that the strongest methods in MIL are attention pooling based and operate at the bag/embedding level, not the individual instance level. How do the results presented in this paper affect methods such as [1]? Why should we care about instance based methods in 2024? Explainability or interpretability motivations? The paper should make the case for why these results are important given the current state-of-the-art in the field.\n- The methodology is well supported by the proofs. However, the empirical evaluation is not very thorough. It would be nice to see if the methodology has practical implications in standard MIL benchmarks used in new methods. Datasets such as the ones used in [1] are basically standard for MIL research. If the authors want to go one step further they could incorporate more elaborate testing frameworks too [2]. More experiments would help to convince me that the theoretical results have grounding in empirical findings and encourage me to raise my score.\n\nRefrences: \n\n[1] Attention-based Deep Multiple Instance Learning, Maximilian Ilse, Jakub M. Tomczak, Max Welling, ICML (2017).\n[2] Reproducibility in multiple instance learning: a case for algorithmic unit tests., Raff, Edward, and James Holt., NeruIPS (2024).\n\nQUESTIONS:\n- The assumptions (referred to as conditions) 4, 6, 7 are crucial for the theorem proofs in the paper. Are these conditions assumed to be true? Or are they proven to be true somewhere? Is there a way to check if these conditions are true for a given model / dataset practically? \n\n- The framework analyzes independently distributed instances within a bag $\\mathcal{D}^{\\text{Ind}}_{XY}$  as well as cases where instances within the same bag are statistically dependent. However, in many cases it is important to model the relation between different bags (or instances inside different bags). For example, thinking of an image as a set of bags where each bag (patch of pixels) is a set of feature instances (pixels), we might want to learn relationships between nearby pixels that fall in different bags. Do the authors have any insight on if their analysis could be expanded in this direction in future work?\n\nLIMITATIONS:\nThe paper discusses its limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses a critical gap in Multiple Instance Learning research by proposing a theoretical framework to evaluate instance-level learnability of MIL algorithms. Utilizing PAC learning theory, the authors derive conditions under which Deep MIL algorithms can achieve instance-level learnability. The framework is then applied to evaluate the learnability of several existing Deep MIL algorithms, and the findings are validated through empirical studies.\n\nSTRENGTHS:\n1. The paper introduces a novel theoretical framework to assess instance-level learnability in MIL, filling a significant gap in current research.\n2. The authors provide clear and precise definitions of MIL domain spaces, hypothesis spaces, and risk, which aids in understanding the theoretical constructs.\n3. The theoretical conditions and their proofs are well-articulated and grounded in PAC learning theory, providing a strong foundation for the proposed framework.\n4. The paper evaluates a range of Deep MIL algorithms against the proposed theoretical conditions, offering a broad validation of the framework.\n5. The study addresses practical concerns in domains requiring expert labeling, such as pathology, highlighting the potential impact of instance-level learnable MIL algorithms.\n\nWEAKNESSES:\n1. The paper’s theoretical sections are dense and may be challenging for readers not well-versed in PAC learning theory.\n2. The framework relies on several assumptions, such as the independence of instances in certain conditions, which may not always hold in real-world applications.\n3. Details on experimental setups and reproducibility of empirical studies are sparse, which could hinder replication efforts.\n\nQUESTIONS:\n1. How do the proposed theoretical conditions translate to improvements in practical MIL applications? Can you provide more detailed real-world examples?\n2. How robust are the theoretical conditions under different assumptions, such as dependent instances within bags?\n3. Can the framework be extended to other types of machine learning algorithms beyond Deep MIL?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}