Multi-Instance Partial-Label Learning with Margin Adjustment

Authors: wei  tang
yin-fang  yang
zhaofei  wang
weijia  zhang
min-ling  zhang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 05:53:57

Abstract: Multi-instance partial-label learning (MIPL) is an emerging learning framework where each training sample is represented as a multi-instance bag associated with a candidate label set. Existing MIPL algorithms often overlook the margins for attention scores and predicted probabilities, leading to suboptimal generalization performance. A critical issue with these algorithms is that the highest prediction probability of the classifier may appear on a non-candidate label. In this paper, we propose an algorithm named MIPLMA, i.e., Multi-Instance Partial-Label learning with Margin Adjustment, which adjusts the margins for attention scores and predicted probabilities. We introduce a margin-aware attention mechanism to dynamically adjust the margins for attention scores and propose a margin distribution
loss to constrain the margins between the predicted probabilities on candidate and non-candidate label sets. Experimental results demonstrate the superior performance of MIPLMA over existing MIPL algorithms, as well as other well-established multi-instance learning algorithms and partial-label learning algorithms.

Main paper: 
 introduction: 
 Weakly supervised learning is a powerful strategy for constructing predictive models with limited supervision. Based on the quality and quantity of supervision, Zhou [1] systematically categorizes weak supervision into three types: inexact, inaccurate, and incomplete supervision. Inexact supervision indicates a coarse alignment between instances and labels, which is a common and challenging issue in real-world tasks. Multi-instance learning (MIL) [2] [3] [4] [5] [6] [7] [8] and partial-label learning (PLL) [9] [10] [11] [12] [13] [14] [15] are two predominant weekly supervised learning frameworks for learning from samples with inexact supervision in the instance space and the label space, respectively. Recently, multi-instance partial-label learning (MIPL) [16] has been introduced to handle dual inexact supervision, where inexact supervision exists in both the instance space and label space. Therefore, MIPL can be seen as a generalized framework of MIL and PLL. In MIPL, a training sample is represented as a multi-instance bag associated with a candidate label set. The candidate label set comprises one true label and the remaining are false positives. The multi-instance bag contains at least one instance corresponding to the true label and does not contain any instance associated with the false positives. Additionally, positive instances refer to the instances that belong to the true label, while negative instances represent the remaining instances in the bag that are not Dual inexact supervision widely exists in many tasks. In the classification of histopathological images, an image is frequently partitioned into a multi-instance bag due to its high resolution [17] [18] [19] 6] and employing domain experts for providing ground truth labels are costly. As a result, the utilization of crowd-sourced candidate label sets proves to be a valuable strategy in substantially mitigating labeling expenses [20] . To address colorectal cancer classification under dual inexact supervision, Tang et al. [21] have introduced the MIPL algorithm named DEMIPL. This approach employs an attention mechanism to aggregate all instances within a bag into a bag-level feature representation and a disambiguation strategy to identify the true label. Following DEMIPL, ELIMIPL algorithm has been proposed to exploit the label information from both candidate and non-candidate label sets [22] . Additionally, the early MIPL algorithm MIPLGP predicts a bag-level label by aggregating all instance-level labels within the bag without utilizing attention mechanisms [16] .However, existing MIPL algorithms fail to consider the dynamics of the margin between attention scores of positive and negative instances, as well as the margin between the candidate and the noncandidate label sets. These oversights could lead to two major issues. First, the attention scores for positive and negative instances can be quite similar, and in some cases, negative instances may even receive higher attention scores than positive ones, as illustrated in Figure 1 (a). Second, the classifier may even assign higher predicted probabilities to non-candidate labels than to candidate labels. Figure 1 (c) illustrates a scenario where ELIMIPL assigns predicted probabilities to the candidate labels that are only marginally higher than the non-candidate ones. Furthermore, ELIMIPL may even output lower predicted probabilities for candidate labels than non-candidate ones, as depicted in Figure 1(d) .Such erroneous predictions may have serious consequences in applications. For example, in medical image classification, misclassifying images of severe conditions as mild or disease-free may cause patients to miss the opportunity for timely treatment. In this paper, we term this phenomenon as margin violations, where the attention scores of negative instances surpass those of positives, or the predicted probabilities for non-candidate labels exceed those for candidate ones. Margin violations occur in both the instance and label spaces, adversely affecting the model's generalization.To overcome margin violations, we propose a novel end-to-end MIPL algorithm named MIPLMA, i.e., Multi-Instance Partial-Label learning with Margin Adjustment. Specifically, to mitigate margin violations in the instance space, we introduce a margin-aware attention mechanism to consolidate each multi-instance bag into a unified feature representation, incorporating dynamic margin adjustments for attention scores. To address margin violations in the label space, we propose a margin distribution loss that adjusts the margin distribution between the model's highest predicted probability for candidate labels and its highest predicted probability for non-candidate labels. In Figure 1 X i = {x i,1 , x i,2 , • • • , x i,9} and the candidate label set S i = {2, 3, 5, 7}, where L d and L m represent the dynamic disambiguation loss and the margin distribution loss, respectively.higher attention scores to positive instances and enlarges the gap between the attention scores of positive and negative instances. As illustrated in Figure 1 (e) and (f), MIPLMA significantly enhances the classifier's highest predicted probability on candidate labels while concurrently reducing the model's highest predicted probability on non-candidate labels. Consequently, our margin adjustment strategy effectively reduces supervision inexactness in both the instance space and the label space.Our contributions can be summarized as follows: First, we identify the phenomenon of margin violations and adjust the margins in both the instance and label spaces to alleviate this issue. Second, our proposed MIPLMA outperforms state-of-the-art methods significantly. Third, the introduced margin-aware attention mechanism enhances the performance of MIL algorithms, while the margin distribution loss improves the generalization ability of PLL algorithms. 
the proposed approach: 
 Formally, we define a MIPL training dataset as D = {(X i , S i ) | 1 ≤ i ≤ m}, comprising m multi-instance bags and their corresponding candidate label sets. Specifically, each candidate label set S i includes one true label, and the remaining are false positives. We denote the instance space as X = R d , and the label space as The overall framework of MIPLMA is depicted in Figure 2 . Initially, we employ a feature extractor ψ to learn instance-level feature representations H i within the multi-instance bag X i . Subsequently, we propose a margin-aware attention mechanism with adjustable margins of attention scores to fuse H i into a unified feature representation z i . Lastly, we utilize a classifier to predict the probabilities pi of the multi-instance bag. To identify the true label from the candidate label set, we introduce the dynamic disambiguation loss L d and the margin distribution loss L m .Y = {1, 2, • • • , k}, covering k class labels. The i-th bag X i = {x i,1 , x i,2 , • • • , x i,ni } consists of n i instancesFor a given multi-instance bagX i = {x i,1 , x i,2 , • • • , x i,ni } ∈ R d×ni comprising n i instances,we utilize a feature extractor ψ to learn instance-level feature representations, which is defined as follows:EQUATIONHere, H i ∈ R l×ni represents the instance-level feature representation of the multi-instance bag X i , and h i,j denotes the feature representation of the j-th instance in the multi-instance bag X i .The subsequent step involves computing attention scores for each instance. In MIPL, attention scores of all instances are closely distributed during the early stages of training. However, as training progresses, attention scores for positive instances gradually become higher than those for negative instances [21] . Due to dual inexact supervision, the attention mechanism struggles to differentiate between positive and negative instances during the initial training phases and calculate their corresponding attention scores. As training continues, the attention mechanism gradually assigns more distinct attention scores to positive and negative instances.Motivated by this observation, we introduce a margin-aware attention mechanism that dynamically adjusts the margin of attention scores to achieve a closer alignment with the model's training process. The computation of attention scores is given by:EQUATIONwhere W , W 1 , and W 2 are learnable parameters. tanh(•) and sigm(•) are the hyperbolic tangent and sigmoid functions, respectively. The operator denotes element-wise multiplication, and τ (t) denotes the temperature parameter of the margin-aware attention mechanism. Specifically, in the early training stages, a larger temperature parameter is employed to smooth the distribution of attention scores, preventing the attention mechanism from assigning high scores to instances that are not unequivocally identified as positive or negative. In the later training stages, a smaller temperature parameter is used to sharpen the distribution of attention scores, thereby widening the gap between attention scores for positive and negative instances. Consequently, throughout the training process, the temperature parameter at the t-th epoch is dynamically represented as follows:EQUATIONwhere τ m and τ (t-1) represent the minimum temperature and the temperature at the (t -1)-th epoch, respectively. Therefore, Eq. ( 3) describes an annealing process for the temperature parameter τ (t) .For multi-instance bags with varying numbers of positive instances, the distribution of attention scores exhibits variations. Consequently, different multi-instance bags require varying temperature parameters. To address this issue, we introduce the following normalization operations for the attention scores:EQUATIONwhere āi = 1 ni ni j=1 a i,j is the mean value of the attention score A i and Āi = [ā i , āi , • • • , āi ] ∈ R 1×ni . Subsequent to obtaining normalized attention scores, we aggregate the instance-level feature representations to compose the bag-level feature representation z i ∈ R l in the following manner:EQUATIONWe now discuss the theoretical properties of the proposed margin-aware attention mechanism. Based on the definitions of the permutation and permutation invariance (Appendix A), the margin-aware attention mechanism can be seen as the operator A. Then, we have the following theorem: Theorem 1. The margin-aware attention mechanism is permutation invariant.Theorem 1 demonstrates that the margin-aware attention mechanism remains unaffected by the order of instances within multi-instance bags. This property is crucial for algorithms that handle set inputs [23, 4] . The proof is provided in Appendix A.With the aggregated bag-level feature representation, we utilize a classifier that synergizes dynamic disambiguation loss and margin distribution loss to identify the true label.The aim of our dynamic disambiguation loss is to progressively identify the true labels by calculating the classification loss, as illustrated below:EQUATIONi,c and p(t) i,c represent the weight and predicted probability, respectively, on the c-th class at the t-th iteration. This weight represents the probability that the corresponding candidate label is the true label, which is initialized as follows:EQUATIONwhere |•| represents the set cardinality. In the t-th epoch, we update the weight as:EQUATIONwhere α (t) = (T -t)/T is a tuning parameter used to balance the update speed of the weight, and T is the maximum number of training epochs.The dynamic disambiguation loss adjusts the classifier's predicted probabilities for the candidate labels, without affecting the probabilities assigned to non-candidate labels. As illustrated in Figure 1 (d), this circumstance may result in the classifier assigning its highest predicted probability to a non-candidate label instead of a candidate label, i.e., margin violations. To mitigate potential issues in model generalization, it is crucial to maintain a significant margin between the highest predicted probabilities for the candidate and non-candidate labels. Therefore, we propose the margin loss to maximize the margin between the highest predicted probability on the candidate label set and on the non-candidate label set, as shown below:EQUATIONwhere max c∈Si p(t) i,c and max c∈ Si p(t) i,c are the highest predicted probabilities on the candidate label set and the non-candidate label set, respectively. However, only considering the mean margin cannot effectively address margin violations, thus affecting the performance. Some recent studies have shown that the model performance can be enhanced by maximizing the margin mean and minimizing the margin variance simultaneously [24] [25] [26] . Therefore, we employ two statistics of the margins i.e., the margin mean and the margin variance, to adjust the margin distribution. Specifically, we can maximize the margin mean and minimize the margin variance between the highest predicted probability on the candidate label set and on the non-candidate label set simultaneously by minimizing the following margin distribution loss:EQUATIONwhereφ i = {1 -(max c∈Si p(t) i,c -max c∈ Si p(t) i,c)} refers to the margin loss of the i-th multi-instance bag. M{•} and V{•} are the mean and the variance of the margin loss, respectively.During training, the full loss is represented as the weighted sum of the dynamic disambiguation loss and the margin distribution loss, as expressed below:EQUATIONwhere λ represents a hyperparameter. 
experiments: 
 3.1 Experimental ConfigurationsFollowing the experimental setup of DEMIPL [21] , we utilize four MIPL benchmark datasets and one real-world dataset. The four benchmark datasets encompass MNIST-MIPL, FMNIST-MIPL, Birdsong-MIPL, and SIVAL-MIPL, spanning diverse application domains such as image analysis and biology [27] [28] [29] [30] . Additionally, the real-world CRC-MIPL dataset is annotated by crowdsourced workers for colorectal cancer classification. The previous works [21, 22] employ four distinct types of multiinstance features and consists of four sub-datasets: CRC-MIPL-Row (C-Row), CRC-MIPL-SBN (C-SBN), CRC-MIPL-KMeansSeg (C-KMeans), and CRC-MIPL-SIFT (C-SIFT). These multi-instance features are generated via four image bag generators [31] , i.e., Row, single blob with neighbors (SBN), k-means segmentation (KMeansSeg), and scale-invariant feature transform (SIFT), respectively. Besides these multi-instance features, we are the first to employ the ResNet [32] to learn the multi-instance features of CRC-MIPL dataset. Specifically, we partition each image into N non-overlapping patches, treating each patch as an instance. Subsequently, the ResNet-34 is employed to acquire feature representations for each patch, resulting in feature representations of dimension 1000 for each patch.In our experiments, the N is 16 and 25, and the resulting datasets are CRC-MIPL-ResNet-34-16 (C-R34-16) and CRC-MIPL-ResNet-34-25 (C-R34-25).The characteristics of the dataset are detailed in Table 1 . It provides the number of multi-instance bags and total instances, denoted as #bag and #ins, respectively. Furthermore, we employ max. #ins, min. #ins, and avg. #ins to express the maximum, minimum, and average instance count within all bags. The dimensionality of each instance-level feature representation is indicated by #dim. #class and avg. #CLs denote the length of the label space and the average length of candidate label sets, respectively. For a comprehensive performance assessment, we vary the number of false positive labels on the benchmark datasets, represented as r (|S i | = r + 1).We conduct a comprehensive comparison of MIPLMA with a wide variety of baselines, covering MIPL, PLL, and MIL algorithms. For MIPL algorithms, we compare with MIPLGP [16] , DEMIPL [21] , and ELIMIPL [22] . In our evaluation, we incorporate seven PLL algorithms, featuring five deep-learning-based approaches: PRODEN [33] , RC [34] , LWS [35] , CAVL [11] , and POP [36] , one feature-aware disambiguation algorithm, PL-AGGD [37] , and two margin-based algorithms, M3PL [38] and PL-SVM [39] . Furthermore, our comparison encompasses seven MIL algorithms. Three of the MIL algorithms are Gaussian processes-based: VWSGP [40] , VGPMIL [41] , and LM-VGPMIL [41] . Additionally, a variational autoencoder-based algorithm, MIVAE [42] , and three attention-based algorithms: ATTEN [4] , ATTEN-GATE [4] , and LOSS-ATTEN [43] , are included.The deep-learning-based PLL algorithms [33] [34] [35] 11 ] can be equipped with either the linear model or multi-layer perceptrons (MLP) as backbone networks. Results obtained from the linear model are presented in the main body of the paper, while additional experiment results are detailed in the Appendix C. Parameters for all compared baselines have been meticulously tuned, drawing from recommendations in the original literature or refined through our pursuit of improved performance.We implement MIPLMA using PyTorch [44] and conduct training with a single NVIDIA Tesla V100 GPU. Employing the stochastic gradient descent (SGD) optimizer, we set the momentum value to 0.9 with a weight decay of 0.0001. To learn the instance-level features, we employ a two-layer convolutional neural network and a fully connected network for the MNIST-MIPL and FMNIST-MIPL datasets. Since the features of the Birdsong-MIPL, and SIVAL-MIPL datasets are preprocessed, we only employ a fully connected network to learn the feature representations. For the CRC-MIPL dataset, the feature extractor is one of the four image bag generators or ResNet-34, followed by a fully connected network. The initial learning rate is chosen from the set {0.01, 0.05} and coupled with a cosine annealing technique. We set the number of epochs to 100 for benchmark datasets and 200 for the CRC-MIPL dataset. The weight of the margin distribution loss is chosen from the set {0.01, 0.05, 0.1, 0.5, 1, 3, 5} for all datasets. For the annealing process of the temperature parameter, the initial temperature parameter τ (0) = 5. Additionally, τ m = 0.1 and τ m = 0.5 are used for benchmark datasets and the CRC-MIPL dataset, respectively. The dataset partitioning method aligns with that of DEMIPL [21] and ELIMIPL [22] . We execute ten random train/test splits, maintaining a ratio of 7 : 3. Mean accuracies and standard deviations from these ten runs are reported. The code of MIPLMA can be found at https://github.com/tangw-seu/MIPLMA.Since PLL algorithms can not directly handle the multi-instance bags, we utilize two data degradation strategies: the Mean strategy and the MaxMin strategy [16] . The former computes the average feature values across all instances within a bag for producing a bag-level feature representation. The latter identifies both the maximum and minimum feature values for each dimension among instances within a multi-instance bag and concatenates these values to form a bag-level feature representation.Table 2 provides a comprehensive comparison of the results achieved by MIPLMA, three MIPL algorithms (ELIMIPL [22] , DEMIPL [21] , and MIPLGP [16] ), five deep-learning-based PLL algorithms (PRODEN [33] , RC [34] , LWS [35] , CAVL [11] , and POP [36] ) with linear model, and the feature-aware disambiguation PLL algorithm (PL-AGGD [37] ). The evaluation is conducted on benchmark datasets with varying numbers of false positive labels (r ∈ {1, 2, 3}).Notably, MIPLMA consistently exhibits higher average accuracy than the three MIPL algorithms in 33 out of 36 cases. For the methods based on the embedding space paradigm, MIPLMA demonstrates superior performance compared to ELIMIPL and DEMIPL in 21 out of 24 cases. Compared to MIPLGP that follows the instance space paradigm, MIPLMA achieves higher average accuracies than it in all cases. Specifically, on the SIVAL-MIPL dataset, the average accuracies of MIPLGP consistently Hence, the two data degradation strategies do not uniformly outperform each other but have their respective advantages. The selection of the degradation strategy is dependent on the characteristics of the dataset. For simpler datasets, a straightforward Mean strategy may suffice, while for more complex datasets, a sophisticated MaxMin strategy may be preferable.Table 3 presents a detailed comparison of results on the CRC-MIPL dataset. Our method, MIPLMA, demonstrates superior performance in all 11 cases when compared to ELIMIPL [22] , DEMIPL [21] , and MIPLGP [16] . In terms of the PLL algorithm, MIPLMA also achieves superior accuracies in all cases. While the PLL algorithms yield satisfactory results on relatively simple datasets like CRC-MIPL-Row and CRC-MIPL-SBN, their performances noticeably deteriorate when handling more complex datasets such as CRC-MIPL-KMeans and CRC-MIPL-SIFT.Moreover, both MIPLMA and ELIMIPL demonstrate significantly better performance on the CRC-MIPL-KMeans and CRC-MIPL-SIFT datasets compared to the CRC-MIPL-Row and CRC-MIPL-SBN datasets. However, this trend is reversed for MIPLGP and the PLL algorithms. We attribute this discrepancy to the incapacity of these algorithms to effectively model complex features. Particularly, the limitations of the PLL algorithms become more apparent when dealing with complex MIPL data. Therefore, there is an urgent need to devise more effective MIPL algorithms.Tang et al. [21] have introduced the CRC-MIPL dataset, extracting multi-instance features using four hand crafted image bag generators [31] . Both DEMIPL [21] and ELIMIPL [22] were evaluated using these multi-instance features in the literature. In this study, we investigate CRC-MIPL with neural network generated features and employ ResNet to learn deep multi-instance features from the CRC-MIPL dataset. The resulting datasets are referred to as C-R34-16 and C-R34-25. Algorithm q = 0.1 q = 0.3 q = 0.5 q = 0.7 q = 0.9 PRODEN-MA .932±.001 .926±.002 .914±.002 .892±.001 .816±.008 PRODEN .906±.002 .900±.001 .884±.005 .876±.010 .772±.017In summary, feature representations learned by the deep feature extractor ResNet-34 exhibit higher discriminative capacity compared to those generated by image bag generators. Our model consistently achieves the highest classification accuracy among these three MIPL algorithms, especially on the C-R34-25 dataset. These observations suggest that our model not only achieves the highest classification accuracy on traditional features but also handles deep features better.To assess the effectiveness of margin adjustment, we introduce three variants of MIPLMA. MIPL-MAINS denotes the margin adjustment of attention scores exclusively, with λ in Eq. ( 11) set to 0. MIPL-MALAB signifies the margin adjustment of predicted probabilities only, where τ (t) in Eq. ( 2) is assigned Figure 3 demonstrates that MIPLMA consistently outperforms its three variants, proving that margin adjustment in the instance and label spaces can significantly enhance model performance. Additionally, adjusting the margin in the label space yields better results than in the instance space, validating the effectiveness of our proposed margin distribution loss. From another perspective, adjusting the margin of predicted probabilities directly impacts classification accuracies, whereas adjusting the margin of attention scores affects the bag-level feature representations, thereby influencing classification accuracies. Consequently, MIPL-MALAB demonstrates superior performance than MIPL-MAINS. By simultaneously adjusting the margins in both the instance and label spaces, i.e., MIPLMA, optimal results can be achieved. This underscores the effectiveness of our margin adjustment strategy in dealing with the inexact supervision of MIPL.1 for t = 1, 2, • • • , T . MIPL-WOMAIn MIPLMA, the margin adjustments for attention scores and predicted probabilities reduce the supervision inexactness in the instance space and the label space, respectively. MIPL is a generalized framework of MIL and PLL. Therefore, this raises a pertinent question: can margin adjustment enhance the performance of MIL and PLL algorithms?To answer this question, we propose a MIL algorithm named MAAM that is a simplified variant of MIPLMA. We compare MAAM with two classical MIL methods incorporating attention mechanisms, namely ATTEN [4] and ATTEN-GATE [4] , on the MNIST-MIPL (MIL) and FMNIST-MIPL (MIL) datasets.During training, we use only the features of multi-instance bags and their corresponding bag-level true labels. The parameters of MAAM for the two datasets are as follows: learning rates of 0.005 for MNIST-MIPL and 0.01 for FMNIST-MIPL. Additionally, τ (0) = 5 and τ m = 0.1 for both datasets. Table 5 presents the classification accuracies over ten runs, indicating that MAAM outperforms both ATTEN and ATTEN-GATE, particularly on the FMNIST-MIPL dataset. These results demonstrate the effectiveness of MAAM, confirming its superior performance.Moreover, we equip the PLL algorithm PRODEN with the margin distribution loss, resulting in the variant PRODEN-MA. Table 6 presents the classification accuracies of PRODEN-MA and PRODEN on the Kuzushiji-MNIST dataset with varying flipping probability q ∈ {0.1, 0.3, 0.5, 0.7, 0.9}. The only difference between PRODEN-MA and PRODEN lies in that PRODEN-MA includes the margin distribution loss with the weight of 1. We employ MLP as the backbone network for both PRODEN-MA and PRODEN, and keep all other parameters consistent. Experimental results indicate that PRODEN-MA outperforms PRODEN across all scenarios. Notably, under higher disambiguation difficulty, i.e., q = 0.9, the superiority of PRODEN-MA is more pronounced.In summary, adjusting the margins of attention scores improves classification performance for MIL algorithms. Similarly, margin adjustment in the label space enhances the performance of PLL algorithms, particularly in challenging disambiguation scenarios. 
conclusion: 
 This paper investigates the margin adjustments in both the instance space and the label space for MIPL. We propose MIPLMA, which incorporates a margin-aware attention mechanism and a margin distribution loss to adjust the margins for attention scores and predicted probabilities, respectively. Experimental results on the benchmark and real-world datasets illustrate the superiority of our proposed MIPLMA algorithm over a diverse set of baselines, encompassing MIPL, PLL, and MIL algorithms. Specifically, MIPLMA achieves superior performances compared to baselines in 96.4% of cases. These results underscore the effectiveness and significance of our margin adjustment strategy.However, MIPLMA has several limitations. First, similar to other attention-based MIL and MIPL methods, it cannot process multiple multi-instance bags simultaneously. Second, MIPLMA demonstrates a slight overfitting problem on the relatively simple MNIST-MIPL dataset. Third, MIPLMA is not suitable for instance-level classification tasks. In the future, we will delve into designing MIPL algorithms capable of instance-level classification and parallel algorithms that can handle multiple multi-instance bags concurrently. 
 
 Appendix: A Proof of Theorem 1For notation simplicity, we denote the instance-level features H i as H for the rest of this section. First, we definite the permutation and permutation invariant as follows. Definition 1. (Permutation). We characterize Q : R l×n → R l×n as a permutation operation, denoted by Q(H) = HQ, where Q ∈ R n×n is a permutation matrix employed to rearrange the instance order within H. Specifically, Q is an orthogonal matrix, ensuring Q Q = I. Definition 2. (Permutation Invariant). An operator A : R l×n → R l is permutation invariant concerning the instance order inH if Q(A(H)) = Q(H) is satisfied [23].Theorem 1. The margin-aware attention mechanism is permutation invariant. Extract the instance-level features according to Eq. (1)1 Y * = MIPLMA (D, λ, T , X * ) Inputs: D: the MIPL training set {(X i , S i ) | 1 ≤ i ≤ m}, where X i = {x i,1 , x i,2 , • • • , x i,ni }, x i,j ∈ X , X = R d , S i ⊂ Y, Y = {1, 2, • • • ,Calculate the attention scores with the temperature τ (t) as stated by Eqs. ( 2) and (3)7:Normalize the attention scores as stated by Eq. ( 4)Aggregate the instance-level feature representations into the bag-level feature representation according to Eq. ( 5)Classify the multi-instance bag with predicted probabilities. Calculate the full loss L according to Eq. (11) 13:Update the model Φ by the optimizer 14: end for 15: Extract the instance-level features of X * according to Eq. ( 1) 16: Calculate the attention scores and map the instance-level features into a single vector representation z * according to Eqs. ( 2), ( 3), (4), and ( 5 B Pseudo-Code of MIPLMA Algorithm 1 describes the complete procedure of MIPLMA. First, the algorithm uniformly initializes the weights on the candidate label set (Step 1). In each epoch, the training set is divided into multiple mini-batches (Step 3). Then, instance-level feature representations are extracted for each mini-batch and aggregated into bag-level feature representation (Steps 5-8). The subsequent step involves classify the multi-instance bag and updating the weights on the candidate label set (Steps 9-10). Last, the full loss is calculated, and the model is updated (Steps 12-13). For an unseen multi-instance bag, instancelevel feature representations are learned and aggregated into a bag-level feature representation via the feature extractor and the margin-aware attention mechanism, respectively (Steps 15-16). The predicted label is the category corresponding to the highest prediction probability (Step 17).In the field of PLL, there exist several disambiguation algorithms that rely on the maximum margin criteria, with PL-SVM and M3PL emerging as notable algorithms. PL-SVM achieves disambiguation by maximizing the margin mean between the highest prediction probability on the candidate label set and that on the non-candidate label set. In contrast, M3PL maximizes the margin mean between the highest and second-highest prediction probabilities of the classifier for disambiguation. Consequently, we conduct a comparative analysis of classification performance using the MIPLMA algorithm against PL-SVM and M3PL. Figures A1 and present the average accuracy and standard deviation of the three algorithms over ten runs on benchmark and real-world datasets, with Mean and MaxMin representing the corresponding data degradation strategies. MIPLMA consistently outperforms PL-SVM and M3PL algorithms on both benchmark and CRC-MIPL datasets. This superiority can be attributed to two primary factors. First, within MIPLMA, the margin distribution loss simultaneously maximizes the mean margin and minimizes the variance of predicted probabilities. In contrast, PL-SVM and M3PL only maximize the mean margin of predicted probabilities. Therefore, our proposed margin distribution loss provides a more refined approach to optimizing the margins of predicted probabilities. Second, unlike the data degradation strategies employed by PLL algorithms for the indirect treatment of multi-instance bags, MIPLMA aggregates each multi-instance bag into a single bag-level feature representation using the margin-ware attention mechanism. The aggregated feature representations in MIPLMA are more discriminative than those obtained using data degradation strategies. Additionally, PL-SVM and M3PL employ iterative optimization strategies, rendering them challenging to integrate with deep models.Existing MIL algorithms are primarily tailored for addressing binary classification problems and are not readily applicable to solving MIPL problems. To overcome this limitation, we adopt the One vs. Rest (OvR) decomposition strategy employed in MIPLGP [16] for degradation of MIPL data. Specifically, when presented with a multi-instance bag X i linked to a candidate label set S i , we assign each label from the candidate set to the bag, and thus generate |S i | multi-instance bags where each bag is associated with a singular bag-level label. For each class c (c ∈ {1, 2, • • • , k}), we train and test the c-th classifier by transforming the label c to 1 (positive) and all other labels to 0 (negative). In the testing phase, a multi-instance bag yields k predictions from the k classifiers. If only one prediction is positive, the corresponding class label of that positive prediction is considered the classification result for the bag. In cases where there is more than one positive prediction among the k predictions, we choose the class label associated with the classifier demonstrating the highest prediction confidence as the classification result for the bag. If all k predictions are negative, the classification result is the class label with the lowest prediction confidence. In particular, LOSS-ATTEN is a multi-class MIL algorithm, eliminating the need for employing the One vs. Rest decomposition strategy. We can A2: The classification accuracies (mean±std) of MIPLMA and comparative PLL algorithms with MLP on the benchmark datasets (r ∈ {1, 2, 3}). directly assign all candidate labels of a multi-instance bag X i as the label for the multi-instance bag and obtain |S i | multi-instance bags with a singular bag-level label.Table A1 presents the classification accuracy of MIPLMA and seven comparative MIL algorithms. Across all scenarios, MIPLMA consistently demonstrates significantly superior performance compared to the seven MIL algorithms. Notably, among the compared MIL algorithms, LOSS-ATTEN consistently outperforms its counterparts, primarily due to its capability of directly addressing the multi-class MIL problem. While these algorithms yield reasonable results on the MNIST-MIPL and FMNIST-MIPL datasets, their performance diminishes severely on datasets with more intricate features, such as Birdsong-MIPL and SIVAL-MIPL. We attribute this challenge to the noise present in the labels due to data degradation, hindering the effective learning of these MIL algorithms.Among the compared PLL algorithms, PRODEN [33] , RC [34] , LWS [35] , and CAVL [11] can be utilized with either linear model or MLP. Tables 2 and 3 in the main body present the results obtained using the linear model. In this section, we present the results of MIPLMA and the comparative PLL algorithms with MLP on the benchmark and CRC-MIPL datasets in Tables A2 and A3 , respectively.Table A2 presents the performance comparison of MIPLMA with the four PLL algorithms utilizing the MLP on benchmark datasets. MIPLMA consistently outperforms the comparative PLL algorithms in all cases, demonstrating statistically significant superiority. Interestingly, when employing the MLP, the compared PLL algorithms do not consistently achieve superior outcomes compared to the linear model. This observation suggests that the linear model has already captured sufficient features for benchmark datasets, while employing the MLP may lead to overfitting.Table A3 demonstrates that MIPLMA surpasses the comparative PLL algorithms utilizing the MLP in 37 out of 40 cases. Compared to the outcomes obtained using the linear classifiers, the results obtained using the MLP exhibit superior performance. Notably, on the complex CRC-MIPL-KMeans dataset, the enhancement with the MLP is more pronounced. This indicates that the linear model is inadequate for comprehensive feature learning on the CRC-MIPL dataset, while the MLP demonstrates a more comprehensive capability in feature learning.To ensure result reliability, we conduct pairwise t-tests with a significance level of 0.05. To investigate the interpretability of attention mechanisms, we compare MIPLMA with ELIMIPL and DEMIPL regarding the attention scores across three test multi-instance bags in the FMNIST-MIPL dataset (r = 1). Each row in Figure A2 represents a multi-instance bag, with the first to third columns depicting the attention scores produced by MIPLMA, ELIMIPL, and DEMIPL across the three multiinstance bags. It is noteworthy that since the sum of attention scores computed by DEMIPL does not equal 1, we initially normalize them to sum to 1 before visualization.Figure A2 reveals several important findings. First, on the Bags 1 and 2, both our MIPLMA and ELIMIPL accurately identify positive instances. However, compared to ELIMIPL, MIPLMA assigns higher attention scores to positive instances and lower scores to negative ones, confirming the efficacy of our margin-aware attention mechanism. Second, the Bag 3 contains four positive instances, of which MIPLMA only identifies three, erroneously assigning high attention scores to several negative instances. However, the highest attention score assigned to a negative instance is still lower than the scores of these three positives. In contrast, ELIMIPL assigns significantly higher attention scores to two negative instances compared to the scores of the four positives, leading to a pronounced influence of negative instances on the aggregated bag-level features. Third, DEMIPL computes attention scores using the sigmoid function followed by normalization, while MIPLMA and ELIMIPL directly utilize the softmax function for attention score computation. Results indicate that attention scores computed by MIPLMA and ELIMIPL better conform to the distribution of positive and negative instances.In summary, our margin-aware attention mechanism yields more accurate attention scores. Moreover, when distinguishing negative instances within multi-instance bags proves challenging, our attention mechanism effectively mitigates their influence on bag-level features.To adjust the margin for the predicted probabilities in the label space, we first propose the margin loss λ ml as shown in Eq. ( 9). Whereas the margin loss aims to maximize the mean margin of predicted probabilities, the margin distribution loss endeavors to concurrently maximize this mean margin and minimize the variance in these probabilities. Therefore, the margin distribution loss is a generalized formulation of the margin loss. There are several significant connections between the two loss functions. V{•} equals 0. Consequently, the margin distribution loss reduces to the margin loss. (b) When the variability of φ i is low, as indicated by a small variance V{•}, the margin distribution loss slightly exceeds the mean of margin loss M{•}. This observation implies that when φ i exhibits minimal variation across different samples, the margin distribution loss marginally surpasses the margin loss. (c) When the variability of φ i is high, denoted by a large variance V{•}, the margin distribution loss M{•}/(1 -V{•}) can significantly exceed the mean of margin loss M{•}. In extreme cases, if V{•} approaches 1, then the margin distribution loss becomes notably large. This suggests that when φ i exhibits substantial variation across different samples, the margin distribution loss could notably surpass the margin loss.To compare the magrin loss L ml and the margin distribution loss L m , we substitute the L m in Eq. ( 11) with L ml , resulting in a variant named MIPL-MAMM. Table A5 presents the classification accuracies of MIPLMA and MIPL-MAMM on the benchmark datasets, where the weight of the margin loss L ml is tuned to achieve preferable performances. MIPLMA outperforms MIPL-MAMM in 10 of the 12 scenarios. On the MNIST-MIPL dataset, MIPLMA performs worse than MIPL-MAMM when r = 1 and 3. We speculate that this discrepancy may be attributed to the model slightly overfitting due to the consideration of margin distribution on relatively simple MNIST-MIPL dataset. Notably, the performance advantage of MIPLMA is more pronounced on the more challenging Birdsong-MIPL and SIVAL-MIPL datasets. This demonstrates that optimizing the margin distribution is more effective than only optimizing the margin mean, thus confirming the superiority of the margin distribution loss.In summary, the margin distribution loss is associated with the variability of the margin loss across different samples, effectively indicating the concentration of the margin loss within a dataset. In the MIPL datasets, the margin distribution loss outperforms the margin loss in most cases.As shown in Eq. ( 11), λ denotes the weighting factor for the margin distribution loss. Figure A3 depicts the average accuracies from ten runs. On the Birdsong-MIPL dataset, experimental findings encompass λ values ranging from {2, 3, • • • , 10}. The obtained accuracy remains relatively stable when λ ranges between 3 and 7. However, when λ exceeds 7, accuracy decreases. Therefore, it is noteworthy that within a certain range of λ, MIPLMA can maintain a stable performance on the Birdsong-MIPL dataset. In the main experiment, λ is set to 5 for the Birdsong-MIPL dataset. 3) describes the annealing process of the temperature parameter τ (t) , with the initial temperature parameter τ (0) manually specified. To investigate the influence of different initial temperature parameters on MIPLMA, we vary the initial temperature parameter across {4, 5, 6, 7, 8, 9, 10}, while maintaining all other parameters constant.Figure A4 presents the average accuracies obtained from ten runs on the MNIST-MIPL (left) and Birdsong-MIPL (right) datasets. The experimental results suggest that the performance of MIPLMA remains relatively stable within the range of {4, 5, 6, 7, 8, 9, 10}, particularly noticeable when r = 1 and r = 2 on the MNIST-MIPL dataset. However, MIPLMA exhibits fluctuations in average accuracy when r = 3 on the MNIST-MIPL dataset, and the standard deviation is significantly higher compared to when r = 1 and r = 2. The MNIST-MIPL is a five-class dataset, and r = 3 represents an extremely challenging scenario. Therefore, we believe that the fluctuations in average accuracy observed in Figure A4 (left) are unavoidable. Furthermore, as shown in Figure 1 , when τ (0) varies within the range of 5 to 10, the performance of the Birdsong-MIPL dataset does not exhibit significant changes.In summary, the results presented in Figure A4 highlight the robustness of MIPLMA to variations in the initial temperature parameter within the range of {5, 6, 7, 8, 9, 10}. In our experiments, the initial temperature parameter τ (0) is consistently set to 5 for all datasets. Multi-instance learning has its roots in drug activity prediction [45] , and it has found applications in a variety of fields ranging from text classification [46, 42] , object detection [47] , and video anomaly detection [48] . In contemporary multi-instance learning methodologies, a prevalent strategy involves incorporating attention mechanisms to aggregate features from each multi-instance bag into a unified feature representation, subsequently fed into a classifier. Ilse et al. [4] introduced both the plain attention mechanism and gated attention mechanism to effectively enhance the performance of binary multi-instance learning. An extension of this paradigm is the loss-based attention mechanisms [43] , providing a solution for multi-class tasks. Owing to the exceptional performance of the attention mechanisms, multi-instance learning methods based on attention mechanisms have gained widespread adoption in tasks such as histopathological image classification [49, 50] . These attention mechanisms typically fall under the category of soft attention mechanisms, wherein the weighted sum of attention scores for all instances in a multi-instance bag yields a bag-level feature representation. Conversely, Li et al. [51] introduced hard attention mechanisms, which focus on selecting a subset of instances from multi-instance bags to construct the feature representation.Despite the considerable performance advancements achieved by these algorithms in multi-instance learning tasks, their direct application in multi-instance partial-label learning scenarios is impeded by their inability to handle inexact label information directly.Partial-label learning has widespread applications in diverse real-world scenarios, encompassing facial age estimation [37] , face naming [9] , object classification [52] , and bioinformatics [29, 53] . Margin violations also exist in partial-label learning, prompting researchers to introduce the maximum margin criteria as a viable solution. Nguyen and Caruana [39] employed the maximum margin criterion to augment the distinction between the model's highest predicted probability on candidate labels and its highest predicted probability on non-candidate labels. Similarly, Yu and Zhang [38] focused on maximizing the margin between the model's predicted probability on the true label and its highest predicted probability on labels other than the true one. However, these methods require an alternating optimization, contributing to the intricacies of the optimization procedure. In recent years, numerous deep learning-based partial-label learning algorithms have emerged. Lv et al. [33] utilize linear classifiers or multi-layer perceptrons to learn feature representations from instances, employing progressive disambiguation strategies to identify true labels. Following this line of thought, Feng et al. [34] delved into the generation process of partial-label data and proposed two theoretically guaranteed partial-label learning algorithms. Similarly, Wen et al. [35] introduced a weighted loss for disambiguation, serving as a generalized version across multiple algorithms.While these algorithms exhibit considerable efficacy in tackling partial-label learning problems, they encounter limitations in directly handling inexact supervision within the instance space. Consequently, they cannot be directly applied to multi-instance partial-label learning problems.MIPL is an extension that encompasses both MIL and PLL. Its objective is to tackle the challenge of inexact supervision simultaneously present in both instance and label spaces. To our knowledge, only three viable MIPL algorithms, i.e., MIPLGP [16] , DEMIPL [21] , and ELIMIPL [22] , currently exist. Tang et al. [16] have introduced the MIPL framework, with MIPLGP adopting an instance-space paradigm. MIPLGP is structured in three steps. Firstly, it augments a negative class for each candidate label set. Secondly, it treats the candidate label set of each multi-instance bag as that of each instance within the bag. Finally, it employs the Dirichlet disambiguation strategy and the Gaussian processes regression model for disambiguation. On the other hand, DEMIPL follows the embedded-space paradigm and consists of two steps. Initially, it aggregates each multi-instance bag into a unified feature representation through the disambiguated attention mechanism. Subsequently, it employs a momentum-based disambiguation strategy to discern true labels from candidate label sets. Following this way, Tang et al. [22] have proposed ELIMIPL to exploit the information from candidate and non-candidate label sets via three loss functions. Specifically, ELIMIPL learns the mappings fromThe implementations of the compared algorithms are publicly available. Table A6 includes the URLs of all compared algorithms in this paper, while the source code of our proposed MIPLMA is included in the supplementary material. The MIPL datasets can be accessed publicly at http: //palm.seu.edu.cn/zhangml/.The proposed MIPLMA has several potential societal impacts, both positive and negative.The positive impacts of MIPLMA include: (a) Medical diagnosis: MIPLMA could enhance the accuracy of medical diagnoses where obtaining exact labels is challenging. For example, in medical image classification tasks, when experts lack confidence in the provided results, MIPLMA can yield more accurate outcomes. (b) Privacy-preserving surveys: Similar to PLL methods, MIPLMA can be applied in scenarios where respondents are hesitant to disclose sensitive information. By allowing respondents to select a set of candidate labels instead of providing a single label, MIPLMA can facilitate data collection while respecting privacy. This can be particularly useful in fields such as mental health, where patients may feel uncomfortable disclosing exact symptoms or conditions.Conversely, there may also exist some negative impacts of MIPLMA. (a) Misuse in surveillance: There is a risk that our method could be exploited in surveillance systems to infer sensitive information about individuals without their explicit consent. For example, an adversary could use our algorithm to analyze multi-instance data collected from social media or other sources to infer private details about individuals, leading to potential breaches of privacy. (b) Job displacement: As MIPLMA improves the efficiency and effectiveness of learning from inexact data, it might reduce the demand for human annotators. This could lead to job displacement involved in data labeling and annotation. Consequently, efforts should be made to retrain and upskill these workers to mitigate the impact. 


Decision: The paper introduces MIPLMA, a Multi-Instance Partial Label algorithm that aims to maximize margins in both instance and label spaces. In instance space, it leverages a temperature parameter in the attention mechanism to increase the gap between attention scores for positive and negative instances. In label space, it employs a margin loss that combines two objectives: maximizing the margin between the highest predicted probability in the candidate label set versus the non-candidate label set, and minimizing the variance of these margins across bags. Empirical results on four benchmarks and one real world dataset show satisfactory performance compared with existing methods.

The reviewers appreciated the margin based formulation of the problem and found the empirical results to be promising. The technical novelty is on the low side but overall the paper presents a well executed study showing the benefits of different margin objectives for MIPL problems.

Reviews: review: paper_summary: paper_summary: The paper proposes MIPLMA, a new Multi-Instance Partial Label algorithm that focuses on margin adjustments in both instance and label space. While computing the bag level representations, it introduces a temperature parameter in the margin-aware attention mechanism to widen the gap between attention scores for positive and negative instances. It also employs a margin loss to maximise the margin between the highest predicted probability on the candidate label set and on the non-candidate label set.
strength_weakness: strength_weakness: - In many cases, either the number of classes is low or the number of false positives is low, which raises concerns regarding the generalization of the algorithm

- Paper has not been compared with some of the recent PLL algorithms like PICO [A], etc.  A discussion on why such comparisons were not done would be helpful. Maybe adaptation was difficult, or they focused on a single modality?

- Contributions are limited, it is a direct adaptation of margin-based losses to the problem

- The applications of this setup need to be more convincing to me. They do give one application on a Cancer detection dataset, however, I believe it could be achieved through an architecture capable of processing higher-resolution images. 

[A] Wang et al. PiCO: Contrastive Label Disambiguation for Partial Label Learning. ICLR 2022
questions: questions: - The creation of the synthetic datasets is unclear. While Table 1 presents details regarding the number of instances in a bag, the distribution proportion of true positives in a bag is not mentioned. This could have implications for the performance of the algorithm.


- In section 3.1.2 (line 200), It is mentioned that “PLL algorithms can be equipped with either the linear model or multi-layer perceptrons (MLP) as backbone networks”. However, DNNs like ResNet can be used to learn feature representations. Ideally, the comparison should be made using the same feature extractors, the ones mentioned in lines 208 - 213.

- I would willing to reconsider my ratings post the author rebuttal
limitations: limitations: Yes
paper_summary: paper_summary: In this paper, the learning scenario of multi-instance partial learning (MIPL) is studied. The paper proposes a new approach for MIPL, whose key technical idea is margin regularization. The paper points out that an important issue for previous MIPL approaches is the ignorance of margin information in both the outputs of attention modules and final predictors, leading to incorrect attention weights and prediction confidence. To address this issue, the paper proposes a novel margin loss for MIPL, whose effectiveness is verified under various MIPL benchmark datasets.
strength_weakness: strength_weakness: 1. The discussion on the scheme of modifying the strength of margin regularization can be augmented.
questions: questions: I suggest including more discussions on setting the parameter \lambda. In my view, this parameter could significantly affect the learning performance. In my understanding, this parameter can be kept small at the beginning and be gradually increased. The reason is discussed in the paper: at the beginning, the model is less accurate, making the margin less informative, and the thing turns different as learning process proceeds. I find that currently, \lambda is kept as a constant. So I would like to ask for more discussions on the interpretations for this choice.
limitations: limitations: Yes
paper_summary: paper_summary: This paper deals with an emerging learning framework, i.e., multi-instance partial-label learning framework, which can be regarded as an extension and combination of multi-instance learning and partial-label learning. Overall, such dual inexact supervision makes the MIPLL difficult to resolve. This paper introduces a margin-aware attention mechanism and margin-compliant loss to deal with such issue.
strength_weakness: strength_weakness: 1. I am confused with the definition of MIPLL as the authors claimed in the paper: "positive instances refer to the instances that belong to the true label in the setting of MIPL"，I agree with the definition of positive instances. However, for the negative instances, " while negative instances represent the remaining instances in the bag that are not associated with any label in the label space". Does the author indicates that the negative instances are not associated with the whole label space or just the candidate label set for the given bag?  I think the authors should be careful with this definition. I don't think the whole label space is the correct description w.r.t the negative instances.
2. The so called margin-aware attention mechanism and the margin-compliant loss are not novel, which in my opinion have already been utilized in other well-defined issues. So the novelty cannot reach the bar of NeruIPS.
3. In addition, although the multi-instance partial label learning framework is relatively novel and interesting, there has already existed at least two pioneer works, which again decreases the novelty of this work. 
4. As compared with existing methods, the proposed algorithm cannot achieve the dominant performance on all the datasets under all the settings.
questions: questions: N/A
limitations: limitations: See the weakness.
paper_summary: paper_summary: In this paper, the authors observe the presence of ‘margin-violations’ in the MIPL problem, which manifest in two aspects: between positive and negative instances, and between candidate and non-candidate labels. Therefore, the paper proposes modifications to the attention mechanism and enhancement of the loss function to rectify and widen the margin.
strength_weakness: strength_weakness: 1. In Figure 1, the authors compare the performances of MIPLMA and DEMIPL, however, according to the results in Table 2 and Table 3, ELIMIPL surpasses DEMIPL on FMNIST-MIPL, and both ELIMIPL and MIPLGP perform better than DEMIPL on CRC-MIPL-Row dataset, so the comparison with these two methods would be more beneficial.
2. When discussing the linear-decay strategy for temperature in Equation 3, the sudden introduction of the constant coefficient 0.95 requires some clarification for better understanding.
3. In Sec.3.1.1, the authors claim that they are the first to employ ResNet on CRC-MIPL dataset, so it is suggested that the authors should list the detailed baseline performances on C-R34-16 and C-R34-25 datatset bag generators, just as in Table.3.
4. The authors assert that “features learned by ResNet-34 outperform those obtained via image bag generators in terms of classification performance.” However, it remains unclear whether this still holds on other benchmarks such as FMNIST-MIPL. To strengthen this claim, it would be beneficial to add further experiments or discussions on this.
questions: questions: See weaknesses
limitations: limitations: NA
paper_summary: paper_summary: The paper proposes an approach to tackle the multi-instance partial-label learning (MIPL) problem. In MIPL, each training sample is represented as a bag of instances associated with a set of candidate labels. The existing MIPL algorithms may gives the high prediction probability on non-candidate label, which can lead to suboptimal performance. They propose MIPLMA  algorithm addresses this issue by introducing a margin-aware attention mechanism and a margin-compliant loss function. The algorithm dynamically adjusts the margins for attention scores, while the loss function ensures that the margins between predicted probabilities on candidate and non-candidate label sets are properly constrained. The paper presents experimental results on benchmark and real-world datasets that demonstrate the superior performance of MIPLMA compared to existing MIPL algorithms, as well as other multi-instance algorithms and partial-label learning algorithms.
strength_weakness: strength_weakness: 1. The margin-aware attention is performed by setting a changing temperature parameter which seems a trival idea. And the details of how the gap of attention scores help the model to decrease the prediction probability on non-candidate labels are also not given.
2. The form of margin-compliant loss seems not easy to be optimized in neural network framework, the paper should give more details on the proposed loss function.
questions: questions: 1. The initial weights are uniformly distributed. After the training in first epoch, the weights change. Subsequent epochs can only make the margin of the distribution larger, so the results are greatly affected by the first epoch. If there are errors in the training of the first few epochs, Will this error accumulate?
limitations: limitations: Yes.

Comments: title: Further Explanation on the Feature Extractor
Post rebuttal response
Looking Forward to Your Feedback
Looking Forward to Your Feedback
Looking Forward to Your Feedback
Looking Forward to Your Feedback
Looking Forward to Your Feedback
comment: Thank you for your positive feedback!
Thanks for the author's rebuttal, and most of my concerns have been fixed. So I would like to raise my score.
Thank you for your feedback and for updating your rating.

We acknowledge that various PLL methods, such as POP [B], PRODEN [C], and LWS [D], use different backbones like linear models, MLP, and ResNet depending on the dataset. For example, PRODEN and LWS mainly use linear and MLP models for MNIST and FMNIST datasets, while ResNet and convNet are employed for CIFAR-10 dataset. In our experiments, LeNet and ResNet-34 performed worse than our two-layer CNN on the MNIST-MIPL and FMNIST-MIPL datasets. This is likely due to the relatively simple nature of the features in these datasets, where deeper networks like ResNet may not provide significant advantages in weakly supervised scenarios. Additionally, the Birdsong-MIPL and SIVAL-MIPL datasets are preprocessed tabular data, which makes ResNet unsuitable for these datasets.

For the C-R34-16 and C-R34-25 datasets, we used ResNet-34 as the feature extractor for both MIPLMA and the compared PLL algorithms. The table below presents the results of these algorithms, where Mean and MaxMin represent the aggregation strategies used to transform instance-level features into bag-level features. The results indicate that MIPLMA outperforms the compared PLL algorithms, even with the same feature extractor. We will include more detailed experimental results in the revised manuscript. Additionally, we plan to collect higher-resolution WSI datasets and the use of pre-trained backbones, including large language models, for better feature learning.

| Algorithm       |   C-R34-16    |   C-R34-25    |
| --------------- | :-----------: | :-----------: |
| MIPLMA          | **.631±.008** | **.685±.011** |
| LWS (Mean)      |   .593±.018   |   .614±.021   |
| LWS (MaxMin)    |   .462±.014   |   .460±.016   |
| PRODEN (Mean)   |   .537±.015   |   .585±.019   |
| PRODEN (MaxMin) |   .434±.010   |   .444±.012   |
| POP (Mean)      |   .549±.014   |   .591±.014   |
| POP (MaxMin)    |   .450±.009   |   .462±.015   |

References:

[B] Xu et al. Progressive purification for instance-dependent partial label learning. ICML 2023.

[C] Lv et al. Progressive identification of true labels for partial-label learning. ICML 2020.

[D] Wen et al. Leveraged weighted loss for partial label learning. ICML 2021.
The author response addresses some of my concerns and I have updated my rating accordingly. However, the last point is still unclear. Line 200-201, are misleading, why limit to linear or MLP backbone. I see no difficulty in a using a resnet backbone for instance (for example LWS based algorithm do use ResNet backbone in the experiments). And even if the authors are using pre-trained backbones, the adaption and fine-tuning is certainly possible.
Dear Reviewer Unv6,

Thank you for your valuable feedback, which has greatly improved our paper. We have carefully addressed your concerns by providing visualizations of attention scores and bag-level features, detailing the optimization of the margin-compliant loss, and demonstrating how our dynamic disambiguation loss mitigates early errors. Should any issues remain, we are happy to discuss them further.

Best regards,

The Authors
Dear Reviewer b79F,

Thank you for your constructive feedback, which has significantly enhanced our manuscript. We have addressed your concerns by conducting experiments on the temperature parameter decay rate, providing results for the C-R34-16 and C-R34-25 datasets, and offering further explanations on the feature extractor. If there are any remaining issues, we are open to further discussion.

Best regards,

The Authors
Dear Reviewer BX8M,

Thank you for your valuable feedback, which has significantly improved our paper. We have carefully addressed your concerns regarding the definition of negative instances and the novelty of our work. We have reviewed and discussed two related MIPL studies [A, B] with different negative instance setups, which will be discussed in the revised manuscript. Should any issues remain, we welcome further discussion.

Best regards,

The Authors



[A] Wang et al. On learning latent models with multi-instance weak supervision. NeurIPS 2023.

[B] Wang et al. On characterizing and mitigating imbalances in multi-instance partial label learning. arXiv:2407.
Dear Reviewer zb3e,

Thank you for your insightful feedback, which has greatly enhanced our paper. We have carefully addressed your concerns, including conducting experiments on the dynamic adjustment strategy for the parameter $\lambda$. If you have any further questions or concerns, we would be pleased to discuss them.

Best regards,

The Authors
Dear Reviewer H7jM,

Thank you for your thoughtful and detailed feedback, which has greatly improved our manuscript. We have carefully addressed your concerns by providing additional results on false positives in the SIVAL-MIPL dataset, comparing our method with the POP PLL method, and designing a variant of MIPLMA. We also included explanations about MIPL applications and the creation of synthetic datasets. Should any issues remain unresolved, we are happy to discuss them further.

Best regards,

The Authors