Imbalanced Multi-instance Multi-label Learning via Coding Ensemble and Adaptive Thresholds

Authors: xinyue  zhang
tingjin  luo
yueying  liu
chenping  hou
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-24 09:03:15

Abstract: Multi-instance multi-label learning (MIML), which deals with objects with complex structures and multiple semantics, plays a crucial role in various fields. In practice, the naturally skewed label distribution and label dependence contribute to the issue of label imbalance in MIML, which is crucial but rarely studied. Most existing MIML methods often produce biased models due to the ignorance of inter-class variations in imbalanced data. To address this issue, we propose a novel imbalanced multi-instance multi-label learning method named IMIMLC, based on the error-correcting coding ensemble and an adaptive threshold strategy. Specifically, we design a feature embedding method to extract the structural information of each object via Fisher vectors and eliminate inexact supervision. Subsequently, to alleviate the disturbance caused by the imbalanced distribution, a novel ensemble model is constructed by concatenating the error-correcting codes of randomly selected subtasks. Meanwhile, IMIMLC trains binary base classifiers on small-scale data blocks partitioned by our codes to enhance their diversity and then learns more reliable results to improve model robustness for the imbalance issue. Furthermore, IMIMLC adaptively learns thresholds for each individual label by margin maximization, preventing inaccurate predictions caused by the semantic discrepancy across many labels and their unbalanced ratios. Finally, extensive experimental results on various datasets validate the effectiveness of IMIMLC against state-of-the-art approaches.

Main paper: 
 introduction: 
 A complicated object in many practical applications contains several labels simultaneously and its own intrinsic structure, which may be thought of as a bag of instances. Multi-instance multi-label learning (MIML) is a weakly supervised learning paradigm [45] that can handle such samples. By simulating the interaction between instances and the label set, it enhances the generalization capacity of the model by helping it comprehend the multiple meanings and overall structure of the object. With adaptation to different practical scenarios by adjusting the combination of instances, MIML is highly flexible and has been applied to various fields, such as text analysis [4, 42] , audio and video location mining [11, 39, 40] , signal recognition [24] , and medical diagnosis [15, 21] , etc.In practice, it is typical for the label distribution to be inherently skewed, which inevitably raises the problem of label imbalance. For example, in breast medical imaging diagnosis, a single medical image viewed as a bag consists of many tissue pixels, as shown in Fig. 1 . Because cancer patients make up a small percentage of the population overall, there are often one to several orders of magnitude fewer pixels of tumor tissue than of normal tissue. As a result, it has a significant label imbalance issue due to bags and diseaserelated cases being minority samples, which is a typical long-tailed distribution. In addition, label dependence will exacerbate the issue of imbalance. On the one hand, various bags usually match various numbers of label classes. As an illustration, Fig. 1 (a) shows three different kinds of lesions in the breast tissue. Actually, there is only one label for "normal" in an image of normal tissue, while others with more severe and intricate pathological processes include several breast tissue lesion labels. This leads to an increase in the difference in the number of samples that each label class possesses. On the other hand, certain labels have a greater co-occurrence frequency due to label correlation. For example, pathological processes with the same causes often co-occur. Concurrent identification of related lesions can proceed simultaneously when the image contains specific lesion labels. As the number of samples with certain labels increases, there are more samples with related labels, making the problem of imbalance more severe and difficult to solve.Compared with traditional classification tasks, there are two main challenges in handling such imbalanced MIML problems. (1) In MIML tasks, only bag-level labels are available, while labels of instances recording features are unknown. The inconsistency of levels at which features and labels are located makes the task very complicated. (2) In imbalanced distributions, it is more difficult to handle tasks due to the effect of class overlap, data selection bias, and insufficient prior being magnified.To solve these problems, many algorithms have been proposed in the literature. To address coarse-grained labeling issues, there are many MIML methods. Degradation methods transform the original MIML task into either a multi-instance learning (MIL) task or a multi-label learning (MLL) task to eliminate inexact supervision. Some work discretizes bag-level labels to label instances to train an instance-level classifier [48] , and some work represents the bag as a new feature vector using the information provided by instance features to learn a bag-level classifier [16, 18, 37, 42, 48] . In recent years, some work has jointly learned the bag-level classifier and instancelevel classifier to cover both features and labels simultaneously [35, 41] . To solve label imbalance problems, there are many imbalanced learning methods. Resampling methods generate or throw some samples to balance data distribution [2, 8, 30, 38] . Reweighting methods assign different weights to samples to balance the importance of different sizes of the population for the model [5, 17, 22, 44] . Ensemble methods incorporate multiple base models to enhance recognition abilities for minority classes [3, 19, 26] . Meta-learningbased methods reweight samples through self-adaptation to tackle majority classes and minority classes differently [6, 7, 20, 23] .Although these approaches have been successfully applied to various applications, they still face several obstacles when directly used in the imbalanced MIML task. (1) Traditional MIML or imbalanced learning methods only focus on one aspect of our setting and are unable to effectively address our problems. (2) Due to the assumption of balanced label distributions, existing MIML algorithms tend to focus on majority-class samples and ignore the contribution of minority classes, which results in biased models and decreased performance. (3) Traditional methods often assign labels to multisemantic objects using a fixed threshold, resulting in unreliable predictions due to ignoring the semantic discrepancy among multiple labels and the imbalanced ratios of different classes.To solve these problems in this new learning scenario, we propose an imbalanced multi-instance multi-label learning method (IMIMLC), based on the error-correcting coding ensemble and an adaptive threshold strategy, whose main framework is presented in Fig. 2 . Specifically, we conduct feature embedding to extract the structural information of each object via the fisher vector and eliminate the effect of inexact supervision. Subsequently, to alleviate the disturbance of the imbalanced distribution, a novel ensemble strategy is constructed based on concatenating the error-correcting codes of randomly selected subtasks. IMIMLC trains multiple binary base classifiers on small-scale data blocks and learns the more reliable prediction to enhance the diversity of base learners and model robustness for imbalanced data. To avoid unreliable predictions caused by traditional fixed thresholds, we transform the threshold determination problem into learning classifiers with the maximum margin of soft labels, thereby IMIMLC adaptively learning threshold for each label. Finally, extensive experimental results on various MIML datasets verify the effectiveness of IMIMLC against state-ofthe-art MIML approaches in terms of imbalance-specific evaluation metrics. In summary, the main contributions are listed as follows.• We propose IMIMLC to solve a crucial but rarely studied problem, the label imbalance of MIML. As far as we know, this is the first attempt to tackle MIML tasks with the imbalanced label distribution. • IMIMLC establishes a novel ensemble model and trains multiple base classifiers on randomly partitioned small-scale data blocks, which enhances the diversity of base learners and model robustness for the imbalanced distribution issue. Existing MIML methods proposed in the literature can be summarized according to the types of learned classifiers. With the assumption that each instance in a bag contributes equivalently and labels are independent, Zhang et al. [48] have proposed MIML-Boost and MIMLSVMmi algorithms by splitting multi-label sets into several binary classification label pairs to learn the instance-level classifier. To learn the bag-level classifier, Zhang and Zhou [48] have proposed the MIMLSVM algorithm that uses cluster to map for converting MIML samples into MLL samples. Based on this work, an increasing number of algorithms have been proposed for more robust classifiers or more accurate distance measurements, which achieved better performance [18, 37, 48] . Considering highly correlated labels often share information, Li et al. [16] have proposed the KISAR algorithm, which explores the reasonable relationship between input patterns and output labels to identify key instances that trigger labels. Similar to this work, the MIMLwel algorithm [42] tackles the weak labeling problem in MIML tasks where only a part of positive labels are tagged. In recent years, some work has considered learning at both the instance level and the bag level to train a joint classifier, which enriches the algorithms used for handling MIML tasks. Based on active learning, the CM2AL algorithm [35] queries the most probably positive instance-label pairs, labeling the bag based on these feedbacks. Founded upon attention mechanisms and manifold learning, the MIML-LLMC algorithm [41] handles bag structures and learns instance-label relations simultaneously. 
imbalanced learning: 
 In real MIML tasks, imbalance is a common characteristic of datasets. It can be viewed from three perspectives: imbalance within labels, imbalance between labels, and imbalance among the label sets [36] . The learning task with the label imbalance issue is more challenging. Specifically, during training, the model tends to concentrate more on the features and patterns of majority samples while ignoring important information from minority samples. It results in a weak ability of the model to recognize minority classes. Due to such insufficient learning, the model may mistakenly predict minorityclass samples as majority-class samples, resulting in a lower recall for minority classes. To solve these problems, an increasing number of methods have been proposed to adjust label distribution and enhance the ability of the model to recognize minority classes, thereby improving its performance and robustness.Resampling: Resampling methods balance data distribution in the training set by over/undersampling [2, 8, 30, 38] , or by filtering out noise through cleaning resampling [9, 13] . Reweighting: Reweighting methods assign different weights to samples to balance the importance of different sizes of the population for the model [5, 17, 22, 44] . Ensemble Methods: Ensemble methods improve recognition abilities for minority samples by combining multiple specific models [3, 26] . Liu et al. [19] have investigated the combination of the resampling method and the ensemble learning framework, which is proven effective. Meta-learning Methods: Meta-learning, which can also be formulated as a domain-adaptive strategy [10] , has been applied to imbalanced learning [6, 7, 23, 25] . It tackles majority samples and minority samples differently and learns how to reweight through self-adaptation [33] . Liu et al. [20] have proposed the MESA algorithm, which learns the model from the imbalanced distribution based on the meta-sampler boost ensemble that trains the meta-sampler on task-agnostic meta-data and directly learns sampling strategies from the data. 
method: 
 To tackle MIML tasks with imbalanced label distributions, we propose a novel imbalanced multi-instance multi-label learning method (IMIMLC) based on the error-correcting coding ensemble and an adaptive threshold strategy. Our IMIMLC mainly consists of feature embedding, code-based ensemble training, and classification threshold adapting, which are shown in Fig. 2 . Specifically, IMIMLC first extracts structural information from instances and represents bags via fisher vectors to train the ensemble model on randomly partitioned data blocks. The ensemble model updates the bag representation with semantic annotation, based on which IMIMLC self-adapts classification thresholds for imbalanced label distribution. Besides, its computational complexity analysis is provided.In MIML samples, only bag labels are available, while labels of instances that record features are unknown. To eliminate the effect of such inexact supervision, we extract the structural information of each bag by feature embedding.Assume instances Gaussian model. To retain distribution characteristics of instances and remove redundant information, the bag representation x𝑖 can be described as the fisher vector, which whitens the gradient of the log-likelihood of the GMM 𝑝 [31] , i.e.,{𝒙 1 𝑖 , 𝒙 2 𝑖 , • • • , 𝒙 𝑛 𝑖 𝑖 } in B 𝑖EQUATIONwhere L 𝜃 is the Cholesky decomposition for whitening. Besides, similar to [31] , we adopt the same normalization approach for x𝑖 to reduce variance dependence on bag-specific information. Note that x𝑖 , whose dimension is only related to the dimension of parameter 𝜃 [31] . In other words, bags with different numbers of instances can be represented as new feature vectors with a consistent dimension. It is beneficial to algorithm implementation and popularization. More details can be found in the appendix.After obtaining x𝑖 , imbalance is mainly reflected in two aspects. On the one hand, a specific label class may contain a large number of negative samples and a few positive samples. On the other hand, the number of samples that different label classes have varies greatly. In this case, traditional MIML methods pay more attention to majority classes while ignoring important information from minority ones. It results in biased models, which may mistakenly classify minority-class samples as majority-class samples during prediction. To tackle these problems, our IMIMLC constructs the coding-based ensemble model by three stages: encoding, training base classifiers, and decoding. Specifically, to rectify mistakes made by the interference of the label imbalance issue, IMIMLC trains multiple base classifiers on randomly partitioned small-scale data blocks based on error-correcting codes that correspond to subtasks randomly selected from the decomposed MLL task.Encoding: Denote 𝑄 as the transformed MLL task with 𝐶 labels, which can be decomposed into several equally sized subtasks, each of them containing 𝑘 labels, where 𝑘 ≤ 𝐶. Subtask set 𝑄 𝑘 , which involved 𝑘 labels of interest, consists of C 𝑘 𝐶 subtasks. IMIMLC randomly selects 𝑑 subtasks {𝑤 𝑖 |𝑖 = 1, 2, • • • , 𝑑 } from 𝑄 𝑘 without replacement and encodes them. Encoding follows that the one-vs-all encoding strategy (OVA) [28] is applied to selected 𝑘 labels, and "0" is assigned to labels that are not selected. Therefore, we obtain the encoding matrixZ (𝑖 ) ∈ {-1, 0, +1} 𝐶 ×𝑘 , 𝑖 = 1, 2, • • • , 𝑑 of subtasks.Finally, the whole training encoding matrix Z ∈ {-1, 0, +1} 𝐶 ×𝑘𝑑 is concatenated by Z (𝑖 ) with linking codewords from the same label.Training: As shown in Fig. 2 (b), IMIMLC treats each column of Z as the label code and constructs the binary classifier ℎ 𝑗 (•) based on each row of Z. It means that values "0", "1", and "-1" in Z not only represent codewords for labels but also indicate which bags need to be involved in the training of ℎ 𝑗 (•) and play the role of positive or negative categories in this process. In detail, if the label is encoded by "+1", bags related to that label will be treated as positive samples for ℎ 𝑗 (•). If the label is encoded by "-1", bags related to that label and not related to any label encoded by "+1" will be treated as negative samples for ℎ 𝑗 (•). If the label is encoded by "0", bags only related to that label will be excluded from training the base classifier ℎ 𝑗 (•).It is noteworthy that training bags of ℎ 𝑗 (•) are selected by OVAbased coding and relationships among labels. As the number of subtasks increases, label correlation is more fully utilized, and model performance will be improved. Besides, the imbalance problem in small-scale data blocks may not be so serious. Different binary base classifiers are trained on different data blocks, which enhances the diversity of classifiers and recognition abilities for minority classes.Decoding: Conducting all base classifiers on x𝑖 , we can get a code 𝒛 𝑖 composed of corresponding codewords 𝑐 1 𝑖 , 𝑐 2 𝑖 , • • • , 𝑐 𝑘𝑑 𝑖 , which are binary classification results. The matching of x𝑖 and the label set relies on the hamming distance between 𝒛 𝑖 and all label codes. However, elements of the 𝑙-th label code 𝒛 𝑙 may contain "0", as the 𝑙-th label is irrelevant to the training of the corresponding base classifier, and that pair of codewords will certainly be different, resulting in a larger hamming distance. Such meaningless codeword pairs should be excluded to obtain more accurate and reliable values. Therefore, the valid hamming distance between two codewords can be defined asEQUATIONwhere1 𝑐 𝑠 𝑙 = 1 when 𝑐 𝑠 𝑙 = 0, otherwise 1 𝑐 𝑠 𝑙 = 0.Typically, if 𝐷 𝑉 between 𝒛 𝑖 and 𝒛 𝑙 is less than the threshold 𝜂 𝐷 , B 𝑖 can be matched with the 𝑙-th label. As we have analyzed, the matching result is determined by multiple binary classification results. When the imbalanced distribution interferes the learning of the binary classifier, the ensemble model can correct this error and output more reliable results. However, because different labels participate in the training of different numbers of base classifiers, their meaningful codeword lengths are different, as the quantity of meaningful codeword pairs is determined by the number of base classifiers that label participates in. It is unfair to use a fixed threshold 𝜂 𝐷 to cover all label judgments. Hence, we match the bag and label semantics by 𝑡 𝑖 𝑙 ,EQUATIONwhich is related to the number of base classifiers whose training the 𝑙-th label participates in. 1 -𝑡 𝑖 𝑙 can be regarded as the matching degree between the 𝑖-th bag and the 𝑙-th label semantics. Finally, 𝒑 𝒛 𝑖 = 1 𝐶 -𝒕 𝑖 can be viewed as a new feature representation that contains semantic information, or the soft label set of B 𝑖 .In multi-label scenarios, it is not easy to obtain the precise tags of the object based on soft labels. In imbalanced MIML tasks, it is more challenging. The multiple semantics of the bag make it correspond to different numbers of labels. With the increasing dimensionality of the label space, differences among soft labels are tiny. Small changes can lead to misjudgments. Besides, in label imbalance cases, the fixed threshold adopted by traditional methods may incorrectly label the bag as majority classes with high prior probabilities. To obtain more reliable predictions, we adaptively learn classification thresholds for multiple label classes.Concretely, we decompose this labeling task into multiple classification tasks, establishing a binary classifier for each label. The process of adapting the threshold is equivalent to maximizing the margin between two half spaces, the "larger than" one and the "less than" one. Since the inputX 𝑝 = [𝒑 𝒛 1 , 𝒑 𝒛 2 , • • • , 𝒑 𝒛 𝑀 ]of classifiers contains semantics and label correlation has been considered in the training of the ensemble model, it is acceptable to perform decomposition here. Decomposition is beneficial for adapting thresholds for different labels more pertinently without being affected by majority class priors or label dependence. Hence, its objective can be formulated as the following constraint optimization problemEQUATIONwhere 1 ≤ 𝑖 ≤ 𝑀, 𝜙 (•) maps 𝒑 𝒛 𝑖 into a higher-dimensional space, and 𝜆 is the non-negative regularization hyper-parameter. The problem in Eq. ( 4) is a classical convex quadratic problem with inequality constraints. Motivated by [29] , it can be equally transformed into the following unconstrained optimization problem by imposing the Lagrangian multipliers 𝜶 , 𝜷EQUATIONi.e., min on Karush-Kuhn-Tucker (KKT) [12, 14] . Obeying the KKT conditions, the dual problem of Eq. ( 4) can be equally formulated asEQUATIONMany algorithms related to quadratic programming can be used to deal with the problem in Eq. ( 6). We adopt the sequential minimal optimization (SMO) algorithm, which is faster and more efficient [27] , to get the optimal solution 𝜶 * . According to the primal-dual relationship and KKT, optimal 𝝎 * 𝑙 satisfiesEQUATIONFinally, the adaptive threshold for the 𝑙-th label 𝜂 𝑙 𝐴 can be obtained by SMO [27] or calculated asEQUATIONFor a new testing bag B, its structure information from instances { x1 , x2 , . . . , x n } is embedded in x by Eq. ( 1). Then x will be updated as a new vector 𝒑 ẑ with semantic annotation via the coding-based ensemble model proposed in Subsection 3.2. According to trained classifiers {𝝎 * 𝑙 |𝑙 = 1, 2, • • • , 𝐶} on 𝒑 𝒛 𝑖 by Eq. ( 7) and Eq. ( 8), we can compute the prediction score 𝝎 * 𝑇 𝑙 𝜙 (𝒑 ẑ ) and 𝜂 𝑙 𝐴 . The comparison of 𝝎 * 𝑇 𝑙 𝜙 (𝒑 ẑ ) and 𝜂 𝑙 𝐴 determines the positive/negative output under the classifier for the 𝑙-th label. According to T-criterion [1] , if 𝒑 ẑ is judged as positive by the binary classifier, B is marked by the corresponding label; if all classifiers judge 𝒑 ẑ as negative, B will be marked by the label that has the fewest negative marks. In other words, the hard label set of testing bag B isŶ = {𝑙 |𝝎 * 𝑇 𝑙 𝜙 (𝒑 ẑ ) ≥ -𝜂 𝑙 𝐴 , 𝑙 ∈ Y}∪ arg min 𝑙 ∈ Y 𝑀 ∑︁ 𝑖=1 1 -𝜂 𝑙 𝐴 𝝎 * 𝑇 𝑙 𝜙 (𝒑 𝒛 𝑖 ) ,where 1 𝑎 (𝑥) is the indicator function, whose value is 1 when 𝑥 < 𝑎.The computational complexity of IMIMLC can be analyzed in two parts: (1) bag vector representation based on instances, and (2) imbalanced learning based on the coding ensemble and adaptive thresholds. We estimate the complexity as O (𝑁 𝐼 𝐺𝑑 2 (𝑇 1 + 1) + 𝑘𝑑 (𝑇 2 𝑁 3 𝑠 + 𝑁 𝑠 𝑑 1 + 3𝑀𝐶) +𝐶𝑁 3 𝑠 ). More analyses are in the appendix. 
experiments 4.1 datasets: 
 We conducted experiments on six public MIML datasets, including MIML-image, MIML-text, HJA Bird Song, MSRC v2, Letter Carroll, and Isoform Gene Data. Their brief information is reported in Table 1 . Specific information about datasets is in the appendix. Note that different datasets have different extents of label imbalance issues. To measure the level of label imbalance in different datasets, inspired by [19] , we adopt the average imbalance ratio (AIR), which is formulated as where |𝑁 𝑖+ | and |𝑁 𝑖 -| represent the quantity of minority and majority class samples under the 𝑖-th label, respectively. AIR ranges from 0 to 1, and the more balanced the label distribution, the closer this value is to 1, otherwise, to 0. AIRs of datasets are presented in the last column of Table 1 .EQUATIONTo showcase the performance of our IMIMLC, several representatives of MIML algorithms were selected for comparison. MIMLSVMmi [48] degenerates the original MIML task into an MIL task to learn the instance-level classifier. MIMLSVM [43] , MIMLNN [48] , MK-EnMIMLNN [18] , KISAR [16] , and MIMLwel [42] convert original MIML tasks into MLL tasks to learn bag-level classifiers. CM2AL [35] and MIML-LLMC [41] investigate the underlying relationships among instances, bags, and labels to learn joint classifiers. Among them, CM2AL, MK-EnMIMLNN, and MIML-LLMC are state-of-the-art algorithms. To comprehensively quantify the performance of all algorithms, we adopt six evaluation metrics, which are commonly used and listed as follows:• Accuracy (ACC, ↑): The proportion of correctly predicted labels for the testing data. • Area Under the Curve (AUC, ↑): The average area under the Receiver Operating Characteristics (ROC) curve.• F1 score (F1, ↑): The harmonic average of precision and recall for classification. • Hamming Loss (HL, ↓): The proportion of mislabeled predictions, which is the dual metric of accuracy. • One-Error (0-1, ↓): The proportion of samples where the most likely label is not the true label. • Ranking Loss (RL, ↓): The proportion of cases where the false soft labels are ranked higher than the true soft labels.To ensure a fair comparison of classification performance, experiments were conducted with the optimal parameters of all algorithms. For each dataset, 60% of its samples were selected as the training set, 20% as the validation set, and the remaining 20% as the testing set. To eliminate the effect of random factors, experiments were independently repeated 20 times. The average and standard deviation of AUC, F1-score, accuracy, hamming loss, one-error, and ranking loss on all datasets are recorded in Table 2 . As seen from the results in Table 2 , we have several observations.(1) IMIMLC outperforms other algorithms in most cases. For example, on the LC dataset, IMIMLC performed approximately 7% better than the second-best MK-EnMIMLNN in evaluation based on AUC. In terms of F1, IMIMLC even outperformed second-best MIMLSVMwel by approximately 24%, which was a significant improvement. The reason may be that IMIMLC has mitigated the imbalance issue with the coding-based ensemble and an adaptive threshold strategy. (2) Six evaluation metrics, AUC, F1, ACC, 0-1, RL, and HL, are adopted to evaluate the performance of all MIML methods from different aspects. In different tasks and scenarios, the ranks of their classification performance based on different metrics are different. For example, on the TEX dataset, the AUC of our IMIMLC was slightly inferior to that of CM2AL, while its F1 score was significantly better than others. Thus, with imbalanced data, we should adopt multiple evaluation metrics to verify the effectiveness of algorithms. (3) Comparison methods KISAR and MIML-LLMC sometimes performed more competitively than IMIMLC on the IMG dataset. However, these algorithms have only considered one aspect of our scenario setting without taking into account the issue of label distribution imbalance, which makes them less effective than IMIMLC in more experiments. To deeply evaluate the performance of all comparison methods, we conducted statistical comparisons based on parametric and non-parametric tests. For the parametric test, t-tests evaluate the statistical differences between two specific algorithms on individual metrics in each dataset. For the non-parametric test, the Nemenyi tests compare performance based on the average ranks of algorithms in all datasets. Their results are presented in Table 2 and Fig. 3 , from which we have the following observations.(1) The t-test results in the last row of Table 2 demonstrate that the performance of IMIMLC is still better than others in most cases. In paired competitions with the t-test, the win rate of IMIMLC was at least 77.50%, sometimes even 89.58%. Besides, different datasets have different extents of label imbalance issues. Even in datasets with heavy imbalances, IMIMLC can still play to its strengths and win in t-tests. (2) Fig. 3 shows critical difference diagrams of Nemenyi tests based on all metrics. In each subfigure, the average ranks of algorithms are marked along the axis, with lower ranks to the left. IMIMLC achieved the lowest average rank against other comparison approaches in terms of all metrics. It has been proven that IMIMLC is better than others. Besides, there are some specials in these subfigures. In Fig. 3c , although IMIMLC is at the top of the ranking, its rank exceeds "2" and approaches "3". It reminds us that in imbalanced datasets, evaluations based on accuracy are unstable. It is necessary to construct a comprehensive evaluation system.To better verify the effectiveness of our IMIMLC, we conducted ablation studies to investigate the effect of each component. Firstly, we validate the effectiveness of feature embedding based on the fisher vector and fusion coding-based ensemble. Specifically, in stage 1, we replaced our method with a classical method, which extracts features by calculating the average of instance feature values, to validate the effectiveness of the feature embedding approach adopted by IMIMLC. In stage 2, we replaced our strategy with MESA, which adopts decision trees (DTs) and SVMs as base classifiers, to validate the effectiveness of the special ensemble model adopted by IMIMLC for addressing the issue of imbalanced label distribution. Thereby, six variants in the ablation experiment were formed. Results on the IMG and MSRC datasets are presented in Table 3 . In stage 1, feature embedding improves model classification ability to some extent. However, only conducting this in stage 1 does not significantly improve model performance. In stage 2, the adoption of the fusion coding-based ensemble effectively alleviates the imbalance problem and results in excellent performance in most metrics. However, only using our strategy in stage 2, it competed with MESA in terms of F1 score. When both stages used strategies proposed in IMIMLC, classification performance had an overall improvement of approximately 5%-10%, even some metrics increased by 20% on the MSRC dataset, which means that the ability of the model to tackle imbalanced MIML tasks is dramatically enhanced.Besides, to further validate the effectiveness of the adaptive threshold method with classifiers in IMIMLC, we conducted experiments based on fixed thresholds of 0.5, 0.6, 0.7, 0.8, and 0.9 and adaptive thresholds. Experimental results based on three metrics, which are mainly influenced by threshold-based hard labels, on the TEX dataset are recorded in Table 4 , and more results on other datasets can be found in the appendix. It is obvious that our IMIMLC involves two hyper-parameters: subset size 𝑘 and selected subtask number 𝑑. To enhance sample diversity and alleviate label imbalance issues, the number of subtasks in 𝑄 𝑘 , C 𝑘 𝐶 , should be maximized. Similar to [32] , 𝑘 would not be too large, and it was controlled to be within {2, 3, 4, 5, 6}. Besides, 𝑑 shouldn't be too small because small-scale subtasks could not cover sufficient labels. Therefore, 𝑑 was controlled to be within {𝐶, 2𝐶, 3𝐶, 4𝐶}. To better declare the effect of these hyper-parameters on the performance of IMIMLC, we report AUCs based on different parameter combinations in two datasets with high label space dimensions in Fig. 4 . Results based on other datasets are in the appendix. As presented in Fig. 4 , when 𝑘 = 3 or 𝑘 = 4, IMIMLC demonstrates better performance, and as 𝑑 increases, the capability of the algorithm improves. Obviously, the optimal hyper-parameter combination is relatively easy to obtain.To further compare the computational efficiency of algorithms, we have recorded the running times of all algorithms on six datasets in Fig. 5 . As shown in Fig. 5 , on the IMG dataset, IMIMLC is indeed more efficient and faster. Compared to KISAR, which has the secondshortest running time, our method had a speed improvement of approximately 22.3%. Additionally, the number of labels, instances, and dimensions also affect the efficiency of the algorithm. On other datasets, IMIMLC did not perform at the fastest computing speed, but it remained at a relatively high level.  
conclusion: 
 We discuss the MIML task in the context of imbalanced label distribution and propose IMIMLC based on the error-correcting coding ensemble and an adaptive threshold strategy to alleviate the effect of imbalance. IMIMLC trains the ensemble model on randomly selected data blocks to enhance the diversity of base classifiers and adaptively learns its thresholds for each semantic class label to obtain more reliable predictions than the manually pre-specified schemes. Finally, extensive experimental results show the effectiveness of IMIMLC against state-of-the-art MIML methods in imbalanced label distribution scenarios. In the future, we will extend IMIMLC to solve problems in different cases, such as noisy labels, novel classes, etc [34, 46, 47] . Besides, there are many more complex nonlinear features in real-word applications. To handle such tasks, we will study the model that ensembles various types of base nonlinear classifiers, such as decision trees, neural networks, etc. 
 
 Appendix:  


Decision: Reviewers provided 3 distinct views on the topic of imbalanced label distribution in multi-instance multi-label learning. I support the paper to be accepted as a poster, as this is a topic that clearly needs more discussion and exploration in our community.

Reviews: review: paper_summary: paper_summary: The paper explores the problem of imbalanced label distribution in multi-instance multi-label learning and proposes IMIMLC based on the error-correcting coding ensemble and an adaptive threshold strategy. Extensive experiments have demonstrated its outstanding performance.
strength_weakness: strength_weakness: 1. The problem statement (solving imbalanced label distribution for MIML task) makes sense, and I believe it holds significant importance for MIML tasks.
2. The experimental analysis is comprehensive and reasonable.
limitations: limitations: The paper presents a clear and intuitive idea and I do not see critical weaknesses. However, some more analysis can be provided on where and to what extent the proposed method works better. For example:

1. Table 2 shows that the proposed method seems to achieve optimal performance across nearly all metrics. However, it is hard to tell how strong the proposed method can perform for minority classes. Is the performance improvement attributable to improvements in these minority classes? A more detailed analysis of the performance gains the proposed method offers for classes with different proportions can be provided.
2. Similarly, Table 3 presents a comparison of different feature embedding algorithms and imbalanced learning algorithms. Can you provide an analysis of their results for classes with different proportions?
paper_summary: paper_summary: This study introduces an innovative unbalanced multi-instance multi-label learning (IMIMLC) approach, integrating error-correcting coding with adaptive thresholding. Utilizing Fisher vector-based feature embedding, it captures object structure and trains binary classifiers on compact data subsets for enhanced prediction accuracy. The method reframes threshold determination as a classifier optimization task, aiming to maximize soft label boundaries and dynamically assign thresholds per label.
strength_weakness: strength_weakness: (1)This article is composed in a systematic and comprehensive style, indicating that the author has cultivated proficient academic writing skills.
(2)This article rigorously tackles the challenge of label imbalance inherent in Multi-Instance Multi-Label (MIML) learning and proposes a novel methodology that synergizes coding-integrated learning with adaptive classification thresholding techniques.
(3)The experimental section is adequate and complete and persuasive.
limitations: limitations: (1)In the Training part of the Methods section, a more elaborate exposition on the integrated learning module is imperative to enhance comprehension, particularly for individuals outside the field. This entails detailing aspects such as the specific integrated learning model employed, its objective function, relevant formulas, theoretical analysis, among other critical components.
(2)The meaning of Eq4 needs to be explained in more detail, e.g. what is wl?
paper_summary: paper_summary: The paper proposes a novel imbalanced multi-instance multi-label learning method named IMIMLC, which addresses the issue of label imbalance in MIML tasks. The method uses error-correcting coding ensemble and adaptive threshold strategy to handle the imbalance problem. Experimental results on various datasets demonstrate the effectiveness of IMIMLC compared to state-of-the-art approaches.
strength_weakness: strength_weakness: 1	The paper presents extensive experimental results across various datasets, highlighting the superior effectiveness of IMIMLC in comparison to other methods.
2	The motivation and the definition of the problem in the introduction are sufficient.
3	From the point of experiment result, the performance of the IMIMLC is good.
limitations: limitations: 1 From a theoretical point of view, I am not sure of the novelty of the proposal. In fact, it seems that it can be developed further by considering, for instance, Imbalance problems strategy more general than ensemble strategy.
2 The imbalanced learning methods mentioned by the author in Chapter 1 and 2.2 imbalanced learning have nothing to do with multi-label learning. The author should conduct a review and analysis of the existing imbalanced learning methods studied in multi-label learning.
3	The experimental analysis is confusing. The author designed some metrics for comparison. For the reader, he does not know which metric is beneficial.